Giulio rebuffo
a4d7b6d33f
Switched Caplin snapshot format to ZSTD blinded blocks ( #9058 )
...
* Chunked format -> blinded
* LZ4 -> ZSTD
* Implemented parent block root support for history download
* Rationale: Allows to optimize GC collection easily on state
reconstruction and it allows to read fast attestations in historical
states reader
2023-12-23 15:56:35 +01:00
Giulio rebuffo
a0b9d60265
Update Consensus Specification tests ( #9024 )
2023-12-19 09:42:11 +01:00
Giulio rebuffo
ffc26b0f00
Added GET SyncCommittees ( #9003 )
2023-12-18 13:54:15 +01:00
Giulio rebuffo
eeb471d800
Added Finality_Checkpoints endpoint ( #8979 )
2023-12-15 02:27:27 +01:00
Giulio rebuffo
4e83ef0ef5
Optimized beacon state retrieval ( #8963 )
...
* Reconstruct previous epoch without looking at DB: no hindrance to
performance -> removed 15GB
* Store inactivity scores and slashings in MDBX and do not store diffs
for them(they are tiny 700/400 bytes)
* Reduced dumps from every 2048 to 1024 -> Added 5 GB (maybe we should
down it to 768)
* Parallel processing of shuffled sets, 2x performance boost in reading
participation.
* Store balances diffs in a Btree diff matter, see:
https://github.com/ledgerwatch/erigon-documents/blob/master/caplin/design/data-model.md#uint64listuint64vector
2023-12-13 15:23:27 +01:00
Giulio rebuffo
24987878e4
Resumable beacon state reconstruction ( #8918 )
...
* Most of the PR changed files are extra and slightly more complicated
unit tests.
* Fixed Eth1DataVotes not inheriting genesis
* Fixed Attestations simulation using wrong slot when reconstructing
partecipation
* Fixed Copy() operation on BeaconState on Eth1DataVotes
* Used correct ListSSZ type for Eth1DataVotes and HistoricalSummaries
* Fixed wrong []uint64 deltas on empty slots
2023-12-11 14:07:57 +01:00
a
036446777c
max blocks in gossipsource ( #8944 )
2023-12-10 07:40:04 -06:00
Giulio rebuffo
0d2aecf829
Backfill only with flag ( #8913 )
...
Caplin snapshots only enabled with caplin.backfill
2023-12-06 14:22:13 +01:00
Giulio rebuffo
c477281362
Caplin: Parallel historical states reconstruction ( #8817 )
...
What does this PR do:
* Optional Backfilling and Caplin Archive Node
* Create antiquary for historical states
* Fixed gaps of chain gap related to the Head of the chain and anchor of
the chain.
* Added basic reader object to Read the Historical state
2023-12-06 10:48:36 +01:00
Giulio rebuffo
57bcbaa21f
Adds flags to enable/disable backfilling and enable full historical beacon node ( #8813 )
...
* Correct naming of hash func in Eth2
* Customizable mode of operation for Caplin
2023-11-22 13:24:35 +01:00
Giulio rebuffo
a2433455f9
Keep few beacon block headers in mdbx ( #8809 )
...
Now keep few beacon block headers in mdbx
2023-11-22 01:45:15 +01:00
Giulio rebuffo
7d28151a11
Added data model for public keys validator ( #8774 )
...
We store public keys contiguosly and batch update them on forkchoice
2023-11-18 19:03:56 +01:00
Giulio rebuffo
b72b5b95c5
working debug/state/ for early states ( #8767 )
...
Added jsonification to beacon state
2023-11-18 03:08:19 +01:00
Giulio rebuffo
51af060450
Added --beacon.api flags to enable experimental beacon api. ( #8727 )
...
Make it so that erigon can the enable beacon api.
2023-11-15 15:07:16 +01:00
Giulio rebuffo
8d8368091c
Add full support to beacon snapshots ( #8665 )
...
This PR adds beacon blocks snapshots for the following chains:
* Mainnet snapshots
* Sepolia snapshots
2023-11-13 14:10:57 +01:00
Giulio rebuffo
e08b031f74
Updated go.sum and go.mod for snapshots ( #8664 )
2023-11-06 23:39:44 +01:00
Giulio rebuffo
4b580dcc2f
update caplin snapshots hashes ( #8663 )
...
This PR also adds snippets to download caplin snapshots
2023-11-06 21:05:07 +01:00
Giulio rebuffo
e67db34145
Caplin: Bumbed down snapshots from 500 to 100k ( #8657 )
2023-11-06 15:41:19 +01:00
Giulio rebuffo
513fd50fa5
Compress snapshots for Caplin with lz4 level=1 ( #8609 )
2023-10-30 13:48:14 +01:00
Giulio rebuffo
d7448fdb3f
Added functional beacon snapshots reader and generator to Caplin ( #8570 )
...
This PR adds beacon blocks snapshots and beacon blocks snapshot
generator to Caplin, plus a snapshot verifier CLI
2023-10-24 21:32:29 +02:00
Giulio rebuffo
995009ac7b
Added cli too for Snapshots Generations for Caplin ( #8543 )
2023-10-22 19:21:37 +02:00
Giulio rebuffo
8f29ca7405
Same range parallel downloader ( #8554 )
...
will do a more advanced version later
2023-10-22 17:30:27 +02:00
a
436493350e
Sentinel refactor ( #8296 )
...
1. changes sentinel to use an http-like interface
2. moves hexutil, crypto/blake2b, metrics packages to erigon-lib
2023-10-22 01:17:18 +02:00
Giulio rebuffo
343479988c
Caplin <-> MDBX ( #8470 )
2023-10-18 23:10:53 +02:00
Giulio rebuffo
9e42b705ce
Caplin: under the hood block downloading ( #8459 )
2023-10-16 15:35:26 +02:00
Giulio rebuffo
a62796ab4d
Added block snapshot format (attempt for Caplin) ( #8357 )
...
Added block snapshot format
2023-10-03 23:50:43 +02:00
Giulio rebuffo
d62ef03cad
Added states/fork and states/root to beacon API ( #8293 )
...
Added /eth/v1/beacon/states/{state_id}/root and
/eth/v1/beacon/states/{state_id}/fork
2023-09-27 11:15:51 +02:00
a
7a833e5dc5
can't change size of btree during ascend ( #8295 )
2023-09-27 09:42:20 +07:00
Giulio rebuffo
c1961eff2b
Added the option to directly write blocks on io.Writer. ( #8215 )
2023-09-17 13:15:06 +02:00
Giulio rebuffo
c552f993ac
Removed Partial blocks and reduced pruneDepth to 1000 ( #8198 )
2023-09-14 22:14:05 +02:00
Giulio rebuffo
2f8687fe08
Smart SQL transactions ( #8185 )
2023-09-14 02:56:29 +02:00
Giulio rebuffo
22bd173ab3
Updating Libs ( #8147 )
...
* BLS
* Sqlite
2023-09-06 23:20:26 +02:00
Giulio rebuffo
1cfb1eb1e7
Caplin: fixed canonical tracking ( #8120 )
2023-09-03 13:40:10 +02:00
Giulio rebuffo
825be2ac11
Beacon API: added /headers methods ( #8112 )
2023-09-01 15:29:17 +02:00
Giulio rebuffo
901edf9181
Accessors to Caplin indexed database ( #8093 )
2023-08-29 16:47:16 +02:00
Giulio rebuffo
93cdbae2e5
Caplin<->Erigon: Added Range methods ( #8087 )
2023-08-29 02:37:30 +02:00
Giulio rebuffo
8291aa2497
Caplin: Storing Execution Headers in Caplin ( #8080 )
...
This PR stores Execution Headers in Caplin when running without storing
fullblocks
2023-08-26 00:41:57 +02:00
Giulio rebuffo
d223adf114
Better migrations and chain segmentation recovery ( #8079 )
2023-08-25 17:45:00 +02:00
Giulio rebuffo
41185e2518
Added Online pruning to Caplin ( #8058 )
...
Basically, pruning is specified by the user, by default, 1 million (in
the PR set to 100 for pruning purposes). the pruning for the database is
stored inside the db
2023-08-25 14:58:35 +02:00
Giulio rebuffo
db4533acd0
fixed small inconsistency with sqlite3 ( #8054 )
2023-08-23 22:44:22 +02:00
Giulio rebuffo
bd81e15981
Caplin: Implemented SQL beacon indexer ( #8043 )
2023-08-22 01:24:26 +02:00
Giulio rebuffo
cab6d30322
Caplin's Persistence data format ( #8028 )
2023-08-16 23:03:25 +02:00
