Commit Graph

23 Commits

Author SHA1 Message Date
Giulio rebuffo
eaf0348bd0
[Grindmas] Added tests to Beacon API, also fixed stuff. (#9074)
* Testing Beacon API
* Fixed sentinel code (a little bit)
* Fixed sentinel tests
* Added historical state support
* Fixed state-related endpoints (i was drunk when writing them)
2023-12-25 02:34:13 +01:00
Giulio rebuffo
be4036e132
Added tests to sentinel (FINALLY!) (#9066) 2023-12-24 03:54:50 +01:00
Alex Sharov
7107cfed0f
snaps: stop merge to 500K and enjoy immutability (#9034) 2023-12-21 08:04:46 +07:00
Giulio rebuffo
a0b9d60265
Update Consensus Specification tests (#9024) 2023-12-19 09:42:11 +01:00
Giulio rebuffo
ffc26b0f00
Added GET SyncCommittees (#9003) 2023-12-18 13:54:15 +01:00
Giulio rebuffo
eeb471d800
Added Finality_Checkpoints endpoint (#8979) 2023-12-15 02:27:27 +01:00
Giulio rebuffo
4e83ef0ef5
Optimized beacon state retrieval (#8963)
* Reconstruct previous epoch without looking at DB: no hindrance to
performance -> removed 15GB
* Store inactivity scores and slashings in MDBX and do not store diffs
for them(they are tiny 700/400 bytes)
* Reduced dumps from every 2048 to 1024 -> Added 5 GB (maybe we should
down it to 768)
* Parallel processing of shuffled sets, 2x performance boost in reading
participation.
* Store balances diffs in a Btree diff matter, see:
https://github.com/ledgerwatch/erigon-documents/blob/master/caplin/design/data-model.md#uint64listuint64vector
2023-12-13 15:23:27 +01:00
Alex Sharov
d41d523050
Downloader: add ProhibitNewDownloads() (#8939)
"whitelisting" mechanism (list of files - stored in DB) - which
protecting us from downloading new files after upgrade/downgrade was
broken. And seems it became over-complicated with time.
I replacing it by 1 persistent flag inside downloader:
"prohibit_new_downloads.lock"
Erigon will turn downloader into this mode after
downloading/verification of first snapshots.


```
//Corner cases:
	// - Erigon generated file X with hash H1. User upgraded Erigon. New version has preverified file X with hash H2. Must ignore H2 (don't send to Downloader)
	// - Erigon "download once": means restart/upgrade/downgrade must not download files (and will be fast)
	// - After "download once" - Erigon will produce and seed new files
```

------
`downloader --seedbox` is never "prohibit new downloads"
2023-12-12 16:05:56 +07:00
Giulio rebuffo
24987878e4
Resumable beacon state reconstruction (#8918)
* Most of the PR changed files are extra and slightly more complicated
unit tests.
* Fixed Eth1DataVotes not inheriting genesis
* Fixed Attestations simulation using wrong slot when reconstructing
partecipation
* Fixed Copy() operation on BeaconState on Eth1DataVotes
* Used correct ListSSZ type for Eth1DataVotes and HistoricalSummaries
* Fixed wrong []uint64 deltas on empty slots
2023-12-11 14:07:57 +01:00
Giulio rebuffo
0d2aecf829
Backfill only with flag (#8913)
Caplin snapshots only enabled with caplin.backfill
2023-12-06 14:22:13 +01:00
Giulio rebuffo
c477281362
Caplin: Parallel historical states reconstruction (#8817)
What does this PR do:
* Optional Backfilling and Caplin Archive Node
* Create antiquary for historical states
* Fixed gaps of chain gap related to the Head of the chain and anchor of
the chain.
* Added basic reader object to Read the Historical state
2023-12-06 10:48:36 +01:00
Alex Sharov
49890b38b7
caplin: to use 1 worker for snapshots compression (#8892)
Erigon by default must use as minimum resources as possible to build
snapshots. But cli commands - can go opposite and maximize.
2023-12-04 18:55:40 +07:00
Giulio rebuffo
a2433455f9
Keep few beacon block headers in mdbx (#8809)
Now keep few beacon block headers in mdbx
2023-11-22 01:45:15 +01:00
Giulio rebuffo
274f84598c
Automation tool to automatically upload caplin's snapshot files to R2 (#8747)
Upload beacon snapshots to R2 every week by default
2023-11-16 20:59:43 +01:00
Giulio rebuffo
51af060450
Added --beacon.api flags to enable experimental beacon api. (#8727)
Make it so that erigon can the enable beacon api.
2023-11-15 15:07:16 +01:00
Giulio rebuffo
8d8368091c
Add full support to beacon snapshots (#8665)
This PR adds beacon blocks snapshots for the following chains:

* Mainnet snapshots
* Sepolia snapshots
2023-11-13 14:10:57 +01:00
Giulio rebuffo
8f29ca7405
Same range parallel downloader (#8554)
will do a more advanced version later
2023-10-22 17:30:27 +02:00
a
436493350e
Sentinel refactor (#8296)
1. changes sentinel to use an http-like interface

2. moves hexutil, crypto/blake2b, metrics packages to erigon-lib
2023-10-22 01:17:18 +02:00
Giulio rebuffo
343479988c
Caplin <-> MDBX (#8470) 2023-10-18 23:10:53 +02:00
Giulio rebuffo
2f8687fe08
Smart SQL transactions (#8185) 2023-09-14 02:56:29 +02:00
Giulio rebuffo
bd81e15981
Caplin: Implemented SQL beacon indexer (#8043) 2023-08-22 01:24:26 +02:00
Giulio rebuffo
cab6d30322
Caplin's Persistence data format (#8028) 2023-08-16 23:03:25 +02:00
a
521f0df55b
Historical block downloader (#8016) 2023-08-16 04:32:40 +02:00