Commit Graph

113 Commits

Author SHA1 Message Date
Giulio rebuffo
46ecf030f5
Added GET /eth/v1/beacon/rewards/blocks/{block_id} and POST /eth/v1/beacon/rewards/sync_committee/{block_id} (#9102)
* Changed slightly archive format (again)
* Added all of the remaining rewards endpoints
2023-12-30 20:51:28 +01:00
a
78bb3cdca8
[caplin] topic strings (#9000) 2023-12-30 16:55:01 +01:00
Giulio rebuffo
698ee60339
Beacon API: Added attestation rewards endpoint. (#9091) 2023-12-28 13:07:53 +01:00
Giulio rebuffo
cdb4731ae8
Added Validators endpoint to Beacon API (#9080)
* Added /validators retrieval to beacon api
* fixed race in sentinel
* fixed possible UB in gossip management
2023-12-26 00:34:38 +01:00
Giulio rebuffo
eaf0348bd0
[Grindmas] Added tests to Beacon API, also fixed stuff. (#9074)
* Testing Beacon API
* Fixed sentinel code (a little bit)
* Fixed sentinel tests
* Added historical state support
* Fixed state-related endpoints (i was drunk when writing them)
2023-12-25 02:34:13 +01:00
Giulio rebuffo
a4d7b6d33f
Switched Caplin snapshot format to ZSTD blinded blocks (#9058)
* Chunked format -> blinded
* LZ4 -> ZSTD
* Implemented parent block root support for history download
* Rationale: Allows to optimize GC collection easily on state
reconstruction and it allows to read fast attestations in historical
states reader
2023-12-23 15:56:35 +01:00
Giulio rebuffo
0e18866ed9
Added fill-up with pre-existing API (#9042) 2023-12-21 17:49:40 +01:00
Giulio rebuffo
ffc26b0f00
Added GET SyncCommittees (#9003) 2023-12-18 13:54:15 +01:00
Giulio rebuffo
eeb471d800
Added Finality_Checkpoints endpoint (#8979) 2023-12-15 02:27:27 +01:00
Giulio rebuffo
4e83ef0ef5
Optimized beacon state retrieval (#8963)
* Reconstruct previous epoch without looking at DB: no hindrance to
performance -> removed 15GB
* Store inactivity scores and slashings in MDBX and do not store diffs
for them(they are tiny 700/400 bytes)
* Reduced dumps from every 2048 to 1024 -> Added 5 GB (maybe we should
down it to 768)
* Parallel processing of shuffled sets, 2x performance boost in reading
participation.
* Store balances diffs in a Btree diff matter, see:
https://github.com/ledgerwatch/erigon-documents/blob/master/caplin/design/data-model.md#uint64listuint64vector
2023-12-13 15:23:27 +01:00
Giulio rebuffo
24987878e4
Resumable beacon state reconstruction (#8918)
* Most of the PR changed files are extra and slightly more complicated
unit tests.
* Fixed Eth1DataVotes not inheriting genesis
* Fixed Attestations simulation using wrong slot when reconstructing
partecipation
* Fixed Copy() operation on BeaconState on Eth1DataVotes
* Used correct ListSSZ type for Eth1DataVotes and HistoricalSummaries
* Fixed wrong []uint64 deltas on empty slots
2023-12-11 14:07:57 +01:00
Giulio rebuffo
c477281362
Caplin: Parallel historical states reconstruction (#8817)
What does this PR do:
* Optional Backfilling and Caplin Archive Node
* Create antiquary for historical states
* Fixed gaps of chain gap related to the Head of the chain and anchor of
the chain.
* Added basic reader object to Read the Historical state
2023-12-06 10:48:36 +01:00
a
47a6ac16da
[beacon handler] framework (#8851)
adds a two indexes to the validators cache

creates beaconhttp package with many utilities for beacon http endpoint
(future support for ssz is baked in)

started on some validator endpoints
2023-12-05 00:13:52 +01:00
milen
9b74cf0384
metrics: use prometheus histogram and summary interfaces (#8808) 2023-11-24 17:50:57 +00:00
Giulio rebuffo
57bcbaa21f
Adds flags to enable/disable backfilling and enable full historical beacon node (#8813)
* Correct naming of hash func in Eth2
* Customizable mode of operation for Caplin
2023-11-22 13:24:35 +01:00
Giulio rebuffo
073dbe25ac
Added duties/proposer to beacon api (#8777) 2023-11-19 22:03:14 +01:00
Giulio rebuffo
7d28151a11
Added data model for public keys validator (#8774)
We store public keys contiguosly and batch update them on forkchoice
2023-11-18 19:03:56 +01:00
Giulio rebuffo
b72b5b95c5
working debug/state/ for early states (#8767)
Added jsonification to beacon state
2023-11-18 03:08:19 +01:00
Andrew Ashikhmin
d4cd712da0
Revisit getPayloadBodiesByHash (#8758)
Revisit PR #8750
2023-11-17 13:04:02 +01:00
Giulio rebuffo
8d8368091c
Add full support to beacon snapshots (#8665)
This PR adds beacon blocks snapshots for the following chains:

* Mainnet snapshots
* Sepolia snapshots
2023-11-13 14:10:57 +01:00
Giulio rebuffo
513fd50fa5
Compress snapshots for Caplin with lz4 level=1 (#8609) 2023-10-30 13:48:14 +01:00
a
0a8bda751e
mbps (#8564) 2023-10-23 21:23:54 +02:00
Giulio rebuffo
995009ac7b
Added cli too for Snapshots Generations for Caplin (#8543) 2023-10-22 19:21:37 +02:00
Giulio rebuffo
8f29ca7405
Same range parallel downloader (#8554)
will do a more advanced version later
2023-10-22 17:30:27 +02:00
a
436493350e
Sentinel refactor (#8296)
1. changes sentinel to use an http-like interface

2. moves hexutil, crypto/blake2b, metrics packages to erigon-lib
2023-10-22 01:17:18 +02:00
Giulio rebuffo
3f30b96199
Forkchoice states saved on disk (#8546) 2023-10-21 23:10:58 +02:00
Giulio rebuffo
684ca4c030
keep caplin data (#8518) 2023-10-19 00:53:03 +02:00
Giulio rebuffo
343479988c
Caplin <-> MDBX (#8470) 2023-10-18 23:10:53 +02:00
Giulio rebuffo
9e42b705ce
Caplin: under the hood block downloading (#8459) 2023-10-16 15:35:26 +02:00
Giulio rebuffo
bf2ef04c46
Optimized the checkpoint state layout (#8482) 2023-10-15 18:05:13 +02:00
Giulio rebuffo
1da6dc25f3
Caplin: Resetting cache head hash when new slot is avaiable (#8477) 2023-10-14 20:53:16 +02:00
Giulio rebuffo
54ce971084
Added separate Withdrawal data type to Caplin (#8474)
Reason: JSON formatting
2023-10-14 17:59:52 +02:00
Giulio rebuffo
10746eb376
Added BLS_TO_EXECUTION_CHANGES pool (#8332)
Adds BLS_TO_EXECUTION_CHANGES
2023-10-01 17:16:55 +02:00
Giulio rebuffo
72ba18bd36
Beacon: Added basic operations pool (#8309)
Added operation pools for beacon chain. operations are the equivalent of
txs for eth2

Added operation pools for:

* Attester Slashings
* Proposer Slashings
* VoluntaryExits
* BLSExecutionToChange
* Postponed to later: Attestations (or maybe not)
2023-09-29 23:42:07 +02:00
Giulio rebuffo
d62ef03cad
Added states/fork and states/root to beacon API (#8293)
Added /eth/v1/beacon/states/{state_id}/root and
/eth/v1/beacon/states/{state_id}/fork
2023-09-27 11:15:51 +02:00
Giulio rebuffo
a95914df26
Added blocks API and attestations API (#8279) 2023-09-24 01:44:26 +02:00
Giulio rebuffo
13720af42c
Caplin: Implemented internal tree for ValidatorSet (#8234) 2023-09-22 12:16:19 +02:00
Giulio rebuffo
2aa060f454
Fixed forkchoice nil exception (#8223) 2023-09-17 17:06:44 +02:00
Giulio rebuffo
0b307e7aca
Fixed accidental chain gap (#8222)
A chain gap can occur when peers may not give us enough info
2023-09-17 16:25:17 +02:00
Giulio rebuffo
c1961eff2b
Added the option to directly write blocks on io.Writer. (#8215) 2023-09-17 13:15:06 +02:00
Giulio rebuffo
c552f993ac
Removed Partial blocks and reduced pruneDepth to 1000 (#8198) 2023-09-14 22:14:05 +02:00
Giulio rebuffo
2f8687fe08
Smart SQL transactions (#8185) 2023-09-14 02:56:29 +02:00
Giulio rebuffo
346b278a3b
Caplin: Improved logging (#8169) 2023-09-10 22:10:21 +02:00
alex.sharov
6c1cf0a9b0 save 2023-09-10 15:49:13 +07:00
Alex Sharov
2318138c6c
Downloader: don't fail when see unusual file, skip it (backward/forward compatibility) (#8170) 2023-09-10 15:46:30 +07:00
Giulio rebuffo
4733f7fb52
Added json tags to beacon block (#8162) 2023-09-08 22:26:02 +02:00
Mark Holt
8ea0096d56
moved metrics sub packages types to metrics (#8119)
This is a non functional change which consolidates the various packages
under metrics into the top level package now that the dead code is
removed.

It is a precursor to the removal of Victoria metrics after which all
erigon metrics code will be contained in this single package.
2023-09-03 08:09:27 +07:00
alex.sharov
e53df00849 split grafana board to: user-facing and dev-facing (internals) 2023-09-01 12:09:30 +07:00
Mark Holt
a4cfbe0d56
Heimdall metrics + Metrics HTTP server rationalization (#8094)
This is an update of:

https://github.com/ledgerwatch/erigon/pull/7846

which uses a local fork of victoria metrics to include the changes that
https://github.com/anshalshukla added to the original for we where
using.

It also includes code to address the duplicate metrics issue identified
here:

https://github.com/ledgerwatch/erigon/issues/8053

It has one more associated fix which is to correctly add a metadata
label to counters, these where previously labelled as gauges.

e.g. 

```
# TYPE p2p_peers counter
p2p_peers 0
```
rather than

```
# TYPE p2p_peers gauge
p2p_peers 0
```

---------

Co-authored-by: Anshal Shukla <53994948+anshalshukla@users.noreply.github.com>
Co-authored-by: Anshal Shukla <shukla.anshal85@gmail.com>
2023-08-31 09:04:27 +01:00
Giulio rebuffo
901edf9181
Accessors to Caplin indexed database (#8093) 2023-08-29 16:47:16 +02:00