Giulio rebuffo
825be2ac11
Beacon API: added /headers methods ( #8112 )
2023-09-01 15:29:17 +02:00
alex.sharov
e53df00849
split grafana board to: user-facing and dev-facing (internals)
2023-09-01 12:09:30 +07:00
Mark Holt
a4cfbe0d56
Heimdall metrics + Metrics HTTP server rationalization ( #8094 )
...
This is an update of:
https://github.com/ledgerwatch/erigon/pull/7846
which uses a local fork of victoria metrics to include the changes that
https://github.com/anshalshukla added to the original for we where
using.
It also includes code to address the duplicate metrics issue identified
here:
https://github.com/ledgerwatch/erigon/issues/8053
It has one more associated fix which is to correctly add a metadata
label to counters, these where previously labelled as gauges.
e.g.
```
# TYPE p2p_peers counter
p2p_peers 0
```
rather than
```
# TYPE p2p_peers gauge
p2p_peers 0
```
---------
Co-authored-by: Anshal Shukla <53994948+anshalshukla@users.noreply.github.com>
Co-authored-by: Anshal Shukla <shukla.anshal85@gmail.com>
2023-08-31 09:04:27 +01:00
Giulio rebuffo
e7dbc69571
Middleware for Caplin Beacon API ( #8103 )
2023-08-31 01:18:12 +02:00
Giulio rebuffo
901edf9181
Accessors to Caplin indexed database ( #8093 )
2023-08-29 16:47:16 +02:00
Giulio rebuffo
93cdbae2e5
Caplin<->Erigon: Added Range methods ( #8087 )
2023-08-29 02:37:30 +02:00
Giulio rebuffo
8291aa2497
Caplin: Storing Execution Headers in Caplin ( #8080 )
...
This PR stores Execution Headers in Caplin when running without storing
fullblocks
2023-08-26 00:41:57 +02:00
Giulio rebuffo
d223adf114
Better migrations and chain segmentation recovery ( #8079 )
2023-08-25 17:45:00 +02:00
Giulio rebuffo
41185e2518
Added Online pruning to Caplin ( #8058 )
...
Basically, pruning is specified by the user, by default, 1 million (in
the PR set to 100 for pruning purposes). the pruning for the database is
stored inside the db
2023-08-25 14:58:35 +02:00
Giulio rebuffo
db4533acd0
fixed small inconsistency with sqlite3 ( #8054 )
2023-08-23 22:44:22 +02:00
Giulio rebuffo
bd81e15981
Caplin: Implemented SQL beacon indexer ( #8043 )
2023-08-22 01:24:26 +02:00
Alex Sharov
863a706858
caplin: use tmpdir inside datadir ( #8041 )
2023-08-20 15:26:34 +02:00
Giulio rebuffo
6d2a9aed2f
Fixed startup panic ( #8040 )
2023-08-18 22:16:30 +02:00
Giulio rebuffo
38db2ed2e7
[Caplin Phase 2]: Initial working implementation ( #8033 )
...
caplin just using execution.proto
2023-08-18 15:43:22 +02:00
Giulio rebuffo
cab6d30322
Caplin's Persistence data format ( #8028 )
2023-08-16 23:03:25 +02:00
Giulio rebuffo
c8db11f150
Sentinel: improvements to block download ( #8023 )
2023-08-16 17:02:30 +02:00
a
521f0df55b
Historical block downloader ( #8016 )
2023-08-16 04:32:40 +02:00
Giulio rebuffo
114dd684eb
Caplin Optimization: Efficient Processing of Epoch and Reuse of checkpoint states ( #8020 )
2023-08-15 12:45:48 +02:00
Giulio rebuffo
98db1e97b5
Better caplin logging ( #8009 )
2023-08-12 22:29:30 +02:00
Giulio rebuffo
de5706dbc6
Direct sentinel instead of over the network sentinel ( #8006 )
2023-08-12 01:54:45 +02:00
Giulio rebuffo
e3a59ed902
Better caplin logging and logic ( #7992 )
2023-08-10 22:34:58 +02:00
a
db5b348673
Caplin block persistence ( #7941 )
...
Co-authored-by: Giulio <giulio.rebuffo@gmail.com>
2023-08-09 01:21:19 +02:00
Andrew Ashikhmin
d69b20bc4e
Support engine_forkchoiceUpdatedV3 with ParentBeaconBlockRoot (EIP-4788) ( #7969 )
...
Prerequisites: https://github.com/ledgerwatch/interfaces/pull/187 &
https://github.com/ledgerwatch/erigon-lib/pull/1069 . Also implement
https://github.com/ethereum/execution-apis/pull/426 .
2023-08-06 11:54:14 +02:00
Andrew Ashikhmin
e1f1c081fc
Remove failing TestGnosisForkDigest ( #7959 )
...
I'm not sure what caused this failure (for instance, CI for PR #7888 was
green), but currently we have the following test failure in `devel`:
```
--- FAIL: TestGnosisForkDigest (0.00s)
fork_test.go:79:
Error Trace: github.com/ledgerwatch/erigon/cl/fork/fork_test.go:79
Error: Not equal:
expected: [4]uint8{0x82, 0x4b, 0xe4, 0x31}
actual : [4]uint8{0x21, 0xa6, 0xf8, 0x36}
```
2023-08-02 14:07:34 +02:00
Somnath Banerjee
d91f4263dc
EIP-4788: Beacon block root and stateful precompile ( #7888 )
2023-08-01 16:01:26 +05:30
Giulio rebuffo
443757edbd
Consensus separation for Engine API (Working on Sepolia) ( #7945 )
...
This makes the experimental consensus separation functional on sepolia.
2023-07-30 23:35:55 +02:00
racytech
af884204cb
eip-4844: hive tests: RLP decode order and newPayloadV3 to expect versionedHashes ( #7943 )
2023-07-30 15:49:31 +02:00
Andrew Ashikhmin
7d35c6b737
EIP-4844: Rename "data gas" to "blob gas" ( #7937 )
...
See https://github.com/ethereum/EIPs/pull/7354 &
https://github.com/ethereum/consensus-specs/pull/3461 . Prerequisite:
https://github.com/ledgerwatch/erigon-lib/pull/1058
2023-07-28 12:12:05 +02:00
a
dc47f43f4d
[caplin] block persistence experimentation ( #7935 )
2023-07-28 00:55:14 +02:00
Giulio rebuffo
b4ecd7f524
Consensus Separation: Separated GetPayload using an execution service ( #7933 )
...
Miracoulously, hive tests pass first try. YIPPIE.
Also for the future, I added `--experimental.modular` which enables a
secondary engine API for consensus separation.
Now block building is responsibility of the execution module.
2023-07-27 14:37:49 +02:00
a
60a200fd46
[caplin] basic persistence layer ( #7918 )
2023-07-25 22:24:52 +02:00
Giulio rebuffo
f5f833fd70
Proper Error handling in direct communication with engine ( #7929 )
2023-07-25 14:15:50 +02:00
a
3ab373787e
[caplin] extracting beacon state interface ( #7910 )
...
we need to extract this interface from the struct.
i need to also break down the interface more, to better show what parts
the caching is used, move some functions from the cache state to the
underlying.
don't merge
2023-07-20 00:20:33 +02:00
Giulio rebuffo
862faf054b
Fixed JWT race and panic in Caplin ( #7904 )
2023-07-18 09:47:38 +01:00
Giulio rebuffo
0239dfd654
Better error reporting Caplin <-> Erigon ( #7902 )
2023-07-16 23:31:09 +02:00
Giulio rebuffo
23f5377614
updated interfaces ( #7891 )
2023-07-14 02:51:47 +02:00
Giulio rebuffo
bd63cb7c8c
Removed GRPC layer from Engine API ( #7878 )
2023-07-12 18:11:41 +02:00
Giulio rebuffo
20b8e156db
moved cmd/rpcdaemon/commands -> turbo/jsonrpc ( #7858 )
2023-07-08 19:01:26 +02:00
Enrique Jose Avila Asapche
bff1e0e901
[Caplin] Added json rpc cl ( #7836 )
...
Allows for Caplin to be used with other ELs from aside of Erigon
---------
Co-authored-by: Giulio <giulio.rebuffo@gmail.com>
2023-07-08 16:42:30 +02:00
Giulio rebuffo
84f31c873e
Separation of Engine from Ethbackend ( #7821 )
...
This PR separates ENGINE from Ethbackend. It makes it so:
1) EthBackend not a god class
2) We can abstract away engine API so that we can make it CL-like and
enable Consensus-Execution driven design
3) Objective is Json-RPC -> Engine Consensus Module -> Execution module.
2023-07-06 18:09:52 +02:00
Andrew Ashikhmin
9f5808b055
Schedule Shapella for Gnosis Chain ( #7784 )
...
Schedule the
[Shanghai](https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/shanghai.md )/[Capella](https://github.com/ethereum/consensus-specs/tree/dev/specs/capella )
upgrade for Gnosis Chain. The [Gnosis withdrawals
spec](https://github.com/gnosischain/specs/blob/master/execution/withdrawals.md )
documents the difference between Gnosis Chain and Ethereum mainnet.
Refer to https://github.com/gnosischain/specs/pull/10 and
https://github.com/gnosischain/configs/pull/12 .
2023-06-22 15:36:07 +02:00
Alex Sharov
5ff8246e2a
e4: release some tests ( #7729 )
2023-06-14 10:01:00 +07:00
Giulio rebuffo
cefbf18088
Fixed grpcing of block ( #7717 )
2023-06-12 21:55:13 +02:00
Giulio rebuffo
63006611ec
Added concurrent bls ( #7709 )
2023-06-12 00:40:03 +02:00
a
4d35e776da
[caplin] abstract transition ( #7661 )
...
start of abstracting the transition layer.
incremental movements.
2023-06-11 23:50:02 +02:00
Enrique Jose Avila Asapche
eac9ead6fe
[Caplin] added deneb into converting payload to grpc ( #7708 )
...
Added so now we can have version 3 payloads preparing for Deneb
2023-06-11 19:34:06 +02:00
Giulio rebuffo
8f8da14814
Optimized THE CAPLIN a lil bit ( #7687 )
2023-06-08 15:52:09 +02:00
Enrique Jose Avila Asapche
1cb1c303d4
added a listener for beacon api and added get genesis ( #7659 )
...
Added two new flags beacon.api.port and beacon.api.addr
Now we can listen for beacon api and get beacon genesis
---------
Co-authored-by: Giulio <giulio.rebuffo@gmail.com>
2023-06-08 09:43:27 +02:00
Giulio rebuffo
07e2100510
Added Regeression testing for Caplin ( #7679 )
2023-06-07 20:01:32 +02:00
Giulio rebuffo
5e682aa5d5
correct data pruning ( #7678 )
2023-06-06 22:29:51 +02:00
Giulio rebuffo
3a8d5c4d0a
improved freezer for Caplin ( #7670 )
2023-06-06 03:24:59 +02:00
Giulio rebuffo
e45ed7d139
Added record mode for Caplin ( #7660 )
2023-06-05 01:52:55 +02:00
a
21625652f3
[caplin] record mode ( #7651 )
...
interface for recording things
2023-06-04 03:13:58 +02:00
Giulio rebuffo
e11c8192f8
fixed deneb for devnet 6 ( #7650 )
2023-06-04 00:36:16 +02:00
Giulio rebuffo
999c0ba214
Caplin: Fixed bad block handling ( #7649 )
2023-06-03 19:47:05 +02:00
Andrew Ashikhmin
190bc9dfd8
EIP-4844: add data_gas_used ( #7639 )
...
Implement https://github.com/ethereum/EIPs/pull/7062 and
https://github.com/ethereum/EIPs/pull/7095 . Pick up
https://github.com/ledgerwatch/erigon-lib/pull/1006 .
2023-06-02 21:26:19 +01:00
Giulio rebuffo
91dd39a7c1
Added support back for internal cl for testnets and gnosis ( #7641 )
2023-06-02 20:26:46 +02:00
Giulio rebuffo
6602c89e32
[Caplin] Fixed checkpoints on orphaned chains ( #7613 )
2023-06-02 14:54:40 +02:00
Alex Sharov
5fb31ae95f
e3: release some e4 parts ( #7628 )
2023-06-02 10:35:26 +07:00
Giulio rebuffo
8bc1fda4ea
Caplin: mutable public key for validator for 0 copy ( #7622 )
...
Co-authored-by: a <a@tuxpa.in>
2023-06-02 01:21:15 +02:00
Enrique Jose Avila Asapche
94276deaf6
Caplin: added tests ( #7623 )
...
Added CL tests to have test coverage over 60% in all files covered by
running ssh ./cl/coverage
2023-06-02 01:21:08 +02:00
Giulio rebuffo
d77f057cbc
removed state_encoding package from caplin ( #7621 )
2023-06-01 20:25:37 +02:00
Giulio rebuffo
c0042d570c
Added coverage checker for caplin packages ( #7609 )
2023-05-31 18:12:18 +02:00
Giulio rebuffo
027d83b556
Caplin: skip finalized error ( #7595 )
2023-05-28 22:48:17 +02:00
Giulio rebuffo
2d3b3c3b8d
Caplin: Proper validator set ( #7591 )
...
Adding proper flat buffer to Caplin
2023-05-28 17:11:18 +02:00
a
3c2b99e19c
[caplin] incremental improvements to p2p layer ( #7582 )
...
we update observability in the p2p layer for handlers, and also properly
encode error codes, close streams.
---------
Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
Co-authored-by: Giulio <giulio.rebuffo@gmail.com>
2023-05-28 07:54:04 +07:00
Alex Sharov
ead45694ff
torrent-related goroutine leack ( #7545 )
...
can't merge - because `torrent` lib dropped go1.19 support
2023-05-26 17:12:47 +07:00
Giulio rebuffo
faf62a0c11
Caplin: Adding SSZ generics ( #7542 )
...
This PR makes the SSZ encodings less demented and more compact/flexible
overall
2023-05-23 20:58:34 +02:00
ledgerwatch
b0117a7c30
[devnet] separate logging - p2p ( #7547 )
...
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-19 18:41:53 +01:00
ledgerwatch
b382f96f6c
Introduce logger into etl ( #7537 )
...
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-18 21:20:07 +01:00
a
761524a986
eth2 shuffle ( #7522 )
...
moves eth2shuffle package to erigon.
2023-05-18 01:06:25 +02:00
Giulio rebuffo
10848adb89
Embedded CL specific support to mainnet ( #7533 )
2023-05-17 15:32:47 +02:00
Andrew Ashikhmin
ea9712f858
Schedule Shapella on Chiado ( #7518 )
...
See https://github.com/NethermindEth/nethermind/pull/5160 ,
https://github.com/NethermindEth/nethermind/pull/5688 , and
https://github.com/gnosischain/configs/pull/11
2023-05-16 23:50:09 +02:00
Giulio rebuffo
53b116ccff
Fixed finality sync back in gnosis ( #7527 )
2023-05-16 13:23:08 +02:00
Giulio rebuffo
6f758fd0b9
added chiado and gnosis support to caplin ( #7523 )
2023-05-16 00:47:20 +02:00
Giulio rebuffo
4cbbe65f31
Resp/Req support for Deneb ( #7519 )
2023-05-15 20:07:27 +02:00
Giulio rebuffo
dfadf495cf
Flat buffers for Attestations and Cached roots ( #7512 )
2023-05-15 00:12:24 +02:00
a
cda14447ad
[caplin] pkg refactor ( #7507 )
...
Co-authored-by: Giulio <giulio.rebuffo@gmail.com>
2023-05-13 23:44:07 +02:00
Enrique Jose Avila Asapche
9cf2b42478
[Caplin] Block transition ( #7485 )
...
Added process of blob kzg commits for block transition following:
https://github.com/ethereum/consensus-specs/blob/dev/specs/deneb/beacon-chain.md#beacon-chain-state-transition-function
#7389
2023-05-13 21:52:49 +02:00
a
233a06a8a0
[caplin] bug fix in bitlist ( #7505 )
2023-05-13 15:56:55 +02:00
a
fd6acd4b31
[Caplin] beginnings of instrumentation ( #7486 )
...
this pr is ready for review, but it is waiting on this PR
https://github.com/VictoriaMetrics/metrics/pull/45
so that we do not need to use a replace directive.
2023-05-11 18:38:56 +02:00
Roberto Bayardo
8356f3a320
use ssz from updated erigon-lib ( #7490 )
...
ssz/clonable interfaces have been moved to erigon-lib so they can be
shared with the txpool code there
2023-05-11 12:54:20 +01:00
Giulio rebuffo
d4a2298998
Added GC optimization to the ssz list object ( #7488 )
2023-05-11 01:02:23 +02:00
a
37da9ec1e3
[caplin] ssz byteobjects ( #7454 )
...
instead of converting from ssz -> struct -> ssz, it may be better to
just stay as ssz, then use methods to read the data.
this pr explores this concept, while maintaining compatiblity with the
existing codebase.
2023-05-10 21:37:50 +02:00
Enrique Jose Avila Asapche
4f56247610
made KZGCommitment constant and SSZ compatiable ( #7465 )
...
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2023-05-10 14:40:19 +07:00
Andrew Ashikhmin
771c6fc202
Rename Serenity consensus engine to Merge ( #7475 )
...
[EIP-2982](https://eips.ethereum.org/EIPS/eip-2982 ) "Serenity Phase 0"
was superseded by [EIP-3675](https://eips.ethereum.org/EIPS/eip-3675 )
"Upgrade consensus to Proof-of-Stake"
2023-05-09 18:45:33 +01:00
Giulio rebuffo
5b7643c57b
Added dencun networking ( #7453 )
2023-05-07 01:37:53 +02:00
Enrique Jose Avila Asapche
fbd18516ae
Deneb container changes beacon-chain ( #7420 )
...
Adding container changes for Deneb:
https://github.com/ethereum/consensus-specs/blob/dev/specs/deneb/beacon-chain.md
Changed to use pointers for BlobSideCar
#7389
2023-05-06 18:50:58 +02:00
Giulio rebuffo
43b09a7f55
Refactor of ssz interface ( #7439 )
...
Also added a better way to copy and the hasher
2023-05-05 11:19:24 +02:00
a
30430d585a
begin refactor of beacon state ( #7433 )
...
this first major move separates the transient beacon state cache from
the underlying tree.
leaf updates are enforced in the setters, which should make programming
easier.
all exported methods of the raw.BeaconState should be safe to call
(without disrupting internal state)
changes many functions to consume *raw.BeaconState in perparation for
interface
beyond refactor it also:
adds a pool for the leaves of the validator ssz hash
adds a pool for the snappy writers
removed the parallel hash experiment (high memory use)
2023-05-04 15:18:42 +02:00
a
9644e6d220
Implement SpecTests in native go, add fork_choice handler ( #7422 )
...
a few TODO: remain to make this not a draft
---------
Co-authored-by: Giulio <giulio.rebuffo@gmail.com>
2023-05-02 16:19:22 +02:00
Enrique Jose Avila Asapche
f363eb1026
adding deneb p2p containers ( #7401 )
...
This PR adds the new p2p containers from Deneb into Caplin
#7389
2023-04-29 21:35:33 +02:00
Giulio rebuffo
250a0a4a65
Caplin: Remotion of bad peers giving us bad blocks ( #7411 )
2023-04-29 21:32:33 +02:00
Giulio rebuffo
01ce968361
Fixed networking issue in Caplin ( #7388 )
2023-04-26 14:33:21 +01:00
Giulio rebuffo
515aedda7b
Added Caplin Phase 1 ( #7290 )
...
* Introduces full beacon validation
* Removes light client
* NOTE: slow on purpose, I want everything to be BLS verified for now.
2023-04-17 18:06:50 +00:00
Andrew Ashikhmin
2a356babfe
Update TestMainnetFork to Capella fork ID ( #7309 )
2023-04-13 15:09:13 +00:00
Giulio rebuffo
c1cf58ef93
Added forkchoice rule ( #7281 )
...
Added fork choice rule to Erigon-CL
2023-04-08 01:01:10 +00:00
alex.sharov
cd2c5fe0fd
sepolia url fix
2023-04-04 11:42:12 +07:00
Alex Sharov
a8e8bf4528
remove simd lib, because it doesn't work with ghcr.io/goreleaser/goreleaser-cross (which producing release binaries) ( #7229 )
...
@shyba hi, seems this lib doesn't work with
ghcr.io/goreleaser/goreleaser-cross (which producing release binaries)
removing it for now, feel free to add it in future - if can make it work
with goreleaser-cross
see: https://github.com/ledgerwatch/erigon/issues/7210
2023-03-31 05:07:43 +00:00
Peter Davies
e912ae01d8
cl: Update checkpoint sync endpoints ( #7223 )
...
I have replaced the `ethdevops.io` endpoints with `ethpandaops.io`. The
Sepolia endpoint was failing with `no such host` and the checkpoint sync
documentation shows `ethpandaops.io` to be the official source for EF
checkpoint sync endpoints.
2023-03-30 20:59:12 +00:00
