Commit Graph

76 Commits

Author SHA1 Message Date
Giulio rebuffo
cea51685fe
attempt at fixing lightclient performance leaks (#6169) 2022-12-02 00:21:33 +01:00
Giulio rebuffo
025839afd5
moved out protobuffers for sentinel (#6145) 2022-11-29 00:00:40 +01:00
Giulio rebuffo
05be20b70b
smol refactor of lightclient CI + logs demotion (#6144) 2022-11-28 23:29:48 +01:00
Giulio rebuffo
8c67c55cc7
Created Erigon beacon downloader (#6130) 2022-11-25 16:38:22 +01:00
Giulio rebuffo
7ea8de4a3f
added checkpoint saving (#6060) 2022-11-16 16:19:40 +01:00
Giulio rebuffo
3a2d3be080
Added persistence in LightClient (#6056) 2022-11-15 20:39:18 +01:00
Andrew Ashikhmin
32629bdce4
Upgrade urfave/cli to v2 (#6047)
See also
https://github.com/urfave/cli/blob/main/docs/migrate-v1-to-v2.md
2022-11-14 17:33:57 +01:00
Giulio rebuffo
ae01026ca9
Customizable networking for Erigon Lightclient (#6032) 2022-11-11 23:22:47 +01:00
Giulio rebuffo
63a8b21bb8
switch BLS library (#5950) 2022-11-03 23:55:59 +01:00
Giulio rebuffo
5c3245d4e0
Added Checkpoint sync for CL full client (#5934) 2022-11-02 13:12:44 +01:00
Giulio rebuffo
adf5fc7b82
refactored files regarding lightclient (#5904) 2022-10-29 21:51:32 +02:00
Mike Neuder
dab1ba44d0
Test req/resp domain response decoding (#5886)
https://github.com/ledgerwatch/erigon/issues/5884 has more context on
this issue. we are trying to understand the encoding of the responses.

from the Ethereum R&D discord, AgeManning said that the prefix is
supposed to be a varint representing the length of the payload:
https://developers.google.com/protocol-buffers/docs/encoding#varints.

that works for the `&cltypes.MetadataV1{}` type, but not for
`&cltypes.MetadataV2{}` or `&cltypes.LightClientFinalityUpdate{}`.
2022-10-29 13:02:26 +02:00
Mike Neuder
29ff8daa92
req/resp heartbeat handler modifications (#5859)
addressing some aspects of the req/resp portion of the CL spec:
https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#reqresp-interaction.

this PR does a few things: 
1. moves `blocksByRangeHandler` and `beaconBlocksByRootHandler` handlers
to a different file in the `handlers` package. these are going to be the
more complicated handlers so they will be better in their own files.
2. makes `pingHandler` a method on the `*ConsensusHandlers` receiver and
starts returning the sequence number instead of the request. (see
https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#ping
`Peers request and respond with their local metadata sequence number`).
3. adds a `goodbyeHandler` to respond with a status message of 1. (see
https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#goodbye).
4. makes `statusHandler` a method on the `*ConsensusHandlers` receiver.
(the rest of this handler is still not implemented.
5. refactored the `heartbeats_test` into a table driven test. this makes
the test much more readable:
https://dave.cheney.net/2019/05/07/prefer-table-driven-tests.
2022-10-25 19:30:11 +02:00
Mike Neuder
4b5880e8d3
Adding VoluntaryExit, ProposerSlashing, and AttesterSlashing topics to pubsub service (#5851)
This is a follow on PR in support of
https://github.com/ledgerwatch/erigon/issues/5824. See
https://github.com/ledgerwatch/erigon/pull/5841 for the
`beacon_aggregate_and_proof` topic.

This PR adds support for the `voluntary_exit, proposer_slashing, and
attester_slashing` topics as defined in the phase 0 spec
https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#global-topics.

The CL types were already defined so this just adds the listening to the
pub sub service.
2022-10-24 17:52:49 +02:00
Mike Neuder
db1c2d2d82
Adding the AggregateAndProof global topic to the sentinel pubsub service (#5841)
This is the first PR in support of
https://github.com/ledgerwatch/erigon/issues/5824.

The phase 0 sepc
https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#global-topics
specifies 6 global pubsub topics that CL nodes need to handle.

This PR implements the `beacob_aggregate_and_proof` topic: 

https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#beacon_aggregate_and_proof.

The `AggregateAndProof` and `SignedAggregateAndProof` types are defined
here:
https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/validator.md#aggregateandproof.

I followed the implementation of `SignedBeaconBlockBellatrix`, which has
the following references:

1. cmd/lightclient/cltypes/types.go‎: defines the struct with relevant
SSZ annotations on the fields.
2. cmd/lightclient/cltypes/clone.go: this just returns a reference to an
empty object, so not super clear to me if it is necessary:
3. cmd/lightclient/rpc/common.go: this decodes gossip data, switching on
the type of gossip message that is received.
4. ‎cmd/lightclient/sentinel/service/service.go: this listens on the
pubsub channel and notifies when a packet of the relevant type comes in.
5. cmd/lightclient/sentinel/pubsub.go: this defines the gossip topic
struct.
6. cmd/lightclient/lightclient/subscriber.go: this is the lightclient
interface for the incoming messages that come from the sentinel.
2022-10-23 20:22:33 +02:00
Enrique Jose Avila Asapche
a874459835
got rid of time.Sleep (#5840) 2022-10-22 17:38:29 +01:00
Enrique Jose Avila Asapche
199a9a633c
lowered the amount of max peers (#5828)
Change the amount of max peers from 200 -> 33
2022-10-22 02:10:00 +02:00
Alex Sharov
86551e82f4
e3: parallel build missed indices (#5817) 2022-10-21 13:45:20 +07:00
Giulio rebuffo
889164e847
Strenghtened safety system for LC (#5811) 2022-10-20 16:07:17 +02:00
Leonard Chinonso
4c376b06da
Tests for lightclient/sentinel (#5772) 2022-10-20 15:41:28 +02:00
Giulio rebuffo
39dd749db0
Added verbosity flag to lightclient (#5785)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-18 12:46:48 +02:00
Giulio rebuffo
147c1d0460
fixed messaage id (more efficient gossip, reduction in bandwidth). (#5770)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-17 19:13:23 +02:00
Enrique Jose Avila Asapche
19f0776a20
Adding flags (#5759) 2022-10-16 19:54:12 +02:00
Giulio rebuffo
527c13fb90
fixed portion of CL verification for NextSyncCommittee (#5764) 2022-10-16 18:31:41 +02:00
Giulio rebuffo
ab3a0b0fbd
Switched to validating lightclient (#5761)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-16 00:07:27 +02:00
Giulio rebuffo
d075618ecc
Importing of ETH2 blocks from Lightclient (#5760) 2022-10-15 23:01:52 +02:00
Giulio rebuffo
6f1c2be4e5
Implemented clause 3 of phase 4 (#5758) 2022-10-15 19:03:31 +02:00
Giulio rebuffo
45f1c99b50
Commented out BLS validation because it breaks CI (#5755)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-15 12:38:37 +02:00
Giulio rebuffo
8e1b559ad3
implemented clauses 4 (i) and (ii) (#5752) 2022-10-15 00:14:04 +02:00
Giulio rebuffo
f6e7cc4d3e
Full BLS Validatition for lightclient update (#5751) 2022-10-14 20:32:33 +02:00
Giulio rebuffo
497d414b36
Added Lightclient update base validation without BLS (#5746)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-14 15:24:44 +02:00
Giulio rebuffo
5785f4ecec
Lightclient Checkpoint Sync (#5738) 2022-10-13 18:26:29 +02:00
Giulio rebuffo
5734065e36
retrieving finalized checkpoint (#5722) 2022-10-13 01:05:01 +02:00
Giulio rebuffo
513e149dc4
separated sentinel_node executable (#5721)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-12 23:47:06 +02:00
Giulio rebuffo
01975a4761
Added Bootstrap/Update handlers request (#5718)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-12 19:55:43 +02:00
Giulio rebuffo
327abb3c13
fixed lightclient finality/optimistic updates (#5706)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-11 23:44:02 +02:00
Giulio rebuffo
f09084f45d
Refactored and got rid of ton of useless code (#5690)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-10 20:14:07 +02:00
Giulio rebuffo
e1dedf4220
Added Finalty/Optimistic requests (#5688)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-10 15:58:31 +02:00
Giulio rebuffo
6193d02543
fixed no handlers responses (MetadataV1/MetadataV2) (#5679)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-09 20:28:49 +02:00
Giulio rebuffo
b8d7219d9d
refactored streaming with protobuffers (#5677)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-09 17:49:27 +02:00
Giulio rebuffo
a8338e4dd2
Infra refactoring (#5673)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-09 13:41:15 +02:00
Giulio rebuffo
067ad9a59b
Licensing new files in cmd/lightclient (#5672)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-08 16:15:44 +02:00
Giulio rebuffo
e36b1206f7
Resolve panic on ctx.done when using --experimental.lightclient (#5670)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-08 15:07:04 +02:00
Giulio rebuffo
5493ce864d
Removed code duplications for ping handlers (#5668)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-08 01:02:56 +02:00
Giulio rebuffo
7c7400512e
Fixed Ping Req/Resp (#5665)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-07 22:37:08 +02:00
Enrique Jose Avila Asapche
ff2e763a98
fixed disconnecting and now nimbus happy with us (#5662)
Added no disconnecting with metadata requests, and we answer now with a
bogus status call and proper prefix for both status and metadata
request. This changes allows for nimbus nodes to connect to us and
actually stay connected.
2022-10-07 18:40:18 +02:00
Giulio rebuffo
bbc85efb4f
Fixed Encoding of requests + Added unit tests (#5658)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-07 14:38:12 +02:00
Enrique Jose Avila Asapche
6a55337266
Metadata resp (#5634)
Now we are sending correct metadata responses
We disconnect from peer no matter if we fail to respond to them once we
receive a goodbye message
We close all writers once we are done writing to signal our peer
2022-10-06 22:53:24 +02:00
Giulio rebuffo
ebe63cfa30
Added tests for config functions and further refactoring (#5646)
Co-authored-by: giuliorebuffo <giuliorebuffo@system76-pc.localdomain>
2022-10-06 18:23:46 +02:00
Enrique Jose Avila Asapche
d566d4af09
leaving reqresp stream open (#5636)
We were closing our reqresp stream, which caused the peers to disconnect
from us.
2022-10-06 16:16:02 +01:00