Commit Graph

361 Commits

Author SHA1 Message Date
realbigsean
a5addf661c
Rename eip4844 to deneb (#4129)
* rename 4844 to deneb

* rename 4844 to deneb

* move excess data gas field

* get EF tests working

* fix ef tests lint

* fix the blob identifier ef test

* fix accessed files ef test script

* get beacon chain tests passing
2023-03-26 11:49:16 -04:00
Pawan Dhananjay
b276af98b7
Rework block processing (#4092)
* introduce availability pending block

* add intoavailableblock trait

* small fixes

* add 'gossip blob cache' and start to clean up processing and transition types

* shard memory blob cache

* Initial commit

* Fix after rebase

* Add gossip verification conditions

* cache cleanup

* general chaos

* extended chaos

* cargo fmt

* more progress

* more progress

* tons of changes, just tryna compile

* everything, everywhere, all at once

* Reprocess an ExecutedBlock on unavailable blobs

* Add sus gossip verification for blobs

* Merge stuff

* Remove reprocessing cache stuff

* lint

* Add a wrapper to allow construction of only valid `AvailableBlock`s

* rename blob arc list to blob list

* merge cleanuo

* Revert "merge cleanuo"

This reverts commit 5e98326878c77528d0c4668c5a4db4a4b0fbaeaa.

* Revert "Revert "merge cleanuo""

This reverts commit 3a4009443a5812b3028abe855079307436dc5419.

* fix rpc methods

* move beacon block and blob to eth2/types

* rename gossip blob cache to data availability checker

* lots of changes

* fix some compilation issues

* fix compilation issues

* fix compilation issues

* fix compilation issues

* fix compilation issues

* fix compilation issues

* cargo fmt

* use a common data structure for block import types

* fix availability check on proposal import

* refactor the blob cache and split the block wrapper into two types

* add type conversion for signed block and block wrapper

* fix beacon chain tests and do some renaming, add some comments

* Partial processing (#4)

* move beacon block and blob to eth2/types

* rename gossip blob cache to data availability checker

* lots of changes

* fix some compilation issues

* fix compilation issues

* fix compilation issues

* fix compilation issues

* fix compilation issues

* fix compilation issues

* cargo fmt

* use a common data structure for block import types

* fix availability check on proposal import

* refactor the blob cache and split the block wrapper into two types

* add type conversion for signed block and block wrapper

* fix beacon chain tests and do some renaming, add some comments

* cargo update (#6)

---------

Co-authored-by: realbigsean <sean@sigmaprime.io>
Co-authored-by: realbigsean <seananderson33@gmail.com>
2023-03-24 17:30:41 -04:00
Diva M
25a2d8f078
Merge branch 'eip4844' into deneb-free-blobs 2023-03-24 14:38:29 -05:00
Diva M
1b9cfcc11b
Merge branch 'unstable' into eip4844 2023-03-24 13:32:50 -05:00
Diva M
7fad926b65
Merge commit '65a5eb829264cb279ed66814c961991ae3a0a04b' into eip4844 2023-03-24 13:24:21 -05:00
ethDreamer
d1e653cfdb
Update Blob Storage Structure (#4104)
* Initial Changes to Blob Storage

* Add Arc to SignedBlobSidecar Definition
2023-03-21 15:33:06 -04:00
Paul Hauner
3ac5583cf9 Set Capella fork epoch for Mainnet (#4111)
## Issue Addressed

NA

## Proposed Changes

Sets the mainnet Capella fork epoch as per https://github.com/ethereum/consensus-specs/pull/3300

## Additional Info

I expect the `ef_tests` to fail until we get a compatible consensus spec tests release.
2023-03-21 05:15:01 +00:00
ethDreamer
65a5eb8292 Reconstruct Payloads using Payload Bodies Methods (#4028)
## Issue Addressed

* #3895 

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2023-03-19 23:15:59 +00:00
Diva M
78414333a2
Merge branch 'eip4844' into deneb-free-blobs 2023-03-17 16:39:17 -05:00
Diva M
607242c127
Merge branch 'unstable' into eip4844 2023-03-17 16:26:51 -05:00
Jimmy Chen
1301c62436
Validator blob signing for the unblinded flow (#4096)
* Implement validator blob signing (full block and full blob)

* Fix compilation error and remove redundant slot check

* Fix clippy error
2023-03-17 09:29:25 -04:00
Paul Hauner
c3e5053612 Reduce false positive logging for late builder blocks (#4073)
## Issue Addressed

NA

## Proposed Changes

When producing a block from a builder, there are two points where we could consider the block "broadcast":

1. When the blinded block is published to the builder.
2. When the un-blinded block is published to the P2P network (this is always *after* the previous step).

Our logging for late block broadcasts was using (2) for builder-blocks, which was creating a lot of false-positive logs. This is because the builder publishes the block on the P2P network themselves before returning it to us and we perform (2). For clarity, the logs were false-positives because we claim that the block was published late by us when it was actually published earlier by the builder.

This PR changes our logging behavior so we do our logging at (1) instead. It also updates our metrics for block broadcast to distinguish between local and builder blocks. I believe the metrics change will be natively compatible with existing Grafana dashboards.

## Additional Info

One could argue that the builder *should* return the block to us faster, however that's not the case. I think it's more important that we don't desensitize users with false-positives.
2023-03-17 00:44:03 +00:00
realbigsean
fb7d729d92
migrate types to API crate 2023-03-15 16:03:36 -04:00
realbigsean
b303d2fb7e
lints 2023-03-15 15:32:22 -04:00
Diva M
34cea6d1c3
fmt 2023-03-15 12:37:53 -05:00
realbigsean
91a06ba484
Merge pull request #4083 from jimmygchen/post-block-and-blobs
Implement POST beacon block for EIP-4844
2023-03-15 13:35:37 -04:00
Diva M
4a39e43f96
Merge branch 'eip4844' into deneb-free-blobs 2023-03-15 12:26:30 -05:00
realbigsean
2e075c0a80
Merge pull request #4091 from realbigsean/get-validator-block-and-blobs
Get validator block and blobs
2023-03-15 12:22:03 -04:00
realbigsean
3d99e1f14d
move block contents to api crate, rename blob sidecars list 2023-03-15 12:15:08 -04:00
Jimmy Chen
2ef3ebbef3
Update SignedBlobSidecar container (#4078) 2023-03-15 11:03:56 -05:00
Jimmy Chen
775ca89801
Add blobs publishing 2023-03-15 15:57:30 +11:00
Diva M
7f2e9b80bb
Merge branch 'unstable' into eip4844 2023-03-14 12:00:32 -05:00
Jimmy Chen
a8978a5f69
Implement publish block and blobs endpoint (WIP) 2023-03-14 17:03:45 +11:00
Jimmy Chen
9ba390f7c5
Merge branch 'update-signed-blob-sidecar' into post-block-and-blobs 2023-03-14 11:52:25 +11:00
Jimmy Chen
e81a5029b8
Update SignedBlobSidecar container 2023-03-14 11:48:58 +11:00
Jimmy Chen
6ec0ce6070
Implement get validator block endpoint for EIP-4844 2023-03-13 16:50:08 +11:00
Michael Sproul
90cef1db86 Appease Clippy 1.68 and refactor http_api (#4068)
## Proposed Changes

Two tiny updates to satisfy Clippy 1.68

Plus refactoring of the `http_api` into less complex types so the compiler can chew and digest them more easily.

Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2023-03-13 01:40:03 +00:00
Divma
3898cf7be8
Modify lighthouse_network gossip types to free the blobs (#4064)
* Modify blob topics

* add signedblol type

pubsun messages are signed

* improve subnet topic index

* improve display code

* fix parse code

---------

Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
2023-03-10 16:23:36 +05:30
Divma
545532a883
fix rpc types to free the blobs (#4059)
* rename to follow name in spec

* use roots and indexes

* wip

* fix req/resp types

* move blob identifier to consensus types
2023-03-07 16:28:45 -05:00
Diva M
63011c5bca
fmt 2023-03-06 18:09:31 -05:00
Diva M
75a0d52ee0
get back the Blobs pub use to fix compilation issues 2023-03-06 17:58:00 -05:00
Pawan Dhananjay
39d4f0a1f3
Add BlobSidecar type 2023-03-06 17:10:12 -05:00
Diva M
d93753cc88
Merge branch 'unstable' into off-4844 2023-03-02 15:38:00 -05:00
Divma
047c7544e3 Clean capella (#4019)
## Issue Addressed

Cleans up all the remnants of 4844 in capella. This makes sure when 4844 is reviewed there is nothing we are missing because it got included here 

## Proposed Changes

drop a bomb on every 4844 thing 

## Additional Info

Merge process I did (locally) is as follows:
- squash merge to produce one commit
- in new branch off unstable with the squashed commit create a `git revert HEAD` commit
- merge that new branch onto 4844 with `--strategy ours`
- compare local 4844 to remote 4844 and make sure the diff is empty
- enjoy

Co-authored-by: Paul Hauner <paul@paulhauner.com>
2023-03-01 03:19:02 +00:00
Paul Hauner
eed7d65ce7
Allow for withdrawals in max block size (#4011)
* Allow for withdrawals in max block size

* Ensure payload size is counted
2023-02-21 18:03:10 +11:00
Paul Hauner
b72f273e47
Capella consensus review (#4012)
* Add extra encoding/decoding tests

* Remove TODO

The method LGTM

* Remove `FreeAttestation`

This is an ancient relic, I'm surprised it still existed!

* Add paranoid check for eip4844 code

This is not technically necessary, but I think it's nice to be explicit about
EIP4844 consensus code for the time being.

* Reduce big-O complexity of address change pruning

I'm not sure this is *actually* useful, but it might come in handy if we see a
ton of address changes at the fork boundary. I know the devops team have been
testing with ~100k changes, so maybe this will help in that case.

* Revert "Reduce big-O complexity of address change pruning"

This reverts commit e7d93e6cc7cf1b92dd5a9e1966ce47d4078121eb.
2023-02-21 15:33:27 +11:00
Michael Sproul
066c27750a
Merge remote-tracking branch 'origin/staging' into capella-update 2023-02-17 12:05:36 +11:00
realbigsean
b805fa6279
merge with upstream 2023-02-15 14:20:12 -05:00
Michael Sproul
2fcfdf1a01 Fix docker and deps (#3978)
## Proposed Changes

- Fix this cargo-audit failure for `sqlite3-sys`: https://github.com/sigp/lighthouse/actions/runs/4179008889/jobs/7238473962
- Prevent the Docker builds from running out of RAM on CI by removing `gnosis` and LMDB support from the `-dev` images (see: https://github.com/sigp/lighthouse/pull/3959#issuecomment-1430531155, successful run on my fork: https://github.com/michaelsproul/lighthouse/actions/runs/4179162480/jobs/7239537947).
2023-02-15 11:51:46 +00:00
Michael Sproul
fd379ae2e2
Upgrade sqlite3 2023-02-15 09:44:37 +01:00
Michael Sproul
918b688f72
Simplify payload traits and reduce cloning (#3976)
* Simplify payload traits and reduce cloning

* Fix self limiter
2023-02-15 14:17:56 +11:00
Emilia Hane
73c7ad73b8
Disable use of system time in tests 2023-02-14 13:33:38 +01:00
Michael Sproul
10d32ee04c
Quote Capella BeaconState fields (#3967) 2023-02-14 14:41:28 +11:00
realbigsean
28702c9d5d
merge upstream, add back get_blobs logic 2023-02-13 16:29:21 -05:00
ethDreamer
39f8327f73
Properly Deserialize ForkVersionedResponses (#3944)
* Move ForkVersionedResponse to consensus/types

* Properly Deserialize ForkVersionedResponses

* Elide Types in from_value Calls

* Added Tests for ForkVersionedResponse Deserialize

* Address Sean's Comments & Make Less Restrictive

* Utilize `map_fork_name!`
2023-02-10 08:49:25 -06:00
Emilia Hane
2653f88b5f
Fix conflicts rebasing eip4844 2023-02-10 15:35:00 +01:00
Emilia Hane
4d3ff347a3
Fixes after rebasing eip4844 2023-02-10 15:34:58 +01:00
Emilia Hane
5437dcae9c
Fix conflicts rebasing eip4844 2023-02-10 15:34:58 +01:00
Emilia Hane
d292a3a6a8
Fix conflicts rebasing eip4844 2023-02-10 09:41:23 +01:00
Emilia Hane
09370e70d9
Fix rebase conflicts 2023-02-10 09:41:19 +01:00