Commit Graph

1389 Commits

Author SHA1 Message Date
Potuz
67dccc5e43
Break out several helpers from postBlockProcess (#13419)
* Break out several helpers from `postBlockProcess`

In addition fix a bug found by @terencechain where we should use a slot
context instead of the parent context in the second FCU call.

* Remove calls for tracked proposer

getPayloadAttribute already takes care of this
Also compute correctly the time into voting window

* call with attributes only when incoming block is canonical

* check for empty payload instead of only nil

* add unit tests

* move log for non-canonical block

* return early if the incoming block does not change head

* Pass fcuArgs as arguments

* lint
2024-01-06 02:29:07 +00:00
Potuz
396b8bf970
Simplify fcu 4 (#13403)
* send two FCU when proposing

* compute voting window at runtime
2024-01-04 13:43:57 +00:00
Nishant Das
7e6fd5fd8b
Make Reorging Of Late Blocks The Permanent Default (#13405)
* make it the permanent default

* gaz

* fix merge conflicts
2024-01-03 14:46:58 +00:00
Potuz
31c72672d7
Remove the getPayloadAttribute call from updateForkchoiceWithExecution (#13402)
* Remove the getPayloadAttribute call from updateForkchoiceWithExecution

* Move log
2024-01-03 12:43:40 +00:00
Potuz
8c1e180dd1
Simplify fcu 2 (#13400)
* change getPayloadAttribute signature

* Unify different FCU arguments
2024-01-02 22:45:55 +00:00
Potuz
a602acf492
Remove getPayloadAttributes from FCU call (#13399) 2024-01-02 17:37:18 +00:00
Potuz
9809f5ac77
Simplify fcu 1 (#13387)
* Remove unsafe proposer indices cache

* Simplify FCU #1

This PR starts the process of gradually simplifying FCU
It removes the responsibility of getting the state and block from this
function and informing if head has changed. It is only called when the
imported block has actually become head.

* Add a call to FCU in edge cases
2023-12-30 12:20:20 +00:00
Potuz
cff5e2b5fe
Remove unsafe proposer indices cache (#13385) 2023-12-30 12:20:02 +00:00
Potuz
d4cc6fcf4a
update shuffling caches before calling FCU on epoch boundaries (#13383)
* update shuffling caches before calling FCU on epoch boundaries

* Terence's review
2023-12-28 15:19:09 +00:00
Potuz
e80db9554d
Use advanced epoch cache when preparing proposals (#13377) 2023-12-27 12:42:51 +00:00
Potuz
b7e0819f00
refactor Payload Id caches (#12987)
* init

- getLocalPayload does not use the proposer ID from the cache but takes
  it from the block

- Fixed tests in blockchain package
- Fixed tests in the RPC package
- Fixed spectests

EpochProposers takes 256 bytes that can be avoided to be copied, but
this optimization is not clear to be worth it.

assginmentStatus can be optimized to use the cached version from the
TrackedValidatorsCache

We shouldn't cache the proposer duties when calling getDuties but when
we update the epoch boundary instead

* track validators on prepare proposers

* more rpc tests

* more rpc tests

* initialize grpc caches

* Add back fcu log

Also fix two existing bugs wrong parent hash on pre Capella and wrong
blockhashes on altair

* use beacon default fee recipient if there is none in the vc

* fix validator test

* radek's review

* push always proposer settings even if no flag is specified in the VC

* Only register with the builder if the VC flag is set

Great find by @terencechain

* add regression test

* Radek's review

* change signature of registration builder
2023-12-22 18:47:51 +00:00
Nishant Das
eb713d1177
Refactor Network Config Into Main Config (#13364)
* change parameters to main config

* add more changes

* change to accepted format

* fix changes in config

* gaz

* fix test

* fix test again
2023-12-19 14:59:30 +00:00
Potuz
9efaa832cd
use different keys for the proposer indices cache (#13272)
* use different keys for the proposer indices cache

* Add a way to get the proposer indices from a checkpoint

* fix fuzzing tests

* use htr instead of body root

* move comment
2023-12-19 13:14:55 +00:00
terence
0eff83cb9d
Use a cache of one entry to build attestation (#13300)
* Use a cache of one entry to build attestation

* Gazelle

* Enforce on RPC side

* Rm unused var

* Potuz feedback, dont use pointer

* Fix tests

* Init fetcher

* Add in-progress

* Add back missing lock

* Potuz feedback

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

---------

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2023-12-18 16:12:43 +00:00
Nishant Das
b45a6664be
Enable Deneb For E2E Scenario Tests (#13317)
* fix all cases

* update web3signer

* current progress

* fix it finally

* push it back to capella

* remove hard-coded forks

* fix failing tests

* gaz

* fix dumb bug

* fix bad test setup

* change back
2023-12-16 11:37:44 +00:00
Preston Van Loon
db096488b0
fixing sa4006 (#13350) 2023-12-15 16:49:27 +00:00
Justin Traglia
1962cca69e
Fix error string generation for missing commitments (#13338) 2023-12-15 04:03:45 +00:00
terence
ddcf0c18dc
Excluse DA wait time for chain processing time (#13335)
* Excluse DA wait time for chain processing time

* Rename
2023-12-14 22:46:48 +00:00
Justin Traglia
52b9b65adb
Add sanity checks for bundle from builder (#13319)
* Add sanity checks for bundle from builder

* Add more checks to BlobsBundle.ToProto()

* Fix minor typo

* Fix tests & add new ones

* Add tests for ToProto

* Add "not" to error message

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-12-13 15:54:00 +00:00
Potuz
ce7452c97a
update spectests to 1.4.0-beta.5 (#13318)
* update spectests to 1.4.0-beta.5

* add spec config
2023-12-12 18:27:48 +00:00
Nishant Das
6a45323ab7
only run metrics for canonical blocks (#13289) 2023-12-07 11:03:23 +00:00
kasey
4008ea736f
Verify roblobs (#13245)
* scaffolding for verification package

* WIP blob verification methods

* lock wrapper for safer forkchoice sharing

* more solid cache and verification designs; adding tests

* more test coverage, adding missing cache files

* clearer func name

* remove forkchoice borrower (it's in another PR)

* revert temporary interface experiment

* lint

* nishant feedback

* add comments with spec text to all verifications

* some comments on public methods

* invert confusing verification name

* deep source

* remove cache from ProposerCache + gaz

* more consistently early return on error paths

* messed up the test with the wrong config value

* terence naming feedback

* tests on BeginsAt

* lint

* deep source...

* name errors after failure, not expectation

* deep sooource

* check len()==0 instead of nil so empty lists work

* update test for EIP-7044

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2023-12-07 02:36:25 +00:00
terence
243bcb03ce
Fix FFG LMD Consistency Check (Option 2) (#13258)
* Fix FFG LMD Consistency Check with TargetRootForSlot

* Add test, removed  implementation

* convert to epoch and fix self target

---------

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2023-12-04 15:31:18 +00:00
Preston Van Loon
b68a4e12aa
Update bazel and other CI improvements (#13246)
* Update bazel to 6.4.0, review flags

* Remove problematic/slow targets

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-01 22:20:54 +00:00
Potuz
461af4baa6
Add test helpers to produce commitments and proofs (#13242)
* Add test helpers to produce commitments and proofs

* go mod tidy

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-01 19:51:19 +00:00
Potuz
83ce7e3607
Verify lmd without ancestor (#13250) 2023-12-01 17:31:27 +00:00
terence
6c85587d14
Update broadcast method to use BlobSidecar instead of SingedBlobSidecar (#13221)
* Update broadcast method to use BlobSidecar instead of SingedBlobSidecar

* Fix test
2023-11-24 07:18:00 +00:00
Potuz
07ee42660a
lock RecentBlockSlot (#13212)
* lock RecentBlockSlot

* Kasey's fix
2023-11-22 16:58:00 -03:00
kasey
1123df7432
Verified roblobs (#13190)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-21 18:44:38 +00:00
terence
8f5ae760ee
Add concurrency test for getting attestation state (#13196)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-16 17:52:59 +00:00
Radosław Kapka
d1dd8471a3
Debug HTTP endpoints (#13164)
* Debug HTTP endpoints

* register endpoints

* tests

* small fixes

* config test fix
2023-11-03 15:33:46 +00:00
Terence
57eda1de63
Add RO blob sidecar (#13144) 2023-11-01 10:03:49 -07:00
Potuz
dc9d34b41b
fix head slot in log (#13139) 2023-10-30 16:56:20 -03:00
Potuz
2ea5bff9c0
Log when sending FCU with payload attributes (#13137)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-30 12:30:20 +00:00
terencechain
c2433ff854
Update spectest and changed minimal preset for field elements (#13090)
* update trusted setup

* update dependencies

* Update workspace

---------

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-30 11:41:58 +00:00
Preston Van Loon
82640b3d88
Enable profile guided optimization for beacon-chain (#13035)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-30 05:52:36 +00:00
terencechain
203dc5f63b
Check blob index duplication for blob notifier (#13123)
* Check blob index duplication for blob notifier

* Better locks and test

* Better locks and test

* Kasey's feedback

* Fix init
2023-10-27 03:26:34 +00:00
Delweng
00826e8858
beacon-chain/blockchain: fix some datarace in go test (#13036)
* beacon-chain/blockchain: mockBeaconNode with mutex

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-node/forkchoice: bool -> atomic.Bool

Signed-off-by: jsvisa <delweng@gmail.com>

* beacon-chain/blockchain: datarace in concurrent postBlock

Signed-off-by: jsvisa <delweng@gmail.com>

* Revert "beacon-node/forkchoice: bool -> atomic.Bool"

This reverts commit 4aad095b0fc52bd707cc0d042fc0728f1cb2ce4c.

Signed-off-by: jsvisa <delweng@gmail.com>

---------

Signed-off-by: jsvisa <delweng@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-23 21:55:47 +00:00
Potuz
0920fb1f61
Return early from ReceiveBlock if already sycned (#13089)
* Return early from ReceiveBlock if already sycned

* Fix bad setup test
2023-10-22 18:31:50 -03:00
terencechain
823f8ee3a2
Fix redundant type converstion (#13076)
* Fix redundant type converstion

* Revert generated changes

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-20 15:07:10 +00:00
Potuz
e231d88ca0
Remove sidecars with invalid proofs (#13070)
* Remove sidecars with invalid proofs

* unhandled error

* Add missing delete
2023-10-19 12:33:19 -03:00
terencechain
cb5ce74a23
Add pending blobs queue for missing parent block (#13005)
* Add pending blobs queue for missing parent block

* Prune sidecars older than previous slot

* Prune sidecar based on time

* Tests

* Fix state notifier

* Wait for chain to start

* Remove logs

* Remove bad logs

* James feedback

* Fix conflict

* Rm outdated check

* Potuz's feedback

* Kasey's feedback

* Use 11s mark

* Use secs

* Add pending blobs queue for missing parent block

* Prune sidecars older than previous slot

* Prune sidecar based on time

* Tests

* Fix state notifier

* Wait for chain to start

* Remove logs

* Remove bad logs

* James feedback

* Fix conflict

* Rm outdated check

* Potuz's feedback

* Kasey's feedback

* Use 11s mark

* Use secs

* Add test case for duplicates

* Radek's feedback

* Fix test
2023-10-17 14:42:15 +00:00
Radosław Kapka
44973b0bb3
Using the multi value slice for the beacon state (#12549)
* in progress...

* in progress...

* remove log

* log root

* Revert "Auxiliary commit to revert individual files from f12a609ea2a2f1e87e97321f3a717cd324b5ae97"

This reverts commit 5ae35edb6477d8d0ea4e94b273efc6590484da85.

* cleanup

* remove log

* remove whitespace

* remove logs

* more stuff

* copy

* always rebuild trie

* revert

* add state

* init state

* fix all

* uintptr

* move slice to new package

* lock in `Detach`

* remove constraint

* reorder

* blockroots and stateroots

* fill roots in empty()

* fix hasher

* implement slice for balances and inactivity scores

* detach in setters

* Revert "implement slice for balances and inactivity scores"

This reverts commit 59eb9df8d766cb1c44a7eb5b3f5e3c042249943d.

# Conflicts:
#	beacon-chain/state/state-native/setters_validator.go

* use counter to track states

* typos

* rename interface

* balances

* gauge

* some improvements

* first try with map

* fix

* inactivity scores in progress

* fix test

# Conflicts:
#	beacon-chain/state/state-native/helpers_test.go

* test fixes

* ToProto fix

* copy roots

* validators

* build fixes

* fix bug in `ToProto`

* fix fuzz test

* fix bug in slice getters

* fix state equality checks

* make tests pass

* make tests pass

* more test updates

* Revert "Auxiliary commit to revert individual files from 34e7344bff08a589e6341bb1829e3cb74159e878"

This reverts commit ecd64efa8917f37ca41460e0356ff007fe55dd9d.

* Revert "make tests pass"

This reverts commit 0cf00f19eecf4678cd2b866dd107f3179d0426ef.

* Revert "make tests pass"

This reverts commit 521b65e1d2e13be3d720f333008b6838a8e78878.

* pass tests

* deepequal identifiable types

* Deflake `cloners_test.go`

* feature flag for block roots

* feature flag

* remove recursive locks

* reduce complexity of rootSelector

* fix randao mixes root

* some fixes

* revisit tests

* revert change to field trie helpers

* initialize field map for tests

* remove whitespace

* initialize roots with proper length

* more fixes

* out of bounds message fix

* optimize length calculation

* remove call to Len in PubkeyAtIndex

* don't log deposits

* unit tests

* unit tests

* fix

* comments

* test fixes

* id

* remove Enumerator interface

* review feedback

* simplify field trie

* bring back fieldtrie package

* fix bazel file

* use handle32ByteArrays for root computation

* fix locks

* metrics

* bzl

* simplify some things

* use htr in state test

* remove code from require package

* gzl

* more htr

* Fuzzing of the multi-value slice

* assert values

* getter optimizations

* use At when reading from validators

* Nishant's review

* restore safe copy

* remove empty line

* build fix

* restore how we get root at index for deafult mode

* more review comments

* optimize default behavior

* simplify Slice calls

* test fix

* remove unnecessary package

* remove unused setter

* make fieldMap unexported

* some improvements in state package

* call `Slice` instead of manually copying

* unlock in ReadFromEveryValidator

* Potuz's comments

* lock the state when reading from all validators

# Conflicts:
#	beacon-chain/state/state-native/getters_validator.go

* add back preston's changes

* add index

---------

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-10-06 01:10:05 +00:00
terencechain
f9a40ef111
Handle wrapped invalid block error (#12982)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-04 18:16:35 +00:00
Potuz
56f3dafb54
Lock forkchoice on late block tasks (#12978)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-10-02 17:07:03 +00:00
Nicolás Pernas Maradei
70380660b3
[1/5] Light client sync protocol (#12853)
* generate SyncCommittee SSZ

* refactor error handling

* rewards: use http2.HandleError

* add light client protobuf files

* add light client helpers
2023-10-02 15:34:34 +00:00
Potuz
b4e72f1e19
Deneb spectests release v1.4.0-beta.2-hotfix (#12959)
* Update and use max per epoch churn limit

* Update spec tests

* Fix e2e test

* deneb fork epoch condition

* Fix lint and better casting

* fix ordering

* fix check

* gaz

* Fix more tests

* Apply proposer boost to first block in equivocation

* Increase timeout

* Don't increase timeout, it's not the reason

* implement deneb forkchoice spectests

expose ReceiveBlob from the blockchain package

* spin_off_helper

* remove minimal tests

* Terence's review

* Add process register test for Deneb

* Terence's suggestion

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* fix forkchoice minimal

* fix minimal sha

* general sha

* different repos

* different repos

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2023-09-28 21:05:23 +00:00
Nishant Das
42c192d97d
Fix More Racy Tests in Blockchain (#12957) 2023-09-26 09:17:11 +00:00
Nishant Das
70225186ff
Fix TestService_ReceiveBlock (#12953)
* fix tests

* radek's review

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-25 15:24:32 +00:00
james-prysm
723f73795f
Deneb: blob sidecar events (#12928)
* adding in deneb blob event triggers

* fixing linting

* kasey's feedback

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-22 21:54:10 +00:00