Commit Graph

142 Commits

Author SHA1 Message Date
terence
5a66807989
Update to V5 (#13622)
* First take at updating everything to v5

* Patch gRPC gateway to use prysm v5

Fix patch

* Update go ssz

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-15 05:46:47 +00:00
Potuz
79bb7efbf8
Check init sync before getting payload attributes (#13479)
* Check init sync before getting payload attributes

This PR adds a helper to forkchoice to return the delay of the latest
imported block. It also adds a helper with an heuristic to check if the
node is during init sync. If the highest imported node was imported with
a delay of less than an epoch then the node is considered in regular
sync. If on the other hand, in addition the highest imported node is
more than two epochs old, then the node is considered in init Sync.

The helper to check this only uses forkchoice and therefore requires a
read lock. There are four paths to call this

1) During regular block processing, we defer a function to send the
   second FCU call with attributes. This function may not be called at
all if we are not regularly syncing
2) During regular block processing, we check in the path
   `postBlockProces->getFCUArgs->computePayloadAttributes` the payload
attributes if we are syncing a late block. In this case forkchoice is
already locked and we add a call in `getFCUArgs` to return early if not
regularly syncing
3) During handling of late blocks on `lateBlockTasks` we simply return
   early if not in regular sync (This is the biggest change as it takes
a longer FC lock for lateBlockTasks)
4) On Attestation processing, in UpdateHead, we are already locked so we
   just add a check to not update head on this path if not regularly
syncing.

* fix build

* Fix mocks
2024-01-17 15:39:28 +00:00
Potuz
67dccc5e43
Break out several helpers from postBlockProcess (#13419)
* Break out several helpers from `postBlockProcess`

In addition fix a bug found by @terencechain where we should use a slot
context instead of the parent context in the second FCU call.

* Remove calls for tracked proposer

getPayloadAttribute already takes care of this
Also compute correctly the time into voting window

* call with attributes only when incoming block is canonical

* check for empty payload instead of only nil

* add unit tests

* move log for non-canonical block

* return early if the incoming block does not change head

* Pass fcuArgs as arguments

* lint
2024-01-06 02:29:07 +00:00
Nishant Das
7e6fd5fd8b
Make Reorging Of Late Blocks The Permanent Default (#13405)
* make it the permanent default

* gaz

* fix merge conflicts
2024-01-03 14:46:58 +00:00
Potuz
31c72672d7
Remove the getPayloadAttribute call from updateForkchoiceWithExecution (#13402)
* Remove the getPayloadAttribute call from updateForkchoiceWithExecution

* Move log
2024-01-03 12:43:40 +00:00
Potuz
9809f5ac77
Simplify fcu 1 (#13387)
* Remove unsafe proposer indices cache

* Simplify FCU #1

This PR starts the process of gradually simplifying FCU
It removes the responsibility of getting the state and block from this
function and informing if head has changed. It is only called when the
imported block has actually become head.

* Add a call to FCU in edge cases
2023-12-30 12:20:20 +00:00
Nishant Das
eb713d1177
Refactor Network Config Into Main Config (#13364)
* change parameters to main config

* add more changes

* change to accepted format

* fix changes in config

* gaz

* fix test

* fix test again
2023-12-19 14:59:30 +00:00
terence
243bcb03ce
Fix FFG LMD Consistency Check (Option 2) (#13258)
* Fix FFG LMD Consistency Check with TargetRootForSlot

* Add test, removed  implementation

* convert to epoch and fix self target

---------

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2023-12-04 15:31:18 +00:00
Potuz
83ce7e3607
Verify lmd without ancestor (#13250) 2023-12-01 17:31:27 +00:00
Potuz
27d8b1c358
add more descriptive log on FFG-LMD consistency (#12763)
* add more descriptive log on FFG-LMD consistency

* Update beacon-chain/blockchain/receive_attestation.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* Update beacon-chain/blockchain/receive_attestation.go

---------

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-08-21 14:06:59 +00:00
Potuz
dd14d5cef0
refactor slot tickers with intervals (#12440)
* refactor slot tickers with intervals

* GoDoc

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-08-03 20:55:16 -03:00
Nishant Das
dd403f830c
clean up logging (#12653) 2023-07-24 20:29:50 +08:00
Potuz
b9917807d8
Ignore some untimely attestations (#12387)
* Ignore some untimely attestations

* correct child slot check

* consider tips as viable for checkpoints

* deal with canonical blocks

* forkchoice unit tests

* blockchain readonly beacon state

* Ignore some untimely attestations

* correct child slot check

* consider tips as viable for checkpoints

* deal with canonical blocks

* forkchoice unit tests

* blockchain readonly beacon state

* Fix AttestationTargetState mock

* Fix ineffectual assignment lint

* Fix blockchain tests

* Fix build

* Add Nil check

* add comment on lock

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-05-12 16:55:33 +00:00
kasey
918129cf36
Replace statefeed Initialize (#12285)
* refactor initialization to blocking startup method

* require genesisSetter in blockchain, fix tests

* work-around gazelle weirdness

* fix dep gazelle ignores

* only call SetGenesis once

* fix typo

* validator test setup and fix to return right error

* move waitForChainStart to Start

* wire up sync Service.genesisWaiter

* fix p2p genesisWaiter plumbing

* remove extra clock type, integrate into genesis

and rename

* use time.Now when no Nower is specified

* remove unused ClockSetter

* simplify rpc context checking

* fix typo

* use clock everywhere in sync; [32]byte val root

* don't use DeepEqual to compare [32]byte and []byte

* don't use clock in init sync, not wired up yet

* use clock waiter in blockchain as well

* use cancelable contexts in tests with goroutines

* missed a reference to WithClockSetter

* Update beacon-chain/startup/genesis.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/blockchain/service_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* more clear docs

* doc for NewClock

* move clock typedef to more logical file name

* adding documentation

* gaz

* fixes for capella

* reducing test raciness

* fix races in committee cache tests

* lint

* add tests on Duration slot math helper

* startup package test coverage

* fix bad merge

* set non-zero genesis time in tests that call Start

* happy deepsource, happy me-epsource

* replace Synced event with channel

* remove unused error

* remove accidental wip commit

* gaz!

* remove unused event constants

* remove sync statefeed subscription to fix deadlock

* remove state notifier

* fix build

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2023-05-03 04:34:01 +00:00
Potuz
b56f3e70df
log correctly the head changed when it actually changes (#12324)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-04-27 17:13:58 +00:00
Preston Van Loon
3894d60282
blockchain: add more spans to analyze block production timing (#12332)
* blockchain: add more spans to analyze block production timing

* There is a reason that we do not use the incoming context for forkchoice updates
2023-04-25 15:45:08 +00:00
Nishant Das
5cbbd26df4
Fallback in Update Head On Error (#12199)
* return if head call fails

* potuz's feedback

* potuz's feedback
2023-03-28 23:06:44 +08:00
Potuz
797cc360c7
Isolate forkchoice to blockchain package (#12174)
* Isolate Forkchoice to blockchain package

* Nishant's feedback

* Terence's feedback

* Nishant saving the day
2023-03-22 09:12:54 +08:00
Potuz
3536669a70
lock forkchoice on calls to Ancestor (#12162)
Co-authored-by: terencechain <terence@prysmaticlabs.com>
2023-03-20 15:51:42 -03:00
terencechain
d17996f8b0
Update to V4 🚀 (#12134)
* Update V3 from V4

* Fix build v3 -> v4

* Update ssz

* Update beacon_chain.pb.go

* Fix formatter import

* Update update-mockgen.sh comment to v4

* Fix conflicts. Pass build and tests

* Fix test
2023-03-17 18:52:56 +00:00
Potuz
93514de00f
Proposer head v5 (#12075)
* Use ShouldOverrideFCU in regular sync

* fix build

* fix tests

* add feature flag gate in updatehead

* fix rpc tests

* fix grcp tests

* deepsource

* add locks and reuse isNewProposer

* flip flag

* Fix ticker for late blocks

* implement get_proposer_head in GetBeaconBlock

* fix unit tests
2023-03-04 20:19:23 -03:00
Potuz
0a87210514
Forkchoice external locks v2 (#12036)
* write locks

* fix forkchoice tests

* blockchain locks

* lock on IsOptimistic

* use forkchoice instead of chaininfo within savehead

* Use forkchoice HasNode to check if a block is consistent with finality

* interface fix

* Use forkchoice HasNode to check if a block is consistent with finality

* interface fix

* fix tests

* remove VerifyFinalizedBlkDescendant

* don't write lock wrappers

* fix validateBeaconBlock

* Terence's review and more missing locks

* add lock for InForkChoice

* lock head on fillMissingBlockPayload

* fix lock on IsOptimisticForRoot

* fix lock in fillMissingBlockPayloadId

* extra comments

* lock proposerBoost on spectests

* nishant's review

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-03-02 09:10:52 -03:00
Potuz
ec13d52f03
Remove VerifyFinalizedBlkDescendant (#12046)
* Use forkchoice HasNode to check if a block is consistent with finality

* interface fix

* fix tests

* remove VerifyFinalizedBlkDescendant

* fix validateBeaconBlock
2023-02-25 13:39:13 -03:00
Potuz
ae76240f83
Use forkchoice HasNode to check if a block is consistent with finality (#12040)
* Use forkchoice HasNode to check if a block is consistent with finality

* interface fix

* fix tests
2023-02-23 21:21:34 +00:00
Potuz
c7f0a94b19
Remove state balance cache (#11964)
* Remove state balance cache

* remove unused metrics

* add missing balance update

* add comment about locks

* update balance only when updataing the justified checkpoint on unrealization

* update checkpoint balances at launch

* fix unit tests

* fix remaining test

* review

* fix build file

* fix test

---------

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-02-15 14:19:46 +00:00
terencechain
c070283bf1
Call FCU with attribute on non head block (#11919)
* Call FCU with attribute on non head block

* Fix condition

* Filer save head

* Refactor

* Rm

* Add tests

* Fix race test

* Potuz feedback

* Reorder r != currentHeadRoot

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-01-31 19:17:16 +00:00
Potuz
2817f8e8d6
update head should go even without attestations (#11503)
* update head should go even without attestations

* add unit tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-30 13:29:18 +00:00
terencechain
e3df25f443
Fix attestations update head error logging (#11514) 2022-09-29 12:16:42 -07:00
terencechain
e1c1d0f864
Don't return out of routine when update head errors out (#11470) 2022-09-20 17:31:59 +00:00
Potuz
7a8d780869
do not return on error during on_tick (#11463) 2022-09-19 08:10:11 -03:00
terencechain
c638e114db
Add new metrics (#11374)
* Better batch block warning

* New metrics

* Revert "Better batch block warning"

This reverts commit e21fcfcebec3a7a15f39c6336de2aba58f87c625.

* More metrics

* Add activation and exit queues

* Gaz
2022-08-31 18:05:50 -04:00
Raul Jordan
d077483577
Add V3 Suffix to All Prysm Packages (#11083)
* v3 import renamings

* tidy

* fmt

* rev

* Update beacon-chain/core/epoch/precompute/reward_penalty_test.go

* Update beacon-chain/core/helpers/validators_test.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/iface/BUILD.bazel

* Update beacon-chain/db/kv/kv.go

* Update beacon-chain/db/kv/state.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

* Update beacon-chain/sync/initial-sync/service.go

* fix deps

* fix bad replacements

* fix bad replacements

* change back

* gohashtree version

* fix deps

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-08-16 12:20:13 +00:00
terencechain
7fd2c08be3
Propagate FCU invalid error (#10997)
* Wrap fcu error

* Wrap fcu error

* Wrap error better

* More test

* Add else

* Potuz feedback

* Propagate the correct root for fcu

* Always return true for invalid

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-09 20:51:03 +00:00
Potuz
f8b4d8c57a
Deprecate store in blockchain pkg (#10903)
* Deprecate store WIP

* fix spectests build

* mock right interface

* sync tests build

* more tests builds

* blockchain tests

- TestFinalizedCheckpt_GenesisRootOk
- TestCurrentJustifiedCheckpt_CanRetrieve
- TestJustifiedCheckpt_GenesisRootOk
- TestHeadRoot_CanRetrieve
- TestHeadRoot_UseDB
- TestService_ProcessAttestationsAndUpdateHead
- TestService_VerifyWeakSubjectivityRoot
- TestVerifyFinalizedConsistency_InconsistentRoot_ProtoArray
- TestVerifyFinalizedConsistency_InconsistentRoot_DoublyLinkedTree
- TestVerifyFinalizedConsistency_Ok
- TestStore_OnBlock_ProtoArray
- TestStore_OnBlock_DoublyLinkedTree
- TestStore_OnBlockBatch_ProtoArray
- TestStore_OnBlockBatch_DoublyLinkedTree
- TestStore_OnBlockBatch_NotifyNewPayload
- TestCachedPreState_CanGetFromStateSummary_ProtoArray
- TestCachedPreState_CanGetFromStateSummary_DoublyLinkedTree

* more blockchain tests

- TestStore_OnBlockBatch_PruneOK_Protoarray
- TestFillForkChoiceMissingBlocks_CanSave_ProtoArray
- TestFillForkChoiceMissingBlocks_CanSave_DoublyLinkedTree
- TestFillForkChoiceMissingBlocks_RootsMatch_Protoarray
- TestFillForkChoiceMissingBlocks_RootsMatch_DoublyLinkedTree
- TestFillForkChoiceMissingBlocks_FilterFinalized_ProtoArray
- TestFillForkChoiceMissingBlocks_FilterFinalized_DoublyLinkedTree
- TestVerifyBlkDescendant
- Test_verifyBlkFinalizedSlot_invalidBlock
- TestChainService_SaveHeadNoDB

* update best justified from genesis

* deal with nil head on saveHead

* initialize prev justified checkpoint

* update finalization correctly

* update finalization logic

* update finalization logic

* track the wall clock on forkchoice spectests

* export copies of checkpoints from blockchain package

* do not use forkchoice's head on HeadRoot

* Remove debug remain

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* terence's review

* add forkchoice types deps

* wtf

* debugging

* init-sync: update justified and finalized checkpoints on db

* call updateFinalized instead of only DB

* remove debug symbols

* safe copy headroot

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-25 03:57:52 +00:00
Potuz
e439f4aff6
Forkchoice duplicate store (#10840)
* forkchoice tests

* blockchain tests

* no fatal errors

* use ZeroHash for genesis

* deal with zerohashes at genesis

* avoid nil best justified checkpoint on new store

* unit tests

* Debug log

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* Terence's review

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-16 18:21:40 +00:00
David
adabd1fa4f
service.head race condition fix (#10741)
* added various read mutex locks for service.head

* added RLocks around all calls to s.headRoot()

* added RLocks around all calls to s.headBlock()

* reduce lock surface-> Stop(),handleEpochBoundary()

* refactor Stop() to +performance, -lock_surface

* Apply suggestions from code review

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* fixed indentation

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-05-27 14:24:43 +00:00
terencechain
370cf1a6c8
Chain info: Return err if checkpoint is nil (#10729)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-20 18:41:33 +00:00
Potuz
61033ebea1
handle failure to update head (#10651)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-12 16:36:46 +00:00
terencechain
4ad1c4df01
Cache and use justified and finalized payload block hash (#10657)
* Cache and use justified and finalized payload block hash

* Fix tests

* Use real byte

* Fix conflicts

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-10 21:20:28 +00:00
terencechain
bbdf19cfd0
Process atts and update head before proposing (#10653)
* Process atts and updeate head

* Fix ctx

* New test and old tests

* Update validator_test.go

* Update validator_test.go

* Update service.go

* Rename to UpdateHead

* Update receive_attestation.go

* Update receive_attestation.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-09 23:12:50 +00:00
terencechain
dda48c452d
Move head change attestation log (#10636) 2022-05-05 15:09:58 +00:00
Potuz
025d053fa4
Change log when head changes (#10633) 2022-05-05 11:40:21 +00:00
terencechain
7af89a82c9
Use justified hash as safe block hash (#10627)
* Use justified hash as safe block hash

* Remove debug log

* Feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-05 00:36:09 +00:00
terencechain
0faf7ac01f
Debug log when new head is not in db (#10591) 2022-04-29 14:26:39 -07:00
terencechain
b7a82d0fd1
Can retrieve cached initial sync block and db block (#10568)
* Save cached initial sync blocks before getting head block

* Add better abstraction to get block

* Move unlock read to a better location

* Feedbacks

* Add head changed logging

* Harder hasBlock requirement

* Update beacon-chain/blockchain/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update receive_attestation.go

* Don't process head if the block is unknown

* Use a helper method

* Fix test

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-04-29 07:56:31 +00:00
Radosław Kapka
2060f876b1
Code cleanup in blockchain module (#10566)
* Code cleanup in `blockchain` module

* revert comment fix

* remove test

* forkchoice test fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-28 11:34:25 +00:00
Potuz
56187edb98
Use forkchoice first when checking canonical status (#10516) 2022-04-12 21:34:13 -03:00
terence tsao
0411b7eceb
Payload ID caching (#10481) 2022-04-06 16:36:52 -07:00
terence tsao
bdab34fd01
Use correct head for notifyForkchoiceUpdate (#10485)
* Use correct head for notifyForkchoiceUpdate

* Fix existing tests

* Update receive_attestation_test.go
2022-04-06 19:05:53 +08:00
Potuz
ce15823f8d
Save head after FCU (#10466)
* Update head after FCU

* rename function

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-04-01 00:08:01 +00:00