67 Commits

Author SHA1 Message Date
terence
5a66807989
Update to V5 (#13622)
* First take at updating everything to v5

* Patch gRPC gateway to use prysm v5

Fix patch

* Update go ssz

---------

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2024-02-15 05:46:47 +00:00
Potuz
79bb7efbf8
Check init sync before getting payload attributes (#13479)
* Check init sync before getting payload attributes

This PR adds a helper to forkchoice to return the delay of the latest
imported block. It also adds a helper with an heuristic to check if the
node is during init sync. If the highest imported node was imported with
a delay of less than an epoch then the node is considered in regular
sync. If on the other hand, in addition the highest imported node is
more than two epochs old, then the node is considered in init Sync.

The helper to check this only uses forkchoice and therefore requires a
read lock. There are four paths to call this

1) During regular block processing, we defer a function to send the
   second FCU call with attributes. This function may not be called at
all if we are not regularly syncing
2) During regular block processing, we check in the path
   `postBlockProces->getFCUArgs->computePayloadAttributes` the payload
attributes if we are syncing a late block. In this case forkchoice is
already locked and we add a call in `getFCUArgs` to return early if not
regularly syncing
3) During handling of late blocks on `lateBlockTasks` we simply return
   early if not in regular sync (This is the biggest change as it takes
a longer FC lock for lateBlockTasks)
4) On Attestation processing, in UpdateHead, we are already locked so we
   just add a check to not update head on this path if not regularly
syncing.

* fix build

* Fix mocks
2024-01-17 15:39:28 +00:00
kasey
ce2344301c
forkchoice.Getter wrapper with locking wrappers (#13244)
* forkchoice.Getter wrapper with locking wrappers

* comments

* lint

* only expose fast fc getters

* potuz feedback re rlock

* update mocks for new fc method

* appease deepsource

* add missing exported func comment

* yeet errors to make the linter happy

* even more devious _discard

* rm TargetRoot

* derp

* handle nil error in _discard

* deep source

---------

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-04 21:01:39 +00:00
terence
243bcb03ce
Fix FFG LMD Consistency Check (Option 2) (#13258)
* Fix FFG LMD Consistency Check with TargetRootForSlot

* Add test, removed  implementation

* convert to epoch and fix self target

---------

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2023-12-04 15:31:18 +00:00
Potuz
cf8e554981
track target in forkchoice (#13249) 2023-12-01 16:30:34 +00:00
Radosław Kapka
d1dd8471a3
Debug HTTP endpoints (#13164)
* Debug HTTP endpoints

* register endpoints

* tests

* small fixes

* config test fix
2023-11-03 15:33:46 +00:00
Potuz
3394bbe359
forkchoice return last canonical root of epoch (#12954)
* forkchoice return last canonical root of epoch

* move inside loop

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-25 16:36:43 +00:00
Potuz
b9917807d8
Ignore some untimely attestations (#12387)
* Ignore some untimely attestations

* correct child slot check

* consider tips as viable for checkpoints

* deal with canonical blocks

* forkchoice unit tests

* blockchain readonly beacon state

* Ignore some untimely attestations

* correct child slot check

* consider tips as viable for checkpoints

* deal with canonical blocks

* forkchoice unit tests

* blockchain readonly beacon state

* Fix AttestationTargetState mock

* Fix ineffectual assignment lint

* Fix blockchain tests

* Fix build

* Add Nil check

* add comment on lock

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-05-12 16:55:33 +00:00
Potuz
300a827027
Use same signature for unrealized justified hash and justified hash (#12314)
* Use same signature for unrealized justified hash and justified hash

* fix build
2023-04-20 15:59:12 +00:00
terencechain
17488028a6
Use justified hash as safe block hash when passing attribute (#12196)
* With attribute, use justified hash as safe block hash

* Use UnrealizedJustifiedPayloadBlockHash

* Cleanups

* Add test to forkchoice

* Block proposal

* Fix mock

* Fix test

* Fix rest of the tests

* Potuz feedback

* Fix test

* Use unrealized justified hash for regular fcu
2023-03-29 23:38:36 +00:00
terencechain
d17996f8b0
Update to V4 🚀 (#12134)
* Update V3 from V4

* Fix build v3 -> v4

* Update ssz

* Update beacon_chain.pb.go

* Fix formatter import

* Update update-mockgen.sh comment to v4

* Fix conflicts. Pass build and tests

* Fix test
2023-03-17 18:52:56 +00:00
Potuz
99997fa2e3
forkchoice changes (#12126)
* forkchoice changes

* gazelle

* remove best justified from forkchoice dump

* keep protobufs and update comment

* remove phase0 tests

* bumb spectests version

* missing sha

* Fix sha256 for spec

* rpc tests

* Mark field 3 as reserved so it will never be used it again

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2023-03-16 14:27:30 -03:00
terencechain
78d49fda13
Minor cleanup to forkchoice pkg (#12078)
* Minor cleanup to forkchoice pkg

* Rm implementation

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-03-06 17:01:36 +00:00
Potuz
93514de00f
Proposer head v5 (#12075)
* Use ShouldOverrideFCU in regular sync

* fix build

* fix tests

* add feature flag gate in updatehead

* fix rpc tests

* fix grcp tests

* deepsource

* add locks and reuse isNewProposer

* flip flag

* Fix ticker for late blocks

* implement get_proposer_head in GetBeaconBlock

* fix unit tests
2023-03-04 20:19:23 -03:00
Potuz
0a87210514
Forkchoice external locks v2 (#12036)
* write locks

* fix forkchoice tests

* blockchain locks

* lock on IsOptimistic

* use forkchoice instead of chaininfo within savehead

* Use forkchoice HasNode to check if a block is consistent with finality

* interface fix

* Use forkchoice HasNode to check if a block is consistent with finality

* interface fix

* fix tests

* remove VerifyFinalizedBlkDescendant

* don't write lock wrappers

* fix validateBeaconBlock

* Terence's review and more missing locks

* add lock for InForkChoice

* lock head on fillMissingBlockPayload

* fix lock on IsOptimisticForRoot

* fix lock in fillMissingBlockPayloadId

* extra comments

* lock proposerBoost on spectests

* nishant's review

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2023-03-02 09:10:52 -03:00
Potuz
166f7119ec
Implement should_override_forkchoice_update (#11981)
* Implement should_override_forchoice_update

* add tests

* remove unused exported function

* go mod tidy

* fix go mod

* Fix go mod tidy

* Fix context import

* mod tidy

* fix test

* Terence's review

* fix test

---------

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2023-02-18 07:37:03 -03:00
Potuz
b687d29d02
Forkchoice interface cleanup (#12010)
* remove AllTipsAreInvalid

* rename InsertOptimisticChain

* remove proposer boost interface

* remove HasParent
2023-02-17 17:36:52 +00:00
Potuz
c7f0a94b19
Remove state balance cache (#11964)
* Remove state balance cache

* remove unused metrics

* add missing balance update

* add comment about locks

* update balance only when updataing the justified checkpoint on unrealization

* update checkpoint balances at launch

* fix unit tests

* fix remaining test

* review

* fix build file

* fix test

---------

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-02-15 14:19:46 +00:00
Radosław Kapka
77d3ccb9ad
Clean up state types (#11916)
* Clean up state types

* rename package
2023-01-26 14:40:12 +00:00
Potuz
66b6177ed3
Log forkchoice weights on reorgs (#11849) 2023-01-05 20:06:40 -03:00
Radosław Kapka
c7a3cf8563
GetForkChoice API endpoint (#11680)
* proto

* middleware

* change structure

* fix all issues

* test

* validity field
2022-11-28 19:17:53 +00:00
Potuz
ad65c841c4
Add a justified balance getter to forkchoice (#11513)
* init

* unit test

* DeepSource complains

* gaz

* shutting deepsource down

* change var name and use handler type

* Kasey's name suggestion

* fix stategen

* interface signature

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-09-30 06:39:07 -03:00
terencechain
9fefb33cdf
Forkchoice: track highest received root (#11434) 2022-09-12 16:29:01 +02:00
Potuz
6cf4f3c260
Report voted fractions for a given root (#11421)
* Report voted fractions for a given root

* unused parameters

* comment update

* change test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-11 19:04:40 +00:00
Radosław Kapka
30cd158ae5
Move forkchoice dump to eth namespace (#11325)
* protos

* server code

* rename v2 to v1 in endpoint

* middleware

* test fix

* test fix

* oops

* remove duplicated import

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-26 14:54:32 +00:00
Potuz
52271cf0ba
Report depth and distance on reorgs (#11315)
* Report depth and distance on reorgs

* rename to CommonAncestor

* change event feed

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-25 23:59:08 +00:00
Potuz
e1f56d403c
Restore forkchoice dump endpoint. (#11312)
* Restore forkchoice dump endpoint.

Only working on doubly-linked-tree.

* unit test

* revert proto changes

* protoarray

* Deepsource

* shut up!

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-25 23:37:23 +00:00
Raul Jordan
d077483577
Add V3 Suffix to All Prysm Packages (#11083)
* v3 import renamings

* tidy

* fmt

* rev

* Update beacon-chain/core/epoch/precompute/reward_penalty_test.go

* Update beacon-chain/core/helpers/validators_test.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/iface/BUILD.bazel

* Update beacon-chain/db/kv/kv.go

* Update beacon-chain/db/kv/state.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

* Update beacon-chain/sync/initial-sync/service.go

* fix deps

* fix bad replacements

* fix bad replacements

* change back

* gohashtree version

* fix deps

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-08-16 12:20:13 +00:00
Potuz
155d60dd60
Remove forkchoice debug endpoint (#11105)
* Remove forkchoice debug endpoint

* change protos

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-08-15 21:04:39 +00:00
terencechain
1117118435
Add highest slot and blocks received last epoch tracking (#11146)
* Add highest slot and blocks received last epoch tracking

* Comments

* Potuz feedback

* Fix tesT

* Potuz feedback, use clock time

* Raul's feedback

* Raul's feedback

* Use field params

* Fix minimal value

* Fix constant

* Fix constant

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-10 15:58:52 +00:00
Potuz
19e4cd30cf
Forkchoice: expose if no tip is viable (#11153)
* Forkchoice: expose if no tip is viable

This PR changes the behavior on when the node is considered optimistic.
A call to `blockchain.IsOptimistic()` relies solely on forkchoice, if
all tips are invalid, then it's optimistic. If the current headroot is
not in forkchoice then it's optimistic.

A call to `blockchain.IsOptimisticForRoot()` will return true if the
requested root is headroot and it's not found in forkchoice

* update comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-08-03 13:59:03 +00:00
Potuz
5d29ca4984
Experimental disable boundary checks (#10936)
* init

* bellatrix + altair tests passing

* Add Phase0 support

* add feature flag

* phase0 test

* restore testvectors

* mod tidy

* state tests

* gaz

* do not call precompute

* fix test

* Fix context

* move to own's method

* remove spectests pulltips

* time import

* remove phase0

* mod tidy

* fix getters

* Update beacon-chain/forkchoice/doubly-linked-tree/types.go

* reviews

* fix workspace

* Recursive rlocks

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-06-29 23:37:21 +00:00
Potuz
f8b4d8c57a
Deprecate store in blockchain pkg (#10903)
* Deprecate store WIP

* fix spectests build

* mock right interface

* sync tests build

* more tests builds

* blockchain tests

- TestFinalizedCheckpt_GenesisRootOk
- TestCurrentJustifiedCheckpt_CanRetrieve
- TestJustifiedCheckpt_GenesisRootOk
- TestHeadRoot_CanRetrieve
- TestHeadRoot_UseDB
- TestService_ProcessAttestationsAndUpdateHead
- TestService_VerifyWeakSubjectivityRoot
- TestVerifyFinalizedConsistency_InconsistentRoot_ProtoArray
- TestVerifyFinalizedConsistency_InconsistentRoot_DoublyLinkedTree
- TestVerifyFinalizedConsistency_Ok
- TestStore_OnBlock_ProtoArray
- TestStore_OnBlock_DoublyLinkedTree
- TestStore_OnBlockBatch_ProtoArray
- TestStore_OnBlockBatch_DoublyLinkedTree
- TestStore_OnBlockBatch_NotifyNewPayload
- TestCachedPreState_CanGetFromStateSummary_ProtoArray
- TestCachedPreState_CanGetFromStateSummary_DoublyLinkedTree

* more blockchain tests

- TestStore_OnBlockBatch_PruneOK_Protoarray
- TestFillForkChoiceMissingBlocks_CanSave_ProtoArray
- TestFillForkChoiceMissingBlocks_CanSave_DoublyLinkedTree
- TestFillForkChoiceMissingBlocks_RootsMatch_Protoarray
- TestFillForkChoiceMissingBlocks_RootsMatch_DoublyLinkedTree
- TestFillForkChoiceMissingBlocks_FilterFinalized_ProtoArray
- TestFillForkChoiceMissingBlocks_FilterFinalized_DoublyLinkedTree
- TestVerifyBlkDescendant
- Test_verifyBlkFinalizedSlot_invalidBlock
- TestChainService_SaveHeadNoDB

* update best justified from genesis

* deal with nil head on saveHead

* initialize prev justified checkpoint

* update finalization correctly

* update finalization logic

* update finalization logic

* track the wall clock on forkchoice spectests

* export copies of checkpoints from blockchain package

* do not use forkchoice's head on HeadRoot

* Remove debug remain

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* terence's review

* add forkchoice types deps

* wtf

* debugging

* init-sync: update justified and finalized checkpoints on db

* call updateFinalized instead of only DB

* remove debug symbols

* safe copy headroot

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-25 03:57:52 +00:00
Potuz
20b4c60bcb
Apply Proposer Boost from forkchoice (#10893)
* Apply proposer boost at block insertion

* gaz

* fix tests

* revert time change

* conflict

* change genesis time in forkchoice on spectests

* Terence Review

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-22 13:56:07 +00:00
Potuz
2d0fdf8b4a
prune within forkchoice (#10896) 2022-06-17 12:53:17 -03:00
Potuz
e439f4aff6
Forkchoice duplicate store (#10840)
* forkchoice tests

* blockchain tests

* no fatal errors

* use ZeroHash for genesis

* deal with zerohashes at genesis

* avoid nil best justified checkpoint on new store

* unit tests

* Debug log

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* Terence's review

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* update capitalization

Co-authored-by: terencechain <terence@prysmaticlabs.com>

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-16 18:21:40 +00:00
Potuz
18fc17c903
Forkchoice checkpoints (#10823)
* double_tree_changes

* protoarray changes

* beacon-chain changes

* spec tests and debug rpc fixes

* more conflicts

* more conflicts

* Terence's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-09 22:28:30 +00:00
Potuz
2536195be0
Change forkchoice API (#10774)
* Change forkchoice API

doubly-linked-tree changes

* protoarray changes

* blockchain tests

* rebase and fix conflicts

* More blockchain fixes

* blockchain test fixes

* doubly linked tree changes again

* protoarray changes v2

* blockchain packages v2

* Radek's review

* Fix on batch processing

* Terence's review

* fill in at start

* Revert "fill in at start"

This reverts commit 8c11db063a02a59df8e0ac6d0af0c8923921dc76.

* wrap error message

* fill in before mutating the state

* wrap nil node errors

* handle unknown optimistic status on init sync

* rename insert function

* prune on batches only after forkchoice insertion

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-05 17:48:21 +00:00
Potuz
b582ca27e6
Forkchoice fill chain (#10783)
* Move chain inclusions to forkchoice handling

* gaz

* return early if no blocks are pending

* Terence's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-31 10:19:48 +00:00
terencechain
364ad3fbda
Reinsert reorg atts (#10767)
* Add common ancestor root for protoarray

* More efficient algo

* Tests

* Fix linting

* Fix linting

* Fix linting

* Fix linting

* Fix linting

* Fix linting

* Apply suggestions from code review

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* Feedbacks

* Revert saveHead changes

* Revert "Revert saveHead changes"

This reverts commit a15fddc2e68b2e370290b1783eb1ee626af17503.

* Fix rest of the tests

* Update beacon-chain/blockchain/head.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-29 19:32:42 +00:00
Potuz
237807b248
Do not update forkchoice checkpoints on calls to Head (#10702)
* Only update forkchoice checkpoints at epoch transition

* gazelle

* blockchain package changes

* fix node

* spectest package

* rpc package

* fix spec test

* Fix spectests

* fix new_slot.test

* gaz

* fix spectests

* fix conflicts

* Update beacon-chain/blockchain/process_block.go

* regression do not update on newSlot

* revert bd64cab

* terence's review

* fix conflicts

* fix latest conflicts

* gaz

* go mod tidy

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-22 18:37:01 +00:00
Potuz
4c869fa587
remove equivocating votes from forkchoice (#10597)
* remove equivocating votes from forkchoice

* shutup deepsource

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-02 19:19:03 +00:00
Potuz
f9113dfd06
Forkchoice use parent hash when removing invalid blocks (#10594) 2022-05-02 06:53:22 -07:00
Raul Jordan
84916672c6
Remove Eth2-Types Dependency in Prysm (#10578)
* replace eth2 types

* replace protos

* regen proto

* replace

* gaz

* deps

* amend

* regen proto

* mod

* gaz

* gaz

* ensure build

* ssz

* add dep

* no more eth2 types

* no more eth2

* remg

* all builds

* buidl

* tidy

* clean

* fmt

* val serv

* gaz

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-04-29 10:32:11 -04:00
Potuz
83a83279d4
Remove synced tips and use last valid hash (#10439)
* Remove synced tips

use last valid hash in removing invalid nodes.

* add test

* Remove unused code

* More unused parameters

* Fix proposer boost

* terence's review #1

* Fix conflicts

* terence's review 2

* rename argument

* terence's review #3

* rename optimistic -> status

* Minor clean up

* revert loop variable change

* do not mark lvh as valid

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-04-06 14:18:30 +00:00
Raul Jordan
eb6d68a4b1
Deterministic Proposer Root Boosting in Forkchoice (#10427)
* begin refactor

* fix doubly linked tree tests

* pass

* fix up

* gaz

* buidl

* buidl more

* comment

* args check

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-28 21:34:41 +00:00
Potuz
8f49167117
return invalid roots from SetOptimisticToInvalid (#10403)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-23 15:55:05 -03:00
terence tsao
c7d64c03ac
Insert payload hash to forkchoice store nodes (#10400)
* Insert payload hash to forkchoice store nodes

* Fix tests

* Update new_slot_test.go

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-03-22 01:20:42 +00:00
Potuz
3f1e3cf82f
remove optimistic control flag (#10340)
* remove optimistic control flag

* Rename

* More

* More

* redundant return

* fix conflict

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-12 17:32:04 +00:00
Potuz
b59484b285
Export SetOptimisticToInvalid (#10330)
* temp

* export remove invalid blocks
2022-03-09 13:50:57 -03:00