Commit Graph

538 Commits

Author SHA1 Message Date
Preston Van Loon
4b033f4cc7
Update go to 1.19.3 (#11630)
* Update go to 1.19.3

* Update other items to 1.19

* Update golangci-lint to latest release

* Run gofmt -s with go1.19

* Huge gofmt changes

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-11-18 19:12:19 +00:00
terencechain
19f6d3bef6
Capella: Add DB changes (#11624)
* Add Capella DB changes

* Add tests
2022-11-07 15:26:27 +00:00
kasey
007c776d8a
tool to search db for a key prefix (#11417)
* tool to query db by a key prefix

* cleanup

* lint and fmt

* db/kv public visibility

We've discussed before that Bazel visibility constraints don't
accomplish much in go, so I'm phasing them out in places where they get
in the way.

* derp

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-10-26 21:28:02 +00:00
int88
57d7207554
fix typo and log error (#11565)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-10-23 22:01:44 +00:00
Radosław Kapka
b7a878d011
Resolve remaining native state tasks (#11561)
* remove ToProto and ToProtoUnsafe wrappers

* TestAppendBeyondIndicesLimit

* change type of genesisValidatorsRoot

* fuzz tests

* check type assertion
2022-10-19 10:37:45 -04:00
Radosław Kapka
cafe0bd1f8
Capella beacon state (#11141)
* fork

* types

* cloners

* getters

* remove CapellaBlind from fork

* hasher

* setters

* spec params, config tests

* generate ssz

* executionPayloadHeaderCapella

* proto state

* BeaconStateCapella SSZ

* saving state

* configfork

* BUILD files

* fix RealPosition

* fix hasher

* SetLatestExecutionPayloadHeaderCapella

* fix error message

* reduce complexity of saveStatesEfficientInternal

* add latestExecutionPayloadHeaderCapella to minimal state

* halway done interface

* remove withdrawal methods

* merge setters

* change signatures for v1 and v2

* fixing errors pt. 1

* paylod_test fixes

* fix everything

* remove unused func

* fix tests

* state_trie_test improvements

* in progress...

* hasher test

* fix configs

* simplify hashing

* Revert "fix configs"

This reverts commit bcae2825fcc8ba45a2b43d68ad0ab57f8eac8952.

* remove capella from config test

* unify locking

* review

* hashing

* fixes

Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-10-12 11:39:19 -05:00
Radosław Kapka
73443208a1
Remove proto state (#11445)
* Remove native state flag and use native state in spectests

* remove feature from tests

* use e2e config in slasher simulator

* use params.BeaconConfig in testutil

* use correct function

* use minimal config in go_test

* fix TestListValidators

* parameterize sync committee bits and aggregation bits

* Fix TestServer_ListIndexedAttestations_GenesisEpoch

(cherry picked from commit 254ab623dde08ae8886b152facdbbd8889ed79db)

* fix more tests

* fix even more

* moreeee

* aaaand more

* one more fix

* one more

* simplify TestGetAltairDuties_UnknownPubkey

* comment out problematic test

* one more fix

* one more

* aaaand one more

* another

* use fieldparams in HydrateBlindedBeaconBlockBodyBellatrix

* create new package for mainnet tests

* TestServer_GetBellatrixBeaconBlock

* change slashed validator index

* clear cache in reward_test.go

* deprecate flag

* create bazel mainnet target

* move attester mainnet test to mainnet target

* "fix" proposer tests

* use minimal config in TestServer_circuitBreakBuilder

* fix TestProposer_ProposeBlock_OK

* more fixes in validator package

* more fixes

* more fixes

* test code

* move TestProposer_GetBeaconBlock_BellatrixEpoch to minimal

* finally

* remove proposer_bellatrix_mainnet_test.go

* fix TestServer_GetBellatrixBeaconBlock_HappyCase

* fix TestServer_GetBellatrixBeaconBlock_BuilderCase

* Preston needs to fix this!

* Revert "Preston needs to fix this!"

This reverts commit b03d97a16e3080e254c7b19d7f193d3c600ca869.

* remove proto state tests

* fix migration tests

* static analysis fix

* review

* remove proto state

* swap state in tests

* fix BUILD file in /proto/testing

* remove metrics test with nil state
2022-09-16 18:17:46 -04:00
Radosław Kapka
0f0ab1327e
Deprecate native state flag (#11268) 2022-09-15 20:47:51 +02:00
terencechain
d860daff75
clean up: godoc comments, redundant castings and more (#11428)
* clean up: Godoc comments, redundant castings and more

* Fix assertion check

* Update beacon-chain/forkchoice/doubly-linked-tree/forkchoice.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/forkchoice/protoarray/store.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-09-12 14:03:20 +00:00
Radosław Kapka
4d90afe944
Fixed size arrays in block interfaces and structs (#11375)
* Fixed size arrays in block fields

* test fix

* fmt

* fix fetcher test

* fix fuzz tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-09-06 14:30:16 +00:00
Raul Jordan
bcaae1c440
Performance Metrics for Prysm (#11377)
* atts performance and blocks

* idiomatic observe

* all attestation related errors

* block metrics

* db metrics

* metrics func

* rem old metrics

* naming

* rem metric

* rem unneeded

* fix

* fix up

* rev

* fix

* rem
2022-09-01 01:26:19 +00:00
Nishant Das
a9ccabf6c9
Remove Bolt Mmap Flag (#11236)
* remove mmap flag

* fix mmap

* fix all build

* gaz

* config

* fix
2022-08-17 12:22:41 +00:00
Raul Jordan
d077483577
Add V3 Suffix to All Prysm Packages (#11083)
* v3 import renamings

* tidy

* fmt

* rev

* Update beacon-chain/core/epoch/precompute/reward_penalty_test.go

* Update beacon-chain/core/helpers/validators_test.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/iface/BUILD.bazel

* Update beacon-chain/db/kv/kv.go

* Update beacon-chain/db/kv/state.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

* Update beacon-chain/sync/initial-sync/service.go

* fix deps

* fix bad replacements

* fix bad replacements

* change back

* gohashtree version

* fix deps

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-08-16 12:20:13 +00:00
Radosław Kapka
5c580d4a1c
Replace version.BellatrixBlind with b.isBlinded (#11159)
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-08-11 00:00:10 +02:00
terencechain
a82325615b
Can recover from missing state summary in DB (#11167)
* Can recover from missing state summary in DB

* Tests

* fix tests

Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-08-04 13:01:07 +00:00
Potuz
2349899658
Wrap missing state for checkpoint error (#11162)
* Wrap missing state for checkpoint error

* terence's review
2022-08-03 16:49:11 +00:00
Radosław Kapka
879e310332
Native Blocks Ep. 2 - Switch usages to new package (#10885)
* panic in SizeSSZ

* moving slowly

* adapt old code to new interfaces

* return interfaces from factory functions

* replace the rest of WrappedSignedBeaconBlock

* WrappedBeaconBlock

* WrappedBeaconBlockBody

* miscellaneous

* Test_BeaconBlockIsNil

* replace usages of BeaconBlockIsNil

* replace usages of mutator

* fix all build errors

* fix some more issues

* mutator changes

* relax assertions when initializing

* revert changes in object_mapping.go

* allow calling Proto on nil

* Revert "allow calling Proto on nil"

This reverts commit ecc84e455381b03d24aec2fa0fa17bddbec71705.

* modify Copy and Proto methods

* remove unused var

* fix block batch tests

* correct BUILD file

* Error when initializing nil objects

* one more error fix

* add missing comma

* rename alias to blocktest

* add logging

* error when SignedBeaconBlock is nil

* fix last test

* import fix

* broken

* working

* test fixes

* reduce complexity of processPendingBlocks

* simplified
2022-08-02 15:30:46 +00:00
terencechain
699bfdfdb4
Rename pow to execution (#11135)
* Rename pow to execution

* Fix complain

* Fix complain
2022-08-01 14:43:47 +00:00
Preston Van Loon
5722a5793c
Tracing: properly overwrite context so that spans can be correctly attributed (#11012)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-07-14 17:00:33 +00:00
Raul Jordan
10fffa6e7c
[Feature] - Store Only Blinded Beacon Blocks Post-Merge (#11010)
* add in flags

* add sync and db items

* bring over all other changes

* enable on

* use feature flag

* powchain

* builds

* fix up tests

* pass iface

* gaz

* enable bellatrix blind in unmarshal only behind flag

* poolside

* pass rpc tests

* rebuilds

* naming

* cleaner func

* check needs resync

* idiomatic

* gaz

* rem

* build

* nicer

* build

* cleaner

* surface error

* wrapping

* unmarshal logs

* fix up

* cleaner

* log

* builds

* Update beacon-chain/blockchain/execution_engine.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* terence feedback

* test added for resync

* nil check

* fmt

Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-07-13 17:18:30 +00:00
terencechain
5d06c14cec
Check validator has registration before getting header (#11023)
* Check validator has registration before calling header

* Check validator has registration before calling header

* Update proposer_bellatrix_test.go
2022-07-11 20:57:58 -05:00
Radosław Kapka
b82e2e7d40
Use prysmaticlabs/fastssz as a direct dependency (#10941)
* Update dependency

* Regenerate SSZ files

* fix BUILD files

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-28 13:03:24 +00:00
Radosław Kapka
7f56ac6355
Massive code cleanup (#10913)
* Massive code cleanup

* fix test issues

* remove GetGenesis mock expectations

* unused receiver

* rename unused params

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-27 13:34:38 +00:00
Radosław Kapka
2b6e86ec1b
Some test improvements (#10928)
* extract DeterministicGenesisStateWithGenesisBlock

(cherry picked from commit a5e3a9c9bbbacb23a644f5c68c92839a315f66a1)

# Conflicts:
#	testing/util/state.go

* part 1

* part 2

* part 3

* fix errors

* db interface public visibility

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-24 17:22:39 +00:00
Raul Jordan
e81f3fed01
Remove Extraneous BoltDB Logs (#10888) 2022-06-16 01:11:07 +00:00
Jie Hou
6c878b1665
Refactor: Continue reducing cognitive complexity (#10862)
* Refactor beacon-chain/db/kv/state.go

* Refactor api/gateway/apimiddleware/process_field.go

* Refactor beacon-chain/sync/initial-sync/blocks_queue.go

* Refactor validator/db/kv/migration_optimal_attester_protection.go

* goimports

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-06-15 18:34:59 +00:00
kasey
1fa864cb1a
use slot:block index correctly (#10820)
* adding splitRoots, refactor to use it

* use splitRoots & work in roots only

the most common use case for this method is to get a list of
candidate roots and check if they are canonical. there isn't a great
reason to look up all the non-canonical blocks, because forkchoice
checks based on the root only, so just return roots and defer the
responsibility of resolving those to full blocks.

* update comment

* clean up shadowing

* more clear non-error return

* add test case for single root in index slot

* fmt

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-07 16:47:42 +00:00
terencechain
5b12f5a27d
Integrate builder client into builder service (#10825)
* Integrate builder client into builder service

* Do nothing if pubkey is not found

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-06 22:38:49 +00:00
kasey
a170fd4bd6
Revert "Change name and return type of HighestSlotBlocksBelow (#10811)" (#10818)
This reverts commit 0e9dfe04a1.

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-06-03 18:36:06 +00:00
terencechain
3e36eacd1e
Add registration DB methods (#10812) 2022-06-03 15:13:04 +00:00
kasey
0e9dfe04a1
Change name and return type of HighestSlotBlocksBelow (#10811)
* HighestSlotBlocksBelow to return a single value

* HighestSlotBlocksBelow->HighestBlockBelowSlot

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-03 07:16:34 +00:00
kasey
c72f1af951
rewind cursor upon nil value (bucket) (#10802)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-06-02 04:06:19 +00:00
kasey
7563bc0444
HighestSlotBlocksBelow, but in reverse (#10772)
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-31 21:10:33 +00:00
Preston Van Loon
6910460173
Refactor migrateStateValidators for better readability (#10727)
* Refactor migrateStateValidators for better readability

* pass test

* rev

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-05-23 23:56:02 +00:00
Raul Jordan
244e670b71
Move BeaconBlockNil Checker Function to Consensus-Types/Wrapper Package (#10731)
* beacon block is nil wrapper

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-20 23:29:16 +00:00
Raul Jordan
dc5fb92b28
Remove Unnecessary State Interfaces (#10707)
* rem other state interfaces

* redundant check

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-20 22:40:03 +00:00
kasey
588dea83b7
Config registry (#10683)
* test coverage and updates to config twiddlers

* LoadChainConfigFile error if SetActive conflicts

* lint

* wip working around test issues

* more fixes, mass test updates

* lint

* linting

* thanks deepsource!

* fix undeclared vars

* fixing more undefined

* fix a bug, make a bug, repeat

* gaz

* use stock mainnet in case fork schedule matters

* remove unused KnownConfigs

* post-merge cleanup

* eliminating OverrideBeaconConfig outside tests

* more cleanup of OverrideBeaconConfig outside tests

* config for interop w/ genesis gen support

* improve var name

* API on package instead of exported value

* cleanup remainders of "registry" naming

* Nishant feedback

* add ropstein to configset

* lint

* lint #2

* ✂️

* revert accidental commented line

* check if active is nil (replace called on empty)

* Nishant feedback

* replace OverrideBeaconConfig call

* update interop instructions w/ new flag

* don't let interop replace config set via cli flags

Co-authored-by: kasey <kasey@users.noreply.github.com>
2022-05-20 07:16:53 +00:00
terencechain
092e9e1d19
Clean up various warnings (#10710)
* Clean up various warnings

* Update beacon-chain/rpc/prysm/v1alpha1/debug/state_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Fix redundant casting genState

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-05-19 04:38:04 +00:00
Radosław Kapka
21bdbd548a
Deduplicate native state (a.k.a. One State to rule them all) (#10483)
* v0

* getters/setters

* init and copy

* hasher

* all the nice stuff

* make bazel happy

* remove tests for smaller PR

* remove old states

* move files

* import fixes

* custom MarshalSSZ

* fixed deadlock

* copy version when copying state

* correct issues in state_trie

* fix Copy()

* better e2e comment

* add code to minimal state

* spectest test

* Revert "Auxiliary commit to revert individual files from 84154423464e8372f7e0a03367403656ac5cd78e"

This reverts commit 9602599d183081291dfa0ba4f1036430f63a7822.

* native state assert

* always error

* always log

* more native state usage

* cleanup

* remove empty line

* Revert "spectests"

This reverts commit 1c49bed5d1cf6224afaf21e18562bf72fae5d2b6.

# Conflicts:
#	beacon-chain/powchain/service.go
#	beacon-chain/state/v1/state_trie.go
#	beacon-chain/state/v2/state_trie.go
#	beacon-chain/state/v3/state_trie.go
#	testing/spectest/shared/phase0/finality/BUILD.bazel
#	testing/spectest/shared/phase0/finality/runner.go

* dedup field trie

* fix test issues

* cleanup

* use correct field num in FinalizedRootProof

* use existing version constant

* halfway there

* "working" version

* some fixes

* fix field nums in tests

* rename v0types to nativetypes

* Revert "Auxiliary commit to revert individual files from dc549b1cf8e724bd08cee1ecc760ff3771d5592d"

This reverts commit 7254d3070d8693b283fc686a2e01a822ecbac1b3.

* uncomment code

* remove map size

* Revert "Revert "spectests""

This reverts commit 39c271ae6b57d6a3737e2c202cd8407857475e56.

* use reverse map

* Revert "Revert "Revert "spectests"""

This reverts commit 19ba8cf95cdca689357c8234a262e08cccbafef4.

* finally found the bug

(cherry picked from commit a5414c4be1bdb61a50b391ea5301895e772cc5e9)

* simplify populateFieldIndexes

* fix copy

(cherry picked from commit 7da4fb8cf51557ef931bb781872ea52fc6731af5)

* remove native state from e2e

* remove index map

* unsupported functions

* Use ProtobufBeaconState() from native state

* tests

* typo

* reduce complexity of `SaveStatesEfficient`

* remove unused receiver name

* update doc.go

* fix test assertion

* fix test assertion 2

* Phase0 justification bits

* bring back state tests

* rename fieldIndexRev

* versioning of ToProto

* remove version check from unexported function

* hasher tests

* don't return error from JustificationBits

* extract fieldConvertersNative

* helper error function

* use fieldConvertersNative

* Introduce RealPosition method on FieldIndex

* use RealPosition in hasher

* remove unused fields

* remove TestAppendBeyondIndicesLimit

(cherry picked from commit 3017e700282969c30006b64c95c21ffe6b166f8b)

* simplify RealPosition

* rename field interface

* use helper in proofs.go

* Update beacon-chain/core/altair/upgrade.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-05-09 13:02:34 +00:00
kasey
b1b13cfca7
simplify config names, use strings (#10656)
* simplify config names, use strings

* lint

Co-authored-by: kasey <kasey@users.noreply.github.com>
2022-05-06 21:42:27 +00:00
kasey
10e1f04ce4
Checkpoint sync fixes (#10630)
* fixes for checkpoint sync

* improve node version regex; real client test cases

* add test case for checkpoint sync finalization fix

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-05-04 23:41:33 +00:00
terencechain
d4fa490dec
Handle blind block for DB (#10580)
* Handle blind block for DB

* Update blinded_beacon_block_bellatrix_test.go

* Update blocks_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-03 16:55:59 +00:00
Raul Jordan
20ab988a4a
Rename Block Package in Consensus-Types to Interfaces (#10605)
* interfaces package

* builds

* gaz
2022-05-02 14:32:37 -04:00
Raul Jordan
16bbf5602f
Move Consensus Type Wrappers Into Consensus Types Package (#10598)
* builds

* move block to consensus-types

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-02 15:43:40 +00:00
Raul Jordan
84916672c6
Remove Eth2-Types Dependency in Prysm (#10578)
* replace eth2 types

* replace protos

* regen proto

* replace

* gaz

* deps

* amend

* regen proto

* mod

* gaz

* gaz

* ensure build

* ssz

* add dep

* no more eth2 types

* no more eth2

* remg

* all builds

* buidl

* tidy

* clean

* fmt

* val serv

* gaz

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-04-29 10:32:11 -04:00
Preston Van Loon
80ebbcf03e
gocognit: Lower complexity threshold to 100, fix a few complexity issues (#10542)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-19 07:12:24 +00:00
Håvard Anda Estensen
d2f4a8cc7c
Replace ioutil with io and os (#10541)
* Replace ioutil with io and os

* Fix build errors
2022-04-18 20:42:07 +00:00
terence tsao
3833f78803
Remove invalid nodes (#10399) 2022-04-06 14:24:00 -07:00
kasey
de0143e036
save origin block root before finalize (#10463)
* save origin block root before finalize

* add test for SaveOrigin

* goimports :(

* signature to LoadGenesis changed in a diff PR

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-04-05 17:13:53 +00:00
kasey
0df8d7f0c0
refactor genesis state flag handling, support url (#10449)
* refactor genesis state flag handling, support url

* lint fix

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-03-30 22:23:34 +00:00