Commit Graph

214 Commits

Author SHA1 Message Date
Potuz
0b06c48ed0
Fix typo in comment for LoadGenesis (#8685) 2021-03-30 00:45:14 +00:00
Nishant Das
7f0c92504f
Add DB To P2P Service (#8676)
* add files

* goimports
2021-03-26 09:51:58 -05:00
Preston Van Loon
e477fdfd6d
Embedded mainnet genesis state + load genesis.ssz from file (#8614)
* Update rules_go and fix proto conflicts

* gaz

* Update generated code

* First pass inclusion of using baked states

* more emptypb fixes

* remove testnet genesis files, only embed mainnet

* Refactoring for SaveGenesisData, fix tests that use mainnet config but do not support mainnet genesis values

* a bit more refactoring, load genesis from a file. Needs tests still

* Add method to ensure an embedded genesis file also has the appropriate genesis block

* gofmt

* more clear error

* Check genesis fork version to ensure testnet config matches genesis file

* viz

* test for SaveGenesisData

* More genesis db method tests

* Merge

* Minor tweaks, lint, fmt, etc

* Add more test to genesis db methods

* Revert beacon-chain/state/stateV0/BUILD.bazel

* Update beacon-chain/db/iface/errors.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* PR feedback

* Update beacon-chain/db/kv/genesis.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* fmt.Errorf works better for nil errors

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-03-18 21:00:00 +00:00
terence tsao
ce725ceec3
Move state pkg to stateV0 pkg (#8620)
* Move state pkg to stateV0 pkg

* Build.bazel

* Remove unused RootsArrayHashTreeRoot

* Revert "Remove unused RootsArrayHashTreeRoot"

This reverts commit bf0bda30d1a8eb7a071f6e3ce9ee85041b45aca6.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-17 19:49:49 +00:00
terence tsao
7b16601399
State: Return interface{} for pb related methods (#8617)
* Return interface{} instead of *pbp2p.BeaconState

* Comment
2021-03-16 22:26:17 -05:00
Victor Farazdagi
548b471b8a
Simplify nil assignments (#8587)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-10 20:38:16 +00:00
terence tsao
bdf2b2019b
Use beacon state interfaces across (#8552) 2021-03-08 14:37:33 -08:00
Nishant Das
565d51009d
Make DB Backup More Efficient (#8543)
* checkpoint

* add test

* Update beacon-chain/db/kv/backup_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-03 15:37:13 +00:00
terence tsao
1db3c86b68
Use DeepSSZEqual to satisfy go tests (#8499)
* Use DeepSSZEqual to satisfy go tests

* New line

* New line

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-23 22:20:11 +00:00
Victor Farazdagi
a069738c20
ETH2 Types: Slot (#8408)
* update shared/params

* update eth2-types deps

* update protobufs

* update shared/*

* fix testutil/state

* update beacon-chain/state

* update beacon-chain/db

* update tests

* fix test

* update beacon-chain/core

* update beacon-chain/blockchain

* update beacon-chain/cache

* beacon-chain/forkchoice

* update beacon-chain/operations

* update beacon-chain/p2p

* update beacon-chain/rpc

* update sync/initial-sync

* update deps

* update deps

* go fmt

* update beacon-chain/sync

* update endtoend/

* bazel build //beacon-chain - works w/o issues

* update slasher code

* udpate tools/

* update validator/

* update fastssz

* fix build

* fix test building

* update tests

* update ethereumapis deps

* fix tests

* update state/stategen

* fix build

* fix test

* add FarFutureSlot

* go imports

* Radek's suggestions

* Ivan's suggestions

* type conversions

* Nishant's suggestions

* add more tests to rpc_send_request

* fix test

* clean up

* fix conflicts

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2021-02-16 07:45:34 +00:00
Raul Jordan
36b6a71af4
Configurable DB Mmap Size for Beacon Node and Validator Client (#8448)
* add flag to beacon and validator

* gaz

* fuzz

* add dep viz

* add to tools
2021-02-15 20:29:47 +00:00
terence tsao
28631e7791
Verify nil block helper (#8447) 2021-02-15 15:11:25 +00:00
Victor Farazdagi
a8e501b3cf
ETH2 Types: Epoch (#8373)
* update deps

* update deps

* update protos/*

* update deps

* reset protos

* update protos

* update shared/params/config

* update protos

* update /shared

* update shared/slotutil and shared/testutil

* update beacon-chain/core/helpers

* updates beacon-chain/state

* update beacon-chain/forkchoice

* update beacon-chain/blockchain

* update beacon-chain/cache

* update beacon-chain/core

* update beacon-chain/db

* update beacon-chain/node

* update beacon-chain/p2p

* update beacon-chain/rpc

* update beacon-chain/sync

* go mod tidy

* make sure that beacon-chain build suceeds

* go fmt

* update e2e tests

* update slasher

* remove redundant alias

* update validator

* gazelle

* fix build errors in unit tests

* go fmt

* update deps

* update fuzz/BUILD.bazel

* fix unit tests

* more unit test fixes

* fix blockchain UTs

* more unit test fixes
2021-02-09 10:05:22 +00:00
Radosław Kapka
86a9d4c6a4
Configurable testutil's BeaconState (#8407)
* Configurable testutil's BeaconState

* fix shared and fuzz tests

* return state copy

* use mainnet config values for default state

* handle error in block fuzz

* goimports

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-08 20:00:09 +00:00
Nishant Das
9f62405a81
Update Prombolt Collector (#8339)
* update

* block big buckets

* add more buckets

* patch go.mod

* clean up

* Update beacon-chain/db/kv/kv.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-01-26 15:32:46 +00:00
terence tsao
d5ec248691
Rename getter functions to be idiomatic (#8320)
* Rename getter functions

* Rename new

* Radek's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-25 21:27:30 +00:00
Radosław Kapka
7842fd9da6
Replace bytes.Equal with assert.DeepEqual in tests (#8318)
* beacon chain

* format imports

* Update beacon-chain/db/kv/migration_archived_index_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/db/kv/migration_block_slot_index_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* remove unused imports

* Update beacon-chain/core/state/skip_slot_cache_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/core/state/skip_slot_cache_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/core/state/skip_slot_cache_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/core/state/skip_slot_cache_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/db/kv/migration_archived_index_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Update beacon-chain/db/kv/migration_block_slot_index_test.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-01-22 15:15:40 +00:00
Radosław Kapka
f89fd67952
Better receiver names in beacon chain module (#8286) 2021-01-19 13:21:32 +00:00
pinglamb
d7d2c6354b
Blocks filtering should return genesis when startSlot=0 and endSlot=0 (#8270)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-01-15 18:36:17 +00:00
terence tsao
daf6da5beb
Add hydrate indexed att test helper (#8261)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-13 22:05:57 +00:00
pinglamb
9f423617cb
Added db.BlocksBySlot and db.BlockRootsBySlot (#8184)
* Added blockBySlot and blockRootBySlot

* Changed to BlocksBySlot and BlockRootsBySlot

* Updated to use BlocksBySlot and BlockRootsBySlot

* Added missing passthrough to karfa exporter

* Return hasBlocks/hasBlockRoots in the new getters

* Fixed CI lint

* Replace call to bytes.Compare with bytes.Equal

* Reordered the returns of the new getters

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-12 18:31:15 +00:00
Radosław Kapka
5fd03f8fb0
Unify the pattern of using a package-level logger (#8245)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-01-11 20:03:28 +00:00
terence tsao
9a1866b735
Hydrate header test helper (#8234)
* Hydrate headers and fix tests

* Gazelle

* Fix a slashing test

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-01-09 16:45:45 +00:00
Victor Farazdagi
1abe92fd8b
Remove redundant parentheses around nil values (#8199)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-05 13:29:46 +00:00
terence tsao
20dede7532
Move state summary cache to DB (#8101) 2020-12-16 08:56:21 -08:00
terence tsao
dc27cd7a1e
Pass context to NewKVStore (#8125)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-15 22:07:01 +00:00
terence tsao
70c0bb106b
Beacon node code health improvements (#8109)
* Apply code health fixes after code inspect

* Remove attestationRoot

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-14 23:41:24 +00:00
Radosław Kapka
9d7052796b
Restore database CLI command (#8061)
* restore beacon node db

* revert image name

* move restore out of the kv folder

* remove files from kv folder

* go mod tidy

* Remove usage of prometheus testutil

* add yes/no to prompt text

* restore slasher db

* organize imports

* go mod tidy

* restore validator db

* close slasher db

* defer close backup db in tests

* simplify function literal
2020-12-07 21:36:43 +01:00
Raul Jordan
14e1f08208
Add Backup Webhooks to All Prysm Services With DBs (#8025)
* integrate backup webhooks

* pass slasher tests

* fix node

* cmd

* gaz

* read test passes

* radek feedback

* added comment

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-12-03 22:28:57 +00:00
Preston Van Loon
0c9389a438
Fix instances of "The result of append is not used anywhere SCC-SA4010" (#7812) 2020-11-13 22:54:12 +00:00
Victor Farazdagi
2f11e55869
Use t.TempDir() in tests (#7769)
* use t.TempDir()

* remove redundant delete

* simplify setupDB()

* simplify db/testing/setup_db

* fix tests
2020-11-10 22:45:17 +00:00
terence tsao
be40e1a3b9
Update delete state(s) functions (#7754) 2020-11-09 15:37:36 -08:00
Raul Jordan
d4c954648c
Prevent Usage of Stdlib File/Dir Writing With Static Analysis (#7685)
* write file and mkdirall analyzers

* include analyzer in build bazel

* comments to the single entrypoint and fix validator references

* enforce 600 for files, 700 for dirs

* pass validator tests

* add to nogo

* remove references

* beaconfuzz

* docker img

* fix up kv issue

* mkdir if not exists

* radek comments

* final comments

* Try to fix file problem

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-11-09 14:27:03 -06:00
Nishant Das
8cac198692
Keep Non Finalized States (#7742)
* keep non finalized states

* Update beacon-chain/db/kv/state.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-07 18:18:29 +00:00
terence tsao
4dcae8707a
Startup routine to clean up dirty states in DB (#7729)
* Clean up dirty states in DB

* Add Passthrough

* Fix finalized check

* Add tests

* Remove unused test vars

* Update beacon-chain/db/kv/state.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Use non-strict inequality

* Prevent overflow and test

* Use slotsPerArchivedPoint/3 to loosen criteria

* Update test

* Background

* Handle ctx cancellation

* Update beacon-chain/state/stategen/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/db/kv/state.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Handle error

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-06 21:05:44 +00:00
terence tsao
386bfdd6eb
Remove deprecated headstate method in DB (#7680)
* Remove head state in DB

* Clean up tests

* Fix pow tests

* Add stateGen to pow service
2020-10-30 15:11:08 +00:00
Preston Van Loon
5f80754013
Revert "Remove Top Jaeger Span Offenders" (#7657)
This reverts commit ec8eab21ae.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-28 06:07:14 +00:00
Raul Jordan
ec8eab21ae
Remove Top Jaeger Span Offenders (#7655) 2020-10-28 05:14:54 +00:00
Preston Van Loon
424488bf8a
Deprecate db.HeadState (#7653)
* Deprecate db.HeadState

* Deprecate db.HeadState
2020-10-27 16:25:52 +00:00
Raul Jordan
a468a12ef0
Allow Specifying Output Directory When Creating DB Backups (#7630)
* move flags

* backup db output dir flag

* fix build

* fix up broken backup test

* Radek's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-26 18:20:54 +00:00
Nishant Das
6eb022ffa1
Clean Up Block Retrieval Methods (#7593)
* fix and add tests

* return for 0

* add test case

* fix

* fix fail

* add comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-23 04:23:15 +00:00
Raul Jordan
d2f7240255
Context Cancelation Handling for Hot States (#7596)
* attempt deadline for state info

* cancel check
2020-10-21 17:06:48 +00:00
Nishant Das
b502876e98
Fix Mismatched BlockRoots In State Replay (#7559)
* fix mismatched roots

* alternate approach

* preston's review

* comment
2020-10-18 09:39:27 -07:00
Victor Farazdagi
a005c77b3f
Update error usage patterns to go1.13+ (#7529)
* rely on errors.Is

* wrap errors
2020-10-14 17:39:52 +00:00
Victor Farazdagi
a019a0db4c
Combines func params of the same type (#7500)
* combines func params

* update leftovers

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-12 15:43:19 +00:00
Preston Van Loon
7cc32c4dda
Various code inspection resolutions (#7438)
* remove unused code

* remove defer use in loop

* Remove unused methods and constants

* gofmt and gaz

* nilness check

* remove unused args

* Add TODO for refactoring subscribeWithBase to remove unused arg. It seems too involved to include in this sweeping PR. https://github.com/prysmaticlabs/prysm/issues/7437

* replace empty slice declaration

* Remove unnecessary type conversions

* remove redundant type declaration

* rename receivers to be consistent

* Remove bootnode query tool. It is now obsolete by discv5

* Remove relay node. It is no longer used or supported

* Revert "Remove relay node. It is no longer used or supported"

This reverts commit 4bd7717334dad85ef4766ed9bc4da711fb5fa810.

* Delete unused test directory

* Delete unsupported gcp startup script

* Delete old k8s script

* build fixes

* fix build

* go mod tidy

* revert slasher/db/kv/block_header.go

* fix build

* remove redundant nil check

* combine func args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-12 08:11:05 +00:00
Victor Farazdagi
d98a6dda8f
Verify interface compliance using typed nil instead of dereferencing and conversion (#7489)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-10 00:36:48 +00:00
Preston Van Loon
d23b247a82
DB: Return error when saving nil eth1data (#7396)
* Return an error on saving nil eth1data
* gofmt
* Merge branch 'master' into return-error-on-nil
* Merge refs/heads/master into return-error-on-nil
* Merge refs/heads/master into return-error-on-nil
2020-09-30 23:23:22 +00:00
Nishant Das
3b5ef50733
Fix Finalized Epoch Checks (#7364)
* fix up epoch checks

* build

* comp tests

* preston's comment
2020-09-30 08:25:25 +08:00
Radosław Kapka
dca93ce641
Unnecessary Slice-to-Slice Conversion analyzer (#7321)
* analyzer with tests
* fix bazel file
* modify analyzer to fix build issues
* add analyzer to tool chain
* remove arrays from inspections
* fix redundant [:] operator
* Merge branch 'master' into use-slice-directly
* Merge branch 'master' into use-slice-directly
* fix another inspection
* add package-level comment
2020-09-23 16:14:34 +00:00