Commit Graph

6169 Commits

Author SHA1 Message Date
terence tsao
41ab27402f
Sync committee subnet cache (#9350)
* Add sync subnet id cache

* Update BUILD.bazel

* Update BUILD.bazel

* Update beacon-chain/cache/sync_subnet_ids.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Add comments

* add more comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2021-08-11 00:26:20 +00:00
terence tsao
9668ab0fea
Clean up duplicated imports (#9354) 2021-08-10 14:55:24 -05:00
Radosław Kapka
7a9c717ae0
Reduce locking time in recomputeFieldTrie (#9353)
* Reduce locking time in `recomputeFieldTrie`

* add comment

* reuse variable and improve comment
2021-08-10 09:54:35 +00:00
Nishant Das
a5998fab2f
Clean Up State Representation (#9339)
* checkpoint all work

* add new fixes

* fixes

* gaz

* terence's review

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-09 21:54:24 +00:00
terence tsao
8df6854a74
Core: upgrade state to Altair (#9349)
* Add upgrade to altair

* Update BUILD.bazel

* Update beacon-chain/core/altair/upgrade.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Move getter and setter outside the loop

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-09 21:27:51 +00:00
Jim McDonald
3de168b806
Fix state state transition logging. (#9343)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-09 20:05:46 +00:00
Preston Van Loon
bfdaf8a3f5
Fix config loader tests (#9348)
* don't mutate mainnet config

* add missing minimal config params

* Ensure all yaml fields were set and are correct

* gofmt

* Add sanity check that some empty yaml wasn't given

* Gazelle and deepsource feedback
2021-08-09 18:40:06 +00:00
terence tsao
38787353a2
Attestation: get participation status (#9340)
* Add participation flag indics matching helpers

* Gazelle

* Tests

* Update BUILD.bazel

* Update attestation.go
2021-08-09 16:35:46 +00:00
Nishant Das
170e2e115b
Fix Flaky Sync Committee Test (#9337)
* fix flake

* Update beacon-chain/core/altair/sync_committee_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-09 14:49:13 +00:00
terence tsao
be85b7f909
Cleanup: remove test only process block/attestation (#9345)
* Remove test only process block attestation code

* Rm fuzz process block

* Revert "Rm fuzz process block"

This reverts commit 5d1b0f0741eb1dd7489aa6998bdf8987a26e20e0.

* Still can test fuzz

* Fix return

* More unused funcs
2021-08-08 22:01:06 -05:00
terence tsao
ad048a8750
Rewards/penalties: epoch processing for Altair (#9334)
* Add epoch precompute methods

* Better name

* Bring expensive computation outside

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-06 20:02:35 +00:00
Raul Jordan
90ea158301
Add Some More Traces to Blockchain Service (#9336)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-06 15:02:36 +00:00
Radosław Kapka
ef9eff29b8
Tiny state management improvements (#9338)
* Tiny state management improvements

* fix indices length calculation

* review comments

* revert function rename
2021-08-06 10:47:50 +00:00
terence tsao
4cce4f1ee5
Refactor slash validator func's signature (#9321)
* Refactor slash validator func's signature

* Named arguments

* Update beacon-chain/core/blocks/proposer_slashing.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-06 04:06:49 +00:00
Raul Jordan
ce10be9a16
Add Latest Slasher KV Methods to DB Subpackage (#9328)
* tests pass

* use end slot for epoch pruning

* amended to endslot of epoch

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-08-05 21:27:05 +00:00
Raul Jordan
4f0b00894b
More Precise Metrics E2E (#9331)
* metrics test copute slots since genesis

* gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 20:44:56 +00:00
Radosław Kapka
57fac9b74f
Improve API Middleware design (#9324)
* remove superfluous WriteHeader call

* move custom handlers outside hooks

* rename OnPostStart hook

* function that returns a default endpoint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 20:21:43 +00:00
Mohamed Zahoor
395feeb35d
Cross check db state bucket after migration (#9311)
* added state db check after migration

* satisfy deepsource

* added db compare after migration sub-command

* fomt issue

* point ot bolt issue

* gazel fix

* maxUint84

* revert accidental deletion of WORKSPACE file

* added sorting support

* cleanup debug

* gazel fix

* satisfy deepsource

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 18:30:05 +00:00
terence tsao
7bc22e4ddf
Cache: sync committee index position (#9317)
* Add sync committee index cache

* Update BUILD.bazel

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 17:44:13 +00:00
Radosław Kapka
016beb51c0
Make SubmitAttestations v1 implementation similar to v1alpha1 (#9319)
* Make `SubmitAttestations` v1 implementation similar to v1alpha1

* fix test

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 08:01:20 +00:00
Radosław Kapka
456b4d463e
Implement PrepareBeaconCommitteeSubnet in the validator API (#9318)
* validator migration

* status helpers

* implementation

* fix existing tests

* build file

* tests

* run endpoint through API Middleware

* reduce helper package visibility

* empty caches before each test

* assign all validators to subnets

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-05 07:23:52 +00:00
terence tsao
4d33068496
Core helper: validate sync message time (#9325)
* Add validate sync message time

* Update BUILD.bazel
2021-08-05 12:39:32 +08:00
terence tsao
6e9faa3618
Epoch: precompute functions for Altair part 1 (#9309)
* Add epoch precompute methods

* Split into part 1 for better review

* Update epoch_precompute_test.go

* Preston's feedback

* Skip if the valiator is not eligible

* Add regression test for non eligible

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-04 15:13:05 +00:00
Nishant Das
fd42cb712f
Fix Incorrect Conditional Check (#9323)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-04 13:50:27 +00:00
Radosław Kapka
c5b68b4f03
Implement PublishAggregateAndProofs in the validator API (#9294)
* grpc implementation

* build file

* full implementation

* remove unused function

* tests

* wrap top-level array

* Update beacon-chain/rpc/eth/v1/validator/validator.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-04 09:01:38 +00:00
Mohamed Zahoor
239d904d27
Fix memory usability (#9310)
* fix cache usability

* fix the name of the prometheus flag

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-04 02:33:28 +00:00
terence tsao
461be99fac
Block: process sync aggregate (#9308)
* Add altair base reward funcs

* Update BUILD.bazel

* Add process sync aggregate

* Update BUILD.bazel

* Update block.go

* Update block_test.go

* Feedback, refactor and more tests

* Further refactor sync committee votes

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-03 16:14:15 +00:00
Preston Van Loon
9572add171
Fail fast when using old define=ssz=<value> (#9316)
* Fail fast when using old define=ssz=<value>

* remove print statement
2021-08-02 19:04:30 +00:00
ahadda5
17cf2e5377
Fixing .bat file issues (#9266)
* replacing [] with "" for ifs
using !! vs %% for var with delayed expansion
Check first for 404 then download.

* remove the goto after if are fixed

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-02 16:09:09 +00:00
Radosław Kapka
e4388c7fce
Update grpc-gateway dependency to proper version (#9315)
* logging

* properly update grpc-gateway dependency

* Revert "logging"

This reverts commit 6c59a87ac0c313d8136281ee78aab4cb9d2ade90.

# Conflicts:
#	go.mod

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-02 14:52:54 +00:00
Mohamed Mansour
47c28e6cd6
Show error in logs if passing invalid flags in yaml (#9033)
* Show error in logs if passing invalid flags in yaml

The YAML configs pass if an invalid flag is set, this causes frustration
because the flag could have been mispelled causing a huge mystery.

This will strictly check if the flags are deserialized, if a flag
doesn't exist, it will error out as a Fatal error with appropriate
reasoning which flag is wrong.

* Fix review comments to make some of them non-fatal

* Make yaml.TypeError a pointer

* Remove UnmarshalStrict from spectests since they use a different YAML package

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-01 03:26:24 +00:00
Preston Van Loon
0c1fc65604
Core/deposit: refactor blocks.ProcessDeposit for less complex altair implementation (#9305)
* Refactor blocks.ProcessDeposit to return whether or not the deposit resulted in a new validator entry.

* Only mark validator as new after successfully appending it to the state

* fix merge conflict

* fix merge conflict

* gofmt

* fix merge conflict

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-31 23:42:07 +00:00
terence tsao
5c36d59672
Reward: altair base reward (#9307)
* Add altair base reward funcs

* Update BUILD.bazel

* Better tests

* Update BUILD.bazel
2021-07-30 19:17:20 +00:00
Nishant Das
7576442ef0
Only Create Deposit Proofs Pre-Genesis (#9289)
* remove proof gen

* fix pre-genesis case

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-30 18:33:31 +00:00
terence tsao
d130bf0035
Fix more duplicated import prefix (#9306) 2021-07-30 18:05:38 +00:00
Mohamed Zahoor
0d818a5709
Reduce memory overhead during DB migrations (#9298)
* wip

* migration happening

* wip

* fix memory utilization, add the testcases

* migration issues

* the gazel

* remove debug logging

* goimport

* gazel
2021-07-30 08:38:54 -05:00
terence tsao
560fe69425
Core: sync committee helpers (#9269)
* Add sync committee helpers

* Create BUILD.bazel

* Update BUILD.bazel

* Update BUILD.bazel

* Fix visibility

* Preston's review
2021-07-29 23:19:32 +00:00
terence tsao
012d279663
Fix duplicated imports (#9304)
* Fix duplicated imports

* Fix metrics test
2021-07-29 16:45:17 -05:00
terence tsao
fcd4938ec0
Head: validator index getters (#9303) 2021-07-29 20:05:48 +00:00
Radosław Kapka
4db0dfc4f1
Properly broadcast attestations (#9301)
* better handling of gateway response

* empty response tests

* build file

* broadcast attestations properly

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-29 18:33:18 +02:00
terence tsao
cf4ff3d6f9
Clean up phase 0 spec tests (#9299) 2021-07-29 09:01:18 -05:00
Raul Jordan
9145310647
Eliminate Proto V2 Namespace (#9297)
* get rid of v2 in prysm codebase

* replace block2

* builds

* terence comments

* gazelle
2021-07-28 21:23:44 +00:00
Mohamed Zahoor
21c79f4dfa
Protect the state objects in memory when saving them (#9291)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-28 18:27:27 +00:00
Raul Jordan
98f7f82b2f
Copy Altair Changes Into v1alpha1 Namespace (#9296)
* move the v2 changes into v1alpha1

* v1alpha1 builds

* regenerate pbs

* regenerate ssz

* top level builds
2021-07-28 18:05:07 +00:00
terence tsao
f16b9859e7
Test: add altair test utility (#9287)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-28 17:15:23 +00:00
Nishant Das
a372e30c8f
Fix Spaced Arguments for Subcommands (#9292)
* fix args for commands

* fix nasty edge case
2021-07-28 14:36:48 +00:00
Nishant Das
415973fca3
remove flag (#9290) 2021-07-28 16:11:22 +08:00
Raul Jordan
2d9915d2d9
Define Unimplemented V2 Services (#9286)
* generate ssz bindings for protos

* define services and unimplemented

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-27 20:42:14 +00:00
Mohamed Zahoor
c6bd7a2f83
Efficient Historical State Representation (#9155)
* added migration in to new schema

* gazel fix

* goimports fix

* construct state with validator entries from a seperate bucket

* save state with validator entry indirection

* fixed save and retreieve issues

* fixed more test cases related to DeepEqual

* added save benchmark

* gazel fix

* organize benchmarks

* add cache and improve state construction  performance

* gazel fix

* check type

* remove cache checking from Has

* fix decoding when cache is off

* fix slice bugs

* add migration testcases

* linter fix

* fix few review comments

* fix review feedback

* gazel fix

* satisfy deepsource

* added the feature flag

* lint fix

* fix usage of featureconfig Inti in testcases

* call resetConfig method

* add feature flag in  migration testcases

* fix formatting

* change bucket name from blockRootValidatorKeysIndexBucket to blockRootValidatorHashesBucket

* remove from cache when state deleted

* fixed few more comments

* added to devModeFlags

* added cache delete under the feature flag

* fix lint

* change cache sizes and improve documentation

* fiexed few more review coments

* not using hash anymore and using a new SaveStates function

* satisfu deepsource

* run gazel

* fix feature flag related stuff

* fixing merge conflict fix

* few goodies

* improve UX and dont swallow error

* merge fix

* import format fix

* fix migrationion when flag not given issue

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-07-27 20:02:53 +00:00
terence tsao
7129af5ccf
Core/deposit: altair processing methods and tests (#9227) 2021-07-27 12:10:08 -07:00