Commit Graph

5959 Commits

Author SHA1 Message Date
terence tsao
ad048a8750
Rewards/penalties: epoch processing for Altair (#9334)
* Add epoch precompute methods

* Better name

* Bring expensive computation outside

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-06 20:02:35 +00:00
Raul Jordan
90ea158301
Add Some More Traces to Blockchain Service (#9336)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-06 15:02:36 +00:00
Radosław Kapka
ef9eff29b8
Tiny state management improvements (#9338)
* Tiny state management improvements

* fix indices length calculation

* review comments

* revert function rename
2021-08-06 10:47:50 +00:00
terence tsao
4cce4f1ee5
Refactor slash validator func's signature (#9321)
* Refactor slash validator func's signature

* Named arguments

* Update beacon-chain/core/blocks/proposer_slashing.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-06 04:06:49 +00:00
Raul Jordan
ce10be9a16
Add Latest Slasher KV Methods to DB Subpackage (#9328)
* tests pass

* use end slot for epoch pruning

* amended to endslot of epoch

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-08-05 21:27:05 +00:00
Raul Jordan
4f0b00894b
More Precise Metrics E2E (#9331)
* metrics test copute slots since genesis

* gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 20:44:56 +00:00
Radosław Kapka
57fac9b74f
Improve API Middleware design (#9324)
* remove superfluous WriteHeader call

* move custom handlers outside hooks

* rename OnPostStart hook

* function that returns a default endpoint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 20:21:43 +00:00
Mohamed Zahoor
395feeb35d
Cross check db state bucket after migration (#9311)
* added state db check after migration

* satisfy deepsource

* added db compare after migration sub-command

* fomt issue

* point ot bolt issue

* gazel fix

* maxUint84

* revert accidental deletion of WORKSPACE file

* added sorting support

* cleanup debug

* gazel fix

* satisfy deepsource

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 18:30:05 +00:00
terence tsao
7bc22e4ddf
Cache: sync committee index position (#9317)
* Add sync committee index cache

* Update BUILD.bazel

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 17:44:13 +00:00
Radosław Kapka
016beb51c0
Make SubmitAttestations v1 implementation similar to v1alpha1 (#9319)
* Make `SubmitAttestations` v1 implementation similar to v1alpha1

* fix test

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-05 08:01:20 +00:00
Radosław Kapka
456b4d463e
Implement PrepareBeaconCommitteeSubnet in the validator API (#9318)
* validator migration

* status helpers

* implementation

* fix existing tests

* build file

* tests

* run endpoint through API Middleware

* reduce helper package visibility

* empty caches before each test

* assign all validators to subnets

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-05 07:23:52 +00:00
terence tsao
4d33068496
Core helper: validate sync message time (#9325)
* Add validate sync message time

* Update BUILD.bazel
2021-08-05 12:39:32 +08:00
terence tsao
6e9faa3618
Epoch: precompute functions for Altair part 1 (#9309)
* Add epoch precompute methods

* Split into part 1 for better review

* Update epoch_precompute_test.go

* Preston's feedback

* Skip if the valiator is not eligible

* Add regression test for non eligible

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-04 15:13:05 +00:00
Nishant Das
fd42cb712f
Fix Incorrect Conditional Check (#9323)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-04 13:50:27 +00:00
Radosław Kapka
c5b68b4f03
Implement PublishAggregateAndProofs in the validator API (#9294)
* grpc implementation

* build file

* full implementation

* remove unused function

* tests

* wrap top-level array

* Update beacon-chain/rpc/eth/v1/validator/validator.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-04 09:01:38 +00:00
Mohamed Zahoor
239d904d27
Fix memory usability (#9310)
* fix cache usability

* fix the name of the prometheus flag

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-04 02:33:28 +00:00
terence tsao
461be99fac
Block: process sync aggregate (#9308)
* Add altair base reward funcs

* Update BUILD.bazel

* Add process sync aggregate

* Update BUILD.bazel

* Update block.go

* Update block_test.go

* Feedback, refactor and more tests

* Further refactor sync committee votes

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-03 16:14:15 +00:00
Preston Van Loon
9572add171
Fail fast when using old define=ssz=<value> (#9316)
* Fail fast when using old define=ssz=<value>

* remove print statement
2021-08-02 19:04:30 +00:00
ahadda5
17cf2e5377
Fixing .bat file issues (#9266)
* replacing [] with "" for ifs
using !! vs %% for var with delayed expansion
Check first for 404 then download.

* remove the goto after if are fixed

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-02 16:09:09 +00:00
Radosław Kapka
e4388c7fce
Update grpc-gateway dependency to proper version (#9315)
* logging

* properly update grpc-gateway dependency

* Revert "logging"

This reverts commit 6c59a87ac0c313d8136281ee78aab4cb9d2ade90.

# Conflicts:
#	go.mod

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-02 14:52:54 +00:00
Mohamed Mansour
47c28e6cd6
Show error in logs if passing invalid flags in yaml (#9033)
* Show error in logs if passing invalid flags in yaml

The YAML configs pass if an invalid flag is set, this causes frustration
because the flag could have been mispelled causing a huge mystery.

This will strictly check if the flags are deserialized, if a flag
doesn't exist, it will error out as a Fatal error with appropriate
reasoning which flag is wrong.

* Fix review comments to make some of them non-fatal

* Make yaml.TypeError a pointer

* Remove UnmarshalStrict from spectests since they use a different YAML package

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-08-01 03:26:24 +00:00
Preston Van Loon
0c1fc65604
Core/deposit: refactor blocks.ProcessDeposit for less complex altair implementation (#9305)
* Refactor blocks.ProcessDeposit to return whether or not the deposit resulted in a new validator entry.

* Only mark validator as new after successfully appending it to the state

* fix merge conflict

* fix merge conflict

* gofmt

* fix merge conflict

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-31 23:42:07 +00:00
terence tsao
5c36d59672
Reward: altair base reward (#9307)
* Add altair base reward funcs

* Update BUILD.bazel

* Better tests

* Update BUILD.bazel
2021-07-30 19:17:20 +00:00
Nishant Das
7576442ef0
Only Create Deposit Proofs Pre-Genesis (#9289)
* remove proof gen

* fix pre-genesis case

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-30 18:33:31 +00:00
terence tsao
d130bf0035
Fix more duplicated import prefix (#9306) 2021-07-30 18:05:38 +00:00
Mohamed Zahoor
0d818a5709
Reduce memory overhead during DB migrations (#9298)
* wip

* migration happening

* wip

* fix memory utilization, add the testcases

* migration issues

* the gazel

* remove debug logging

* goimport

* gazel
2021-07-30 08:38:54 -05:00
terence tsao
560fe69425
Core: sync committee helpers (#9269)
* Add sync committee helpers

* Create BUILD.bazel

* Update BUILD.bazel

* Update BUILD.bazel

* Fix visibility

* Preston's review
2021-07-29 23:19:32 +00:00
terence tsao
012d279663
Fix duplicated imports (#9304)
* Fix duplicated imports

* Fix metrics test
2021-07-29 16:45:17 -05:00
terence tsao
fcd4938ec0
Head: validator index getters (#9303) 2021-07-29 20:05:48 +00:00
Radosław Kapka
4db0dfc4f1
Properly broadcast attestations (#9301)
* better handling of gateway response

* empty response tests

* build file

* broadcast attestations properly

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-29 18:33:18 +02:00
terence tsao
cf4ff3d6f9
Clean up phase 0 spec tests (#9299) 2021-07-29 09:01:18 -05:00
Raul Jordan
9145310647
Eliminate Proto V2 Namespace (#9297)
* get rid of v2 in prysm codebase

* replace block2

* builds

* terence comments

* gazelle
2021-07-28 21:23:44 +00:00
Mohamed Zahoor
21c79f4dfa
Protect the state objects in memory when saving them (#9291)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-28 18:27:27 +00:00
Raul Jordan
98f7f82b2f
Copy Altair Changes Into v1alpha1 Namespace (#9296)
* move the v2 changes into v1alpha1

* v1alpha1 builds

* regenerate pbs

* regenerate ssz

* top level builds
2021-07-28 18:05:07 +00:00
terence tsao
f16b9859e7
Test: add altair test utility (#9287)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-28 17:15:23 +00:00
Nishant Das
a372e30c8f
Fix Spaced Arguments for Subcommands (#9292)
* fix args for commands

* fix nasty edge case
2021-07-28 14:36:48 +00:00
Nishant Das
415973fca3
remove flag (#9290) 2021-07-28 16:11:22 +08:00
Raul Jordan
2d9915d2d9
Define Unimplemented V2 Services (#9286)
* generate ssz bindings for protos

* define services and unimplemented

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-27 20:42:14 +00:00
Mohamed Zahoor
c6bd7a2f83
Efficient Historical State Representation (#9155)
* added migration in to new schema

* gazel fix

* goimports fix

* construct state with validator entries from a seperate bucket

* save state with validator entry indirection

* fixed save and retreieve issues

* fixed more test cases related to DeepEqual

* added save benchmark

* gazel fix

* organize benchmarks

* add cache and improve state construction  performance

* gazel fix

* check type

* remove cache checking from Has

* fix decoding when cache is off

* fix slice bugs

* add migration testcases

* linter fix

* fix few review comments

* fix review feedback

* gazel fix

* satisfy deepsource

* added the feature flag

* lint fix

* fix usage of featureconfig Inti in testcases

* call resetConfig method

* add feature flag in  migration testcases

* fix formatting

* change bucket name from blockRootValidatorKeysIndexBucket to blockRootValidatorHashesBucket

* remove from cache when state deleted

* fixed few more comments

* added to devModeFlags

* added cache delete under the feature flag

* fix lint

* change cache sizes and improve documentation

* fiexed few more review coments

* not using hash anymore and using a new SaveStates function

* satisfu deepsource

* run gazel

* fix feature flag related stuff

* fixing merge conflict fix

* few goodies

* improve UX and dont swallow error

* merge fix

* import format fix

* fix migrationion when flag not given issue

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-07-27 20:02:53 +00:00
terence tsao
7129af5ccf
Core/deposit: altair processing methods and tests (#9227) 2021-07-27 12:10:08 -07:00
Raul Jordan
f700808c8b
Generate SSZ Bindings for Latest Protos (#9285) 2021-07-27 17:57:11 +00:00
Raul Jordan
1ea1c69174
Copy V1Alpha1 Protos and API Endpoints To V2 Namespace (#9284)
* v2 builds

* generate go code

* fix build
2021-07-27 16:54:49 +00:00
terence tsao
54772596e0
State/V2: rest of pkg (#9283)
* Add rest of state v2

* Update BUILD.bazel
2021-07-27 15:47:03 +00:00
Nishant Das
a556cf27c4
Use Only Checkpoint Cache for Processing Attestations (#9282)
* use only one cache

* clean up
2021-07-27 07:26:53 -07:00
terence tsao
5b5c8f4d3f
State/V2: setters (#9279)
* Add state v2 getters

* Update BUILD.bazel

* Add state v2 setters

* Update BUILD.bazel
2021-07-26 17:27:05 -05:00
terence tsao
245dc23cde
State/V2: getters (#9228)
* Add state v2 getters

* Update BUILD.bazel
2021-07-26 20:55:30 +00:00
Radosław Kapka
837cd4eb8f
Implement ProduceAttestationData in the validator API (#9271)
* functionality

* test

* build file

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-26 19:22:58 +00:00
Radosław Kapka
8c8f1bb9c1
Regression test for #9264 (#9277) 2021-07-26 11:48:51 +00:00
Kaan
a8c49c50ad
v1alpha1 validators: catch possible reqState nil case (#9275)
* v1alpha1 validators: catch possible reqState nil case

* Minor format of error log

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-07-26 13:50:40 +08:00
Lorem Ipsum
14d0d9195f
Fixing wrong node information in fallback log (#9258) (#9272) 2021-07-25 23:46:54 +02:00