Commit Graph

896 Commits

Author SHA1 Message Date
Preston Van Loon
fe6e6909e6
Validator Attestation History Pruning: Use a bolt cursor instead of bucket.ForEach, exit early when applicable (#8627)
* Use a bolt cursor instead of bucket.ForEach, exit early when applicable

* Refactor to reuse the same code

* gofmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-18 22:39:28 +00:00
Radosław Kapka
1f8171d069
Dynamic key reload for remote keymanager (#8611)
* Dynamic key reload for remote keymanager

* fix failing keymanager test

* keymanager tests

* define RemoteKeymanager interface

* WaitForActivation tests

* gzl

* handle error in test
2021-03-16 10:00:05 -05:00
Radosław Kapka
dc6dee3f4e
Fix issues with dynamic key reload for imported/derived keymanager (#8585)
* implementation of handling reload

* proper implementation

* fix WaitForActivation tests

* HandleKeyReload tests

* runner tests

* do not reuse log hook between tests

* gzl

* Drop unnecessary use of the blank identifier

* move mock_validator to testutil package

* Reorganize validator client package

* reduce package visibility

* revert account changes

* gzl

* describe nil parameter

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-12 17:23:56 +00:00
Nishant Das
a3c96c2f44
Prune Better For Larger Buckets (#8599)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-03-12 15:05:20 +00:00
Nishant Das
3b6b3f6ef6
ignore syncing freelist for validator db (#8601) 2021-03-12 08:06:20 -06:00
Preston Van Loon
eb694ab5d5
Update rules_go and fix proto conflicts (#8596)
* Update rules_go and fix proto conflicts

* gaz

* Update generated code

* more emptypb fixes

* gaz

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-03-11 18:03:19 -06:00
Raul Jordan
286444a2ec
Shorter Pruning Period for Slashing Protection (#8590)
* begin a simpler pruning process

* shorter pruning

* pass all pruning tests

* passing tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-11 15:51:16 +00:00
Radosław Kapka
5374d07c4d
Reorganize validator client package (#8592)
* Reorganize validator client package

* reduce package visibility
2021-03-11 11:57:41 +00:00
Benoit Perroud
9980ca3b7e
Add metrics per keys for next scheduled attestation and proposal (#8583)
* Add metrics per keys for next scheduled attestation and proposal

* Found a better place where to update the counters

* Wrap with emitAccountMetrics flag
2021-03-09 21:13:11 +00:00
Raul Jordan
ab301aa4fe
Remove Accounts Enable/Disable Code (#8576)
* rem

* remove all enable disable code

* fix broken build

* fix more tests

* fix broken tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-09 15:46:50 +00:00
Nishant Das
2bb0a602e4
create the bucket only once (#8579)
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2021-03-09 08:22:52 -06:00
Preston Van Loon
f074c5ee89
Fix validator activation monitoring with inactive keys (#8558)
* refactor / move waiting for activation updates

* Commentary

* Update test to follow the full code path

* gofmt and goimports

* manual imports fixes

* Apply suggestions from code review

typo fixes

* Remove redundant handleAccountsChanged and chan. Thanks @nisdas

* var sub = to sub :=

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-03-05 18:33:39 +00:00
Radosław Kapka
e36c3dd668
Ignore "safe" secrets in DeepsSource analysis (#8555) 2021-03-04 13:52:55 +00:00
Raul Jordan
ff329df808
Add All Other Prysm CLI Entrypoints to "cmd" Folder (#8545)
* other cmd

* comments

* gaz

* gaz ignored file

* build

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-03 17:05:37 +00:00
Preston Van Loon
c9858b5e6b
Validator: Remove optional and duplicated dial opts for streaming middleware (#8549) 2021-03-03 10:28:13 -06:00
Raul Jordan
cdea2debc9
Add Goland Standard "cmd" Pattern for Validator Binaries (#8541)
* validator cmd

* imports

* more imports

* e2e viz

* alias

* use native alias

* add actual

* fix macro

* work on fix e2e

* add viz

* gaz

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-03-02 12:58:40 -06:00
Panagiotis Georgiadis
46f6bd6d08
A bit of cleanup based on goconst tool (#8529)
* Make 1 occurence of 'foo', 'bar' and 'fizz!'

* Make 1 occurence of 'merkleizing list that is too large, over limit' string

* Limit password occurrences

* Make only 1 occurence of 'strongPass'

* Limit testMnemonic occurrences

* Limit expected epoch error messages

* Rename errors and use constant

* Update bazel dependencies

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-03-02 11:39:54 +00:00
Preston Van Loon
b3dcbfe2a4
Update snappy to fix ARM64 with go.1.16 (Reverts #8538) (#8539)
* Revert "Revert "Update to go 1.16" (#8538)"

This reverts commit a92b20d2bb.

* Update snappy
2021-03-02 16:54:16 +08:00
Nishant Das
a92b20d2bb
Revert "Update to go 1.16" (#8538)
This reverts commit f4adc0ea86.
2021-03-02 05:35:08 +00:00
terence tsao
c30ee6c166
Ran go import (#8528)
* Ran go import

* Sort
2021-02-28 13:49:00 +01:00
ahadda5
29d1959b81
Recursively Find Validator Database File In Slashing Protection Commands (#8518)
* issue/feature 8500 find validator.db automatically

* gazelle build

* remove less! why there?

* fixed errors import

* fixed errors import

* unit tested

* adding the find validator.db fileutil func to export and import slashing

* deleted the comment

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-26 11:04:18 -06:00
Eduard Klementiev
878bc15229
Add --list-validator-indices flag (#8520)
* Add --list-validator-indices flag

* Fix style issues

* Print pubkeys alongside the validator indexes

* Improve list-indices output

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-02-26 15:00:05 +00:00
Nishant Das
b4648f1df9
Update Test Case in Batching Public Keys (#8523) 2021-02-26 01:44:42 +00:00
Preston Van Loon
f4adc0ea86
Update to go 1.16 (#8521)
* Update to go 1.16

* Also update go.mod

* rm go.sum then run go mod tidy

* go mod tidy with go 1.16

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-25 21:38:19 +00:00
terence tsao
b400098296
Remove unused code (#8517)
* Remove unused code

* Go mod tidy

* Gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-25 18:13:13 +00:00
pinglamb
5db5ca7056
Use eth2-types SSZUint64 (#8514) 2021-02-25 05:51:26 -08:00
Josh Yudaken
f0eb843138
Graffiti ordered index (#8482)
* Added ordered option to graffiti file

* Updated validator to use Ordered graffiti

* Track graffiti ordered index in db

* Update `ordered` to only emit each graffiti once

Co-authored-by: pinglamb <pinglambs@gmail.com>
2021-02-24 22:50:47 +00:00
Radosław Kapka
4d28d5e4d2
Listen for account changes only when required (#8503)
* initial implementation

* remove listening for changes from wallet creation

* goimports

* test fix

* more goimports

* listen for changes when initializing wallet through gRPC

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-24 18:05:46 +00:00
Nishant Das
08b938982b
Propagate Cancellation To Beacon Node Server (#8512)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-24 17:10:25 +00:00
Preston Van Loon
6e831920bf
Fix slice out of bounds error in validator db migration (#8510)
* Fix slice out of bounds error in validator db migration #8509

* Add regression testing

* make it double just in case

* gofmt
2021-02-24 15:02:39 +00:00
Raul Jordan
0c599521b1
Better Handling of Subscriber Errors in Logs Streams (#8505)
* handle subscriber error and increase buffer sizes

* operation order in unsub and close
2021-02-23 15:33:23 -06:00
terence tsao
3edfa8cb88
Use Custom Type ValidatorIndex Across Prysm (#8478)
* Use ValidtorIndex across Prysm. Build ok

* First take at fixing tests

* Clean up e2e, fuzz... etc

* Fix new lines

* Update beacon-chain/cache/proposer_indices_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/core/helpers/rewards_penalties.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/core/helpers/shuffle.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/graffiti/parse_graffiti_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Raul's feedback

* Fix downcast int -> uint64

* Victor's feedback

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-23 00:14:50 +00:00
Victor Farazdagi
b577869ed6
Fixes import aliases (#8497)
* Fixes import aliases

* another fix

* reset gw files

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-22 23:20:57 +00:00
Preston Van Loon
ad9cd1933a
Add nil check for validator db migration (#8493)
* Add nil check for validator db migration

* continue if source bucket is nil
2021-02-22 14:28:34 +00:00
Radosław Kapka
7d3e53f3e4
Constrain read/write channels (#8479) 2021-02-19 15:39:27 +00:00
terence tsao
dc1bec79ed
Use Eth2 type CommitteeIndex (#8477)
* Use types.CommitteeIndex

* Go fmt

* Update validator pkg

* Fix e2e

* Happy fuzz tests

* Sync with upstream ethereumapi

* Go mod tidy

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-18 20:11:20 +00:00
Raul Jordan
4f9752bb3e
Stop Early in Validator Surround Vote Local Protection (#8460)
* include migration and logic for stopping early in slashing protection checks

* remove commented code

* extract methods

* migration logic tested up

* migration up and down tests

* Update validator/db/kv/attester_protection.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* added in pruning and batched migrations

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-02-17 19:23:59 +00:00
Preston Van Loon
558b16275d
Validator: Annotate attestation error spans (#8451)
* Annotate attestation spans

* Annotate attestation spans

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-16 09:06:52 +00:00
Victor Farazdagi
a069738c20
ETH2 Types: Slot (#8408)
* update shared/params

* update eth2-types deps

* update protobufs

* update shared/*

* fix testutil/state

* update beacon-chain/state

* update beacon-chain/db

* update tests

* fix test

* update beacon-chain/core

* update beacon-chain/blockchain

* update beacon-chain/cache

* beacon-chain/forkchoice

* update beacon-chain/operations

* update beacon-chain/p2p

* update beacon-chain/rpc

* update sync/initial-sync

* update deps

* update deps

* go fmt

* update beacon-chain/sync

* update endtoend/

* bazel build //beacon-chain - works w/o issues

* update slasher code

* udpate tools/

* update validator/

* update fastssz

* fix build

* fix test building

* update tests

* update ethereumapis deps

* fix tests

* update state/stategen

* fix build

* fix test

* add FarFutureSlot

* go imports

* Radek's suggestions

* Ivan's suggestions

* type conversions

* Nishant's suggestions

* add more tests to rpc_send_request

* fix test

* clean up

* fix conflicts

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2021-02-16 07:45:34 +00:00
Preston Van Loon
aef5a7b428
Validator: Choose a reasonable cap for the attested target epochs slice (#8452)
* Choose a reasonable cap for the attested target epochs slice

* revert bazelrc

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-16 01:37:35 +00:00
Raul Jordan
36b6a71af4
Configurable DB Mmap Size for Beacon Node and Validator Client (#8448)
* add flag to beacon and validator

* gaz

* fuzz

* add dep viz

* add to tools
2021-02-15 20:29:47 +00:00
Raul Jordan
7c3827a9a4
add test (#8444) 2021-02-15 11:55:28 -06:00
Radosław Kapka
0716519be9
Fix error formatting inside fmt.Errorf (#8439)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-12 23:44:46 +00:00
terence tsao
068f758f49
Code inspect - clean ups (#8445) 2021-02-12 17:04:45 -06:00
Preston Van Loon
e2c5ae53e7
Validator: Safer pending attestation records flushing (#8433)
* Add a debug log to show duration

* merge from dev

* use safe pending attestation records struct

* fix build, use atomic bool

* Add deadline checks to CheckSlashableAttestation

* Go fmt

* Add test for in-progress log

* GoDocs

* Rename pending attestation records to queued attestation records

* rename and add commentary on log

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-12 20:19:01 +00:00
Preston Van Loon
473172ca8b
Validator: Make read operation use db.view instead of db.update (#8434)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-12 19:06:58 +00:00
Radosław Kapka
47fdb3b99a
Revert "Rename NewService to New (#8337)" (#8440)
* Revert "Rename `NewService` to `New` (#8337)"

This reverts commit d121b19145.

# Conflicts:
#	beacon-chain/sync/initial-sync/round_robin_test.go

* fix name in test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-12 17:45:22 +00:00
Raul Jordan
25caa6d1be
Add Mutex and Block Profiling (#8435) 2021-02-11 20:20:19 +00:00
Preston Van Loon
d551c8e1d0
Validator: add a DEBUG log to show batch attestation save duration (#8432)
* Add a debug log to show duration

* Autofix issues in 1 file

Resolved issues in validator/db/kv/attester_protection.go via DeepSource Autofix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2021-02-11 18:27:35 +00:00
Raul Jordan
e236dedc32
Add Ability to Specify All Public Keys When Exiting Validators (#8399)
* add programmatic voluntary exit

* add exit all flag

* test

* lint

* add multiple exits test

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-11 16:50:16 +00:00