Commit Graph

171 Commits

Author SHA1 Message Date
Victor Farazdagi
0f08bd288c
fix tests in //beacon-chain/rpc/beacon (#5559)
* assert new state is not disabled
* Merge refs/heads/master into fix-list-indexed-atts-test
* add helper method
* Merge branch 'fix-list-indexed-atts-test' of github.com:prysmaticlabs/prysm into fix-list-indexed-atts-test
* xx
* xx
* Merge refs/heads/master into fix-list-indexed-atts-test
2020-04-21 17:44:18 +00:00
Ivan Martinez
25102e0978
Use WaitForSynced in validator client for startup (#5465)
* Add WaitForSynced to beacon-chain
* Fix build issues
* Fix comment
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-validator
* Fix tests
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-validator
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-validator
* Change validator client to use WaitForSynced
* Change prysm script to support windows
* Update prysm.sh
* Merge branch '0xKiwi-patch-1' of https://github.com/prysmaticlabs/prysm into simplify-val-client
* Merge branch 'master' into simplify-val-client
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-val-client
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-val-client
* Add to mock
* Merge branch 'simplify-val-client' of https://github.com/0xKiwi/Prysm into simplify-val-client
* Fix mocks
* Add feature flag for WaitForSynced
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-val-client
* Fix flag
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-val-client
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-val-client
* Remove comment
* Merge branch 'master' into simplify-val-client
* Merge branch 'master' into simplify-val-client
* Merge branch 'master' into simplify-val-client
* Merge branch 'master' into simplify-val-client
2020-04-20 23:16:53 +00:00
Victor Farazdagi
b663086a88
add --dev flag (#5539)
* add --dev flag
* Merge refs/heads/master into dev-flag
* Update shared/featureconfig/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/featureconfig/flags.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/master into dev-flag
2020-04-20 17:27:24 +00:00
Shay Zluf
95f1676d5f
Fix broadcast slashing flag (#5515) 2020-04-19 16:41:59 -04:00
Nishant Das
626b3e0c66
Add Reference Copying (#5245)
* add state changes
* add new changes
* add flag
* lint
* add flag
* change to correct bool
* fixing and consolidating trie
* lint
* Apply suggestions from code review
* Merge refs/heads/master into addBetterCopying
* Merge branch 'master' into addBetterCopying
* refCopy -> stateRefCopy
* Merge refs/heads/master into addBetterCopying
* tests whether unexpected mutation of validators within state is avoided
* Merge branch 'addBetterCopying' of github.com:prysmaticlabs/prysm into addBetterCopying
* remove unnecessary fields
* gazelle
* updates test
* avoid unexpected mutation in block roots on refcopy
* avoid unexpected mutation in state roots on refcopy
* Merge refs/heads/master into addBetterCopying
* Merge branch 'master' into addBetterCopying
* fix test
* randao tests
* simplify tests
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* test cur/prev attestations mutation
* Merge branch 'addBetterCopying' of github.com:prysmaticlabs/prysm into addBetterCopying
* gazelle
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* fixes tests
* minor naming update
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
2020-04-19 13:57:43 +00:00
terence tsao
cb045dd0e3
Align code base to v0.11 (#5127)
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* add in new patch and workspace
* update cloners
* Handle rewards overflow (#5122)

* Refactoring of initial sync (#5096)

* implements blocks queue

* refactors updateCounter method

* fixes deadlock on stop w/o start

* refactors updateSchedulerState

* more tests on schduler

* parseFetchResponse tests

* wraps up tests for blocks queue

* eod commit

* fixes data race in round robin

* revamps fetcher

* fixes race conditions + livelocks + deadlocks

* less verbose output

* fixes data race, by isolating critical sections

* minor refactoring: resolves blocking calls

* implements init-sync queue

* udpate fetch/send buffers in blocks fetcher

* blockState enum-like type alias

* refactors common code into releaseTicket()

* better gc

* linter

* minor fix to round robin

* moves original round robin into its own package

* adds enableInitSyncQueue flag

* fixes issue with init-sync service selection

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* initsyncv1 -> initsyncold

* adds span

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Handle rewards overflow

* Revert "Refactoring of initial sync (#5096)"

This reverts commit 3ec2a0f9e0.

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* updated block operations
* updated validator client
* Merge refs/heads/master into v0.10.1
* updated block operations test
* skip benchmark test
* updated transition test
* updated db kv tests
* updated ops tests
* updated ops tests
* updated slashing tests
* updated rpc tests
* updated state utils
* updated test utils and miscs
* Temp skips minimal spec tests
* Fixed proposer slashing test
* Gaz
* Skip 2 more minimal tests
* Skip 2 more minimal tests
* Update readme
* gaz
* Conflict
* Fix import and not use
* Update workspace for new spec test
* Fix workspace
* Merge refs/heads/master into v0.10.1
* Update workspace with new ethapi commit
* Unblock a few tests
* Merge refs/heads/master into v0.10.1
* fixed block op test
* gaz
* Merge refs/heads/master into v0.10.1
* Skip gen state test (test setup issue
* Updated hysteresis config
* Updated epoch processing for new hyteresis
* Updated tests
* regen proto beacon
* update state util for state root
* update state types
* update getter and setters
* update compute domain and get domain and tests
* update validators
* Add forkdata proto
* Updated compute domain api, moved it to helper pkg
* Merge refs/heads/master into v0.10.1
* Fixed all core tests
* Fixed all the sync tests
* Fixed all the rpc tests
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Fixed conflict
* Fixed conflict
* Conflict fix
* visibility
* Fixed validator tests
* Fixing test util
* Fixed rest of non spec tests
* Fixed a bug proposer index wasn't included
* gaz
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Updated eth1 data voting period to epoch based
* Fixed failed tests
* fix bug
* fix error
* Fixed more misc tests
* Add new SignedAggregateAndProof to pass spec test
* Update minimalConfig.PersistentCommitteePeriod
* allow to rebuild trie
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Skip e2e tests
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Align aggregator action with v0.11 (#5146)
* Remove Head Root from Beacon Block by Range Request (#5165)

* make proto changes
* remove head root
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* add back herumi's library
* Update ethapi in workspace, started fixing test. Hand off to Nishant
* fix build
* All tests passing
* Align finalized slot check with v0.11 (#5166)
* Merge branch 'master' into v0.11
* Add DoS resistance for v0.11 (#5158)
* Add Fork Digest Helper (#5173)
* Extend DoS prevention to rest of operation objects (#5174)

* Update mapping

* Add caches

* Update seen block in validation pipeline

* Update seen att in validation pipeline

* Update seen att in validation pipeline

* Fixed rest of tests

* Gazelle

* Better writes

* Lint

* Preston's feedback

* Switched to LRU cache and fixed tests

* Gazelle

* Fix test

* Update proposer slashing

* Update proposer slashing

* Fixed a block test

* Update exit

* Update atteser slashing

* Raul's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add remote keymanager (#5133)

* Add remote keymanager

* Add generic signRoot() helper

* Add tests for remote keymanager

* NewRemote -> NewRemoteWallet

* signRoot -> signOject, to increase reuse

* Fix end-to-end compile error

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Add Snappy Framing to the Encoder (#5172)

* change to framing

* more fixes

* fix everything

* add stricter limits

* preston feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Move Subnet Functionality to its Own File (#5179)

* move subnets to their own file

* fix build fail

* build

* Update beacon-chain/p2p/discovery_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Sync with master
* Verify proposer signature in sync (#5206)
* Fix Signed Attestation In Sync (#5207)
* Add Eth2 Fork ENR Functionality (#5181)

* add fork entry enr

* add in fork

* add the required fork entry to node

* add and retrieve fork entry

* await state initialized

* utilize new structure

* more progress, utilizing a config map instead

* send the genesis validators root via the event feed

* struct method for discovery

* fix broken builds

* fixed up more tsts using state feed initializer

* fix up most tests

* only one more failing test

* almost done with tests

* p2p tests all pass

* config fix

* fix blockchain test

* gaz

* add in todo

* lint

* add compare func

* ensure fork ENR versions match between peers

* add in test for discovery

* test name

* tests complete

* tests done

* done

* comments

* fix all flakes

* addressed comments

* build using ssz gen

* marshal record

* use custom ssz

* deduplicate import

* fix build

* add enr proto

* p2p tests done

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Verify aggregator signature in sync (#5208)
* Add Fork Digest For Gossip Topics (#5191)

* update for the day

* fix remaining failing test

* fix one more test

* change message

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

* implement fork digest'

* align digest to interface'

* passed all tests

* spawn in goroutine

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fix Incorrect Attester Slashing Method (#5229)
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Remove keystore keymanager from validator (#5236)

* Remove keystore keymanager from validator

* Update dependency

* Update validator/flags/flags.go

* Update validator/flags/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* fix broadcaster
* update metrics with fork digest for p2p (#5251)

* update metrics with fork digest for p2p

* update p2p metrics

* update metrics using att values

* wrapped up

* fix bug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix incorrect domain type comments (#5250)

* Fix incorrect domain type comments
* resolve conflicts
* fix broken broadcast test
* fix tests
* include protocol suffix
* fix confs
* lint
* fix test
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* resolve broken slasher test'
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' into v0.11
* fix config override
* Remove deprecated parameters (#5249)
* Avoid div by zero in extreme balance case (#5273)

* Return effective balance increment instead of 1

* Update to new spec tests v0.11.1

* Revert "Regen historical states for `new-state-mgmt` compatibility (#5261)"

This reverts commit df9a534826.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Remove deprecated parameters (#5249)" (#5276)

This reverts commit 7d17c9ac3455ee15c67b3645485693309216bc97.
* Verify block proposer index before gossip  (#5274)

* Update pipeline

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add in Proposer Index to Custom HTR (#5269)

* fix test

* Update beacon-chain/state/stateutil/blocks_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Resolve Flakey P2P Tests (#5285)

* double time for flakey test

* fix test flakeyness in p2p:

* flakey

* time tolerance

* greater tolerance
* Merge branch 'master' into v0.11
* release resources correctly (#5287)
* Merge refs/heads/master into v0.11
* Enable NOISE Handshake by Default v0.11 (#5272)

* noise handshakes by default

* fix build

* noisy noise everywhere

* deprecated noisy noise flag with more noise

* add secio as fallback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
* Merge refs/heads/master into v0.11
* new ports
* fix broken build
* Make `new-state-mgmt` canonical  (#5289)

* Invert the flags
* Update checking messages
* Fixed all db tests
* Fixed rest of the block chain tests
* Fix chain race tests
* Fixed rpc tests
* Disable soudns better...
* Merge branch 'v0.11' into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Fix export
* Merge branch 'invert-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-new-state-mgmt
* Fix conflict tests
* Gazelle
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge branch 'master' into v0.11
* resolve flakeyness
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Detect Proposer Slashing Implementation (#5139)

* detect blocks

* detect blocks

* use stub

* use stub

* use stub

* todo

* fix test

* add tests and utils

* fix imports

* fix imports

* fix comment

* todo

* proposerIndex

* fix broken test

* formatting and simplified if

* Update slasher/detection/service.go

* Update slasher/detection/testing/utils.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fixed up final comments

* better naming

* Update slasher/detection/service.go

* Update slasher/detection/service.go

* Update slasher/detection/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* no more named args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Add Metadata And Ping RPC methods (#5271)

* add new proto files

* add flag and helper

* add initializer

* imports

* add ping method

* add receive/send ping request

* add ping test

* refactor rpc methods and add ping test

* finish adding all tests

* fix up tests

* Apply suggestions from code review

* lint

* imports

* lint

* Update beacon-chain/p2p/service.go

* Update shared/cmd/flags.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Updates for remote keymanager (#5260)
* Merge branch 'spec-v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge remote-tracking branch 'origin' into v0.11
* Update to slash by slot instead of epoch (#5297)

* change to slash by slot instead of epoch

* gaz

* fix test

* fix test

* fix infinite loop on error parse
* Sync with master
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Update proposer protection to v0.11 (#5292)

* Complete most of changes

* Fix other tests

* Test progress

* Tests

* Finish tests

* update pbs

* Fix mocked tests

* Gazelle

* pt 2

* Fix

* Fixes

* Fix tests wit hwrong copying
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Implement `SubscribeCommitteeSubnet` method (#5299)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append
* Merge refs/heads/master into v0.11
* resolve confs
* Validator subscribe subnet to next epoch (#5312)

* Alert to subscribe to next epoch

* Fixed tests

* Comments

* Fixed tests

* Update validator/client/validator.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)

This reverts commit 47e5a2cf96f5add151bf135a5352c2dad7922615.
* Aggregate on demand for v0.11 (#5302)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Wait 1/3 on validator side

* Lint

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Fixed duplication

* Tests

* Skip e2e

* Update beacon-chain/rpc/validator/aggregator.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Refactor Dynamic Subscriptions (#5318)

* clean up

* comment

* metrics

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Fix listindexed attestations and detect historic attestations (#5321)

* fix list indexed attestations

* fix tests

* goimports

* names
* Add check for slot == 0 (#5322)
* Change attester protection to return default if DB is empty (#5323)

* Change how default values are set

* Remove unused imports

* Remove wasteful db call

* Fix db tests

* Fix db test
* Merge refs/heads/master into v0.11
* fix it (#5326)
* V0.11 run time fixes to use interop config (#5324)

* Started testing
* Bunch of fixes
* use-interop
* Sync with v0.11
* Conflict
* Uncomment wait for activation
* Move pending block queue from subscriber to validator pipeline
* Merge branch 'v0.11' into use-interop-config
* passing tests
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Nil Checks in Process Attestation v0.11 (#5331)

* Started testing

* Bunch of fixes

* use-interop

* Sync with v0.11

* Uncomment wait for activation

* Move pending block queue from subscriber to validator pipeline

* passing tests

* nil checks to prevent panics

* lint

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Validator batch subscribe subnets (#5332)

* Update both beacon node and validator

* Comments

* Tests

* Lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Validator smarter subscribe (#5334)
* Fix incorrect proposer index calculation (#5336)

* Use correct parent state

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* enhance error
* enhance error
* Update P2P Service to Handle Local Metadata (#5319)

* add metadata to ENR

* add new methods

* glue everything

* fix all tests and refs

* add tests

* add more tests

* Apply suggestions from code review

* fix method

* raul's review

* gaz

* fix test setup

* fix all tests

* better naming

* fix broken test

* validate nil

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Revert "Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)" (#5343)

This reverts commit e5aef1686e582fc2077767c42187c8527f3a742f.
* Wait for Genesis Event to Start P2P (#5303)

* use event feed for state initialized events

* add in handler for tests

* wait till genesis for p2p

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Avoid duplicated aggregation request (#5346)

* Avoid duplicated aggregation request

* Test and lock

* Gaz
* Fix Validate For Metadata (#5348)

* return true

* shay's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Multiple Proposer Slots Allowed Per Epoch for Validators (#5344)

* allow multiple proposer slots

* multi propose

* proposer indices to slots map

* remove deprecated comm assign

* Apply suggestions from code review

* resolve broken tests, add logic in validator client

* fix val tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Networking Fixes (#5349)

* close stream later

* add ping method

* add method

* lint
* More efficient aggregation on demand (#5354)
* Return Nil Error if Pre-Genesis in P2P Service Healthz Check (#5355)

* pregenesis healthz check:

* optimal

* right order

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* no comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Release DiscoveryV5 for Testnet Restart (#5357)

* release discv5

* fix build
* Fix Overflow in Status Check (#5361)

* fix overflow

* Apply suggestions from code review
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* fix after merge
* Merge refs/heads/master into v0.11
* Make Mainnet Config Default, No More Demo Config  (#5367)

* bye bye demo config

* gaz

* fix usage

* fix dep

* gaz

* Update default balance for sendDeposits

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Use FastSSZ Marshal/Unmarshal for DB Encodings in v0.11.1 (#5351)

* try

* use marshaler structure for db instead of proto

* white list types

* attempt

* revert

* testutil.NewBeaconState()

* Fully populate fields for round trip ssz marshal

* fix //beacon-chain/db/kv:go_default_test

* more passing tests

* another test target passed

* fixed stategen

* blockchain tests green

* passing sync

* more targets fixed

* more test fixes in rpc/validator

* most rpc val

* validators test fixes

* skip round robin old

* aggregate test

* whitelist done

* Update beacon-chain/rpc/validator/attester_test.go

* edit baz

* Fixed tests

* Fixed getblock test

* Add back init

* reduce test size

* fix broken build

* tests pass

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Reconnect slasher streams on beacon node shutdown (#5376)

* restart streams on beacon node shutdown

* fix comment

* remove export

* ivan feedback

* ivan feedback

* case insensitive

* Update slasher/beaconclient/receivers.go

* raul feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' into v0.11
* Merge refs/heads/master into v0.11
* Amend Faucet to Offer 32.5 ETH for v0.11 (#5378)

* deposit amount in faucet

* fix eth amount

* gas cost
* unskip exec transition test
* Revert "Enable NOISE Handshake by Default v0.11 (#5272)" (#5381)

This reverts commit a8d32d504a8f923cdf7fa9dfc2684f8804fbab92.
* Merge refs/heads/master into v0.11
* use string for deposit flag
* Update Bootnode to v0.11 (#5387)

* fix bootnode

* add changes

* gaz

* fix docker
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* build fix
* fix flaky test
* Merge refs/heads/master into v0.11
* Unskip E2E for V0.11 (#5386)

* Begin work on fixing e2e for v0.11

* Start bootnode work

* Begin implementing bootnode into e2e

* Fix E2E for v0.11

* Remove extra

* gaz

* Remove unused key gen code

* Remove trailing multiaddr code

* add skip for slashing

* Fix slashing e2e

* Fix docker image build
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.11
* Merge refs/heads/master into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* Update beacon-chain/p2p/broadcaster_test.go
* Merge refs/heads/master into v0.11
* Pass E2E Tests for v0.11 and Enable Attestation Subnets By Default (#5407)
* Update README.md

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/p2p/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/keystore/deposit_input.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update tools/faucet/server.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update proto/beacon/p2p/v1/BUILD.bazel

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/bls/spectest/aggregate_verify_test.go
* Addressed feedback. All test passing
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Update beacon-chain/core/blocks/block_operations_fuzz_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update shared/testutil/helpers.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/helpers/signing_root.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Resolve Misc v0.11 Items (Raul) (#5414)

* address all comments

* set faucet

* nishant feedback

* Update beacon-chain/p2p/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert keymanager changes (#5416)

* Revert "Updates for remote keymanager (#5260)"

This reverts commit bbcd895db50ce5e7c0ecb64210471cf56f63b373.

* Revert "Remove keystore keymanager from validator (#5236)"

This reverts commit 46008770c162e741251e13772fd7356b43a9af87.

* Revert "Update eth2 wallet keymanager (#4984)"

This reverts commit 7f7ef43f218598a671aaeb327342d7e5130fe8b1.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update BLS and limit visibility (#5415)

* remove duplicated BLS, add golang.org/x/mod

* Update BLS and restrict visibility

* fix build
* Fix eth1data test and fix order of ops (#5413)
* use multiaddr builder (#5419)
* Unskip benchutil and minor v0.11 fixes (#5417)

* Unskip benchutil tests

* Remove protos and gaz

* Fixes
* Networking Fixes (#5421)

* check

* fix test

* fix size

* fix test

* more fixes

* fix test again
* Update ethereum APIs with latest master
* Error handling for v0.11 tests (#5428)

* Proper err handling for tests

* Lint

* Fixed rest of the tests

* Gaz

* Fixed old master tests
* Sync with master
* Rm old aggregate_test.go
2020-04-14 20:27:03 +00:00
Preston Van Loon
c76034147e
Set ssz cache enabled by default (#5398)
* Set ssz cache enabled by default
* gofmt
2020-04-12 21:33:26 +00:00
Preston Van Loon
7bdd1355b8
Add maligned struct static check (#5296)
* Add maligned static check
* Add file, oops
* lint
2020-04-03 05:09:15 +00:00
terence tsao
3544ed2818
Invert init-sync-batch-save-blocks flag for v0.11 (#5293) 2020-04-02 14:46:14 -07:00
Victor Farazdagi
b43e43b4a9
Init sync release queue (#5286)
* fix naming slot -> epoch

* better handling of long periods w/o finality

* bazel

* fixes issue with pointer goint to far ahead

* adds func comment

* hides original sync behind --disable-init-sync-queue

* adds func comment

* deprecated

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-04-02 11:06:32 -05:00
Raul Jordan
f385a1dea6
Release Skip Slot Cache to All (#5280)
* no more skip slot cache

* imports

* deprecated

* fix flakeyness

* disable in e2e

* build

* fix viz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-01 19:09:54 -07:00
Preston Van Loon
8376fb36ca
Enable slashing protection in validator by default (#5278)
* Invert slashing protection validator flags for issue #5267
* remove from e2e flags
* Make error level
* Merge refs/heads/master into flip-propose
2020-04-01 23:17:32 +00:00
terence tsao
c5f186d56f
Batch save blocks for initial sync. 80% faster BPS (#5215)
* Starting a quick PoC
* Rate limit to one epoch worth of blocks in memory
* Proof of concept working
* Quick comment out
* Save previous finalized checkpoint
* Merge branch 'master' of github.com:prysmaticlabs/prysm into batch-save
* Test
* Merge branch 'prev-finalized-getter' into batch-save
* Minor fixes
* Use a map
* More run time fixes
* Remove panic
* Feature flag
* Removed unused methods
* Fixed tests
* E2e test
* Merge branch 'master' into batch-save
* comment
* Merge branch 'master' into batch-save
* Compatible with current initial sync
* Merge branch 'batch-save' of github.com:prysmaticlabs/prysm into batch-save
* Merge refs/heads/master into batch-save
* Merge refs/heads/master into batch-save
* Merge refs/heads/master into batch-save
* Merge branch 'master' of github.com:prysmaticlabs/prysm into batch-save
* Feedback
* Merge branch 'batch-save' of github.com:prysmaticlabs/prysm into batch-save
* Merge refs/heads/master into batch-save
2020-03-30 18:04:10 +00:00
Nishant Das
cdac3d61ea
Custom Block HTR (#5219)
* add custom htr

* fix root

* fix everything

* Apply suggestions from code review

* Update beacon-chain/state/stateutil/blocks.go

* Update beacon-chain/blockchain/receive_block.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/blockchain/process_block.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-26 13:10:22 -05:00
Preston Van Loon
2ab4b86f9b
Allow setting flags via yaml config file. (#4878) 2020-03-19 14:46:44 -07:00
Raul Jordan
3043d4722f
Attestation Dynamic Committee Subnets (#5123)
* initiate cache
* imports fix
* add in feature config flag
* utilize a dynamic set of subnets
* Merge branch 'master' into att-subnets
* add in feature config flag
* Merge branch 'att-subnets' of github.com:prysmaticlabs/prysm into att-subnets
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into att-subnets
* shift
* more changes
* gaz
* Update beacon-chain/rpc/validator/assignments.go
* Update beacon-chain/rpc/validator/assignments.go
* add flag
* Merge branch 'att-subnets' of https://github.com/prysmaticlabs/geth-sharding into att-subnets
* Merge branch 'master' into att-subnets
* Merge refs/heads/master into att-subnets
* no double flag
* Merge branch 'att-subnets' of github.com:prysmaticlabs/prysm into att-subnets
* amend committee ids to better name
* gaz
2020-03-18 23:13:37 +00:00
Nishant Das
5450b3155e
Integrate Field Tries into Current State (#5082)
* add new methods
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* new field trie
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* perform better copying
* fix bug
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* add support for variable length arrays
* get it running
* save all new progress
* more fixes
* more fixes
* more cleanup
* some more clean up
* new memory pool
* remove lock
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* use wrapper
* remove redundant methods
* cleanup
* cleanup
* remove unused method
* change field
* Update beacon-chain/state/types.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Update beacon-chain/state/types.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
2020-03-18 04:52:08 +00:00
Victor Farazdagi
3ec2a0f9e0
Refactoring of initial sync (#5096)
* implements blocks queue

* refactors updateCounter method

* fixes deadlock on stop w/o start

* refactors updateSchedulerState

* more tests on schduler

* parseFetchResponse tests

* wraps up tests for blocks queue

* eod commit

* fixes data race in round robin

* revamps fetcher

* fixes race conditions + livelocks + deadlocks

* less verbose output

* fixes data race, by isolating critical sections

* minor refactoring: resolves blocking calls

* implements init-sync queue

* udpate fetch/send buffers in blocks fetcher

* blockState enum-like type alias

* refactors common code into releaseTicket()

* better gc

* linter

* minor fix to round robin

* moves original round robin into its own package

* adds enableInitSyncQueue flag

* fixes issue with init-sync service selection

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* initsyncv1 -> initsyncold

* adds span

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-03-17 12:27:18 -05:00
terence tsao
e529f5b1d6
Part 1 of integrating new state mgmt to run time (#5108) 2020-03-16 12:07:07 -07:00
shayzluf
f937713fe9
Broadcast slashing (#5073)
* add flag
* broadcast slashings
* Merge branch 'master' of github.com:prysmaticlabs/prysm into broadcast_slashing

# Conflicts:
#	beacon-chain/rpc/beacon/slashings_test.go
* fix tests
* goimports
* goimports
* Merge branch 'master' into broadcast_slashing
* Merge branch 'master' into broadcast_slashing
* Merge branch 'master' into broadcast_slashing
2020-03-12 20:29:23 +00:00
Nishant Das
300d072456
Add Config Change for Validator (#5038)
* add config for validator
* gaz
* Merge refs/heads/master into configureValidator
* Merge refs/heads/master into configureValidator
2020-03-08 06:45:36 +00:00
Nishant Das
345ec1bf8c
Fix Custom Delay Flag (#5026)
* fix flag
* Merge refs/heads/master into fixFlag
* Merge refs/heads/master into fixFlag
* Merge refs/heads/master into fixFlag
* Merge refs/heads/master into fixFlag
* Merge refs/heads/master into fixFlag
* fix config
* Merge branch 'fixFlag' of https://github.com/prysmaticlabs/geth-sharding into fixFlag
2020-03-07 03:52:40 +00:00
prylabs-bulldozer[bot]
e635e5b205
Feature flag to gate prune state upon start up (#5011)
* Added feature flag to gate prune state upon start up
2020-03-05 06:24:59 +00:00
Nishant Das
0093218e41
Add Noise Support To Prysm (#4991)
* add dep
* add feature config
* Merge branch 'master' into addNoiseSupport
* gaz and victor's review
* Merge branch 'addNoiseSupport' of https://github.com/prysmaticlabs/geth-sharding into addNoiseSupport
2020-03-03 17:45:51 +00:00
Nishant Das
3989b65667
Add Flag for Checking HeadState (#4974)
* gate feature
* imports
* add flag
* Merge branch 'master' into gateFeature
2020-03-02 06:06:20 +00:00
terence tsao
b1231f3ddf
Replay blocks and generate state without sig verification (#4943)
* New file

* Add transition_stategen.go

* Update ProcessBlockForStateRoot

* Feature flags

* Fixed tests

* Gaz

* Make them private

* E2e flags

* Gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-26 10:40:33 -06:00
Preston Van Loon
b647ca5dd2
Release --initial-sync-cache-state (#4938)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-24 19:20:00 -06:00
Preston Van Loon
c0f1a1d674
Validator: cache domain data calls (#4914)
* Use a domain data cache to reduce the number of calls per epoch

* fix fakevalidator

* Refactor to use a feature flag, use proto.clone, move interceptor to its own file

* gofmt

* fix comment

* tune cache slightly

* log if error on domain data

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-24 13:02:45 -08:00
Nishant Das
4389e9d3c9
Add mempool feature flag (#4824) (#4903)
* Add mempool feature flag

* gate put too

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-19 14:29:49 +08:00
Preston Van Loon
db68c8a57b
Enable attestation cache flag by default, deprecate feature flag (#4873)
* Enable attester flag by default
2020-02-16 01:07:14 +00:00
Preston Van Loon
661e48f549
Revert state copy PR #4811 (#4825)
* Revert "Add mempool feature flag (#4824)"

This reverts commit b3f2a330dc.
* Revert "Optimize Copying of Fields (#4811)"

This reverts commit 4f654d30ac.
2020-02-11 01:47:31 +00:00
Preston Van Loon
b3f2a330dc
Add mempool feature flag (#4824)
* Add mempool feature flag

* gate put too

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-10 16:20:48 -06:00
Preston Van Loon
0ed8246e28
Release flag to aggregate attestations in fork choice. (#4820)
* Release flag to aggregate attestations in fork choice.
* Merge refs/heads/master into aggregate-atts-in-fc
* fix test
* Merge branch 'aggregate-atts-in-fc' of github.com:prysmaticlabs/prysm into aggregate-atts-in-fc
* Merge refs/heads/master into aggregate-atts-in-fc
2020-02-10 19:57:30 +00:00
terence tsao
8a02003d4b Feature flag to disable head update on attestation basis (#4802) 2020-02-09 11:44:17 -08:00
Preston Van Loon
34178aff2a
Strict verify attestations in pubsub (#4782)
* Verify attestations before putting them into the pool
* use existing method
* Validate aggregated ones too
* Revert "Validate aggregated ones too"

This reverts commit a55646d131813c3e65e0539ef5b2e5bda19a5e91.
* Merge branch 'master' of github.com:prysmaticlabs/prysm into verify-all-atts
* Add feature flag
* The remaining shared reference fields with conditional copy on write
* Merge branch 'master' into better-copy-2
* Merge branch 'better-copy-2' of github.com:prysmaticlabs/prysm into verify-all-atts
* gaz
* fix build, put into validate
* lint
* Merge branch 'master' of github.com:prysmaticlabs/prysm into verify-all-atts
* why does goland do this to me
* revert unrelated change
* fix tests
* Update shared/featureconfig/config.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Merge refs/heads/master into verify-all-atts
* Update beacon-chain/blockchain/testing/mock.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* gofmt
2020-02-07 03:21:55 +00:00
terence tsao
5b83dffbe4
Use proto array forkchoice as default (#4778)
* Starting

* Removing feature flag

* Minor touchups service.go

* Conflict

* Started fixing test

* Init fork choice store for tests
2020-02-06 13:03:26 -06:00
Preston Van Loon
bfa103317e
Disable forkchoice pre-processing of attestations (#4774)
* Disable forkchoice pre-processing of attestations
* space
* gaz
2020-02-06 07:53:08 +00:00
Nishant Das
fb7a75d2c3
Release Proposer Index Cache (#4717)
* release cache

* gaz

* fix all tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-03 10:23:04 -06:00
Ivan Martinez
c80ffc640f
Fix flag bug (#4690) 2020-01-29 16:16:37 -06:00
Ivan Martinez
9149c2e4f4 Replace no-genesis-delay with custom-genesis-delay (#4678)
* Change NoGenesisDelay to CustomGenesisDelay
* Implement flag
* gazelle
* Fix docs
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into custom-genesis-delay
* Fix
* Gazelle
* Add case to fix bad math
* Merge branch 'master' into custom-genesis-delay
2020-01-28 21:07:43 +00:00
Ivan Martinez
127f05d531
Allow easy plugin of featureflags into E2E (#4659)
* Enable easy plugin of featureflags into E2E

* Gazelle

* Fix text

* Fix whitespace
2020-01-26 21:42:10 -05:00
terence tsao
5eece9a507 Integrate proto array forkchoice to run time (#4649)
* Run time

* Fixed pruning

* Fixed test

* Fixed test

* Process attestations during init sync

* Raul's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-25 14:22:25 -06:00
Ivan Martinez
ed3ab828a1
Implement attester protection into validator client (#4598)
* Add flag for attester protection

* Remove flags

* Add attestation history DB functions to validator client

* Fix comments

* Update interface to new funcs

* Fix test

* Add flags

* Implement most of attester protection

* Fix tests

* Add test for pruning

* Add more test cases for prunes

* Remove todo comment

* Fix comments

* Rename functions

* Fix logs

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-01-23 00:52:01 -05:00
Raul Jordan
eb5e814eb4 Disable Fork Choice Feature Flag (#4574) 2020-01-20 17:45:37 -08:00
Preston Van Loon
aae27749d4 Release save deposits flag (#4581)
* release save deposits flag
* Merge refs/heads/master into release-save-deposits
2020-01-18 20:25:29 +00:00
Raul Jordan
ed529965af Fix Up SSZ Cache Branch Recomputation (#4558)
* e2e ssz cache busting
* use ssz cache for e2e
* caching ensure
* fix up the cache even more
* gazelle
* formatting
* formatting
* add back cache for val registry
* sync
* fix up commented item
* add attestations
* Merge branch 'master' into e2e-ssz-cache
* Merge branch 'master' into e2e-ssz-cache
* Merge branch 'e2e-ssz-cache' of github.com:prysmaticlabs/prysm into e2e-ssz-cache
* formatting
* gaz
* Merge branch 'master' into e2e-ssz-cache
* resolve comments
* Merge branch 'master' into e2e-ssz-cache
* naming of test
* Merge refs/heads/master into e2e-ssz-cache
* Merge refs/heads/master into e2e-ssz-cache
2020-01-16 21:40:09 +00:00
Ivan Martinez
452cadc286
Cleanup featureconfig, make naming consistent (#4557)
* Cleanup featureconfig, make naming consistent

* Fix rename

* Change test package name
2020-01-15 18:41:40 -05:00
Ivan Martinez
2a4c89827d
Add double proposal protection to validator client (#4460)
* Add double proposal protection to client

* Add mock test cases for past proposals, and after pruning

* Fix error

* Add force clear db to val in e2e

* Fix val tests

* Move saving proposal history to after broadcasting block

* Add featureflag

* Goimports

* Unexport flag

* Add flag to tests

* gazelle

* Move conditionals
2020-01-15 17:23:39 -05:00
terence tsao
884d2a159d Cache proposer indices (#4528)
* Precompute and plug it into run time
* Run time fix
* Testing
* More logging to debug
* More logging to debug
* This should fix it
* Clean up debug logs
* Removed last bit of debug log
* Comments
* Tests
* Merge branch 'master' into cache-proposer-index
* Gaz
* Merge branch 'cache-proposer-index' of git+ssh://github.com/prysmaticlabs/prysm into cache-proposer-index
* Merge refs/heads/master into cache-proposer-index
* Merge refs/heads/master into cache-proposer-index
* Merge refs/heads/master into cache-proposer-index
2020-01-14 04:08:32 +00:00
terence tsao
a8edfa42cc Cache filtered block tree (#4515)
* Cache filtered block tree
* Merge refs/heads/master into cache-filtered-tree
* Merge refs/heads/master into cache-filtered-tree
* Add locks
* Merge branch 'cache-filtered-tree' of git+ssh://github.com/prysmaticlabs/prysm into cache-filtered-tree
* Confligt
* Merge refs/heads/master into cache-filtered-tree
* Merge refs/heads/master into cache-filtered-tree
* Update shared/featureconfig/flags.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Rlock
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm into cache-filtered-tree
* Merge branch 'cache-filtered-tree' of git+ssh://github.com/prysmaticlabs/prysm into cache-filtered-tree
2020-01-13 04:12:50 +00:00
Preston Van Loon
1cb0edac00 PR #4502 take two (#4522)
* PR #4502 take two
2020-01-12 23:46:14 +00:00
Raul Jordan
88bce4af34 Revert "Deprecate new cache feature flag" (#4520)
* Revert "Deprecate new cache feature flag (#4502)"

This reverts commit 5287ddc114.
2020-01-12 23:15:08 +00:00
terence tsao
5287ddc114 Deprecate new cache feature flag (#4502)
* Starting to deprecate new cache flag
* All tests passing
* Fixed minimal test
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm into deprecate-flag
* Fixed mainnet spec test
* Fixed a typo
* Merge refs/heads/master into deprecate-flag
* Merge refs/heads/master into deprecate-flag
* Merge refs/heads/master into deprecate-flag
2020-01-12 22:46:30 +00:00
shayzluf
39b2570af5 Beacon node slasher client infrastructure (#4111) 2020-01-08 20:49:32 -08:00
terence tsao
6d2a2ebadf Update run time to v0.9.3 (#4154)
* Remove custody (#3986)

* Update proto fields

* Updated block operations

* Fixed all block operation tests

* Fixed tests part 1

* Fixed tests part 1

* All tests pass

* Clean up

* Skip spec test

* Fixed ssz test

* Skip ssz test

* Skip mainnet tests

* Update beacon-chain/operations/attestation.go

* Update beacon-chain/operations/attestation.go
* Decoy flip flop check (#3987)
* Bounce attack check (#3989)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Update beacon-chain/blockchain/forkchoice/process_block.go
* Conflict
* Unskip mainnet spec tests (#3998)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update beacon-chain/core/blocks/block_operations.go
* Unskip minimal spec tests (#3999)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update workspace

* Unskip all minimal spec tests

* Update workspace for general test
* Unskip test (#4001)
* Update minimal seconds per slot to 6 (#3978)
* Bounce attack tests (#3993)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Slots since epoch starts tests

* Update justified checkpt tests

* Conflict

* Fixed logic

* Update process_block.go

* Use helper
* Conflict
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.1
* Conflict
* Fixed failed tests
* Lower MinGenesisActiveValidatorCount to 16384 (#4100)
* Fork choice beacon block checks (#4107)

* Prevent future blocks check and test

* Removed old code
* Update aggregation proto (#4121)

* Update def
* Update spec test
* Conflict
* Update workspace
* patch
* Resolve conflict
* Patch
* Change workspace
* Update ethereumapis to a forked branch at commit 6eb1193e47
* Fixed all the tests
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into conflict
* fix patch
* Need to regenerate test data
* Merge branch 'master' into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Enable snappy compression for all (#4157)

* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* Validate aggregate and proof subscriber (#4159)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Conflict
* Update workspace
* Conflict
* Conflict
* Conflict
* Merge branch 'master' into v0.9.2
* Merge branch 'master' into v0.9.2
* Conflict
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Remove migrate to snappy  (#4205)
* Feature flag: Deprecate --prune-states, release to all (#4204)

* Deprecated prune-states, release to all

* imports

* remote unused import

* remove unused import

* Rm prune state test

* gaz
* Refactoring for dynamic pubsub subscriptions for non-aggregated attestations (#4189)

* checkpoint progress

* chkpt

* checkpoint progress

* put pipeline in its own file

* remove unused imports

* add test, it's failing though

* fix test

* remove head state issue

* add clear db flag to e2e

* add some more error handling, debug logging

* skip processing if chain has not started

* fix test

* wrap in go routine to see if anything breaks

* remove duplicated topic

* Add a regression test. Thanks @nisdas for finding the original problem. May it never happen again *fingers crossed*

* Comments

* gofmt

* comment out with TODO
* Sync with master
* Sync with master
* RPC servers use attestation pool (#4223)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Refactor RPC to Fully Utilize Ethereum APIs (#4243)

* include attester as a file in the validator server

* remove old proposer server impl

* include new patch and properly sync changes

* align with public pbs

* ensure matches rpc def

* fix up status tests

* resolve all broken test files in the validator rpc package

* gazelle include

* fix up the duties implementation

* fixed up all get duties functions

* all tests pass

* utilize new ethereum apis

* amend validator client to use the new beacon node validator rpc client

* fix up most of validator items

* added in mock

* fix up test

* readd test

* add chain serv mock

* fix a few more validator methods

* all validator tests passingggg

* fix broken test

* resolve even more broken tests

* all tests passsssss

* fix lint

* try PR

* fix up test

* resolve broken other tests
* Sync with master
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Aggregate and proof subscriber (#4240)

* Added subscribers

* Fixed conflict

* Tests

* fix up patch

* Use upstream pb

* include latest patch

* Fmt

* Save state before head block
* skip tests (#4275)
* Delete block attestations from the pool (#4241)

* Added subscribers
* Clean up
* Fixed conflict
* Delete atts in pool in validate pipeline
* Moved it to subscriber
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into use-att-pool-3
* Test
* Fixed test
* Initial work on voluntary exit (#4207)

* Initial implementation of voluntary exit: RPC call

* Update for recent merges

* Break out validation logic for voluntary exits to core module

* RequestExit -> ProposeExit

* Decrease exit package visibility

* Move to operation feed

* Wrap errors
* Fix critical proposer selection bug #4259 (#4265)

* fix critical proposer selection bug #4259

* gofmt

* add 1 more validator to make it 5

* more tests

* Fixed archivedProposerIndex

* Fixed TestFilterAttestation_OK

* Refactor ComputeProposerIndex, add regression test for potential out of range panic

* handle case of nil validator

* Update validators_test.go
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Leftover merge files, oops
* gaz
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* Fixes Duplicate Validator Bug (#4322)

* Update dict

* Test helper

* Regression test

* Comment

* Reset test cache
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fixes after PR #4328
* Complete attestation pool for run time (#4286)

* Added subscribers

* Fixed conflict

* Delete atts in pool in validate pipeline

* Moved it to subscriber

* Test

* Fixed test

* New curl for forkchoice attestations

* Starting att pool service for fork choice

* Update pool interface

* Update pool interface

* Update sync and node

* Lint

* Gazelle

* Updated servers, filled in missing functionalities

* RPC working with 1 beacon node 64 validators

* Started writing tests. Yay

* Test to aggregate and save multiple fork choice atts

* Tests for BatchAttestations for fork choice

* Fixed exisiting tests

* Minor fixes

* Fmt

* Added batch saves

* Lint

* Mo tests yay

* Delete test

* Fmt

* Update interval

* Fixed aggregation broadcast

* Clean up based on design review comment

* Fixed setupBeaconChain

* Raul's feedback. s/error/err
* resolve conflicts
* Merge branch 'v0.9.2' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Removed old protos and fixed tests (#4336)
* Merge refs/heads/master into v0.9.2
* Disallow duplicated indices and test (#4339)
* Explicit use of GENESIS_SLOT in fork choice (#4343)
* Update from 2 to 3 (#4345)
* Remove verify unaggregated attestation when aggregating (#4347)
* use slot ticker instead of run every (#4348)
* Add context check for unbounded loop work (#4346)
* Revert "Explicit use of GENESIS_SLOT in fork choice (#4343)" (#4349)

This reverts commit d3f6753c77f8f733563d00ab649c5159b2c2926f.
* Refactor Powchain Service (#4306)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* fix lint

* raul's comments

* use one method

* fix test

* raul's comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Ensure best better-justification is stored for fork choice (#4342)

* Ensure best better-justification is stored. Minor refactor
* Tests
* Merge refs/heads/v0.9.2 into better-best-justified
* Merge refs/heads/v0.9.2 into better-best-justified
* Ensure that epoch of attestation slot matches the target epoch (#4341)

* Disallow duplicated indices and test
* Add slot to target epoch check to on_attestation
* Add slot to target epoch check to process_attestation
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into no-dup-att-indices
* Fixed TestProcessAttestations_PrevEpochFFGDataMismatches
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Update beacon-chain/blockchain/forkchoice/process_attestation_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Filter viable branches in fork choice (#4355)
* Only activate upon finality (#4359)

* Updated functions
* Tests
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into queue-fix-on-finality
* Comment
* Merge refs/heads/v0.9.2 into queue-fix-on-finality
* Fixed failing test from 4359 (#4360)

* Fixed
* Skip registry spec tests
* Wait for state to be initialized at least once before running slot ticker based on genesis time (#4364)
* Sync with master
* Fix checkpoint root to  use genesis block root (#4368)
* Return an error on nil head state in fork choice (#4369)

* Return error if nil head state

* Fixed tests. Saved childen blocks state

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Update metrics every epoch (#4367)
* return empty slice if state is nil (#4365)
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Pubsub: Broadcast attestations to committee based subnets (#4316)

* Working on un-aggregated pubsub topics

* update subscriber to call pool

* checkpointing

* fix

* untested message validation

* minor fixes

* rename slotsSinceGenesis to slotsSince

* some progress on a unit test, subscribe is not being called still...

* dont change topic

* need to set the data on the message

* restore topic

* fixes

* some helpful parameter changes for mainnet operations

* lint

* Terence feedback

* unskip e2e

* Unit test for validate committee index beacon attestation

* PR feedbacK

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into resolveConflicts
* remove condition
* Remove unused operation pool (#4361)
* Merge refs/heads/master into v0.9.2
* Aggregate attestations periodically  (#4376)
* Persist ETH1 Data to Disk (#4329)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* adding in new proto message

* remove outdated vars

* add new changes

* remove latest eth1data

* continue refactoring

* finally works

* lint

* fix test

* fix all tests

* fix all tests again

* fix build

* change back

* add full eth1 test

* fix logs and test

* add constant

* changes

* fix bug

* lint

* fix another bug

* change back

* Apply suggestions from code review

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fixed VerifyIndexedAttestation (#4382)
* rm signing root (#4381)

* rm signing root

* Fixed VerifyIndexedAttestation

* Check proposer slashed status inside ProcessBlockHeaderNoVerify

* Fixed TestUpdateJustified_CouldUpdateBest

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove Redundant Trie Generation (#4383)

* remove trie generation
* remove deposit hashes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* fix build
* Conflict
* Implement StreamAttestations RPC Endpoint (#4390)

* started attestation stream

* stream attestations test

* on slot tick test passing

* imports

* gaz

* Update beacon-chain/rpc/beacon/attestations_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

Co-authored-by: shayzluf <thezluf@gmail.com>
* Fixed goimport (#4394)
* Use custom stateutil ssz for ssz HTR spec tests (#4396)

* Use custom stateutil ssz for ssz HTR spec tests

* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* set mainnet to be the default for build and run (#4398)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* gracefully handle deduplicated registration of topic validators (#4399)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* SSZ: temporarily disable roots cache until cache issues can be resolved (#4407)

* temporarily disable roots cache until cache issues can be resolved

* Also use custom ssz for spectests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove process block attestations as separate routine (#4408)

* Removed old save/process block atts

* Fixed tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save Deposit Cache to Disk (#4384)

* change to protos

* fix build

* glue everything together

* fix test

* raul's review

* preston's comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix activation queue sorting (#4409)

* Removed old save/process block atts

* Fixed tests

* Proper sorting by eligibility epoch then by indices

* Deleted old colde
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' into v0.9.2
* Merge refs/heads/master into v0.9.2
* stop recursive lookup if context is cancelled (#4420)
* Fix proposal bug (#4419)
* Add Pending Deposits Safely (#4422)

* safely prune cache

* use proper method

* preston's,terence's reviews and comments

* revert change to build files

* use as feature config instead
* Release custom state ssz (#4421)

* Release custom state ssz, change all HTR of beacon state to use custom method

* typo

* use mainnet config

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Update initial sync save justified to align with v0.9.3 (#4432)
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fix build
* don't blacklist on pubsub (#4435)
* Fix Flakey Slot Ticker Test (#4434)

* use interface instead for the slot ticker

* fixed up flakey tests

* add gen time

* get duties comment

* fix lifecycle test

* more fixes
* Configurable min genesis delay (#4437)

* Configurable min genesis delay based on https://github.com/ethereum/eth2.0-specs/pull/1557

* remove feature flag for genesis delay

* fix

* demo config feedback
* patch readme
* save keys unencrypted for validators (#4439)
* Add new demo configuration targeting mainnet scale (#4397)

* Add new demo configuration targeting mainnet, with 1/10th of the deposit value

* reduce quotant by 1/10th. Use 1/10th mainnet values

* only change the inactivity quotant

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save justified checkpoint state (#4433)

* Save justified checkpoint state

* Lint

* Feedback

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update shared/testutil/deposits.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update proto/testing/ssz_regression_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/epoch/epoch_processing.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/kv/forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/proposer.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/prepare_forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/aggregator/server.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/cache/depositcache/pending_deposits.go
* Update beacon-chain/cache/depositcache/pending_deposits_test.go
* Update beacon-chain/rpc/validator/proposer.go
* Merge refs/heads/master into v0.9.2
* Fix e2e genesis delay issues (#4442)

* fix e2e genesis delay issues

* register flag

* typo

* Update shared/featureconfig/config.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* skip demo e2e

* fix validator

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Batch Eth1 RPC Calls (#4392)

* add new methods

* get it working

* optimize past deposit logs processing

* revert change

* fix all tests

* use mock

* lint

* lint

* check for nil

* stop panics

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Terence's Review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-01-07 18:47:39 +00:00
Preston Van Loon
9052620453 Release feature --fast-assignments (#4416)
* Deprecated --fast-assignments
* gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into release-fast-assignments
2020-01-06 02:57:52 +00:00
Preston Van Loon
0174397f6e Release --enable-bls-pubkey-cache (#4417)
* Release bls pubkey-cache
2020-01-05 20:08:49 +00:00
Preston Van Loon
2c28e4e7a3 Improvements to Committee Assignments for multiple key requests (#4294)
* Add committees helper, benchmark, results show 62ms for 8k validators which was previously 4 minutes
* Add regression test with same data
* fix epoch conversion
* lint
* undo and lint
* Merge branch 'master' of github.com:prysmaticlabs/prysm into zoom-zoom-assignments
* remove validaotr index span
* fix comment, add test to test against spec definition method for consistency.
* Deprecate CommitteeAssignment, delete unused reference to CommitteeAssignment
* Merge branch 'master' of github.com:prysmaticlabs/prysm into zoom-zoom-assignments
* remove new line
* make test be more complicated with validators activated in an epoch transition
* add feature flag for fast-assignments
* Merge branch 'master' of github.com:prysmaticlabs/prysm into zoom-zoom-assignments
* gaz, gofmt, add deprecated code back
* Update beacon-chain/core/helpers/committee.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge refs/heads/master into zoom-zoom-assignments
* Merge refs/heads/master into zoom-zoom-assignments
* Merge refs/heads/master into zoom-zoom-assignments
2019-12-17 03:05:26 +00:00
terence tsao
a5881f924f Deprecate active count and committee cache flags (#4276)
* Deprecate active count and committee cache flags
* Merge branch 'master' into deprecate-flags
2019-12-13 14:00:29 +00:00
Preston Van Loon
c0b3767757 remove old cache for active indices. this is not used in production and will soon be replaced (#4264) 2019-12-11 15:48:48 -06:00
Nishant Das
667466020e Change All Caches To Ristretto (#4208)
* new caches
* goimports, gaz
* fix all tests
* Merge branch 'swapP2PCaches' of https://github.com/prysmaticlabs/geth-sharding into swapP2PCaches
* remove from bls
* remove ccache
* fix handshake
* Merge branch 'master' into swapP2PCaches
* gofmt
* Merge branch 'master' into swapP2PCaches
2019-12-06 20:06:37 +00:00
Preston Van Loon
d3375d98a8 Kafka exporter (#3840)
* abstract db interface, kafka build, work in progress
* checkpoint
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* feature flag
* move passthrough
* flag change
* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* missing db methods
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* fix interface
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* try using cmake built from source
* lint godocs
* lint godocs
* lint godocs
* Update BUILD.bazel
* Merge branch 'master' into es-exporter
* Merge branch 'master' into es-exporter
* Merge branch 'master' into es-exporter
* Merge branch 'master' of github.com:prysmaticlabs/prysm into es-exporter
* gaz
2019-12-06 02:05:58 +00:00
terence tsao
9d4c7cb4f7 Use cache state during init sync (#4199)
* Initial sync state cache
* Gaz
* Gaz
* Don't save head root
* Fix config validator
* Uncomment save head
* Merge branch 'master' into initial-sync-no-verify
* Minor refactor
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Merge branch 'master' into initial-sync-no-verify
* Tests
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Merge branch 'master' into initial-sync-no-verify
* Add lock
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Tests
* Removed save head
* One more test
* Merge branch 'master' into initial-sync-no-verify
* Raul's feedback
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Comment
* Gazelle
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* revert
* Update beacon-chain/blockchain/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Fixed test
* Fixed feature flag
* Merge branch 'master' into initial-sync-no-verify
* Fixed cache gensis state test
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
2019-12-06 00:49:19 +00:00
Raul Jordan
9e5cc81340 Implement Prysm-Specific HashTreeRootState (#4077)
* new ssz hash tree root
* Merge branch 'master' into new-ssz-state
* better comments on func
* add errors instead of panic in state
* utilize errors wrap everywhere
* include bench
* added bench info
* equality test
* dup
* gaz
* use new hash tree root in state transition
* fix build
* separate test package
* three targets failign
* single target fails
* please test targets...pass for me
* revert
* Merge branch 'master' into new-ssz-state
* rev
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
* broken build
* Merge branch 'master' into new-ssz-state
* gaz
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
* ssz workspace
* master ssz
* Merge branch 'master' into new-ssz-state
* resolve conf
* resolve some conflicts and fix up broken file
* fix up build file issues and sync
* eth1 data votes included
* further abstractions, simplifications
* Merge branch 'master' into new-ssz-state
* gaz
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
* feature flag gating
* add field count test
* Merge branch 'master' into new-ssz-state
* resolving ivan feedback
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
* gaz
* Merge branch 'master' into new-ssz-state
* addressed
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
2019-12-04 19:20:33 +00:00
Preston Van Loon
32245a9062 Deprecate --init-sync-no-verify, make it the default (#4179)
* deprecated --init-sync-no-verify, make it the default
* Merge branch 'master' into deprecate-init-sync-verify-flag
* add more flag info
* Merge branch 'deprecate-init-sync-verify-flag' of github.com:prysmaticlabs/prysm into deprecate-init-sync-verify-flag
* gofmt
* Merge refs/heads/master into deprecate-init-sync-verify-flag
* Merge refs/heads/master into deprecate-init-sync-verify-flag
2019-12-03 18:46:04 +00:00
terence tsao
7495961d6b Prune boundary state (#4139)
* Delete epoch boundary slot of last finalized epoch
* Case to cover start slot is skipped
* Test
* Feature flag
* feature gate the new functionality only
* Update DB for migration
* Test
* Fmt
* Fixed test
* Gazelle
2019-11-28 23:05:47 +00:00
Preston Van Loon
e203f66fe0 DB Improvements: Snappy compression, remove some unnecessary batch / goroutines (#4125)
* do not use batch for SaveAttestations
* use snappy compression
* Encode / decode everything with snappy
* Add snappy migration path
* batch is probably fine...
* fix test
* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into remove-batch-attestations
* add sanity check
* remove that thing
* gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into remove-batch-attestations
2019-11-27 06:32:56 +00:00
Preston Van Loon
f717c5d852 Release --prune-finalized-states to all (#4082)
* Release --prune-finalized-states to all
* Merge branch 'master' into deprecate-ff-prune-finalized-states
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-prune-finalized-states
2019-11-22 20:29:46 +00:00
Preston Van Loon
0cec0ee6c3 Release --optimize-process-epoch to all (#4080)
* Release optimize-proces-epoch to all
* Merge branch 'master' into deprecate-ff-optimize-process-epoch
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
2019-11-22 09:40:53 +00:00
Preston Van Loon
bdbd0aaeb8 Deprecate feature flag --scatter (#4079)
* deprecate --scatter. issue #4031
* forgot one for #4061
* use deprecatedUsage
* hidden
* Merge branch 'master' into deprecate-ff-scatter
* Merge branch 'master' into deprecate-ff-scatter
2019-11-22 04:08:49 +00:00
Ivan Martinez
2d7802c637 Rename featureconfig.Flag to Flags (#4063) 2019-11-19 21:03:00 -06:00
Nishant Das
6e90931837
Handle Missing Blocks (#3880)
* add fixes

* more changes

* simplify the solution

* add shuffle

* remove prom metric

* add to mock

* go doc

* use non default source

* add test

* terence's review

* all tests passing
2019-11-05 10:37:18 +08:00
Preston Van Loon
7d1633230d Feature gate all caches (#3894) 2019-11-03 17:20:08 -08:00
Jim McDonald
485fc538c3 Utility to parallelise functions over arrays (#3813)
* Initial version of scatter

* Add mutex for scatter batch

* Provide mutex to worker processes

* Embed mputil

* Add notes on scatter

* Tidy-up

* Add mutex test

* Simplify scatter for users

* Tidy-ups/code coverage

* Gazelle update

* Add benchmark

* Mutex to RWMutex

* Add test against internal functions

* Bazel fixes

* Fix benchmark

* Benchmark values to constants

* Update shared/mputil/scatter.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Update shared/mputil/scatter.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Moved struct to top of file

* Add featureconfig for scatter

* Gate scatter behind feature flag

* Lint fixes

* fmt
2019-11-03 16:25:52 -05:00
Preston Van Loon
cc16a10a33
Remove --enable-finalized-block-root-index (#3887)
* Deprecated do not use

* remove finalized block indexing feature flag

* remove flag some more

* dont rollback a managed tx

* require blocks to be in a chain in tests
2019-11-02 14:57:35 -04:00
Nishant Das
49e0ddf861 Cache Skipped Slots (#3879)
* use LRU cache on skip slots

* add a metric

* gaz

* more metrics

* temporary log

* added more logging

* fix

* only update, if its higher than the cache

* only update, if its higher than the cache

* increase the cache size, just in case

* split cache related stuff into its own file

* add feature flag

* clean up

* add new test

* make test proper

* godoc

* preston's review
2019-10-31 13:38:51 -04:00
Preston Van Loon
2b444ea954 Add feature flag to enable shuffled index cache (#3893)
* add feature flag to enable shuffled index cache

* gaz'
2019-10-30 10:26:05 +08:00
Preston Van Loon
749f4b776b
Add flag deprecation (#3881)
* add deprecated flags

* move flags to their own list

* gofmt

* more warning
2019-10-29 11:30:14 -04:00
Preston Van Loon
0481eb4872
Track finalized block root index and filter in blocks by range (#3873)
* checkpoint progress

* feature flag and test

* return true if feature flag is not on

* add lint msg

* rollback tx if failure

* Revert validator/BUILD.bazel
2019-10-29 11:14:17 -04:00
Nishant Das
f1627c0b67 Make No Genesis Delay the Default (#3878) 2019-10-29 07:31:10 -07:00
Preston Van Loon
acf11262de Add feature flag for state pruning (#3845)
* Add feature flag for state pruning

* gaz
2019-10-23 23:45:52 -07:00
terence tsao
a41ac6b498 Implement ProcessEpochPrecompute (#3798)
* Implemented ProcessSlashingsPrecompute

* Tests for ProcessSlashingsPrecompute

* Gaz

* Lint

* Feature flags

* Hook in ProcessEpochPrecompute

* Hook it to spec test

* Seperate test into it's own package to avoid circular dependency

* Lint

* Gazelle

* Preston's feedback

* Nishant's feedback
2019-10-18 16:38:47 +08:00
terence tsao
750bc83369
Clean up feature flag namings (#3715) 2019-10-07 14:11:49 +09:00
Preston Van Loon
4bee60826d
add flag to enable BLS pubkey cache (#3699) 2019-10-03 08:26:15 -07:00
Preston Van Loon
7a04ff6368 Add database API for creating backups (#3694)
* Save db backup

* Fix DB backup method

* Add backup db webhook

* gaz

* if err != nil

* more verbose filename

* Don't obliterate everything :)
2019-10-03 17:29:49 +08:00
terence tsao
d02e73c5fe
Feature flag for new caching scheme (#3619) 2019-09-27 13:14:22 -07:00
Raul Jordan
24df2d3e44
Skip BLS With a Feature Flag at Runtime (#3618)
* skip bls verification with a feature flag at runtime

* gazelle

* more bls mocks
2019-09-27 13:28:43 -05:00
Nishant Das
9b008522b8 Refactor Validator Start Routine (#3594)
* make demo default

* make minimal config a flag

* lint

* initialize config at the start

* gaz

* make main method cleaner

* remove interop.go

* fix test

* lint

* gaz

* Update validator/accounts/interop.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fix docker build

* fix docker build
2019-09-26 13:23:25 -05:00
terence tsao
3ef681e649
Initial sync no verify block (#3586) 2019-09-25 20:32:00 -07:00
terence tsao
fb74dae835
Remove dead cache (#3584) 2019-09-25 18:23:15 -07:00
Nishant Das
0e329fc115 Attestation Server Fix (#3570) 2019-09-24 08:19:37 -07:00
Preston Van Loon
ad47817bcd Only write interop ssz states to disk with flag ON (#3566)
* only write SSZ states to disk with flag on

* lint

* also write blocks
2019-09-23 18:36:12 -05:00
Raul Jordan
9683a83750 Properly Use Demo Config (#3494) 2019-09-16 12:55:30 -07:00
Preston Van Loon
a2685245f2 demo-config flag (#3473)
* democonfig

* 3.2
2019-09-15 14:24:08 -05:00
terence tsao
8c7ef61238 Add back configure validator features (#3456) 2019-09-12 14:02:53 -05:00
terence tsao
91f824fe10
Clean up unused flags (#3449) 2019-09-12 11:48:34 -04:00
terence tsao
14c59b2ff9 Remove deprecated services and --next (#3371)
* Save new validators in DB

* Use info

* Add total validator count

* Fixed tests

* Add new test

* Revert light client config

* Add state metrics back

* Gaz

* Mark old ones as deprecated

* Deprecate not --next services

* Fixed all operation tests

* Fixed node test

* All tests passing locally

* Add deprecated-p2p back, blocked by boostrap-query

* Revert message proto

* delete deprecated DB items

* delete all other instances of old db

* gaz

* cycle rem

* clear db
2019-09-05 11:04:06 -05:00
terence tsao
b91639a32e
Deprecate old block chain service (#3268)
* seperate out block chain services

* Fix blockchain service config

* Gazelle

* Fixed tests
2019-08-21 19:14:24 -06:00
Preston Van Loon
acb20e269c Add flags to support new database, new sync (#3252) 2019-08-21 10:04:00 -06:00
Preston Van Loon
0b8cbd06b6
Add flag for testing new p2p (#3243)
* refactor a bit to select p2p

* lint

* fix build

* fix build

* fix build

* fix build

* fix build
2019-08-19 17:20:56 -04:00