Commit Graph

518 Commits

Author SHA1 Message Date
Jim McDonald
3839f577ec Change database *Index() to use slice (#4466)
* Change database *Index() to use slice

* Remove underscore from helper name
2020-01-09 14:45:05 -08:00
Jim McDonald
fc38a0413e Fix gauge description (#4468) 2020-01-09 11:39:11 -08:00
terence tsao
0dd0e23155
Optimize ListBeaconCommittees to use committees cache (#4464) 2020-01-09 08:28:11 -08:00
Ivan Martinez
b30a7d1e19 Fix typos and inconsistencies (#4453)
* Fix typos and inconsistencies

* igoimports

* Gazelle
2020-01-07 20:36:55 -06:00
Ivan Martinez
0d400faea2 Remove unused parameters and unused code (#4452)
* Remove unused parameters
* Remove unused deposit contract config
2020-01-07 23:45:29 +00:00
terence tsao
89eedd2123 Efficient computation of epoch participation (#4430)
* Remove custody (#3986)

* Update proto fields

* Updated block operations

* Fixed all block operation tests

* Fixed tests part 1

* Fixed tests part 1

* All tests pass

* Clean up

* Skip spec test

* Fixed ssz test

* Skip ssz test

* Skip mainnet tests

* Update beacon-chain/operations/attestation.go

* Update beacon-chain/operations/attestation.go
* Decoy flip flop check (#3987)
* Bounce attack check (#3989)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Update beacon-chain/blockchain/forkchoice/process_block.go
* Conflict
* Unskip mainnet spec tests (#3998)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update beacon-chain/core/blocks/block_operations.go
* Unskip minimal spec tests (#3999)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update workspace

* Unskip all minimal spec tests

* Update workspace for general test
* Unskip test (#4001)
* Update minimal seconds per slot to 6 (#3978)
* Bounce attack tests (#3993)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Slots since epoch starts tests

* Update justified checkpt tests

* Conflict

* Fixed logic

* Update process_block.go

* Use helper
* Conflict
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.1
* Conflict
* Fixed failed tests
* Lower MinGenesisActiveValidatorCount to 16384 (#4100)
* Fork choice beacon block checks (#4107)

* Prevent future blocks check and test

* Removed old code
* Update aggregation proto (#4121)

* Update def
* Update spec test
* Conflict
* Update workspace
* patch
* Resolve conflict
* Patch
* Change workspace
* Update ethereumapis to a forked branch at commit 6eb1193e47
* Fixed all the tests
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into conflict
* fix patch
* Need to regenerate test data
* Merge branch 'master' into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Enable snappy compression for all (#4157)

* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* Validate aggregate and proof subscriber (#4159)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Conflict
* Update workspace
* Conflict
* Conflict
* Conflict
* Merge branch 'master' into v0.9.2
* Merge branch 'master' into v0.9.2
* Conflict
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Remove migrate to snappy  (#4205)
* Feature flag: Deprecate --prune-states, release to all (#4204)

* Deprecated prune-states, release to all

* imports

* remote unused import

* remove unused import

* Rm prune state test

* gaz
* Refactoring for dynamic pubsub subscriptions for non-aggregated attestations (#4189)

* checkpoint progress

* chkpt

* checkpoint progress

* put pipeline in its own file

* remove unused imports

* add test, it's failing though

* fix test

* remove head state issue

* add clear db flag to e2e

* add some more error handling, debug logging

* skip processing if chain has not started

* fix test

* wrap in go routine to see if anything breaks

* remove duplicated topic

* Add a regression test. Thanks @nisdas for finding the original problem. May it never happen again *fingers crossed*

* Comments

* gofmt

* comment out with TODO
* Sync with master
* Sync with master
* RPC servers use attestation pool (#4223)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Refactor RPC to Fully Utilize Ethereum APIs (#4243)

* include attester as a file in the validator server

* remove old proposer server impl

* include new patch and properly sync changes

* align with public pbs

* ensure matches rpc def

* fix up status tests

* resolve all broken test files in the validator rpc package

* gazelle include

* fix up the duties implementation

* fixed up all get duties functions

* all tests pass

* utilize new ethereum apis

* amend validator client to use the new beacon node validator rpc client

* fix up most of validator items

* added in mock

* fix up test

* readd test

* add chain serv mock

* fix a few more validator methods

* all validator tests passingggg

* fix broken test

* resolve even more broken tests

* all tests passsssss

* fix lint

* try PR

* fix up test

* resolve broken other tests
* Sync with master
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Aggregate and proof subscriber (#4240)

* Added subscribers

* Fixed conflict

* Tests

* fix up patch

* Use upstream pb

* include latest patch

* Fmt

* Save state before head block
* skip tests (#4275)
* Delete block attestations from the pool (#4241)

* Added subscribers
* Clean up
* Fixed conflict
* Delete atts in pool in validate pipeline
* Moved it to subscriber
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into use-att-pool-3
* Test
* Fixed test
* Initial work on voluntary exit (#4207)

* Initial implementation of voluntary exit: RPC call

* Update for recent merges

* Break out validation logic for voluntary exits to core module

* RequestExit -> ProposeExit

* Decrease exit package visibility

* Move to operation feed

* Wrap errors
* Fix critical proposer selection bug #4259 (#4265)

* fix critical proposer selection bug #4259

* gofmt

* add 1 more validator to make it 5

* more tests

* Fixed archivedProposerIndex

* Fixed TestFilterAttestation_OK

* Refactor ComputeProposerIndex, add regression test for potential out of range panic

* handle case of nil validator

* Update validators_test.go
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Leftover merge files, oops
* gaz
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* Fixes Duplicate Validator Bug (#4322)

* Update dict

* Test helper

* Regression test

* Comment

* Reset test cache
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fixes after PR #4328
* Complete attestation pool for run time (#4286)

* Added subscribers

* Fixed conflict

* Delete atts in pool in validate pipeline

* Moved it to subscriber

* Test

* Fixed test

* New curl for forkchoice attestations

* Starting att pool service for fork choice

* Update pool interface

* Update pool interface

* Update sync and node

* Lint

* Gazelle

* Updated servers, filled in missing functionalities

* RPC working with 1 beacon node 64 validators

* Started writing tests. Yay

* Test to aggregate and save multiple fork choice atts

* Tests for BatchAttestations for fork choice

* Fixed exisiting tests

* Minor fixes

* Fmt

* Added batch saves

* Lint

* Mo tests yay

* Delete test

* Fmt

* Update interval

* Fixed aggregation broadcast

* Clean up based on design review comment

* Fixed setupBeaconChain

* Raul's feedback. s/error/err
* resolve conflicts
* Merge branch 'v0.9.2' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Removed old protos and fixed tests (#4336)
* Merge refs/heads/master into v0.9.2
* Disallow duplicated indices and test (#4339)
* Explicit use of GENESIS_SLOT in fork choice (#4343)
* Update from 2 to 3 (#4345)
* Remove verify unaggregated attestation when aggregating (#4347)
* use slot ticker instead of run every (#4348)
* Add context check for unbounded loop work (#4346)
* Revert "Explicit use of GENESIS_SLOT in fork choice (#4343)" (#4349)

This reverts commit d3f6753c77f8f733563d00ab649c5159b2c2926f.
* Refactor Powchain Service (#4306)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* fix lint

* raul's comments

* use one method

* fix test

* raul's comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Ensure best better-justification is stored for fork choice (#4342)

* Ensure best better-justification is stored. Minor refactor
* Tests
* Merge refs/heads/v0.9.2 into better-best-justified
* Merge refs/heads/v0.9.2 into better-best-justified
* Ensure that epoch of attestation slot matches the target epoch (#4341)

* Disallow duplicated indices and test
* Add slot to target epoch check to on_attestation
* Add slot to target epoch check to process_attestation
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into no-dup-att-indices
* Fixed TestProcessAttestations_PrevEpochFFGDataMismatches
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Update beacon-chain/blockchain/forkchoice/process_attestation_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Filter viable branches in fork choice (#4355)
* Only activate upon finality (#4359)

* Updated functions
* Tests
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into queue-fix-on-finality
* Comment
* Merge refs/heads/v0.9.2 into queue-fix-on-finality
* Fixed failing test from 4359 (#4360)

* Fixed
* Skip registry spec tests
* Wait for state to be initialized at least once before running slot ticker based on genesis time (#4364)
* Sync with master
* Fix checkpoint root to  use genesis block root (#4368)
* Return an error on nil head state in fork choice (#4369)

* Return error if nil head state

* Fixed tests. Saved childen blocks state

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Update metrics every epoch (#4367)
* return empty slice if state is nil (#4365)
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Pubsub: Broadcast attestations to committee based subnets (#4316)

* Working on un-aggregated pubsub topics

* update subscriber to call pool

* checkpointing

* fix

* untested message validation

* minor fixes

* rename slotsSinceGenesis to slotsSince

* some progress on a unit test, subscribe is not being called still...

* dont change topic

* need to set the data on the message

* restore topic

* fixes

* some helpful parameter changes for mainnet operations

* lint

* Terence feedback

* unskip e2e

* Unit test for validate committee index beacon attestation

* PR feedbacK

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into resolveConflicts
* remove condition
* Remove unused operation pool (#4361)
* Merge refs/heads/master into v0.9.2
* Aggregate attestations periodically  (#4376)
* Persist ETH1 Data to Disk (#4329)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* adding in new proto message

* remove outdated vars

* add new changes

* remove latest eth1data

* continue refactoring

* finally works

* lint

* fix test

* fix all tests

* fix all tests again

* fix build

* change back

* add full eth1 test

* fix logs and test

* add constant

* changes

* fix bug

* lint

* fix another bug

* change back

* Apply suggestions from code review

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fixed VerifyIndexedAttestation (#4382)
* rm signing root (#4381)

* rm signing root

* Fixed VerifyIndexedAttestation

* Check proposer slashed status inside ProcessBlockHeaderNoVerify

* Fixed TestUpdateJustified_CouldUpdateBest

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove Redundant Trie Generation (#4383)

* remove trie generation
* remove deposit hashes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* fix build
* Conflict
* Implement StreamAttestations RPC Endpoint (#4390)

* started attestation stream

* stream attestations test

* on slot tick test passing

* imports

* gaz

* Update beacon-chain/rpc/beacon/attestations_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

Co-authored-by: shayzluf <thezluf@gmail.com>
* Fixed goimport (#4394)
* Use custom stateutil ssz for ssz HTR spec tests (#4396)

* Use custom stateutil ssz for ssz HTR spec tests

* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* set mainnet to be the default for build and run (#4398)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* gracefully handle deduplicated registration of topic validators (#4399)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* SSZ: temporarily disable roots cache until cache issues can be resolved (#4407)

* temporarily disable roots cache until cache issues can be resolved

* Also use custom ssz for spectests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove process block attestations as separate routine (#4408)

* Removed old save/process block atts

* Fixed tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save Deposit Cache to Disk (#4384)

* change to protos

* fix build

* glue everything together

* fix test

* raul's review

* preston's comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix activation queue sorting (#4409)

* Removed old save/process block atts

* Fixed tests

* Proper sorting by eligibility epoch then by indices

* Deleted old colde
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' into v0.9.2
* Merge refs/heads/master into v0.9.2
* stop recursive lookup if context is cancelled (#4420)
* Fix proposal bug (#4419)
* Add Pending Deposits Safely (#4422)

* safely prune cache

* use proper method

* preston's,terence's reviews and comments

* revert change to build files

* use as feature config instead
* Release custom state ssz (#4421)

* Release custom state ssz, change all HTR of beacon state to use custom method

* typo

* use mainnet config

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Define framework
* Use participation fetcher
* Build
* Fixed all tests
* Lint
* Update initial sync save justified to align with v0.9.3 (#4432)
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fix build
* don't blacklist on pubsub (#4435)
* Fix Flakey Slot Ticker Test (#4434)

* use interface instead for the slot ticker

* fixed up flakey tests

* add gen time

* get duties comment

* fix lifecycle test

* more fixes
* Fixed rest of the test
* Pass in correct chain service
* Pass in another chain service
* Run time
* Configurable min genesis delay (#4437)

* Configurable min genesis delay based on https://github.com/ethereum/eth2.0-specs/pull/1557

* remove feature flag for genesis delay

* fix

* demo config feedback
* Current -> Prev
* Tests
* patch readme
* save keys unencrypted for validators (#4439)
* Add new demo configuration targeting mainnet scale (#4397)

* Add new demo configuration targeting mainnet, with 1/10th of the deposit value

* reduce quotant by 1/10th. Use 1/10th mainnet values

* only change the inactivity quotant

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save justified checkpoint state (#4433)

* Save justified checkpoint state

* Lint

* Feedback

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update shared/testutil/deposits.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update proto/testing/ssz_regression_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/epoch/epoch_processing.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/kv/forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/proposer.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/prepare_forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/aggregator/server.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/cache/depositcache/pending_deposits.go
* Update beacon-chain/cache/depositcache/pending_deposits_test.go
* Update beacon-chain/rpc/validator/proposer.go
* Merge refs/heads/master into v0.9.2
* Update test
* Conflict
* Update beacon-chain/blockchain/chain_info.go
* Conflict
* Merge branch 'efficient-participation' of git+ssh://github.com/prysmaticlabs/prysm into efficient-participation
* Merge refs/heads/master into efficient-participation
2020-01-07 19:28:25 +00:00
terence tsao
6d2a2ebadf Update run time to v0.9.3 (#4154)
* Remove custody (#3986)

* Update proto fields

* Updated block operations

* Fixed all block operation tests

* Fixed tests part 1

* Fixed tests part 1

* All tests pass

* Clean up

* Skip spec test

* Fixed ssz test

* Skip ssz test

* Skip mainnet tests

* Update beacon-chain/operations/attestation.go

* Update beacon-chain/operations/attestation.go
* Decoy flip flop check (#3987)
* Bounce attack check (#3989)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Update beacon-chain/blockchain/forkchoice/process_block.go
* Conflict
* Unskip mainnet spec tests (#3998)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update beacon-chain/core/blocks/block_operations.go
* Unskip minimal spec tests (#3999)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update workspace

* Unskip all minimal spec tests

* Update workspace for general test
* Unskip test (#4001)
* Update minimal seconds per slot to 6 (#3978)
* Bounce attack tests (#3993)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Slots since epoch starts tests

* Update justified checkpt tests

* Conflict

* Fixed logic

* Update process_block.go

* Use helper
* Conflict
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.1
* Conflict
* Fixed failed tests
* Lower MinGenesisActiveValidatorCount to 16384 (#4100)
* Fork choice beacon block checks (#4107)

* Prevent future blocks check and test

* Removed old code
* Update aggregation proto (#4121)

* Update def
* Update spec test
* Conflict
* Update workspace
* patch
* Resolve conflict
* Patch
* Change workspace
* Update ethereumapis to a forked branch at commit 6eb1193e47
* Fixed all the tests
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into conflict
* fix patch
* Need to regenerate test data
* Merge branch 'master' into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Enable snappy compression for all (#4157)

* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* Validate aggregate and proof subscriber (#4159)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Conflict
* Update workspace
* Conflict
* Conflict
* Conflict
* Merge branch 'master' into v0.9.2
* Merge branch 'master' into v0.9.2
* Conflict
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Remove migrate to snappy  (#4205)
* Feature flag: Deprecate --prune-states, release to all (#4204)

* Deprecated prune-states, release to all

* imports

* remote unused import

* remove unused import

* Rm prune state test

* gaz
* Refactoring for dynamic pubsub subscriptions for non-aggregated attestations (#4189)

* checkpoint progress

* chkpt

* checkpoint progress

* put pipeline in its own file

* remove unused imports

* add test, it's failing though

* fix test

* remove head state issue

* add clear db flag to e2e

* add some more error handling, debug logging

* skip processing if chain has not started

* fix test

* wrap in go routine to see if anything breaks

* remove duplicated topic

* Add a regression test. Thanks @nisdas for finding the original problem. May it never happen again *fingers crossed*

* Comments

* gofmt

* comment out with TODO
* Sync with master
* Sync with master
* RPC servers use attestation pool (#4223)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Refactor RPC to Fully Utilize Ethereum APIs (#4243)

* include attester as a file in the validator server

* remove old proposer server impl

* include new patch and properly sync changes

* align with public pbs

* ensure matches rpc def

* fix up status tests

* resolve all broken test files in the validator rpc package

* gazelle include

* fix up the duties implementation

* fixed up all get duties functions

* all tests pass

* utilize new ethereum apis

* amend validator client to use the new beacon node validator rpc client

* fix up most of validator items

* added in mock

* fix up test

* readd test

* add chain serv mock

* fix a few more validator methods

* all validator tests passingggg

* fix broken test

* resolve even more broken tests

* all tests passsssss

* fix lint

* try PR

* fix up test

* resolve broken other tests
* Sync with master
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Aggregate and proof subscriber (#4240)

* Added subscribers

* Fixed conflict

* Tests

* fix up patch

* Use upstream pb

* include latest patch

* Fmt

* Save state before head block
* skip tests (#4275)
* Delete block attestations from the pool (#4241)

* Added subscribers
* Clean up
* Fixed conflict
* Delete atts in pool in validate pipeline
* Moved it to subscriber
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into use-att-pool-3
* Test
* Fixed test
* Initial work on voluntary exit (#4207)

* Initial implementation of voluntary exit: RPC call

* Update for recent merges

* Break out validation logic for voluntary exits to core module

* RequestExit -> ProposeExit

* Decrease exit package visibility

* Move to operation feed

* Wrap errors
* Fix critical proposer selection bug #4259 (#4265)

* fix critical proposer selection bug #4259

* gofmt

* add 1 more validator to make it 5

* more tests

* Fixed archivedProposerIndex

* Fixed TestFilterAttestation_OK

* Refactor ComputeProposerIndex, add regression test for potential out of range panic

* handle case of nil validator

* Update validators_test.go
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Leftover merge files, oops
* gaz
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* Fixes Duplicate Validator Bug (#4322)

* Update dict

* Test helper

* Regression test

* Comment

* Reset test cache
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fixes after PR #4328
* Complete attestation pool for run time (#4286)

* Added subscribers

* Fixed conflict

* Delete atts in pool in validate pipeline

* Moved it to subscriber

* Test

* Fixed test

* New curl for forkchoice attestations

* Starting att pool service for fork choice

* Update pool interface

* Update pool interface

* Update sync and node

* Lint

* Gazelle

* Updated servers, filled in missing functionalities

* RPC working with 1 beacon node 64 validators

* Started writing tests. Yay

* Test to aggregate and save multiple fork choice atts

* Tests for BatchAttestations for fork choice

* Fixed exisiting tests

* Minor fixes

* Fmt

* Added batch saves

* Lint

* Mo tests yay

* Delete test

* Fmt

* Update interval

* Fixed aggregation broadcast

* Clean up based on design review comment

* Fixed setupBeaconChain

* Raul's feedback. s/error/err
* resolve conflicts
* Merge branch 'v0.9.2' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Removed old protos and fixed tests (#4336)
* Merge refs/heads/master into v0.9.2
* Disallow duplicated indices and test (#4339)
* Explicit use of GENESIS_SLOT in fork choice (#4343)
* Update from 2 to 3 (#4345)
* Remove verify unaggregated attestation when aggregating (#4347)
* use slot ticker instead of run every (#4348)
* Add context check for unbounded loop work (#4346)
* Revert "Explicit use of GENESIS_SLOT in fork choice (#4343)" (#4349)

This reverts commit d3f6753c77f8f733563d00ab649c5159b2c2926f.
* Refactor Powchain Service (#4306)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* fix lint

* raul's comments

* use one method

* fix test

* raul's comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Ensure best better-justification is stored for fork choice (#4342)

* Ensure best better-justification is stored. Minor refactor
* Tests
* Merge refs/heads/v0.9.2 into better-best-justified
* Merge refs/heads/v0.9.2 into better-best-justified
* Ensure that epoch of attestation slot matches the target epoch (#4341)

* Disallow duplicated indices and test
* Add slot to target epoch check to on_attestation
* Add slot to target epoch check to process_attestation
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into no-dup-att-indices
* Fixed TestProcessAttestations_PrevEpochFFGDataMismatches
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Update beacon-chain/blockchain/forkchoice/process_attestation_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Filter viable branches in fork choice (#4355)
* Only activate upon finality (#4359)

* Updated functions
* Tests
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into queue-fix-on-finality
* Comment
* Merge refs/heads/v0.9.2 into queue-fix-on-finality
* Fixed failing test from 4359 (#4360)

* Fixed
* Skip registry spec tests
* Wait for state to be initialized at least once before running slot ticker based on genesis time (#4364)
* Sync with master
* Fix checkpoint root to  use genesis block root (#4368)
* Return an error on nil head state in fork choice (#4369)

* Return error if nil head state

* Fixed tests. Saved childen blocks state

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Update metrics every epoch (#4367)
* return empty slice if state is nil (#4365)
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Pubsub: Broadcast attestations to committee based subnets (#4316)

* Working on un-aggregated pubsub topics

* update subscriber to call pool

* checkpointing

* fix

* untested message validation

* minor fixes

* rename slotsSinceGenesis to slotsSince

* some progress on a unit test, subscribe is not being called still...

* dont change topic

* need to set the data on the message

* restore topic

* fixes

* some helpful parameter changes for mainnet operations

* lint

* Terence feedback

* unskip e2e

* Unit test for validate committee index beacon attestation

* PR feedbacK

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into resolveConflicts
* remove condition
* Remove unused operation pool (#4361)
* Merge refs/heads/master into v0.9.2
* Aggregate attestations periodically  (#4376)
* Persist ETH1 Data to Disk (#4329)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* adding in new proto message

* remove outdated vars

* add new changes

* remove latest eth1data

* continue refactoring

* finally works

* lint

* fix test

* fix all tests

* fix all tests again

* fix build

* change back

* add full eth1 test

* fix logs and test

* add constant

* changes

* fix bug

* lint

* fix another bug

* change back

* Apply suggestions from code review

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fixed VerifyIndexedAttestation (#4382)
* rm signing root (#4381)

* rm signing root

* Fixed VerifyIndexedAttestation

* Check proposer slashed status inside ProcessBlockHeaderNoVerify

* Fixed TestUpdateJustified_CouldUpdateBest

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove Redundant Trie Generation (#4383)

* remove trie generation
* remove deposit hashes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* fix build
* Conflict
* Implement StreamAttestations RPC Endpoint (#4390)

* started attestation stream

* stream attestations test

* on slot tick test passing

* imports

* gaz

* Update beacon-chain/rpc/beacon/attestations_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

Co-authored-by: shayzluf <thezluf@gmail.com>
* Fixed goimport (#4394)
* Use custom stateutil ssz for ssz HTR spec tests (#4396)

* Use custom stateutil ssz for ssz HTR spec tests

* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* set mainnet to be the default for build and run (#4398)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* gracefully handle deduplicated registration of topic validators (#4399)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* SSZ: temporarily disable roots cache until cache issues can be resolved (#4407)

* temporarily disable roots cache until cache issues can be resolved

* Also use custom ssz for spectests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove process block attestations as separate routine (#4408)

* Removed old save/process block atts

* Fixed tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save Deposit Cache to Disk (#4384)

* change to protos

* fix build

* glue everything together

* fix test

* raul's review

* preston's comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix activation queue sorting (#4409)

* Removed old save/process block atts

* Fixed tests

* Proper sorting by eligibility epoch then by indices

* Deleted old colde
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' into v0.9.2
* Merge refs/heads/master into v0.9.2
* stop recursive lookup if context is cancelled (#4420)
* Fix proposal bug (#4419)
* Add Pending Deposits Safely (#4422)

* safely prune cache

* use proper method

* preston's,terence's reviews and comments

* revert change to build files

* use as feature config instead
* Release custom state ssz (#4421)

* Release custom state ssz, change all HTR of beacon state to use custom method

* typo

* use mainnet config

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Update initial sync save justified to align with v0.9.3 (#4432)
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fix build
* don't blacklist on pubsub (#4435)
* Fix Flakey Slot Ticker Test (#4434)

* use interface instead for the slot ticker

* fixed up flakey tests

* add gen time

* get duties comment

* fix lifecycle test

* more fixes
* Configurable min genesis delay (#4437)

* Configurable min genesis delay based on https://github.com/ethereum/eth2.0-specs/pull/1557

* remove feature flag for genesis delay

* fix

* demo config feedback
* patch readme
* save keys unencrypted for validators (#4439)
* Add new demo configuration targeting mainnet scale (#4397)

* Add new demo configuration targeting mainnet, with 1/10th of the deposit value

* reduce quotant by 1/10th. Use 1/10th mainnet values

* only change the inactivity quotant

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save justified checkpoint state (#4433)

* Save justified checkpoint state

* Lint

* Feedback

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update shared/testutil/deposits.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update proto/testing/ssz_regression_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/epoch/epoch_processing.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/kv/forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/proposer.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/prepare_forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/aggregator/server.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/cache/depositcache/pending_deposits.go
* Update beacon-chain/cache/depositcache/pending_deposits_test.go
* Update beacon-chain/rpc/validator/proposer.go
* Merge refs/heads/master into v0.9.2
* Fix e2e genesis delay issues (#4442)

* fix e2e genesis delay issues

* register flag

* typo

* Update shared/featureconfig/config.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* skip demo e2e

* fix validator

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Batch Eth1 RPC Calls (#4392)

* add new methods

* get it working

* optimize past deposit logs processing

* revert change

* fix all tests

* use mock

* lint

* lint

* check for nil

* stop panics

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Terence's Review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-01-07 18:47:39 +00:00
terence tsao
9f5caf8fea
total and target balances metrics (#4414) 2020-01-05 11:12:48 -08:00
terence tsao
ac768207ac Safe delete states (#4401)
* Filter block roots by finalization and head
* Tests
* Comments
* Merge branch 'master' into safe-delete-states
* Fixed exisiting tests
* Merge branch 'safe-delete-states' of git+ssh://github.com/prysmaticlabs/prysm into safe-delete-states
* Merge refs/heads/master into safe-delete-states
* Merge refs/heads/master into safe-delete-states
* Merge refs/heads/master into safe-delete-states
* Merge refs/heads/master into safe-delete-states
2020-01-04 19:20:20 +00:00
terence tsao
77d41024dc
Revert "Use poststate for calculating att votes (#4395)" (#4404) 2020-01-04 09:25:42 -08:00
terence tsao
5fa03edb29 Update committee cache prev epoch (#4402)
* Update cache base on input epoch, not state epoch
* Tests
* Fixed benchmarks
* Use epochs
* One more
2020-01-03 23:47:54 +00:00
terence tsao
220af25bce Use poststate for calculating att votes (#4395)
* Use poststate for votes
* Merge branch 'master' into use-post-state
* Merge branch 'master' into use-post-state
2020-01-03 18:29:26 +00:00
terence tsao
2cb8430ad4 Enhance attester logging (#4380)
* Add validator_log.go
* Use new logging scheme
* Go fmt
* Better name
* Tests
* Tests
* Merge refs/heads/master into better-logging
* Merge branch 'master' of git+ssh://github.com/prysmaticlabs/prysm into better-logging
* Add wg.done, moved logging before span end
* Merge branch 'better-logging' of git+ssh://github.com/prysmaticlabs/prysm into better-logging
2020-01-02 17:04:07 +00:00
Preston Van Loon
bdc4045e23 Add eth1data deposit count metric (#4374) 2019-12-30 08:34:46 -08:00
Preston Van Loon
dc1bd1ef62 Revert 4372 and 4373 (#4375)
* Revert "only add chain start deposits up to min genesis active validator count (#4373)"

This reverts commit 35380dd9bf.
* Revert "Return an error if the wrong number of deposits are provided for genesis state (#4372)"

This reverts commit 9674575892.
2019-12-30 01:21:08 +00:00
Preston Van Loon
9674575892 Return an error if the wrong number of deposits are provided for genesis state (#4372)
* Return an error if the wrong number of deposits are provided for genesis state
* add regression test
2019-12-29 20:10:23 +00:00
terence tsao
30b4b045f5 Add justified check points to chain info getters (#4335)
* Add justified checkpoint getters

* Use it for chainhead

* Mock

* Fixed tests

* Fixed TestServer_StreamChainHead_OnHeadUpdated

* Caught a run time bug. Fixed

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2019-12-22 00:14:23 -06:00
terence tsao
2e4908e7c4
Optimize committee helpers (#4328) 2019-12-19 15:40:51 -08:00
Jim McDonald
20168ad729 More complete validator metrics (#4327)
* More complete validator metrics
* Merge branch 'master' into metrics
* Merge branch 'master' into metrics
2019-12-19 16:14:44 +00:00
terence tsao
5879b26b4b
Hardening Committee Cache for Runtime (#4270) 2019-12-16 10:14:21 -08:00
Preston Van Loon
325a2503f7 AttestingIndices: Make beacon committee be an argument (#4284)
* make beacon committee be an argument
* remove state from ConvertToIndexed
* Merge branch 'master' into refactor-AttestingIndices-committee
* Merge branch 'master' into refactor-AttestingIndices-committee
* Merge branch 'master' into refactor-AttestingIndices-committee
* Merge refs/heads/master into refactor-AttestingIndices-committee
2019-12-15 05:02:50 +00:00
terence tsao
0f4dabfad8
Fix cloning target state for check point state cache (#4288) 2019-12-14 16:06:30 -08:00
Jim McDonald
89e1200b73 Add ticker shared helper (#4285) 2019-12-13 15:14:56 -08:00
terence tsao
9033f6801b Removed active count and shuffling cache (#4266)
* Removed
* All tests pass
* Gaz
* Removed new lines
* A few more lines...
* I think i got them all
* and I didnt : )
* Could this be last...
2019-12-12 01:15:44 +00:00
Preston Van Loon
7a46cc0681 Enforce stronger head state operations (#4216)
* Enforce stronger head state operations

* fix genesis state generation

* one test left to fix

* all tests passing now

* gofmt

* Update beacon-chain/db/kv/state_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/db/kv/state.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* fix tests
2019-12-09 15:35:18 +08:00
Preston Van Loon
0cb681476e Add span for saveCheckpointState (#4227)
* Add span for saveCheckpointState
* Add span for saveCheckpointState
2019-12-08 22:53:09 +00:00
terence tsao
1b8eb16fc7 --initial-sync-cache-state don't need to save head root (#4219)
* Test

* Run time works

* Revert
2019-12-07 16:45:39 -06:00
terence tsao
3e15e2fc1e Add operation feed (#4215)
* Events
* Notifiers
* Refactor
* Gaz
* Fixed rest
* Lint
* Lint
* Visibility
* Typo
* Typo
* Apply suggestions from code review

Co-Authored-By: Nishant Das <nishdas93@gmail.com>
2019-12-07 17:57:26 +00:00
Preston Van Loon
6841d96f36 Remove formatting from error (#4210)
* remove formatting from error
* Fix err
2019-12-06 17:48:38 +00:00
terence tsao
9d4c7cb4f7 Use cache state during init sync (#4199)
* Initial sync state cache
* Gaz
* Gaz
* Don't save head root
* Fix config validator
* Uncomment save head
* Merge branch 'master' into initial-sync-no-verify
* Minor refactor
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Merge branch 'master' into initial-sync-no-verify
* Tests
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Merge branch 'master' into initial-sync-no-verify
* Add lock
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Tests
* Removed save head
* One more test
* Merge branch 'master' into initial-sync-no-verify
* Raul's feedback
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Comment
* Gazelle
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* revert
* Update beacon-chain/blockchain/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
* Fixed test
* Fixed feature flag
* Merge branch 'master' into initial-sync-no-verify
* Fixed cache gensis state test
* Merge branch 'initial-sync-no-verify' of https://github.com/prysmaticlabs/prysm into initial-sync-no-verify
2019-12-06 00:49:19 +00:00
Ivan Martinez
83179376d4 Cleanup testutil and change name scheme to reference deterministic (#4167)
* Clean testutil, change tool names to Deterministic
* Cleanup errors
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into clean-testutil
* Fix bug with generating deposits
* Fix a few tests
* Fix most tests
* Clean up some tests
* Remove err pt. 1
* Remove err pt. 2
* Change tests to use genesis state util
* Remove err from deposits
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Remove circular dependency
* Remove uncompressed signature test
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Merge branch 'master' into clean-testutil
* Goimports
* gazelle
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Add back error handling
* New attestation pool (#4185)

* New pool
* Better namings
* Fmt
* Gazelle
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into define-pool
* Raul's feedback
* Raul's feedback
* Log peer conected log for incoming connections (#4173)

* Log peer conected log for incoming connections
* Merge branch 'master' into peerconnected
* Merge branch 'master' into peerconnected
* Update handshake.go
* Update handshake.go
* Merge branch 'master' into peerconnected
* Merge branch 'master' into peerconnected
* Attestation pool to use go-cache (#4187)
* Update EthereumAPIs  (#4186)

* include new patch targeting latest ethapis master
* ensure project builds
* Merge branch 'master' into update-all-api
* fix up committees
* Merge branch 'update-all-api' of github.com:prysmaticlabs/prysm into update-all-api
* include latest eth apis
* Merge branch 'master' into update-all-api
* update block tests
* Merge branch 'update-all-api' of github.com:prysmaticlabs/prysm into update-all-api
* Merge branch 'master' into update-all-api
* add todos
* Implement GetValidator RPC Endpoint (#4188)

* include new patch targeting latest ethapis master
* ensure project builds
* Merge branch 'master' into update-all-api
* fix up committees
* Merge branch 'update-all-api' of github.com:prysmaticlabs/prysm into update-all-api
* include latest eth apis
* Merge branch 'master' into update-all-api
* update block tests
* Merge branch 'update-all-api' of github.com:prysmaticlabs/prysm into update-all-api
* Merge branch 'master' into update-all-api
* add todos
* implement get validator rpc
* add test for get validator
* table driven test
* fix up test
* fix confs
* tests for more cases
* fix up tests and add out of range
* Slasher optimization (#4172)

* size

* batching and concurrency improvements

* gaz

* merge fixes

* fix comment

* fix test

* fix test

* fix build

* ethpb

* ethpb

* fix test

* fix comment

* add benchmark

* fix benchmark
* Handle error for all testutil uses
* Fix errors
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Revert error handling

Revert "Fix errors"

This reverts commit db081f5486f62c854e7a34e792f5e380cfa922e7.

Revert "Handle error for all testutil uses"

This reverts commit bdabef230632dd184491c2dad63c7d3e74a3861b.

Revert "Add back error handling"

This reverts commit da7e3d2020cd906f45f452d4e441b566f2d5c8aa.
* Change genesis state func to use testing.T
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Fix conflict
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Merge branch 'master' into clean-testutil
* Merge branch 'master' into clean-testutil
* Captialize other logs
* Merge branch 'clean-testutil' of https://github.com/0xKiwi/Prysm into clean-testutil
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into clean-testutil
* Merge branch 'master' into clean-testutil
2019-12-05 19:51:33 +00:00
terence tsao
23dd951e59 Chain head state return nil instead of err (#4193)
* Return nil instead of err
* Preston's feedback
* Merge branch 'master' into return-nil
2019-12-04 23:48:30 +00:00
Raul Jordan
9e5cc81340 Implement Prysm-Specific HashTreeRootState (#4077)
* new ssz hash tree root
* Merge branch 'master' into new-ssz-state
* better comments on func
* add errors instead of panic in state
* utilize errors wrap everywhere
* include bench
* added bench info
* equality test
* dup
* gaz
* use new hash tree root in state transition
* fix build
* separate test package
* three targets failign
* single target fails
* please test targets...pass for me
* revert
* Merge branch 'master' into new-ssz-state
* rev
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
* broken build
* Merge branch 'master' into new-ssz-state
* gaz
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
* ssz workspace
* master ssz
* Merge branch 'master' into new-ssz-state
* resolve conf
* resolve some conflicts and fix up broken file
* fix up build file issues and sync
* eth1 data votes included
* further abstractions, simplifications
* Merge branch 'master' into new-ssz-state
* gaz
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
* feature flag gating
* add field count test
* Merge branch 'master' into new-ssz-state
* resolving ivan feedback
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
* gaz
* Merge branch 'master' into new-ssz-state
* addressed
* Merge branch 'new-ssz-state' of github.com:prysmaticlabs/prysm into new-ssz-state
2019-12-04 19:20:33 +00:00
terence tsao
e87337a97a Update forkchoice spec link to v0.9.0 (#4147)
* Update forkchoice doc link to v0.9.0
* Merge refs/heads/master into update-link
2019-11-30 05:48:18 +00:00
terence tsao
7495961d6b Prune boundary state (#4139)
* Delete epoch boundary slot of last finalized epoch
* Case to cover start slot is skipped
* Test
* Feature flag
* feature gate the new functionality only
* Update DB for migration
* Test
* Fmt
* Fixed test
* Gazelle
2019-11-28 23:05:47 +00:00
terence tsao
e90358cd8e Removed unused mocks (#4135)
* Removed unused mocks

* Lint

* Gaz
2019-11-27 14:52:24 -06:00
Raul Jordan
0326be86b5 Apply Patch Rules to Use EthereumAPIs Generated Protos in Prysm (#4112)
* starting on patch
* finish determining all required patches
* properly redefine the patch rules
* new patch
* rem double semicolon
* fix patch file
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-eth-protos
* building the deps
* test target passes using ethereumapis
* compile gateway
* attempting to build everything
* e2e use ethereumapis
* more fixes for slasher
* other item
* getting closer to compiling slasher
* build slasher package
* Merge branch 'master' into deprecate-eth-protos
* Merge branch 'master' into deprecate-eth-protos
* fix benches
* lint gazelle
* Merge branch 'deprecate-eth-protos' of github.com:prysmaticlabs/prysm into deprecate-eth-protos
* proper gateway
* lint
* Merge branch 'master' into deprecate-eth-protos
* fix build
* Merge branch 'deprecate-eth-protos' of github.com:prysmaticlabs/prysm into deprecate-eth-protos
* use swag
* resolve
* ignore change
* include new patch changes
* fix test
* builds
* fix e2e
* gaz
2019-11-27 05:08:18 +00:00
Nishant Das
b88e6dc918 Speed Up Block Processing In Sync (#4075)
* fix proto
* make them non-batched
* gate behind flag
* fix refs
* fix refs
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* use global archiver flags
* lint
* Merge branch 'master' into speedUpProcessing
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* preston's review
* Merge branch 'speedUpProcessing' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* Merge branch 'master' into speedUpProcessing
* Merge branch 'master' into speedUpProcessing
* Merge branch 'master' into speedUpProcessing
2019-11-26 07:15:54 +00:00
Nishant Das
7d0e5a9dc4 Use Latest Vote Map (#4102)
* add latest vote map
* fix all tests
* remove db crud methods
* Merge branch 'master' into latestVoteMap
* preston's review
* Merge branch 'latestVoteMap' of https://github.com/prysmaticlabs/geth-sharding into latestVoteMap
2019-11-25 16:34:20 +00:00
terence tsao
21deed0fb7 Revert "Add Lock When Accessing Checkpoints" (#4094)
* Revert "Add Lock When Accessing Checkpoints (#4086)"

This reverts commit 2f392544a6.
* Merge branch 'master' into revert-4086-checkpointLock
2019-11-24 17:12:56 +00:00
terence tsao
627791c54e Complete finalization metrics (#4096)
* Complete finaliztion metrics

* Fixed test
2019-11-23 17:12:29 -08:00
Jim McDonald
9e45cffabc Move StateInitialized and ChainStarted to state feed (#4084)
* Migrate ChainStarted and StateInitialized to state notifier
* Provide state notifier to powchain service
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Remove commented line
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Accept err from HeadState() as non-fatal
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
2019-11-23 03:35:47 +00:00
Preston Van Loon
f717c5d852 Release --prune-finalized-states to all (#4082)
* Release --prune-finalized-states to all
* Merge branch 'master' into deprecate-ff-prune-finalized-states
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-prune-finalized-states
2019-11-22 20:29:46 +00:00
Nishant Das
2f392544a6 Add Lock When Accessing Checkpoints (#4086)
* fix data races
* Merge branch 'master' into checkpointLock
* Merge branch 'master' into checkpointLock
* Merge refs/heads/master into checkpointLock
2019-11-22 06:34:42 +00:00
terence tsao
f5cb04012e Aggregator selection from RPC to validator client (#4071)
* Config
* Updated proto
* Updated pool
* Updated RPC
* Updated validator client
* run time works
* Clean ups
* Fix tests
* Visibility
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into aggregator
* Raul's feedback
* Tests for RPC server
* Tests for validator client
* Span
* More tests
* Use go routine for SubmitAggregateAndProof
* Go routines
* Updated comments
* Use array of roles
* Fixed tests
* Build
* Update validator/client/runner.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update validator/client/runner.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* If
* Merge branch 'refactor-validator-roles' of https://github.com/prysmaticlabs/prysm into refactor-validator-roles
* Empty
* Feedback
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into aggregator
* Removed proto/eth/v1alpha1/shard_chain.pb.go?
* Cleaned up
* Revert
* Comments
* Lint
* Comment
* Merge branch 'master' into aggregator
2019-11-22 05:11:38 +00:00
Jim McDonald
49c2dd2cfc Move the state notifier to a different module (#4058)
* Move state notifier to statefeed

* Updates to state notifier

* Create state feed in beacon node

* Formatting
2019-11-19 16:15:48 -06:00
terence tsao
26da7c4114 Nil state fallback in Blockchain.HeadState() (#4042)
* Can build

* All tests pass

* Update beacon-chain/blockchain/chain_info.go

* Fix context
2019-11-19 10:12:50 -06:00
Preston Van Loon
24a5000e47 return to exit select loop rather than break (#4040) 2019-11-19 21:22:45 +08:00
Jim McDonald
65d920e13a Add generic state feed (#4004)
* Initial implementation of state feed

* Add instructions on adding new events

* Tidy up log messages

* Tidy up mock

* Update beacon-chain/core/statefeed/events.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/core/statefeed/events.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Remove unused BlockReceivedData

* Rename BlockHash to BlockRoot in BlockProcessedData

* Punctuation

* Use correct root for block processed event

* StateFeeder -> StateNotifier; fix up tests.

* Add Verified flag to BlockProcessed event

* Fix visibility in Bazel
2019-11-19 17:17:41 +08:00
terence tsao
499f05f34b Return error instead of logging (#4039) 2019-11-18 14:09:26 -08:00
terence tsao
f43a7c67f2 Process attestation to use operation service's pool (#4014)
* Starting

* Routine working

* Single client working

* Fixed all the tests

* Lint

* Gazelle

* 12

* Tests
2019-11-18 11:19:03 -06:00
terence tsao
5acc362f7e
End slot can't be greater than start slot (#4008) 2019-11-15 16:23:43 -08:00
terence tsao
81c53c26fb Update run time to v0.9 (#3935)
* Update seed domains (#3872)

* Remove Transfers (#3870)

* Remove active index roots and compact committee roots (#3869)

* Update inclusion reward (#3886)

* Alter proposer selection logic (#3884)

* Fix early committee bias (#3888)

* Remove shards and committees (#3896)

* Epoch spec tests v0.9 (#3907)

* Block spec test v0.9 (#3905)

* rm'ed in protobuf

* build proto

* build proto

* build proto

* fix core package

* Gazelle

* Fixed all the tests

* Fixed static test

* Comment out spec test for now

* One more skip

* fix-roundRobinSync (#3862)

* Starting but need new seed function

* Revert initial sync

* Updated Proposer Slashing

* Fixed all tests

* Lint

* Update inclusion reward

* Fill randao mixes with eth1 data hash

* Test

* Fixing test part1

* All tests passing

* One last test

* Updated config

* Build proto

* Proper skip message

* Conflict and fmt

* Removed crosslinks and shards. Built

* Format and gazelle

* Fixed all the block package tests

* Fixed all the helper tests

* All epoch package tests pass

* All core package tests pass

* Fixed operation tests

* Started fixing rpc test

* RPC tests passed!

* Fixed all init sync tests

* All tests pass

* Fixed blockchain tests

* Lint

* Lint

* Preston's feedback

* Starting

* Remove container

* Fixed block spec tests

* All passing except for block_processing test

* Failing block processing test

* Starting

* Add AggregateAndProof

* All mainnet test passes

* Update deposit contract (#3906)

* Proto spec tests v0.9 (#3908)

* Starting

* Add AggregateAndProof

* Unskip block util tests (#3910)

* rm'ed in protobuf

* build proto

* build proto

* build proto

* fix core package

* Gazelle

* Fixed all the tests

* Fixed static test

* Comment out spec test for now

* One more skip

* fix-roundRobinSync (#3862)

* Starting but need new seed function

* Revert initial sync

* Updated Proposer Slashing

* Fixed all tests

* Lint

* Update inclusion reward

* Fill randao mixes with eth1 data hash

* Test

* Fixing test part1

* All tests passing

* One last test

* Updated config

* Build proto

* Proper skip message

* Conflict and fmt

* Removed crosslinks and shards. Built

* Format and gazelle

* Fixed all the block package tests

* Fixed all the helper tests

* All epoch package tests pass

* All core package tests pass

* Fixed operation tests

* Started fixing rpc test

* RPC tests passed!

* Fixed all init sync tests

* All tests pass

* Fixed blockchain tests

* Lint

* Lint

* Preston's feedback

* Starting

* Remove container

* Fixed block spec tests

* All passing except for block_processing test

* Failing block processing test

* Starting

* Add AggregateAndProof

* All mainnet test passes

* Unskip block util tests

* Slot processing spec test V0.9 (#3912)

* Starting

* Add AggregateAndProof

* Unskip slot processing mainnet test

* Unskip minimal spec test for finalization (#3920)

* Remove outdated interop tests (#3922)

* Rm outdated interop tests

* Rm test runner

* Gazelle

* Update validator to use proposer slot (#3919)

* Fix committee assignment (#3931)

* Replace shard with committee index (#3930)

* Conflict

* Clean up (#3933)

* Remove shard filter in db (#3936)

* Remove lightouse compatibility test (#3939)

* Update Committee Cache for v0.9 (#3948)

* Updated committee cache

* Removed shuffled indices cache

* Started testing run time

* Lint

* Fixed test

* Safeguard against nil head state

* address edge case

* add test

* Fixed TestRoundRobinSync by doubling the epochs

* Unskip TestProtoCompatability (#3958)

* Unskip TestProtoCompatability

* Update WORKSPACE

* Fix minimal config (#3959)

* fix minimal configs

* fix hardcoded value in test

* Simplify verify att time (#3961)

* update readme for deposit contract, regen bindings for vyper 0.1.0b12 (#3963)

* update readme for deposit contract, regen bindings

* medium

* Check nil base state (#3964)

* Copy Block When Receiving it From Sync (#3966)

* copy block

* clone for other service methods too

* Change logging of Bitfield  (#3956)

* change logging of bits

* preston's review

* Unskip Beacon Server Test (#3962)

* run test till the end

* fix up proto message types

* fmt

* resolve broken tests

* better error handling

* fixing new logic to use archived proposer info

* fix up logic

* clip using the max effective balance

* broken build fix with num arg mismatch

* amend archive

* archival logic changed

* rename test

* archive both proposer and attester seeds

* page size 100

* further experiments

* further experimentation, archivedProposerIndex seems wrong

* test passes

* rem log

* fix broken test

* fix test

* gaz

* fix imports

* ethapis
2019-11-11 17:03:43 -05:00
Nishant Das
6e90931837
Handle Missing Blocks (#3880)
* add fixes

* more changes

* simplify the solution

* add shuffle

* remove prom metric

* add to mock

* go doc

* use non default source

* add test

* terence's review

* all tests passing
2019-11-05 10:37:18 +08:00
Nishant Das
8977e5088e Handle ETH1 Node Failure (#3914)
* refactor powchain config

* send mock eth1 votes when proposing blocks

* fix all tests

* lint

* fix all tests

* fix blockchain test

* Apply suggestions from code review

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul's review

* add warning
2019-11-02 14:56:03 -05:00
Preston Van Loon
cc16a10a33
Remove --enable-finalized-block-root-index (#3887)
* Deprecated do not use

* remove finalized block indexing feature flag

* remove flag some more

* dont rollback a managed tx

* require blocks to be in a chain in tests
2019-11-02 14:57:35 -04:00
Nishant Das
f8c870aa91
Fix Failure Counter (#3850)
* fix bug

* return if not ok

* don't set peer status there

* fix build

* fix test

* fix all other tests

* add regression test

* cosmetic changes
2019-10-29 13:23:55 +08:00
terence tsao
582c382771 Add tracing and annotation for descendent error (#3859)
* Add annotation

* Revert
2019-10-26 20:26:51 -07:00
terence tsao
c5b0b3c326 Prune states since last finalized (#3856)
* Prune states since last finalized

* Fixed test

* Renmaed to prevFinalizedCheckpt
2019-10-25 18:16:10 -07:00
terence tsao
57036d16f9
Fix remove older than last finalized state (#3847) 2019-10-25 09:51:08 -07:00
Preston Van Loon
acf11262de Add feature flag for state pruning (#3845)
* Add feature flag for state pruning

* gaz
2019-10-23 23:45:52 -07:00
Preston Van Loon
2d98902eed
Require state to exist when updating finalized checkpoint (#3843)
* require state to exist before saving finalized checkpoint

* Add comment

* gaz

* fix test

* fix test
2019-10-23 22:50:33 -07:00
terence tsao
ae1e435231 Guard against deleting genesis and finalized state in DB (#3842)
* Don't delete boundary state

* Lint

* Test

* Feedback

* Batch and better comment

* Fix test

* zzzzzz

* rmStatesOlderThanLastFinalized
2019-10-23 22:39:41 -07:00
terence tsao
9a0fb5dca1
Delete DB states older than finalized checkpoint (#3824) 2019-10-23 08:30:21 -07:00
terence tsao
be5451abef
Removed extra save genesis block root (#3829) 2019-10-22 17:49:18 -07:00
terence tsao
e42af4f11d
Check attestations bitfield overlaps (#3804) 2019-10-18 17:41:27 -07:00
Preston Van Loon
fb510a3510 Reject blocks from the future in regular sync (#3799)
* reject blocks from the future in syncing

* fix test and gofmt
2019-10-18 11:30:14 +08:00
Preston Van Loon
6bf14dedcd
Better aggregated attestations pool (#3761)
* WIP of aggregated signatures in DB

* new lines at end

* taking a nap on the plane now

* fix tests

* remove duplication of attestations. so much for that airplane nap lol

* benchmark before flight lands

* gaz

* manual gaz

* fully contained checks

* quick improvement before landing

* new bitlist with fixes

* doesn't need real signatures

* it works, mostly

* print shard too

* some refactoring

* Revert "some refactoring"

This reverts commit 377ce7fbfb02321c20522da2eb27208a20a67669.

* Revert "Revert "some refactoring""

This reverts commit b46a458898acd214da44fc20dfae920d2725b6e8.

These changes are ok, just need to update the expected values

* fix tests

* lint

* lint

* upstream changes

* fix tests

* what

* resolve TODOs

* gofmt

* revert unrelated pb

* remove debug statement
2019-10-16 23:46:07 -07:00
Preston Van Loon
d4001a8b29
Annotate errors / spans in block processing queue (#3751)
* annotation error in span

* added more annotations and spans to process pending blocks

* use diff

* workspace dep
2019-10-10 17:44:24 +09:00
terence tsao
9bf55e53e7 Add context and tracing to attestation pool (#3737)
* Test case for overlapping aggregation bits

* Add ctx and tracing for attestation pool and beyond

* No nil

* Use real ctx

* Use real ctx

* Fix test

* Fix test

* Fix test

* Fix test

* Fixed imports
2019-10-09 06:42:17 +08:00
Preston Van Loon
23764c4640
Abstract verifySlot to helper package (#3731)
* Abstract verifySlot to helper package

* blocks -> slots

* fix test
2019-10-07 16:15:50 +09:00
terence tsao
750bc83369
Clean up feature flag namings (#3715) 2019-10-07 14:11:49 +09:00
Nishant Das
91cb081b7e fix log (#3713) 2019-10-04 14:54:21 -07:00
terence tsao
fb0e504856 Added cov for untested file at chain_info (#3705) 2019-10-03 15:26:24 -07:00
Preston Van Loon
58dbdfb6f5 Remove attestation that fails to verify from queue (#3704) 2019-10-03 13:57:18 -07:00
Jim McDonald
323ee8dfac SetupInitialDeposits() now returns deposit data roots (#3683) 2019-10-02 10:50:34 +08:00
Raul Jordan
d9c0e65cef
Improve Beacon Node Logging UX (#3600)
* info logs beacon node improvements

* prom test fixes

* info logging changes

* wrapped up node info logging

* changed to debug level

* warn logs taken care of

* Terence suggestion

* warn spacing

* better logging in initial sync

* debug level standardized

* complete debug standardization

* participation at epoch end

* fix archive tests

* even more test fixes

* prom test

* ops test

* powtest

* rpc sync test

* rem part

* log formatting
2019-10-01 15:05:17 -05:00
Raul Jordan
3d37a4e038
Optimize Domain Data RPC Request (#3671)
* gaz

* fix broken build

* fix broken test

* fix broken test
2019-10-01 09:36:36 -05:00
terence tsao
944d3b16fd Batch save attestations in state transition (#3672)
* Batch save attestations

* Update test

* Revert config
2019-09-30 16:11:59 -07:00
Raul Jordan
8d215feb25
Large Prysm Performance Improvements (#3622)
* skip bls verification with a feature flag at runtime

* gazelle

* more bls mocks

* block roots efficiency

* db block roots now does not show up on the flame graphs

* save validator latest votes batch

* batch save att

* misc improvements to pprof

* import

* include validator index cache

* error if no filter criteria

* resolved comments

* build fix

* lint

* remove delay global

* attestation and block test fixes

* preston suggestions

* fix db tests

* fix missing broken tests

* tests pass
2019-09-30 15:45:53 -05:00
terence tsao
4f1d2868f8 Save head if diff than prev saved head (#3669)
* Save head if it's diff

* New test for process attestation

* New test for process block

* Fixed loggings

* Fixed all the tests
2019-09-30 15:15:56 -05:00
Nishant Das
8ece8fb44b Expose DB Metrics (#3663)
* add in bolt metrics

* unregister in db teardown

* unregister in Close()

* fix clear db case

* fix test error

* gaz

* remove unregister

* remove gaz
2019-09-30 10:24:47 -05:00
Preston Van Loon
5a92725329
Fix init sync race condition (#3633)
* fix init sync race condition

* grab subscription before checking
2019-09-28 18:42:44 -07:00
terence tsao
4c8269aca3
Part 4 of caching improvement - Use Cache (#3625) 2019-09-27 15:56:08 -07:00
Preston Van Loon
707a816f2b
bootstrap from finalized checkpoint rather than head slot (#3621) 2019-09-27 12:39:32 -07:00
Nishant Das
73938876b1 better log validator votes (#3590) 2019-09-26 06:32:06 -07:00
Preston Van Loon
0a2dfedf0f
Log warning with error instead of returning an error for fork choice votes update (#3587) 2019-09-25 20:39:19 -07:00
terence tsao
3ef681e649
Initial sync no verify block (#3586) 2019-09-25 20:32:00 -07:00
Preston Van Loon
5345ddf686 Initial Sync: Round robin (#3538)
* first pass, step 1 works

* naive from finalized to head

* delete commented code

* checkpoint progress on tests

* passing test

* abstract code slightly

* failure cases

* chkpt

* mostly working, missing a single block and having timeout

* passing tests

* comments

* comments

* gaz

* clarify comments

* progress on a few new cases

* add back bootnode query tool

* bootstrap from DHT

* chunked responses in round robin

* fix tests and deadlines

* add basic counter, time estimation

* hello -> handshakes

* show peers in use during sync

* just one last test failure

* only request blocks starting in the finalized epoch for step 1

* revert that

* comment out test and add better commentary

* move requestBlocks out to pointer receiver

* mathutil

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* PR feedback

* PR feedback
2019-09-25 12:00:04 -05:00
Nishant Das
8ce96428b1 Fixes our Attestation Aggregation Issues (#3579) 2019-09-25 07:39:16 -07:00
terence tsao
9577e2c123 Removed attestation check for votes (#3562) 2019-09-23 15:44:13 -05:00
skillful-alex
b015dc793a HeadFetcher data race fix (#3460)
* HeadFetcher data race fix

* bazel run //:gazelle -- fix

* add the db teardown to test

* add TestChainService_SaveHead_DataRace test

* split race and norace tests

* change testset name

* test CI with 'spectest' tag instead of 'raceon'

* one more test CI with 'spectest' tag instead of 'raceon'

* bazel run //:gazelle -- fix

* set test tag to 'race_on'

* not clone the state
2019-09-23 14:24:42 -05:00
terence tsao
ab2d4e8ad6 Implement attestation pool in memory (#3542)
* Run time bug

* Still failing

* Run time working

* Run time working

* Gazelle

* Fixed all the tests

* Revert config

* Revert back test configs

* Revert config

* Tested run time again, everything is good
2019-09-23 11:11:44 -05:00
terence tsao
4e6ed2744d Fixed 2 cosmetic errors (#3543)
* Fixed

* Treehash

* Updated error msg
2019-09-23 08:30:39 -07:00
terence tsao
315d4f0549 Save aggregated attestations in DB after processing (#3541)
* Added AggregatedAttestation helper

* Implemented `saveNewAttestation` for processing attestation

* Implemented `savesNewBlockAttestations` for processing block

* Tests

* Fix name

* Raul's feedback
2019-09-21 13:57:32 -05:00
terence tsao
2b2ef4f37c
Added AggregatedAttestation helper (#3539) 2019-09-21 09:43:18 -07:00
terence tsao
c29a7be0ec Fix justified check point mutation (#3534)
* first version of the watchtower api

* first version

* delete watchtower

* move to message loop

* roughtime

* one time

* fix test

* Fixed

* Fixed

* Revert unused lock
2019-09-20 10:44:28 -07:00
terence tsao
ac176a5078 Update validators votes from incoming block (#3530)
* first version of the watchtower api

* first version

* delete watchtower

* move to message loop

* roughtime

* one time

* fix test

* Update block attestation votes

* Clean up

* Found a bug

* Confirmed to be working in run time

* Confirmed to be working run time

* Raul's feedback

* Tests
2019-09-19 20:34:57 -05:00
Raul Jordan
4ffef61e1d
Archive Validator Participation on End of Epoch Event (#3524)
* archive participation begin implementation

* validator participation compute

* comments

* compute participation common func

* full test for archiving data

* gazelle

* complete tests

* gaz

* remove double negative grammar in comment

* use head fetcher and root

* tests pass
2019-09-19 15:59:23 -05:00
Raul Jordan
e8b6951591
Complete ListAttestations EthereumAPIs v1alpha (#3452)
* retrieve attestations by block root as well

* add beacon block root filter

* rem err unimpl

* add changes to list atts filter proto

* utilize the new filter attributes

* add filter types

* utilize filters in the api server impl

* tests for filter

* tests pass

* filter test done

* fix test by using head fetcher instead

* gaz

* no panic

* use new ethapis commit

* elim panic

* res panic

* ensure proto compatibility

* fixed broken test
2019-09-18 20:14:26 -05:00