Commit Graph

59 Commits

Author SHA1 Message Date
shayzluf
ade61717a4
Slasher data update from archive (#4563)
* first version

* cli context

* fix service

* starting change to ccache

* ristretto cache

* added test

* test on evict

* remove evict test

* test onevict

* comment for exported flag

* update all span maps on load

* fix setup db

* span cache added to help flags

* start save cache on exit

* save cache to db before close

* comment fix

* fix flags

* setup db new

* data update from archive node

* gaz

* slashing detection on old attestations

* un-export

* rename

* nishant feedback

* workspace cr

* lint fix

* fix calls

* start db

* fix test

* Update slasher/db/db.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* add flag

* fix fail to start beacon client

* mock beacon service

* fix imports

* gaz

* goimports

* add clear db flag

* print finalized epoch

* better msg

* Update slasher/db/attester_slashings.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul feedback

* raul feedback

* raul feedback

* raul feedback

* raul feedback

* add detection in runtime

* fix tests

* raul feedbacks

* raul feedback

* raul feedback

* goimports

* Update beacon-chain/blockchain/process_attestation_helpers.go

* Update beacon-chain/blockchain/receive_block.go

* Update beacon-chain/core/blocks/block_operations_test.go

* Update beacon-chain/core/blocks/block_operations.go

* Update beacon-chain/core/epoch/epoch_processing.go

* Update beacon-chain/sync/validate_aggregate_proof_test.go

* Update shared/testutil/block.go

* Update slasher/service/data_update.go

* Update tools/blocktree/main.go

* Update slasher/service/service.go

* Update beacon-chain/core/epoch/precompute/attestation_test.go

* Update beacon-chain/core/helpers/committee_test.go

* Update beacon-chain/core/state/transition_test.go

* Update beacon-chain/rpc/aggregator/server_test.go

* Update beacon-chain/sync/validate_aggregate_proof.go

* Update beacon-chain/rpc/validator/proposer_test.go

* Update beacon-chain/blockchain/forkchoice/process_attestation.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update slasher/db/indexed_attestations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update slasher/service/data_update.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence feedback

* terence feedback

* goimports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-01-29 07:14:51 +05:30
terence tsao
6d2a2ebadf Update run time to v0.9.3 (#4154)
* Remove custody (#3986)

* Update proto fields

* Updated block operations

* Fixed all block operation tests

* Fixed tests part 1

* Fixed tests part 1

* All tests pass

* Clean up

* Skip spec test

* Fixed ssz test

* Skip ssz test

* Skip mainnet tests

* Update beacon-chain/operations/attestation.go

* Update beacon-chain/operations/attestation.go
* Decoy flip flop check (#3987)
* Bounce attack check (#3989)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Update beacon-chain/blockchain/forkchoice/process_block.go
* Conflict
* Unskip mainnet spec tests (#3998)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update beacon-chain/core/blocks/block_operations.go
* Unskip minimal spec tests (#3999)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update workspace

* Unskip all minimal spec tests

* Update workspace for general test
* Unskip test (#4001)
* Update minimal seconds per slot to 6 (#3978)
* Bounce attack tests (#3993)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Slots since epoch starts tests

* Update justified checkpt tests

* Conflict

* Fixed logic

* Update process_block.go

* Use helper
* Conflict
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.1
* Conflict
* Fixed failed tests
* Lower MinGenesisActiveValidatorCount to 16384 (#4100)
* Fork choice beacon block checks (#4107)

* Prevent future blocks check and test

* Removed old code
* Update aggregation proto (#4121)

* Update def
* Update spec test
* Conflict
* Update workspace
* patch
* Resolve conflict
* Patch
* Change workspace
* Update ethereumapis to a forked branch at commit 6eb1193e47
* Fixed all the tests
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into conflict
* fix patch
* Need to regenerate test data
* Merge branch 'master' into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Enable snappy compression for all (#4157)

* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* Validate aggregate and proof subscriber (#4159)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Conflict
* Update workspace
* Conflict
* Conflict
* Conflict
* Merge branch 'master' into v0.9.2
* Merge branch 'master' into v0.9.2
* Conflict
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Remove migrate to snappy  (#4205)
* Feature flag: Deprecate --prune-states, release to all (#4204)

* Deprecated prune-states, release to all

* imports

* remote unused import

* remove unused import

* Rm prune state test

* gaz
* Refactoring for dynamic pubsub subscriptions for non-aggregated attestations (#4189)

* checkpoint progress

* chkpt

* checkpoint progress

* put pipeline in its own file

* remove unused imports

* add test, it's failing though

* fix test

* remove head state issue

* add clear db flag to e2e

* add some more error handling, debug logging

* skip processing if chain has not started

* fix test

* wrap in go routine to see if anything breaks

* remove duplicated topic

* Add a regression test. Thanks @nisdas for finding the original problem. May it never happen again *fingers crossed*

* Comments

* gofmt

* comment out with TODO
* Sync with master
* Sync with master
* RPC servers use attestation pool (#4223)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Refactor RPC to Fully Utilize Ethereum APIs (#4243)

* include attester as a file in the validator server

* remove old proposer server impl

* include new patch and properly sync changes

* align with public pbs

* ensure matches rpc def

* fix up status tests

* resolve all broken test files in the validator rpc package

* gazelle include

* fix up the duties implementation

* fixed up all get duties functions

* all tests pass

* utilize new ethereum apis

* amend validator client to use the new beacon node validator rpc client

* fix up most of validator items

* added in mock

* fix up test

* readd test

* add chain serv mock

* fix a few more validator methods

* all validator tests passingggg

* fix broken test

* resolve even more broken tests

* all tests passsssss

* fix lint

* try PR

* fix up test

* resolve broken other tests
* Sync with master
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Aggregate and proof subscriber (#4240)

* Added subscribers

* Fixed conflict

* Tests

* fix up patch

* Use upstream pb

* include latest patch

* Fmt

* Save state before head block
* skip tests (#4275)
* Delete block attestations from the pool (#4241)

* Added subscribers
* Clean up
* Fixed conflict
* Delete atts in pool in validate pipeline
* Moved it to subscriber
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into use-att-pool-3
* Test
* Fixed test
* Initial work on voluntary exit (#4207)

* Initial implementation of voluntary exit: RPC call

* Update for recent merges

* Break out validation logic for voluntary exits to core module

* RequestExit -> ProposeExit

* Decrease exit package visibility

* Move to operation feed

* Wrap errors
* Fix critical proposer selection bug #4259 (#4265)

* fix critical proposer selection bug #4259

* gofmt

* add 1 more validator to make it 5

* more tests

* Fixed archivedProposerIndex

* Fixed TestFilterAttestation_OK

* Refactor ComputeProposerIndex, add regression test for potential out of range panic

* handle case of nil validator

* Update validators_test.go
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Leftover merge files, oops
* gaz
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* Fixes Duplicate Validator Bug (#4322)

* Update dict

* Test helper

* Regression test

* Comment

* Reset test cache
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fixes after PR #4328
* Complete attestation pool for run time (#4286)

* Added subscribers

* Fixed conflict

* Delete atts in pool in validate pipeline

* Moved it to subscriber

* Test

* Fixed test

* New curl for forkchoice attestations

* Starting att pool service for fork choice

* Update pool interface

* Update pool interface

* Update sync and node

* Lint

* Gazelle

* Updated servers, filled in missing functionalities

* RPC working with 1 beacon node 64 validators

* Started writing tests. Yay

* Test to aggregate and save multiple fork choice atts

* Tests for BatchAttestations for fork choice

* Fixed exisiting tests

* Minor fixes

* Fmt

* Added batch saves

* Lint

* Mo tests yay

* Delete test

* Fmt

* Update interval

* Fixed aggregation broadcast

* Clean up based on design review comment

* Fixed setupBeaconChain

* Raul's feedback. s/error/err
* resolve conflicts
* Merge branch 'v0.9.2' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Removed old protos and fixed tests (#4336)
* Merge refs/heads/master into v0.9.2
* Disallow duplicated indices and test (#4339)
* Explicit use of GENESIS_SLOT in fork choice (#4343)
* Update from 2 to 3 (#4345)
* Remove verify unaggregated attestation when aggregating (#4347)
* use slot ticker instead of run every (#4348)
* Add context check for unbounded loop work (#4346)
* Revert "Explicit use of GENESIS_SLOT in fork choice (#4343)" (#4349)

This reverts commit d3f6753c77f8f733563d00ab649c5159b2c2926f.
* Refactor Powchain Service (#4306)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* fix lint

* raul's comments

* use one method

* fix test

* raul's comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Ensure best better-justification is stored for fork choice (#4342)

* Ensure best better-justification is stored. Minor refactor
* Tests
* Merge refs/heads/v0.9.2 into better-best-justified
* Merge refs/heads/v0.9.2 into better-best-justified
* Ensure that epoch of attestation slot matches the target epoch (#4341)

* Disallow duplicated indices and test
* Add slot to target epoch check to on_attestation
* Add slot to target epoch check to process_attestation
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into no-dup-att-indices
* Fixed TestProcessAttestations_PrevEpochFFGDataMismatches
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Update beacon-chain/blockchain/forkchoice/process_attestation_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Filter viable branches in fork choice (#4355)
* Only activate upon finality (#4359)

* Updated functions
* Tests
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into queue-fix-on-finality
* Comment
* Merge refs/heads/v0.9.2 into queue-fix-on-finality
* Fixed failing test from 4359 (#4360)

* Fixed
* Skip registry spec tests
* Wait for state to be initialized at least once before running slot ticker based on genesis time (#4364)
* Sync with master
* Fix checkpoint root to  use genesis block root (#4368)
* Return an error on nil head state in fork choice (#4369)

* Return error if nil head state

* Fixed tests. Saved childen blocks state

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Update metrics every epoch (#4367)
* return empty slice if state is nil (#4365)
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Pubsub: Broadcast attestations to committee based subnets (#4316)

* Working on un-aggregated pubsub topics

* update subscriber to call pool

* checkpointing

* fix

* untested message validation

* minor fixes

* rename slotsSinceGenesis to slotsSince

* some progress on a unit test, subscribe is not being called still...

* dont change topic

* need to set the data on the message

* restore topic

* fixes

* some helpful parameter changes for mainnet operations

* lint

* Terence feedback

* unskip e2e

* Unit test for validate committee index beacon attestation

* PR feedbacK

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into resolveConflicts
* remove condition
* Remove unused operation pool (#4361)
* Merge refs/heads/master into v0.9.2
* Aggregate attestations periodically  (#4376)
* Persist ETH1 Data to Disk (#4329)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* adding in new proto message

* remove outdated vars

* add new changes

* remove latest eth1data

* continue refactoring

* finally works

* lint

* fix test

* fix all tests

* fix all tests again

* fix build

* change back

* add full eth1 test

* fix logs and test

* add constant

* changes

* fix bug

* lint

* fix another bug

* change back

* Apply suggestions from code review

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fixed VerifyIndexedAttestation (#4382)
* rm signing root (#4381)

* rm signing root

* Fixed VerifyIndexedAttestation

* Check proposer slashed status inside ProcessBlockHeaderNoVerify

* Fixed TestUpdateJustified_CouldUpdateBest

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove Redundant Trie Generation (#4383)

* remove trie generation
* remove deposit hashes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* fix build
* Conflict
* Implement StreamAttestations RPC Endpoint (#4390)

* started attestation stream

* stream attestations test

* on slot tick test passing

* imports

* gaz

* Update beacon-chain/rpc/beacon/attestations_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

Co-authored-by: shayzluf <thezluf@gmail.com>
* Fixed goimport (#4394)
* Use custom stateutil ssz for ssz HTR spec tests (#4396)

* Use custom stateutil ssz for ssz HTR spec tests

* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* set mainnet to be the default for build and run (#4398)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* gracefully handle deduplicated registration of topic validators (#4399)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* SSZ: temporarily disable roots cache until cache issues can be resolved (#4407)

* temporarily disable roots cache until cache issues can be resolved

* Also use custom ssz for spectests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove process block attestations as separate routine (#4408)

* Removed old save/process block atts

* Fixed tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save Deposit Cache to Disk (#4384)

* change to protos

* fix build

* glue everything together

* fix test

* raul's review

* preston's comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix activation queue sorting (#4409)

* Removed old save/process block atts

* Fixed tests

* Proper sorting by eligibility epoch then by indices

* Deleted old colde
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' into v0.9.2
* Merge refs/heads/master into v0.9.2
* stop recursive lookup if context is cancelled (#4420)
* Fix proposal bug (#4419)
* Add Pending Deposits Safely (#4422)

* safely prune cache

* use proper method

* preston's,terence's reviews and comments

* revert change to build files

* use as feature config instead
* Release custom state ssz (#4421)

* Release custom state ssz, change all HTR of beacon state to use custom method

* typo

* use mainnet config

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Update initial sync save justified to align with v0.9.3 (#4432)
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fix build
* don't blacklist on pubsub (#4435)
* Fix Flakey Slot Ticker Test (#4434)

* use interface instead for the slot ticker

* fixed up flakey tests

* add gen time

* get duties comment

* fix lifecycle test

* more fixes
* Configurable min genesis delay (#4437)

* Configurable min genesis delay based on https://github.com/ethereum/eth2.0-specs/pull/1557

* remove feature flag for genesis delay

* fix

* demo config feedback
* patch readme
* save keys unencrypted for validators (#4439)
* Add new demo configuration targeting mainnet scale (#4397)

* Add new demo configuration targeting mainnet, with 1/10th of the deposit value

* reduce quotant by 1/10th. Use 1/10th mainnet values

* only change the inactivity quotant

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save justified checkpoint state (#4433)

* Save justified checkpoint state

* Lint

* Feedback

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update shared/testutil/deposits.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update proto/testing/ssz_regression_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/epoch/epoch_processing.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/kv/forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/proposer.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/prepare_forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/aggregator/server.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/cache/depositcache/pending_deposits.go
* Update beacon-chain/cache/depositcache/pending_deposits_test.go
* Update beacon-chain/rpc/validator/proposer.go
* Merge refs/heads/master into v0.9.2
* Fix e2e genesis delay issues (#4442)

* fix e2e genesis delay issues

* register flag

* typo

* Update shared/featureconfig/config.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* skip demo e2e

* fix validator

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Batch Eth1 RPC Calls (#4392)

* add new methods

* get it working

* optimize past deposit logs processing

* revert change

* fix all tests

* use mock

* lint

* lint

* check for nil

* stop panics

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Terence's Review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-01-07 18:47:39 +00:00
Preston Van Loon
b08f3f760d Update ethereumapis (#4142)
* update ethereumapis from https://github.com/prysmaticlabs/ethereumapis/pull/55
* add stub for https://github.com/prysmaticlabs/prysm/issues/4141
2019-11-29 16:44:51 +00:00
Raul Jordan
0326be86b5 Apply Patch Rules to Use EthereumAPIs Generated Protos in Prysm (#4112)
* starting on patch
* finish determining all required patches
* properly redefine the patch rules
* new patch
* rem double semicolon
* fix patch file
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-eth-protos
* building the deps
* test target passes using ethereumapis
* compile gateway
* attempting to build everything
* e2e use ethereumapis
* more fixes for slasher
* other item
* getting closer to compiling slasher
* build slasher package
* Merge branch 'master' into deprecate-eth-protos
* Merge branch 'master' into deprecate-eth-protos
* fix benches
* lint gazelle
* Merge branch 'deprecate-eth-protos' of github.com:prysmaticlabs/prysm into deprecate-eth-protos
* proper gateway
* lint
* Merge branch 'master' into deprecate-eth-protos
* fix build
* Merge branch 'deprecate-eth-protos' of github.com:prysmaticlabs/prysm into deprecate-eth-protos
* use swag
* resolve
* ignore change
* include new patch changes
* fix test
* builds
* fix e2e
* gaz
2019-11-27 05:08:18 +00:00
Nishant Das
7d0e5a9dc4 Use Latest Vote Map (#4102)
* add latest vote map
* fix all tests
* remove db crud methods
* Merge branch 'master' into latestVoteMap
* preston's review
* Merge branch 'latestVoteMap' of https://github.com/prysmaticlabs/geth-sharding into latestVoteMap
2019-11-25 16:34:20 +00:00
terence tsao
f5cb04012e Aggregator selection from RPC to validator client (#4071)
* Config
* Updated proto
* Updated pool
* Updated RPC
* Updated validator client
* run time works
* Clean ups
* Fix tests
* Visibility
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into aggregator
* Raul's feedback
* Tests for RPC server
* Tests for validator client
* Span
* More tests
* Use go routine for SubmitAggregateAndProof
* Go routines
* Updated comments
* Use array of roles
* Fixed tests
* Build
* Update validator/client/runner.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update validator/client/runner.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* If
* Merge branch 'refactor-validator-roles' of https://github.com/prysmaticlabs/prysm into refactor-validator-roles
* Empty
* Feedback
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into aggregator
* Removed proto/eth/v1alpha1/shard_chain.pb.go?
* Cleaned up
* Revert
* Comments
* Lint
* Comment
* Merge branch 'master' into aggregator
2019-11-22 05:11:38 +00:00
Nishant Das
a938c305b4 Do Not Reject Attestations in Genesis Epoch (#3955) 2019-11-08 08:09:07 -08:00
terence tsao
768c2bd812
Block validator from getting assignment until beacon node is sync'ed (#3953) 2019-11-07 17:59:11 -08:00
Nishant Das
26af4496c0 Batch Validator Performance Requests (#3520)
* change proto msg types

* change server and client

* regen protos
2019-09-18 11:47:14 -05:00
terence tsao
9ab08e6998
Remove beacon rpc service (#3515)
* first version of the watchtower api

* first version

* delete watchtower

* move to message loop

* roughtime

* one time

* fix test

* Fixed test

* Fixed proposer server

* Gaz

* gaz

* Stuck

* Tests passing

* Fixed all the tests
2019-09-18 06:34:50 -07:00
Nishant Das
d597410d9b MultiLocking in Operations Service (#3470)
* add lock

* add cache

* modify lock retrieval
2019-09-15 13:51:37 -05:00
terence tsao
9d62e542e5
Clean up validator server (#3466) 2019-09-14 13:31:38 -07:00
Nishant Das
590aaaf370 Fix Batched Block Response (#3012) 2019-07-24 07:53:38 -07:00
terence tsao
41e55a6902
Sync with latest eth api definitions (#3059) 2019-07-23 14:33:25 -07:00
terence tsao
1b5b8a57e0 Remove unused proto schemas (#3005)
* Update io_kubernetes_build commit hash to 1246899

* Update dependency build_bazel_rules_nodejs to v0.33.1

* Update dependency com_github_hashicorp_golang_lru to v0.5.1

* Update libp2p

* Update io_bazel_rules_k8s commit hash to e68d5d7

* Starting to remove old protos

* Bazel build proto passes

* Fixing pb version

* Cleaned up core package

* Fixing tests

* 6 tests failing

* Update proto bugs

* Fixed incorrect validator ordering proto

* Sync with master

* Update go-ssz commit

* Removed bad copies from v1alpha1 folder

* add json spec json to pb handler

* add nested proto example

* proto/testing test works

* fix refactoring build failures

* use merged ssz

* push latest changes

* used forked json encoding

* used forked json encoding

* fix warning

* fix build issues

* fix test and lint

* fix build

* lint
2019-07-22 10:03:57 -04:00
Preston Van Loon
e744d1a07e Spec freeze updates (#2312)
* Optimize Shuffled Indices Cache (#2728)

* Refactor Deposit Contract Test Setup (#2731)

* add new package

* fix all tests

* lint

* change hash function (#2732)

* Remove Deprecated Validator Protobuf (#2727)

* Remove deprecated validator protos

* Fix to comments

* Fix most of skipped tests (#2735)

* Cache Active Validator Indices, Count, and Balances (#2737)

* Optimize Base Reward Calculation (#2753)

* benchmark process epoch

* revert prof.out

* Add some optimizations

* beware where we use ActiveValidatorIndices...

* revert extra file

* gaz

* quick commit to get feedback

* revert extra file

* started fixing tests

* fixed broken TestProcessCrosslink_NoUpdate

* gaz

* cache randao seed

* fixed all the tests

* fmt and lint

* spacing

* Added todo

* lint

* revert binary file

* started regression test

* basic tests done

* using a fifo for active indices cache

* using a fifo for active count cache

* using a fifo for total balance cache

* using a fifo for active balance cache

* using a fifo for start shard cache

* using a fifo for seed cache

* gaz

* clean up

* fixing tests

* fixed all the core tests

* fixed all the tests!!!

* lint

* comment

* rm'ed commented code

* cache size to 1000 should be good enough

* optimized base reward

* revert binary file

* Added comments to calculate adjusted quotient outside

* removed deprecated configs (#2755)

* Optimize Process Eth1 Data Vote (#2754)

* Cleanup and Docs update (#2756)

* Add graffiti and update generate seed (#2759)

* Benchmark Process Block with Attestations (#2758)

* Tidying up Godoc for Core Package (#2762)

* Clean up Old RPC Endpoints (#2763)

* Update RPC end point for Proposer (#2767)

* add RequestBlock

* run mockgen

* implemented RequestBlock

* updated proto definitions

* updated tests

* updated validator attest tests

* done

* comment

* todo issue

* removed unused proto

* Update attesting indices v0.6 (#2449)

* sort participants slice

* add bitfield functions and tests

* added BitfieldBit test

* add AttestationParticipantsNew

* revert AttestationParticipants to its previous change

* add tests

* remove verifybitfieldnew

* fix tests and remove multiple tests

* remove duplicate test

* change magic number into ceildiv8

* Implement Justification and finalization Processing (#2448)

* Add convert to indexed (#2519)

* sort participants slice

* add bitfield functions and tests

* added BitfieldBit test

* add AttestationParticipantsNew

* revert AttestationParticipants to its previous change

* add tests

* remove verifybitfieldnew

* fix tests and remove multiple tests

* remove duplicate test

* start work

* convert attestation to indexed attestations

* fix test for convert index

* remove calling getter

* add more tests

* remove underscore

* changes name to signature (#2535)

* update registry updates func (#2521)

* update registry updates func

* added tests and moved to epoch processing

* fixed naming issues

* Update Committee Helpers to v0.6.0 (#2398)

* Update Committee Helper Part 2 (#2592)

* Implement Process Slashings for 0.6 (#2523)

* Update Proposer/Attester Slashings and Slashing Helpers (#2603)

* Implement Final Updates 0.6 (#2562)

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* implemented process_final_updates

* move to the top of the file

* add comment back

* gaz

* Test for process final updates

* fixed tests

* fixed all the tests

* Update Reward Helper v0.6 (#2470)

* added BaseReward

* added rewards helper

* added test for BaseReward

* extra space

* move exported function above

* update to new spec (#2614)

* Update Block Processing Voluntary Exits (#2609)

* Update processEth1Data for v0.6 (#2516)

* Clean up Helper Functions Part 1 (#2612)

* Finalize helper functions for 0.6 (#2632)

* Process Beacon Chain Transfers v0.6 (#2642)

* add transfers

* beacon transfer operations complete

* full cov

* transfer testing

* finished tests

* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/core/blocks/block_operations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Implement Process Crosslink From 0.6 (#2460)

* Process Block Eth1 Data v0.6 (#2645)

* Get attestation data slot v0.6 (#2593)

* attestation.go is ready, tests not

* ready for review

* fixing linter issues

* modified Crosslink and AttestationData proto fields to spec 2.0,marked deprecated fields

* gazelle

* fixed tests

* fixed error

* error msg

* Process Block Deposits v0.6 (#2647)

* imports fixes

* deposits tests pass

* wrapped up gazelle

* spacing

* Implement Crosslink Delta Rewards for 0.6 (#2517)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* merged master

* all tests passing

* start testing

* done

* add ProcessBlockHeader v0.6 (#2534)

* add ProcessBlockHeader

* function has all its dependancies in place

* arange the basic ok test

* gazzele and skip test update

* skip wrong sig test

* fmt imports and change requests

* goimports fmt

* map for struct fields to be location independent

* reorder protobuf fields

* added tests

* gazzle fix

* few change requests fixes

* revert changes in types.proto

* revert changes in types

* fix tests

* fix lint

* fmt imports

* fix gazelle

* fix var naming

* pb update

* var naming

* tarance change request fixes

* fix test

* Add Process Registry for Epoch Processing (#2668)

* update update-process-registry

* added back the old tests

* fmt

* gaz

* Follow up on process block header v0.6 (#2666)

* Putting Crosslink Delta Back (#2654)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* merged master

* all tests passing

* start testing

* done

* fixed tests

* addressed shay's feedback

* Implement Attestation Delta for v0.6 (#2646)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* implemented process_attestation_delta

* comments

* comments

* merged master

* all tests passing

* start testing

* done

* merged master

* fixed tests

* tests, more to come

* tests done

* lint

* spaces over tabs

* addressed shay's feedback

* merged master

* Implement process_rewards_and_penalties for 0.6 (#2665)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* implemented process_attestation_delta

* comments

* comments

* merged master

* all tests passing

* start testing

* done

* merged master

* fixed tests

* tests, more to come

* tests done

* lint

* spaces over tabs

* addressed shay's feedback

* starting but need to merge a few things...

* tests

* fmt

* Update Slot Processing and State Transition v0.6 (#2664)

* edit state transition to add slot processing logic, reorder logic

* fix build

* lint

* tests passing

* spacing

* tests pass

* imports

* passing tests

* Implement Process Epoch for 0.6 (#2675)

* can't find process j f functons

* implemented process_epoch

* tests done

* lint

* nishant's feedback

* stupid goland replace

* goimports

* Update CommitteeAssignment (#2693)

* cleaned up skipped tests for core processing (#2697)

* Process Block Attestations v0.6 (#2650)

* attestation.go is ready, tests not

* ready for review

* fixing linter issues

* modified Crosslink and AttestationData proto fields to spec 2.0,marked deprecated fields

* gazelle

* fixed tests

* fixed error

* add att processing:

* process atts

* error msg

* finish process attestations logic

* spacing

* ssz move

* inclusion delay failure passing

* more attestation tests

* more att tests passing

* more tests

* ffg data mismatching test

* ffg tests complete

* gofmt

* fix testing to match attestation updates

* ssz

* lint

* Fixed Skipped Tests for RPC Server (#2712)

* Remove Obsolete Deposit Proto Objects (#2673)

* remove deposit data

* remove deposit input

* fix references

* remove deposit helpers

* fix all refs

* gaz

* rgene proto

* fix all tests

* remove deposit data deprecated field

* fix remaining references

* fix all tests

* fix lint

* regen proto

* fix test

* Remove Deprecated Protobuf State Fields (#2713)

* Remove Deprecated Protobuf Crosslink/Slashing/Block Fields (#2714)

* Remove Deprecated Beacon Block Proto Fields (#2717)

* Remove Deprecated Attestation Proto Fields (#2723)

* Cache Shuffled Validator Indices (#2682)

* YAML shuffle tests for v0.6 (#2667)

* new shuffle tests

* added comment for exported function

* fix format and print

* added config files handling

* gazelle fix

* shuffle test debugging

* added shuffle list and benchmark

* hash function addition from nishant code

* gazelle fix

* remove unused function

* few minor changes

* add test to test protos optimization

* test a bigger list

* remove commented code

* small changes

* fix spec test and test indices to pass

* remove empty line

* abstraction of repeated code and comment arrangement

* terence change requests

* fix new test

* add small comment for better readability

* change from unshuflle to shuffle

* comment

* better comment

* fix all tests

* Remove Latest Block (#2721)

* lint

* remove latest block

* lint

* add proto

* fix build

* Fix Deposit Trie (#2686)

* remove deposit data

* remove deposit input

* fix references

* remove deposit helpers

* fix all refs

* gaz

* rgene proto

* fix all tests

* remove deposit data deprecated field

* fix remaining references

* fix all tests

* fix lint

* new tests with contract

* gaz

* more tests

* fixed bugs

* new test

* finally fixed it

* gaz

* fix test

* Remove Committee Cache (#2729)

* Benchmark Compute Committee (#2698)

* Fixed Skipped Attestation Tests (#2730)

* Optimize Shuffled Indices Cache (#2728)

* Refactor Deposit Contract Test Setup (#2731)

* add new package

* fix all tests

* lint

* change hash function (#2732)

* Remove Deprecated Validator Protobuf (#2727)

* Remove deprecated validator protos

* Fix to comments

* Fix most of skipped tests (#2735)

* Optimize Base Reward Calculation (#2753)

* benchmark process epoch

* revert prof.out

* Add some optimizations

* beware where we use ActiveValidatorIndices...

* revert extra file

* gaz

* quick commit to get feedback

* revert extra file

* started fixing tests

* fixed broken TestProcessCrosslink_NoUpdate

* gaz

* cache randao seed

* fixed all the tests

* fmt and lint

* spacing

* Added todo

* lint

* revert binary file

* started regression test

* basic tests done

* using a fifo for active indices cache

* using a fifo for active count cache

* using a fifo for total balance cache

* using a fifo for active balance cache

* using a fifo for start shard cache

* using a fifo for seed cache

* gaz

* clean up

* fixing tests

* fixed all the core tests

* fixed all the tests!!!

* lint

* comment

* rm'ed commented code

* cache size to 1000 should be good enough

* optimized base reward

* revert binary file

* Added comments to calculate adjusted quotient outside

* removed deprecated configs (#2755)

* Optimize Process Eth1 Data Vote (#2754)

* Cleanup and Docs update (#2756)

* Add graffiti and update generate seed (#2759)

* Benchmark Process Block with Attestations (#2758)

* Tidying up Godoc for Core Package (#2762)

* Clean up Old RPC Endpoints (#2763)

* Update RPC end point for Proposer (#2767)

* add RequestBlock

* run mockgen

* implemented RequestBlock

* updated proto definitions

* updated tests

* updated validator attest tests

* done

* comment

* todo issue

* removed unused proto

* Cache Active Validator Indices, Count, and Balances (#2737)

* Update Deposit Contract (#2648)

* lint

* add new contract

* change version

* remove log

* generating abi and binary files

* fix tests

* update to current version

* new changes

* add new hash function

* save hashed nodes

* add more things

* new method

* add update to trie

* new stuff

* gaz

* more stuff

* finally fixed build

* remove deposit data

* Revert "remove deposit data"

This reverts commit 9085409e91be0c94550af10290eaad72ad40a6de.

* more changes

* lint and gaz

* lint

* Update Shard Helpers for 0.6 (#2497)

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* Update Genesis State Function to v0.6 (#2465)

* add pseudocode

* make changes

* fix all tests

* fix tests

* lint

* regen protos and mocks

* regenerated protos

* started fixing core

* all core tests passing!

* removed shared/forkutils

* started fixing blockchain package

* lint

* updating rpc package

* add back deleted stuff

* add back deleted stuff that was deleted accidentally

* add back protos and mocks

* fix errors

* fix genesis issue

* fix genesis issue for slot ticker

* fix all genesis errors

* fix build files

* temp change for go-ssz

* fix test

* Revert "temp change for go-ssz"

This reverts commit 3411cb9d6d519cb521181486debc3b21603c8873.

* update to latest go-ssz

* unstaged changes

* Update Attester Server RPC Calls (#2773)

* Update config and function parameters to v0.7 (#2791)

* Minor Updates to 0.7 (#2795)

* Refactor Deposit Flow and Cleanup Tests (#2788)

* More WIP on cleaning deposit flow

* Fix tests

* Cleanup and imports

* run gazelle

* Move deposit to block_operations

* gazelle

* Update beacon-chain/core/blocks/block_operations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Fix docs

* Remove unneeded calculations

* Fix tests

* Fix tests finally (?)

* Optimize Committee Assignment RPC (#2787)

* Update BlockRoot to BlockHash (#2816)

* Fix Final Missing Items in Block Processing v0.6 (#2710)

* override config successfully

* passes processing

* add signing root helper

* blockchain tests pass

* tests blocked by signing root

* lint

* fix references

* fix protos

* proper use of signing root

* only few failing tests now

* fix final test

* tests passing

* lint and imports

* rem unused

* Update beacon-chain/core/blocks/block_operations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* lint

* Update beacon-chain/attestation/service_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/db/block_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* rename to hash tree root

* rename decode to unmarshal

* fix

* use latest ssz

* all tests passing

* lint

* fmt

* Add Config YAML for Spec Tests (#2818)

* Align Protobuf Type Names (#2825)

* gofmt

* Revert "Align Protobuf Type Names (#2825)" (#2827)

This reverts commit 882d067144c674bbf4eeee00404acaa0a9b1fd33.

* Update Domain Related Functions (#2832)

* Add Functions for Compressed and Uncompressed HashG2 With Domain (#2833)

* add tests

* gaz

* lint

* Revert "Add Functions for Compressed and Uncompressed HashG2 With Domain (#2833)" (#2835)

This reverts commit 7fb2ebf3f12f043d7bd12f43500dc2bd133df573.

* Add ConvertToPb to package testutil (#2838)

* Block Processing Bug Fixes (#2836)

* Update types PB with Size Tags (#2840)

* Epoch processing spec tests (#2814)

* Remove Deposit Index (#2851)

* Shuffle tests revisited (#2829)

* first commit

* remove old files, add log

* remove duplicate yaml testing code

* reduce visability

* nishant feedback changes

* skip TestFromYaml_Pass

* added tags to bazel build

* gazelle fix

* remove unused vars

* adda back config

* remove config handling

* remove unused var

* gazelle fix

* SSZ compatibility test for protobufs (#2839)

* update workspace spec sha

* remove yamls from branch

*  BLS spec tests (#2826) (#2856)

* bls spec tests

* add more bls tests

* use ioutil instead of bazel runfiles

* dont read bytes

* skip tests that overflow uint64

* manually fix input data

* add tests

* lint and gaz

* add all new changes

* some refactoring, cleanup, remove new API methods that only exist for tests

* gaz

* Remove yamls, skip test

* Slot processing spec test (#2813)

* eth1data rpc endpoint (#2733)

* eth1data rpc endpoint

* first version

* comment added

* gazelle fix

* new function to go once over the deposit array

* fix tests

* export DepositContainer

* terence feedback

* move structure decleration

* binary search

* fix block into Block

* preston feedback

* keep slice sorted to remove overhead in retrival

* merge changes

* feedback

* update to the latest go-ssz

* revert change

* chnages to fit new ssz

* revert merge reversion

* go fmt goimprts duplicate string

* exception for lint unused doesParentExist

* feedback changes

* latesteth1data to eth1data

* goimports and stop exposing Eth1Data

* revert unneeded change

* remove exposure of DepositContainer

* feedback and fixes

* fix workspace duplicate dependancy

* greatest number of deposits at current height

* add count votes function

* change method name

* revert back to latesteth1data

* latesteth1data

* preston feedback

* seperate function add tests fix bug

* stop exposing voteCountMap

* eth1data comment fix

* preston feedback

* fix tests

* new proto files

* workspace to default version of ssz

* new ssz

* chnage test size

* marshalled  marshaled

* Attesting Indices Fix (#2862)

* add change

* fix one test

* fix all tests

* add test

* clear cache

* removed old chaintest, simulated backend and state generator (#2863)

* Block Processing Sanity Spec Tests (#2817)

* update PrevEpoch

* add new changes

* shift to blocks package

* add more changes

* new changes

* updated pb with size tags

* add new changes

* fix errors

* uncomment code

* more changes

* add new changes

* rename and lint

* gaz

* more changes

* proccess slot SigningRoot instead of HashTreeRoot

* ensure yaml generated structs work

* block sanity all passing

* minimal and mainnet all pass

* remove commented code

* fix one test

* fix all tests

* fix again

* no state comparison

* matching spec

* change target viz

* comments gazelle

* clear caches before test cases

* latest attempts

* clean up test format

* remove debugging log, remove yaml

* unskip attestation

* remove skip, check post state, diff state diffs

* handle err

* add bug fixes

* fixed one more bug

* fixed churn limit bug

* change hashProto to HashTreeRoot

* all tests pass :)

* fix all tests

* gaz

* add regression tests

* fix test bug

* Mutation testing fixes for beacon-chain/core/helpers/attestation.go (#2868)

* mutation testing for attestation.go

* new line

* lint

* revert fmt.Errorf deletion

* gofmt

* Add some fixes for mutation testing on blocks.go (#2869)

* Fix sizes

* gaz

* Spec freeze release candidate spectests

* Align Protobuf Type Names  (#2872)

* Removes some deprecated fields from protobuf (#2877)

* search and replace checkpoints

* fix tests, except spec tests

* Update Configs for Freeze (#2876)

* update configs

* updated minimal configs

* almost there

* all tests passing except for spec tests

* better comment for MinGenesisTime

* done, ready for review

* rm seconds per day

* feedback

* Mutation testing fixes for beacon-chain/core/helpers/committee.go (#2870)

* Add some fixes for mutation testing on blocks.go

* working on mutation testing fo committee.go

* gofmt

* goimports

* update readme target

* update latest sha for spec tests

* fix build

* Update State Transition Function (#2867)

* Change Base Reward Factor (#2888)

* Update Freeze Spec Simplification Section - part 1 (#2893)

* finished changes to attesting_indices

* removed index_count <= 2**40 requirement

* lint

* reverted index_count <= 2**40 check

* added short cut len(a) > len(b)

* Update justification bits (#2894)

* updated all the helper pseudocodes (#2895)

* Make Constants Explicit and Minor Cleanups (#2898)

* Rename outdated configs, make constants explicitly delcared

* Remove activate_validator, not needed

* Remove GenesisSlot and GenesisEpoch

* Remove unused import

* Move Block Operation Length Checks to ProcessOperations (#2900)

* Move block operation length checks to ProcessOperations

* Write tests for each length check in ProcessOperations

* Remove unneeded test

* Move checks to a new function

* Move duplicate check back into ProcessOperations

* reorder proto fields (#2902)

* Slashing Penalty Calculation Change (#2889)

* lint

* change config val

* add max helper

* changes to slashing and process slashing, add a min function for integers

* gaz

* fix failing tests

* fix test

* fixed all tests

* Change Yaml tag

* lint

* remove gc hack

* fix test

* gaz

* preston's comments

* change failing field

* fix and regen proto

* lint

* Implement Compact Committee Root (#2897)

* add tags

* add function

* add new code

* add function

* add all new changes

* lint

* add tests

* fix tests

* fix more tests

* fix all outstanding tests

* gaz

* Update beacon-chain/core/helpers/committee.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* comment

* Remove deprecated fields from attestation data (#2892)

* fix broken tests

* remove comment

* fixes

* Update Deposit Contract (#2903)

* update to new contract

* fix references

* fix tests

* fix some more tests

* fix local deposit trie

* gaz

* shays review

* more changes

* update WORKSPACE to use 0.8 spec tests

* Perform Mutesting in Helpers Package (#2912)

* Perform mutesting on validator

* Mutesting in helpers package

* Mutested eth1data

* s/volundary/voluntary (#2914)

* Update BLS Domain (#2916)

* change from integer to byte slice

* add test

* fix func for bytes4

* Fix Spec tests (#2907)

* fix panics

* handle failed transitions

* remove log

* fix to protos

* new changes

* remove i

* change ssz commit

* new changes

* update epoch tests

* fix epoch testing

* fix shuffle tests

* fix test

* Perform Mutesting in Epoch and State Packages (#2913)

* done with updates (#2890)

* Add Max Size Tag for Protobuf Fields (#2908)

* No more space between ssz-size numbers

* Regen pb.go

* Fixed a few incorrect proto fields (#2926)

* Update Validator Workflow (#2906)

* Justification spec tests (#2896)

* update go-ssz

* Fix SSZ Compatibility Test (#2924)

* figuring out how to seqeeze in multiple fields in a tag for pb

* Added max tags and regenerated pb.go

* updated to new standard

* New bitfield types in proto (#2915)

* Cast bytes to correct bitfield, failing tests now though

* Add forked gogo/protobuf until https://github.com/gogo/protobuf/pull/582

* remove newline

* use proper override for gogo-protobuf

* fix a few tags

* forgot to include custody bits and Slashable not used

* Update yaml struct to use pb

* Update workspace to use latest ssz

* All tests fail

* Use the latest go-ssz commit

* All pass except for state (too long to taste)

* Update test.proto

* Added rest of the tests

* use 1 justification bits

* fix tag test, apply @rauljordan's suggestion

* add IsEmpty and use ssz struct

* delete unused file

* Update zero hash to sha256().digest() (#2932)

* update zero hash

* change zero hash to conform with spec

* goimports

* add test for zero hash

* Revert "Update zero hash to sha256().digest() (#2932)" (#2933)

This reverts commit b926ae0667b18aef3f7e0e8ec8a9b3e98b3d2ccc.

* Fix compress validator (#2936)

* fix compress validator

* update go-ssz

* build without the bytes test

* try minimal

* Block operations spec tests (#2828)

* update PrevEpoch

* debugging proposer slashing tests

* fmt

* add deposit tests

* Added skeleton for attestation minimal test

* remove bazel runfiles thing

* add deposits

* proposer slashing test is done

* comment

* complete test, some failing cases

* sig verify on

* refactor slightly to support mainnet and minimal

* included mainnet stuff

* Add block header tests

* volunary exit done

* transfer done

* new changes

* fix all tests

* update domain functions

* fmt

* fixed lint

* fixed all the tests

* fixed a transfer bug

* finished attester slashing tests and fixed a few bugs

* started fixing...

* cleaned up exit and proposr slashing tests

* attester slashing passing

* refactored deposit tests

* remove yamls, update ssz

* Added todo for invalid sig

* gazelle

* deposits test done!

* transfer tests done and pass!

* fix attesting indices bug

* temporarily disabled signature verification

* cleaned up most of the block ops, except for att

* update committee AttestingIndices

* oops, i dont know how or why i changed this file

* fixed all the rpc tests

* 6 more failing packages

* test max transfer in state package

* replace hashproto with treehash in package blockchain

* gazelle

* fix test

* fix test again

* fixed transition test, 2 more left

* expect an error in attestation tests

* Handle panic when no votes in aggregate attestation

* clear cache

* Add differ, add logging, tests pass yay

* remove todo, add tag

* fixed TestReceiveBlock_RemovesPendingDeposits

* TestAttestationMinimal/success_since_max_epochs_per_crosslink fails now...

* handle panics

* Transfer tests were disabled in https://github.com/ethereum/eth2.0-specs/pull/1238

* more fixes after merge, updating block_operations.yaml.go to match yaml

* figuring out how to seqeeze in multiple fields in a tag for pb

* Added max tags and regenerated pb.go

* updated to new standard

* New bitfield types in proto (#2915)

* Cast bytes to correct bitfield, failing tests now though

* Add forked gogo/protobuf until https://github.com/gogo/protobuf/pull/582

* remove newline

* fix references and test panic

* change to proto objects from custom types

* fix panics in tests

* use proper override for gogo-protobuf

* fix a few tags

* fix tests

* forgot to include custody bits and Slashable not used

* fix tests

* sort again

* Update yaml struct to use pb

* Update workspace to use latest ssz

* All tests fail

* Use the latest go-ssz commit

* All pass except for state (too long to taste)

* Update test.proto

* Added rest of the tests

* use 1 justification bits

* minor fixes

* wrong proto.Equal

* fix tag test, apply @rauljordan's suggestion

* add IsEmpty and use ssz struct

* inverted logic

* update zero hash

* change zero hash to conform with spec

* goimports

* add test for zero hash

* Revert "Update zero hash to sha256().digest() (#2932)"

This reverts commit b926ae0667b18aef3f7e0e8ec8a9b3e98b3d2ccc.

* update ssz, fix import, shard big test

* checkpoint

* fix compress validator

* update go-ssz

* missing import

* missing import

* tests now pass

* been a good day

* update test size

* fix lint

* imports and remove unused const

* update bazel jobs flag

* update bazel jobs flag

* satisfy deprecation warning

* Add ssz regression tests for investigation of test failures in PR #2828 (#2935)

* Adding regression tests for investigation

* add another example

* goimports

* add quick comment about test case 0

* Epoch Process Slashings Spec Tests (#2930)

* updated justification bits, tests passing OK

* regen pb.go, clarify bit operations

* justification and finalization tests; failing

* Add wrapper, so we call the correct method

* checkpoint

* Update tar ref

* TestSlashingsMinimal/small_penalty still failing

* Use bigint instead of () and float

* Revert a bad merge from workspace

* Fmt

* add note about https://github.com/ethereum/eth2.0-specs/issues/1284

* improve tests

* gaz

* Perform Mutesting In core/state Package (#2923)

* Perform mutesting on validator

* Mutesting in helpers package

* Mutested eth1data

* Perform mutesting in epoch and state packages

* Fix voluntary exits test

* Fix typo

* Fix comments

* Fix formatting

* Fix error message

* Handle missing errors

* Handle all errors

* Perform Mutesting In State Package

* Fix block roots size

* Remove comment

* Fix error

* add backend service

* Add ssz compatibility tests for signing root (#2931)

* Added tests for signing root

* imports

* fix lint on travis

* fix bes flag

* Final updates spec tests (#2901)

* set up tests

* need to reorder pbs

* figuring out how to seqeeze in multiple fields in a tag for pb

* Added max tags and regenerated pb.go

* updated to new standard

* New bitfield types in proto (#2915)

* Cast bytes to correct bitfield, failing tests now though

* Add forked gogo/protobuf until https://github.com/gogo/protobuf/pull/582

* remove newline

* use proper override for gogo-protobuf

* fix a few tags

* forgot to include custody bits and Slashable not used

* playing with tags idea, can revert this commit later

* fixes after merge

* reset caches before test

* all epoch tests pass

* gazelle

* Genesis trigger (#2905)

* genesis change

* integrate changes

* bodyroot

* remove unused code

* HasChainStarted

* added isValidGenesisState to ProcessLog

* state fix

* fix gazelle

* uint64 timestamp

* SetupInitialDeposits adds proof

* remove unneeded parts of test

* deposithash

* merkleproof from spec utils

* Revert "merkleproof from spec utils"

This reverts commit 1b0a124352e7b62e3c3220fb0d64e295b474b430.

* fix test failures

* chain started and hashtree root in tests

* simple eth2genesistime

* eth2 genesis time

* fix zero time

* add comment

* remove eth1data and feedback

* fix build issues

* main changes: add fields and methods to track active validator
count

* gaz

* fix test

* fix more tests

* improve test utils

* shift spec method to state package, improve test setup

* fixed log processing tests

* remove log

* gaz

* fix invalid metric

* use better tag names, not latest

* Remove Block Signing Root (#2945)

* replace with hash tree root

* Revert "replace with hash tree root"

This reverts commit 77d8f16a160e42f3c3d598df66c30a66657de1bf.

* replace with signing root instead

* remove one more ref

* Create Test Runner for Genesis State Spec Tests (#2940)

* genesis change

* integrate changes

* bodyroot

* remove unused code

* HasChainStarted

* added isValidGenesisState to ProcessLog

* state fix

* fix gazelle

* uint64 timestamp

* SetupInitialDeposits adds proof

* remove unneeded parts of test

* deposithash

* merkleproof from spec utils

* Revert "merkleproof from spec utils"

This reverts commit 1b0a124352e7b62e3c3220fb0d64e295b474b430.

* fix test failures

* chain started and hashtree root in tests

* simple eth2genesistime

* eth2 genesis time

* fix zero time

* add comment

* remove eth1data and feedback

* fix build issues

* main changes: add fields and methods to track active validator
count

* gaz

* fix test

* fix more tests

* improve test utils

* Start genesis spec tests

* shift spec method to state package, improve test setup

* Add Genesis validity spec test

* Bazel

* fixed log processing tests

* remove log

* gaz

* fix invalid metric

* use json tags

* fix up latest changes

* Fix most of test errors

* Attempts to see whats wrong with genesis validity

* Fix merge

* skip minimal

* fix state test

* new commit

* fix nishant comment

* gaz

* Static check on branch spec-v0.6 (#2946)

* Ran staticcheck and fixed the important complains

* commit

* commit

* Create Test Runner for Genesis State Spec Tests (#2940)

* genesis change

* integrate changes

* bodyroot

* remove unused code

* HasChainStarted

* added isValidGenesisState to ProcessLog

* state fix

* fix gazelle

* uint64 timestamp

* SetupInitialDeposits adds proof

* remove unneeded parts of test

* deposithash

* merkleproof from spec utils

* Revert "merkleproof from spec utils"

This reverts commit 1b0a124352e7b62e3c3220fb0d64e295b474b430.

* fix test failures

* chain started and hashtree root in tests

* simple eth2genesistime

* eth2 genesis time

* fix zero time

* add comment

* remove eth1data and feedback

* fix build issues

* main changes: add fields and methods to track active validator
count

* gaz

* fix test

* fix more tests

* improve test utils

* Start genesis spec tests

* shift spec method to state package, improve test setup

* Add Genesis validity spec test

* Bazel

* fixed log processing tests

* remove log

* gaz

* fix invalid metric

* use json tags

* fix up latest changes

* Fix most of test errors

* Attempts to see whats wrong with genesis validity

* Fix merge

* skip minimal

* fix state test

* new commit

* fix nishant comment

* gaz

* Add Back Eth1Data After Bad Merge (#2953)

* eth1data rpc endpoint

* first version

* comment added

* gazelle fix

* new function to go once over the deposit array

* fix tests

* export DepositContainer

* terence feedback

* move structure decleration

* binary search

* fix block into Block

* preston feedback

* keep slice sorted to remove overhead in retrival

* merge changes

* feedback

* update to the latest go-ssz

* revert change

* chnages to fit new ssz

* revert merge reversion

* go fmt goimprts duplicate string

* exception for lint unused doesParentExist

* feedback changes

* latesteth1data to eth1data

* goimports and stop exposing Eth1Data

* revert unneeded change

* remove exposure of DepositContainer

* feedback and fixes

* fix workspace duplicate dependancy

* greatest number of deposits at current height

* add count votes function

* change method name

* revert back to latesteth1data

* latesteth1data

* preston feedback

* seperate function add tests fix bug

* stop exposing voteCountMap

* eth1data comment fix

* preston feedback

* fix tests

* new proto files

* workspace to default version of ssz

* new ssz

* chnage test size

* marshalled  marshaled

* everything passing again

* add skip reason

* cleanup deposit contract slightly

* remove unused chainstart param (#2957)

* fix breakages from #2957 (#2958)

* fix breakages from #2957

* oops

* Fix deposit input data (#2956)

* fix deposit input data

* fix deposit input data

* gaz and build fix

* Add Tests for Genesis Deposits Caching (#2952)

* remove old method and replace with an improved one

* add new files

* gaz

* add test

* added all these tests

* gaz

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fix merkle proof error

* fix config

* Minor fixes for runtime (#2960)

* Minor fixes for runtime

* use comments

* goimports

* revert beacon-chain/core/state/state.go and fix comment for lint

* fix test too

* Minor runtime fixes  (#2961)

* Add support for bundling binaries and fix ARM64 builds (#2970)

* Add support for bundling binaries and fix ARM64 builds

* Fix exports

* ignore manual targets wrt vis check

* fix graknlabs

* update spec tests (#2979)

* hotfix until https://github.com/graknlabs/bazel-distribution/pull/169

* Overflow slashing calculation fix (#2977)

* Runtime Fixes (#2736)

* first batch of fixes

* add log

* more fixes

* another bug fixed

* update deposit contract and other fixes

* remove logs

* new changes

* fixes

* fix build

* remove config

* more fixes

* add more changes

* add back todo

* make compute state root work

* remove commented out and fix condition

* fix commented code

* fix config

* gaz

* remove flag

* remove init

* new fixes

* fix test

* one more fix

* fix all tests

* change back config

* fix one more bug

* remove logging bool

* Only build test targets when running bazel test //...

* Align prysm to spec v0.8.1 (#2978)

* Bazel problem

* Update zero hash representation to be clear (cosmetic)

* Update minor cosmetic fixes

* Fixed lookahead off by 1

* Update randao.go

* update ssz

* test failures fixed

* test fixes

* fix up workspace

* lint

* fixed errs

* Updated pubkey loggings (#2983)

* Fix proposer assignment (#2984)

* add jvm limits

* add jvm limits

* Removed logging from state transition functions (#2980)

* Match spec on proposer index division (#2985)

* Match spec on proposer index division

* gaz

* fixes

* Fix Default Eth1Data (#2982)

* fix bug

* Update beacon-chain/core/state/transition.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* more fixes

* reg test

* Set ejection balance to 1.6 (#2987)

* improve block processing log (#2990)

* Fix HistoricalRootsLimit (#2989)

* fix historical lenght

* fix genesis state initialization and test

* fix genesis state initialization and test

* fix genesis state initialization and test

* fix genesis state initialization and test

* hack config until https://github.com/prysmaticlabs/prysm/issues/2993

* More Runtime Fixes (#2986)

* local changes

* add val sig

* attester fix

* one more fix

* fixed all tests

* rem validator issue

* fix finality issue (#2994)

* Fix validator prev balance calculation (#2992)

* attester fix

* one more fix

* fixed all tests

* Fix validator prev balances calculation

* go fmt

* 48 bytes

* Fix Querier to handle Deposit Logs Race (#2999)

* fix sync issue

* fix build

* add regression test

* add var for magic number

* Fix eth1data and deposits (#2996)

* Work in progress, eth1data works, deposits are included at the appropriate time, and activation happens at the correct time

* revert blockInfo being public

* git tests to build, not yet pass though

* add tests

* some commentary

* fix comment

* goimports

* fmt and remove unused method

* Update rules go (#2975)

* update rules_go

* Fix some cross compile builds stuff

* add missing deps

* update to 0.19.1

* Update Protobufs to Match Ethereum APIs (#2998)

* add beacon block and attestation files

* add all types

* include all new proto type definitions

* add add all proto definitions

* fix all comments to say 48 bytes

* include latest changes

* readd common

* no swag

* add build file

* deps issue

* right package names

* address feedback, maintain parity between upstream ethereumapis

* delete pb

* bad gens

* Update "Testing Prysm" readme section (#3000)

Resolves invalid url link to golangci-lint.

* Update badge to version 0.8.1

* elaborate on test skip

* revert shared/p2p/options.go

* make travis happy with goimports

* Update beacon-chain/core/blocks/block.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
2019-07-19 19:16:10 -05:00
Nishant Das
d48e0925d0 Add gRPC Gateway Folder (#2793)
* add gateway proto files

* lint
2019-06-12 10:11:04 -05:00
Dan
71b5d5beec Adding the BlockTreeBySlots function for getting the Block tree filte… (#2720) 2019-06-01 22:41:17 -07:00
Nishant Das
eef35996de Blacklist Keys in Cluster PK Manager (#2536)
* initialize rpc client

* add struct

* gaz

* add keymap

* update proto and add rpc methods

* update proto

* add method

* make changes

* add routine

* gaz

* mockgen

* error fix

* prom metric

* Some improvements

* fixes

* fix and working cluster pk manager

* fix and working cluster pk manager

* fix and working cluster pk manager

* fix and working cluster pk manager

* regen mocks and pb.go

* k8s
2019-05-12 16:38:37 -04:00
Raul Jordan
ecef1093eb
Fetch Block Tree from Justified Block to Highest Observed Slot via RPC (#2549)
* test block tree req

* tree improvement

* use the right data

* block tree blocked by children func

* rem file

* imports

* add ctx

* imports

* mock

* check expired context

* added block root

* gazelle

* sace
2019-05-09 12:38:05 -05:00
Raul Jordan
f876df42d7
ValidatorStatus Estimating Activation RPC Server (#2469)
* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename
2019-05-02 14:14:25 -05:00
terence tsao
de6ea81790 Recent Block Roots RPC API (#2455)
* add metrics for corret participation rate

* gaz

* add recent canonical roots API

* gaz

* all tests passed

* comments

* fmt
2019-05-01 10:14:46 -05:00
Preston Van Loon
030f44bb04 Revert "Treat Post & Pre-ChainStart Validators Equally in Assignments Fetching (#2074)" (#2086)
This reverts commit fcc18fac02.
2019-03-26 22:30:22 -05:00
Raul Jordan
fcc18fac02
Treat Post & Pre-ChainStart Validators Equally in Assignments Fetching (#2074)
* treat post/prechainstart validators equally in assignment fetching

* Update validator/client/fake_validator_test.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* status err code

* imports

* beyond activation epoch

* assigned to slot log

* gazelle
2019-03-26 09:03:05 -05:00
shayzluf
6335833c94 Validator: performing for multiple keys - update rpc proto (#2040)
* first version - broken

* Scripts to update mockgen

* working proto changes

* resolve review remarks

* fix goimport issues
2019-03-23 19:46:25 -05:00
Raul Jordan
7a83acf46d
Improve Validator Logging and Default Flags (#2013)
* improve validator logging significantly

* warn validator not yet assigned to epoch

* build added

* report validator logs

* validator performance logging

* all logging improvements

* validator attest better logs

* improved attester logging

* average balances

* dont hash useless hashes

* Update validator/client/validator_propose.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* address some comments

* standardize with block root hash32, renamings

* gazelle

* builds

* improve average eth balance

* eth net gains/losses

* fix tests

* spacing

* goimports

* avg balance

* update pbs

* addressed preston comments

* imports

* gazelle
2019-03-18 09:45:28 -06:00
Raul Jordan
c22e4dc2ae
Wait for Validator Activation Stream Server (#1896)
* create wait for activation stream

* wait for activation server stream open

* complete server side logic

* formatting

* first test passing

* context closed test

* lint

* fix build failure

* imports

* eliminate unused cancel var
2019-03-05 17:06:50 -06:00
terence tsao
b00891880e
Implement Validator Status RPC Methods (#1839) 2019-03-03 21:28:00 -08:00
terence tsao
17682321fe
Clean Up Validator RPC Server (#1798) 2019-03-03 14:55:12 -08:00
terence tsao
4bc324dc95
Make Getting Epoch Assignment Generic (#1746) 2019-02-27 17:16:05 -08:00
Ivan Martinez
136950e0c5 Update some of client to v0.3 spec (#1733)
* Update validator to v0.3 spec

* Fix typos with rename
2019-02-27 12:21:15 -08:00
Raul Jordan
e19920aec1
Miscellaneous Runtime Fixes & Improvements - Raul (#1674)
* fatal if impossible to receive chainstart

* fix tests

* fix

* custom delay

* completed custom delay

* errors

* better logs, nothing at genesis

* use demo in val

* add gazelle

* log

* starting to log stuff

* pass in ops

* avoid printing the large #s for debug, still working on tests..

* all around better logging

* fixed build error in epoch process

* fixed state transiton tests

* fixed block tests

* lint

* verify sigs in randao

* ready for inclusion falg

* only print waiting when slot is not valid

* fix build

* mod config

* fixed last justified slot issue

* fix inclusion

* fixed attestation issue

* using zero hash from params instead

* fix tests

* update balance

* removed swp

* more `- genesis_slot` for logs

* rem unused log

* fix broken tests

* account for skip slots in state root computation

* fixes done

* validator guide bug fixes - 671

* epoch boundary at the last slot of the epoch

* fix epoch issue

* more balance cal logs for debugging

* greater balance

* attestaton fixes

* fixes

* addressed testrun

* fixed ejection balance

* fix tests with far future epoch

* revert sync change

* revert initial sync change

* fix changes

* off by one att fix

* revert the att fix

* address comments

* format

* fix build

* rem file
2019-02-24 20:09:45 -06:00
terence tsao
06bfda24d5 Revert "Remove Redundant Committee Assignment Getters in Validator RPC Service (#1655)" (#1664)
This reverts commit 15f8b6a10d.
2019-02-20 14:46:30 -06:00
terence tsao
15f8b6a10d
Remove Redundant Committee Assignment Getters in Validator RPC Service (#1655) 2019-02-20 11:53:49 -08:00
Raul Jordan
19abe81472
Implement Randao Reveal Signing in Proposer Client (#1650)
* finish the BLS API wrapper

* all tests passing

* unexported comment

* gofmt tests for bls

* block processing test into own package to avoid cycle

* randao tests pass

* blocks test passing

* use common deposit generator

* helper

* resolved import cycle

* setup initial

* builds

* almost done with blockchain tests

* fix blockchain tests

* getting through with chaintests

* revert client change

* lint

* sync master conflict gazelle

* randao test fixes

* randao proposer impl

* tests pass
2019-02-20 12:58:34 -06:00
terence tsao
c30bef0812
Implement Next Epoch Committee Assignment for RPC Server (#1641) 2019-02-19 15:31:04 -08:00
Raul Jordan
c032b0f6b5
Pending Attestations RPC Server/Client Implementation (#1617)
* complete tests for integrating the attestations pool

* fix broken build
2019-02-15 18:36:40 -06:00
Raul Jordan
3d8bb73f15
Attester RPC Server Functions (#1505)
* initial validator attesthead rewrite based on proposer rewrite

* proceed with fetching committees and tree hashing the canonical head at assigned attester slot

* complete filling the properties of attestation data and all associated root hashes

* add when to attest todo

* finish entire attester client logic

* tests with mocks checked in

* tests passing in client

* stubbed out server implementation

* fixed build due to old property

* regen mocks with new mockgen version

* fixed broken tests

* complete bazel build fix

* address some review comments

* deep proto test

* server implementation logic

* attestation info tests

* completed passing tests, tree hash still blocked by error with nil fields unfortunately

* fix todo failure

* gazelle rerun and master merge

* testing full behavior of attestation info server

* finished all required server tests and implementation

* break loop

* fix tests due to genesis slot updates
2019-02-11 10:15:25 -06:00
Raul Jordan
61026103c6
Validator Attester Rewrite (#1487)
* initial validator attesthead rewrite based on proposer rewrite

* proceed with fetching committees and tree hashing the canonical head at assigned attester slot

* complete filling the properties of attestation data and all associated root hashes

* add when to attest todo

* finish entire attester client logic

* tests with mocks checked in

* tests passing in client

* stubbed out server implementation

* fixed build due to old property

* regen mocks with new mockgen version

* fixed broken tests

* complete bazel build fix

* address some review comments

* deep proto test

* tests passing after checking for empty committee and crosslink root

* address nishant comments
2019-02-06 10:20:38 -06:00
Preston Van Loon
4b27b8408b
Fix travis (#1493)
* fix travis

* test 2

* goimports

* Remove unused p2p config

* remove unused var
2019-02-05 10:46:32 -05:00
Preston Van Loon
4add403335
Validator Proposer Rewrite (#1462)
* WIP - with TODOs

* interface w/ test

* basic test for broadcast

* Add computeStateRoot funciton

* remove custody challenge

* resolve TODO lint issues

* more TODOs

* revert new line in types.proto

* broadcaster comment

* one of several failure condition tests

* Add test cases

* handle compute state error test case

* fix config in validator helpers

* fix tests too

* fix conflict

* partial PR feedback

* remove p2p

* gazelle

* package comment

* Better godoc
2019-02-05 08:47:25 -05:00
Raul Jordan
ec9240106e
Implement WaitForChainStart RPC Server and Client (#1421) 2019-01-30 13:28:53 +01:00
Raul Jordan
07c0387be1
Validator Epoch Assignments RPC Implementation (#1419)
* begin reorder

* move into beacon server

* add proposer server

* fix

* add proposer server

* wrap up rpc reorder

* eliminated deprecated RPC endpoints

* formatted nicely

* RPC protos

* fix lint

* integrate hash proto

* epoch assignments

* deprecated committee code

* remove deprecated messages

* fixed mocks, added tests for validator epoch assignments

* gazelle

* fix broken bazel

* nishant comment
2019-01-29 13:56:14 +01:00
Raul Jordan
4c46b02dac
Deprecating Old RPC Methods (#1410)
* begin reorder

* move into beacon server

* add proposer server

* fix

* add proposer server

* wrap up rpc reorder

* eliminated deprecated RPC endpoints

* formatted nicely

* RPC protos

* fix lint

* integrate hash proto

* autoclean

* deprecate all old code

* include the rest of methods mocks

* bazel run

* lint fixes
2019-01-28 20:41:04 +01:00
Preston Van Loon
956721fe43 Simplified validator workflow 2 - updateAssignments (#1371)
* Add attestation p2p messages

* first pass refactor

* lint

* viz fix

* comment unused const for now

* Add opentracing spans

* PR feedback

* added updateAssignments implementation

* update proto

* handle error in runner

* only do updates on epoch start

* gazelle

* goimports
2019-01-25 11:26:03 +08:00
Nishant Das
09d4c3b25d
Modifying Block Proposals for Validator Clients (#1224)
* adding rpc method

* permissions

* adding functions

* gazelle

* adding new methods

* gomocks and test

* tests again

* new changes

* fix tests

* tests and godoc

* gazelle

* lint

* nosec

* addressing review comments

* addressing more review comments

* fix merge issues
2019-01-10 10:56:26 +08:00
Raul Jordan
5d26efcfaa
Deprecate types.BeaconState with pb.BeaconState Generated Proto (#1164) 2018-12-23 16:51:04 -06:00
Preston Van Loon
10f45744d6
Use gogo protobuf as the protoc compiler (#1165)
* first commit, remote att types

* no more agg attestation proto

* regen mock

* only attestations

* proto

* att process

* fix att references

* more tests passing

* use att protos

* complete

* change visibility

* use gogoprotobu
2018-12-23 15:34:59 -05:00
Raul Jordan
1065617087 Replace Attestation Type Primitive With Proto Generated Type (#1149)
* first commit, remote att types

* no more agg attestation proto

* regen mock

* only attestations

* proto

* att process

* fix att references

* more tests passing

* use att protos

* complete

* Update dependency com_github_deckarep_golang_set to v1 (#1159)

* Update dependency com_github_edsrzf_mmap_go to v1 (#1160)

* Update dependency com_github_go_stack_stack to v1 (#1161)

* Update dependency com_github_rs_cors to v1 (#1162)

* Update dependency in_gopkg_urfave_cli_v1 to v1 (#1163)

* change visibility
2018-12-22 15:30:59 -05:00
terence tsao
2566687db7
Validator Client Use Assignment Stream to Determine Responsibility (#566) 2018-09-27 14:46:07 -07:00