Commit Graph

168 Commits

Author SHA1 Message Date
Nishant Das
f16a71f178
Revert "Update Slices More Efficiently" (#4790)
* Revert "Update Slices More Efficiently (#4789)"

This reverts commit 669e1ea787.
2020-02-07 16:58:36 +00:00
Nishant Das
669e1ea787
Update Slices More Efficiently (#4789)
* better cached states

* lint

* jim's review
2020-02-07 09:06:39 -06:00
Raul Jordan
a9d144ad1f
Stream Blocks Functionality for RPC (#4771)
* stream blocks functionality included
* necessary tests for stream blocks and notifier
* gazelle and tests passing
* gazelle and tests passing
* Merge branch 'master' into stream-block
* Update beacon-chain/core/feed/block/events.go
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* naming
* build
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* fix up tests
* Merge branch 'stream-block' of github.com:prysmaticlabs/prysm into stream-block
* Merge refs/heads/master into stream-block
* shay comment
* Merge refs/heads/master into stream-block
* Merge branch 'stream-block' of github.com:prysmaticlabs/prysm into stream-block
* Merge refs/heads/master into stream-block
2020-02-06 20:14:38 +00:00
Raul Jordan
cc741ed8af
Ensure New State Type Tests Pass in Prysm (#4646)
* begin state service

* begin on the state trie idea

* created beacon state structure

* add in the full clone getter

* return by value instead

* add all setters

* new state setters are being completed

* arrays roots exposed

*  close to finishing all these headerssss

* functionality complete

* added in proto benchmark test

* test for compatibility

* add test for compat

* comments fixed

* add clone

* add clone

* remove underlying copies

* make it immutable

* integrate it into chainservice

* revert

* wrap up comments for package

* address all comments and godocs

* address all comments

* clone the pending attestation properly

* properly clone remaining items

* tests pass fixed bug

* begin using it instead of head state

* prevent nil pointer exceptions

* begin using new struct in db

* integrated new type into db package

* add proper nil checks

* using new state in archiver

* refactored much of core

* editing all the precompute functions

* done with most core refactor

* fixed up some bugs in the clone comparisons

* append current epoch atts

* add missing setters

* add new setters

* fix other core methods

* fix up transition

* main service and forkchoice

* fix rpc

* integrated to powchain

* some more changes

* fix build

* improve processing of deposits

* fix error

* prevent panic

* comment

* fix process att

* gaz

* fix up att process

* resolve existing review comments

* resolve another batch of gh comments

* resolve broken cpt state

* revise testutil to use the new state

* begin updating the state transition func to pass in more compartmentalized args

* finish editing transition function to return errors

* block operations pretty much done with refactor

* state transition fully refactored

* got epoch processing completed

* fix build in fork choice

* fixing more of the build

* fix up broken sync package

* it builds nowww it buildssss

* revert registry changes

* Recompute on Read (#4627)

* compute on read

* fix up eth1 data votes

* looking into slashings bug introduced in core/

* able to advance more slots

* add logging

* can now sync with testnet yay

* remove the leaves algorithm and other merkle imports

* expose initialize unsafe funcs

* Update beacon-chain/db/kv/state.go

* lint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* More Optimizations for New State (#4641)

* map optimization

* more optimizations

* use a custom hasher

* comment

* block operations optimizations

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* fixed up various operations to use the validator index map access

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* archiver tests pass

* fixing cache tests

* cache tests passing

* edited validator tests

* powchain tests passing

* halfway thru sync tests

* more sync test fixes

* add in tests for state/

* working through rpc tests

* assignments tests passed

* almost done with rpc/beacon tests

* resolved painful validator test

* fixed up even more tests

* resolve tests

* fix build

* reduce a randao mixes copy

* fixes under //beacon-chain/blockchain/...

* build //beacon-chain/core/...

* fixes

* Runtime Optimizations (#4648)

* parallelize shuffling

* clean up

* lint

* fix build

* use callback to read from registry

* fix array roots and size map

* new improvements

* reduce hash allocs

* improved shuffling

* terence's review

* use different method

* raul's comment

* new array roots

* remove clone in pre-compute

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul's review

* lint

* fix build issues

* fix visibility

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fix visibility

* build works for all

* fix blockchain test

* fix a few tests

* fix more tests

* update validator in slashing

* archiver passing

* fixed rpc/validator

* progress on core tests

* resolve broken rpc tests

* blockchain tests passed

* fix up some tests in core

* fix message diff

* remove unnecessary save

* Save validator after slashing

* Update validators one by one

* another update

* fix everything

* fix more precompute tests

* fix blocks tests

* more elegant fix

* more helper fixes

* change back ?

* fix test

* fix skip slot

* fix test

* reset caches

* fix testutil

* raceoff fixed

* passing

* Retrieve cached state in the beginning

* lint

* Fixed tests part 1

* Fixed rest of the tests

* Minor changes to avoid copying, small refactor to reduce deplicated code

* Handle att req for slot 0

* New beacon state: Only populate merkle layers as needed, copy merkle layers on copy/clone. (#4689)

* Only populate merkle layers as needed, copy merkle layers on copy/clone.

* use custom copy

* Make maps of correct size

* slightly fast, doesn't wait for lock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Target root can't be 0x00

* Don't use cache for current slot (may not be the right fix)

* fixed up tests

* Remove some copy for init sync. Not sure if it is safe enough for runtime though... testing...

* Align with prev logic for process slots cachedState.Slot() < slot

* Fix Initial Sync Flag (#4692)

* fixes

* fix up some test failures due to lack of nil checks

* fix up some test failures due to lack of nil checks

* fix up imports

* revert some changes

* imports

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* resolving further conflicts

* Better skip slot cache (#4694)

* Return copy of skip slot cache state, disable skip slot cache on sync

* fix

* Fix pruning

* fix up issues with broken tests

Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-31 12:57:01 -08:00
Preston Van Loon
62a5931843
Use a client side rate limit to reduce chance of getting banned (#4637)
* Use a client side rate limit to reduce chance of getting banned

* fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-23 17:33:43 -08:00
Preston Van Loon
e7ecd9329a Fix resync (#4585)
* reset synced to false
* comment
2020-01-19 03:29:08 +00:00
Preston Van Loon
d0793f00c5 Partially revert #4477 (#4550)
* Partially revert #4477
2020-01-15 00:29:02 +00:00
Preston Van Loon
18333293d0 Refactor database interface to prefer blockchain.HeadFetcher (#4523)
* start refactoring and deprecation round 1
* Merge branch 'master' of github.com:prysmaticlabs/prysm into single-source-of-truth-1
* Refactoring of database interface. Preferring limited access interface
* revert some changes from 008f992993986f7de4193dd9cddedb465516825b
* Fix tests
* gofmt
* Merge branch 'master' into single-source-of-truth-1
* lint
* Merge refs/heads/master into single-source-of-truth-1
* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Merge refs/heads/master into single-source-of-truth-1
* Merge refs/heads/master into single-source-of-truth-1
* Merge refs/heads/master into single-source-of-truth-1
* Clone head block to avoid mutation
2020-01-13 17:02:20 +00:00
Jim McDonald
3c5d5bfc7b Use helper to calculate epoch (#4497) 2020-01-11 19:06:10 +08:00
Jim McDonald
22a3bf53ad Do not panic if initial sync fails (#4477)
* Do not panic if initial sync fails
* Only consider peers with non-0 finalized epoch
* Additional fixes
* Fix tests
* Merge branch 'master' into mvlog
* Merge branch 'master' into mvlog
2020-01-10 21:36:20 +00:00
terence tsao
6d2a2ebadf Update run time to v0.9.3 (#4154)
* Remove custody (#3986)

* Update proto fields

* Updated block operations

* Fixed all block operation tests

* Fixed tests part 1

* Fixed tests part 1

* All tests pass

* Clean up

* Skip spec test

* Fixed ssz test

* Skip ssz test

* Skip mainnet tests

* Update beacon-chain/operations/attestation.go

* Update beacon-chain/operations/attestation.go
* Decoy flip flop check (#3987)
* Bounce attack check (#3989)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Update beacon-chain/blockchain/forkchoice/process_block.go
* Conflict
* Unskip mainnet spec tests (#3998)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update beacon-chain/core/blocks/block_operations.go
* Unskip minimal spec tests (#3999)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update workspace

* Unskip all minimal spec tests

* Update workspace for general test
* Unskip test (#4001)
* Update minimal seconds per slot to 6 (#3978)
* Bounce attack tests (#3993)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Slots since epoch starts tests

* Update justified checkpt tests

* Conflict

* Fixed logic

* Update process_block.go

* Use helper
* Conflict
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.1
* Conflict
* Fixed failed tests
* Lower MinGenesisActiveValidatorCount to 16384 (#4100)
* Fork choice beacon block checks (#4107)

* Prevent future blocks check and test

* Removed old code
* Update aggregation proto (#4121)

* Update def
* Update spec test
* Conflict
* Update workspace
* patch
* Resolve conflict
* Patch
* Change workspace
* Update ethereumapis to a forked branch at commit 6eb1193e47
* Fixed all the tests
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into conflict
* fix patch
* Need to regenerate test data
* Merge branch 'master' into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Enable snappy compression for all (#4157)

* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* Validate aggregate and proof subscriber (#4159)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Conflict
* Update workspace
* Conflict
* Conflict
* Conflict
* Merge branch 'master' into v0.9.2
* Merge branch 'master' into v0.9.2
* Conflict
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Remove migrate to snappy  (#4205)
* Feature flag: Deprecate --prune-states, release to all (#4204)

* Deprecated prune-states, release to all

* imports

* remote unused import

* remove unused import

* Rm prune state test

* gaz
* Refactoring for dynamic pubsub subscriptions for non-aggregated attestations (#4189)

* checkpoint progress

* chkpt

* checkpoint progress

* put pipeline in its own file

* remove unused imports

* add test, it's failing though

* fix test

* remove head state issue

* add clear db flag to e2e

* add some more error handling, debug logging

* skip processing if chain has not started

* fix test

* wrap in go routine to see if anything breaks

* remove duplicated topic

* Add a regression test. Thanks @nisdas for finding the original problem. May it never happen again *fingers crossed*

* Comments

* gofmt

* comment out with TODO
* Sync with master
* Sync with master
* RPC servers use attestation pool (#4223)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Refactor RPC to Fully Utilize Ethereum APIs (#4243)

* include attester as a file in the validator server

* remove old proposer server impl

* include new patch and properly sync changes

* align with public pbs

* ensure matches rpc def

* fix up status tests

* resolve all broken test files in the validator rpc package

* gazelle include

* fix up the duties implementation

* fixed up all get duties functions

* all tests pass

* utilize new ethereum apis

* amend validator client to use the new beacon node validator rpc client

* fix up most of validator items

* added in mock

* fix up test

* readd test

* add chain serv mock

* fix a few more validator methods

* all validator tests passingggg

* fix broken test

* resolve even more broken tests

* all tests passsssss

* fix lint

* try PR

* fix up test

* resolve broken other tests
* Sync with master
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Aggregate and proof subscriber (#4240)

* Added subscribers

* Fixed conflict

* Tests

* fix up patch

* Use upstream pb

* include latest patch

* Fmt

* Save state before head block
* skip tests (#4275)
* Delete block attestations from the pool (#4241)

* Added subscribers
* Clean up
* Fixed conflict
* Delete atts in pool in validate pipeline
* Moved it to subscriber
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into use-att-pool-3
* Test
* Fixed test
* Initial work on voluntary exit (#4207)

* Initial implementation of voluntary exit: RPC call

* Update for recent merges

* Break out validation logic for voluntary exits to core module

* RequestExit -> ProposeExit

* Decrease exit package visibility

* Move to operation feed

* Wrap errors
* Fix critical proposer selection bug #4259 (#4265)

* fix critical proposer selection bug #4259

* gofmt

* add 1 more validator to make it 5

* more tests

* Fixed archivedProposerIndex

* Fixed TestFilterAttestation_OK

* Refactor ComputeProposerIndex, add regression test for potential out of range panic

* handle case of nil validator

* Update validators_test.go
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Leftover merge files, oops
* gaz
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* Fixes Duplicate Validator Bug (#4322)

* Update dict

* Test helper

* Regression test

* Comment

* Reset test cache
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fixes after PR #4328
* Complete attestation pool for run time (#4286)

* Added subscribers

* Fixed conflict

* Delete atts in pool in validate pipeline

* Moved it to subscriber

* Test

* Fixed test

* New curl for forkchoice attestations

* Starting att pool service for fork choice

* Update pool interface

* Update pool interface

* Update sync and node

* Lint

* Gazelle

* Updated servers, filled in missing functionalities

* RPC working with 1 beacon node 64 validators

* Started writing tests. Yay

* Test to aggregate and save multiple fork choice atts

* Tests for BatchAttestations for fork choice

* Fixed exisiting tests

* Minor fixes

* Fmt

* Added batch saves

* Lint

* Mo tests yay

* Delete test

* Fmt

* Update interval

* Fixed aggregation broadcast

* Clean up based on design review comment

* Fixed setupBeaconChain

* Raul's feedback. s/error/err
* resolve conflicts
* Merge branch 'v0.9.2' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Removed old protos and fixed tests (#4336)
* Merge refs/heads/master into v0.9.2
* Disallow duplicated indices and test (#4339)
* Explicit use of GENESIS_SLOT in fork choice (#4343)
* Update from 2 to 3 (#4345)
* Remove verify unaggregated attestation when aggregating (#4347)
* use slot ticker instead of run every (#4348)
* Add context check for unbounded loop work (#4346)
* Revert "Explicit use of GENESIS_SLOT in fork choice (#4343)" (#4349)

This reverts commit d3f6753c77f8f733563d00ab649c5159b2c2926f.
* Refactor Powchain Service (#4306)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* fix lint

* raul's comments

* use one method

* fix test

* raul's comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Ensure best better-justification is stored for fork choice (#4342)

* Ensure best better-justification is stored. Minor refactor
* Tests
* Merge refs/heads/v0.9.2 into better-best-justified
* Merge refs/heads/v0.9.2 into better-best-justified
* Ensure that epoch of attestation slot matches the target epoch (#4341)

* Disallow duplicated indices and test
* Add slot to target epoch check to on_attestation
* Add slot to target epoch check to process_attestation
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into no-dup-att-indices
* Fixed TestProcessAttestations_PrevEpochFFGDataMismatches
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Update beacon-chain/blockchain/forkchoice/process_attestation_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Filter viable branches in fork choice (#4355)
* Only activate upon finality (#4359)

* Updated functions
* Tests
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into queue-fix-on-finality
* Comment
* Merge refs/heads/v0.9.2 into queue-fix-on-finality
* Fixed failing test from 4359 (#4360)

* Fixed
* Skip registry spec tests
* Wait for state to be initialized at least once before running slot ticker based on genesis time (#4364)
* Sync with master
* Fix checkpoint root to  use genesis block root (#4368)
* Return an error on nil head state in fork choice (#4369)

* Return error if nil head state

* Fixed tests. Saved childen blocks state

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Update metrics every epoch (#4367)
* return empty slice if state is nil (#4365)
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Pubsub: Broadcast attestations to committee based subnets (#4316)

* Working on un-aggregated pubsub topics

* update subscriber to call pool

* checkpointing

* fix

* untested message validation

* minor fixes

* rename slotsSinceGenesis to slotsSince

* some progress on a unit test, subscribe is not being called still...

* dont change topic

* need to set the data on the message

* restore topic

* fixes

* some helpful parameter changes for mainnet operations

* lint

* Terence feedback

* unskip e2e

* Unit test for validate committee index beacon attestation

* PR feedbacK

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into resolveConflicts
* remove condition
* Remove unused operation pool (#4361)
* Merge refs/heads/master into v0.9.2
* Aggregate attestations periodically  (#4376)
* Persist ETH1 Data to Disk (#4329)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* adding in new proto message

* remove outdated vars

* add new changes

* remove latest eth1data

* continue refactoring

* finally works

* lint

* fix test

* fix all tests

* fix all tests again

* fix build

* change back

* add full eth1 test

* fix logs and test

* add constant

* changes

* fix bug

* lint

* fix another bug

* change back

* Apply suggestions from code review

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fixed VerifyIndexedAttestation (#4382)
* rm signing root (#4381)

* rm signing root

* Fixed VerifyIndexedAttestation

* Check proposer slashed status inside ProcessBlockHeaderNoVerify

* Fixed TestUpdateJustified_CouldUpdateBest

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove Redundant Trie Generation (#4383)

* remove trie generation
* remove deposit hashes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* fix build
* Conflict
* Implement StreamAttestations RPC Endpoint (#4390)

* started attestation stream

* stream attestations test

* on slot tick test passing

* imports

* gaz

* Update beacon-chain/rpc/beacon/attestations_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

Co-authored-by: shayzluf <thezluf@gmail.com>
* Fixed goimport (#4394)
* Use custom stateutil ssz for ssz HTR spec tests (#4396)

* Use custom stateutil ssz for ssz HTR spec tests

* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* set mainnet to be the default for build and run (#4398)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* gracefully handle deduplicated registration of topic validators (#4399)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* SSZ: temporarily disable roots cache until cache issues can be resolved (#4407)

* temporarily disable roots cache until cache issues can be resolved

* Also use custom ssz for spectests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove process block attestations as separate routine (#4408)

* Removed old save/process block atts

* Fixed tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save Deposit Cache to Disk (#4384)

* change to protos

* fix build

* glue everything together

* fix test

* raul's review

* preston's comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix activation queue sorting (#4409)

* Removed old save/process block atts

* Fixed tests

* Proper sorting by eligibility epoch then by indices

* Deleted old colde
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' into v0.9.2
* Merge refs/heads/master into v0.9.2
* stop recursive lookup if context is cancelled (#4420)
* Fix proposal bug (#4419)
* Add Pending Deposits Safely (#4422)

* safely prune cache

* use proper method

* preston's,terence's reviews and comments

* revert change to build files

* use as feature config instead
* Release custom state ssz (#4421)

* Release custom state ssz, change all HTR of beacon state to use custom method

* typo

* use mainnet config

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Update initial sync save justified to align with v0.9.3 (#4432)
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fix build
* don't blacklist on pubsub (#4435)
* Fix Flakey Slot Ticker Test (#4434)

* use interface instead for the slot ticker

* fixed up flakey tests

* add gen time

* get duties comment

* fix lifecycle test

* more fixes
* Configurable min genesis delay (#4437)

* Configurable min genesis delay based on https://github.com/ethereum/eth2.0-specs/pull/1557

* remove feature flag for genesis delay

* fix

* demo config feedback
* patch readme
* save keys unencrypted for validators (#4439)
* Add new demo configuration targeting mainnet scale (#4397)

* Add new demo configuration targeting mainnet, with 1/10th of the deposit value

* reduce quotant by 1/10th. Use 1/10th mainnet values

* only change the inactivity quotant

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save justified checkpoint state (#4433)

* Save justified checkpoint state

* Lint

* Feedback

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update shared/testutil/deposits.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update proto/testing/ssz_regression_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/epoch/epoch_processing.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/kv/forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/proposer.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/prepare_forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/aggregator/server.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/cache/depositcache/pending_deposits.go
* Update beacon-chain/cache/depositcache/pending_deposits_test.go
* Update beacon-chain/rpc/validator/proposer.go
* Merge refs/heads/master into v0.9.2
* Fix e2e genesis delay issues (#4442)

* fix e2e genesis delay issues

* register flag

* typo

* Update shared/featureconfig/config.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* skip demo e2e

* fix validator

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Batch Eth1 RPC Calls (#4392)

* add new methods

* get it working

* optimize past deposit logs processing

* revert change

* fix all tests

* use mock

* lint

* lint

* check for nil

* stop panics

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Terence's Review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-01-07 18:47:39 +00:00
Nishant Das
03356fc7b5
Add Ability to Resync Node (#4279)
* add resyncing functionality

* add more validation to status message

* lint and build

* jim's review

* preston's review

* clean up

* remove log

* remove no sync

* change again

* change back

* remove spaces

* Update shared/slotutil/slottime.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Apply suggestions from code review

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* fix refs

* raul's review

* goimports

* goimports

* add counter

* removed condition

* change back

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-02 16:09:28 +08:00
terence tsao
23a6c20dd4 Service as proper names (#4293) 2019-12-16 19:53:55 -06:00
Nishant Das
c5dcf49ded Add Flag For Minimum Handshakes (#4280)
* add flag
* jim and preston's review
* check max peers
* gaz
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into minStatusCount
* remove space
* add references
* add warning log
* change log
* gaz
2019-12-13 15:12:49 +00:00
Jim McDonald
813233373e Advanced peer tracking (#4233)
* Advanced peer status

* Rework errors; add tests

* Gazelle

* time->roughtime

* Update beacon-chain/p2p/handshake.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/p2p/interfaces.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Downgrade log

* Tidy up handshaking logic and commentary

* Downgrade log message

* Protect connected peers from disconnection; increase high water level to avoid bad interactions at maxPeers
2019-12-11 18:31:36 +08:00
terence tsao
3e15e2fc1e Add operation feed (#4215)
* Events
* Notifiers
* Refactor
* Gaz
* Fixed rest
* Lint
* Lint
* Visibility
* Typo
* Typo
* Apply suggestions from code review

Co-Authored-By: Nishant Das <nishdas93@gmail.com>
2019-12-07 17:57:26 +00:00
Preston Van Loon
498417a8fc Wait for 3 peers to start sync (#4194)
* Update service.go
2019-12-04 23:09:47 +00:00
Jim McDonald
3358bde42d Feedfixes (#4093)
* Migrate ChainStarted and StateInitialized to state notifier
* Provide state notifier to powchain service
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Remove commented line
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Accept err from HeadState() as non-fatal
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Explicit unsubscribes from state channels where required
2019-11-23 11:15:02 +00:00
Jim McDonald
9e45cffabc Move StateInitialized and ChainStarted to state feed (#4084)
* Migrate ChainStarted and StateInitialized to state notifier
* Provide state notifier to powchain service
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Remove commented line
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Accept err from HeadState() as non-fatal
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
2019-11-23 03:35:47 +00:00
terence tsao
26da7c4114 Nil state fallback in Blockchain.HeadState() (#4042)
* Can build

* All tests pass

* Update beacon-chain/blockchain/chain_info.go

* Fix context
2019-11-19 10:12:50 -06:00
Nishant Das
6e90931837
Handle Missing Blocks (#3880)
* add fixes

* more changes

* simplify the solution

* add shuffle

* remove prom metric

* add to mock

* go doc

* use non default source

* add test

* terence's review

* all tests passing
2019-11-05 10:37:18 +08:00
Jim McDonald
5000535907 Waiting condition check prior to waiting log entry (#3802) 2019-10-19 13:35:45 -07:00
Raul Jordan
d9c0e65cef
Improve Beacon Node Logging UX (#3600)
* info logs beacon node improvements

* prom test fixes

* info logging changes

* wrapped up node info logging

* changed to debug level

* warn logs taken care of

* Terence suggestion

* warn spacing

* better logging in initial sync

* debug level standardized

* complete debug standardization

* participation at epoch end

* fix archive tests

* even more test fixes

* prom test

* ops test

* powtest

* rpc sync test

* rem part

* log formatting
2019-10-01 15:05:17 -05:00
Preston Van Loon
5a92725329
Fix init sync race condition (#3633)
* fix init sync race condition

* grab subscription before checking
2019-09-28 18:42:44 -07:00
Preston Van Loon
90a02a035b
add a few logs at start of initial sync (#3608) 2019-09-26 19:17:43 -07:00
Preston Van Loon
5345ddf686 Initial Sync: Round robin (#3538)
* first pass, step 1 works

* naive from finalized to head

* delete commented code

* checkpoint progress on tests

* passing test

* abstract code slightly

* failure cases

* chkpt

* mostly working, missing a single block and having timeout

* passing tests

* comments

* comments

* gaz

* clarify comments

* progress on a few new cases

* add back bootnode query tool

* bootstrap from DHT

* chunked responses in round robin

* fix tests and deadlines

* add basic counter, time estimation

* hello -> handshakes

* show peers in use during sync

* just one last test failure

* only request blocks starting in the finalized epoch for step 1

* revert that

* comment out test and add better commentary

* move requestBlocks out to pointer receiver

* mathutil

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* PR feedback

* PR feedback
2019-09-25 12:00:04 -05:00
Nishant Das
0db690df75 Chunked Responses (#3528)
* update naming

* replace with updated version

* more changes

* fixed all tests

* build and lint

* regen protos

* fix test

* remove outdated code

* prestons review

* add chunk size

* more fixes to chunked responses

* handle eof

* fix all tests

* abstract into common method

* add comment

* preston's comments

* preston's review

* preston's review

* lint

* add encoding methods

* gaz

* simplify

* simplify

* lint

* change naming

* update

* handle eof separately

* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* remove def

* preston's review

* preston's review

* add unit tests

* add delay to fix test
2019-09-24 07:56:50 -07:00
Preston Van Loon
9fcc6fc201
Wait until fully synced to process pubsub messages (#3514)
* context timeout for pubsub message processing

* add syncing check

* gofmt

* use a global cache

* lint

* fmt

* fix conflicts

* revert change

* gaz
2019-09-20 10:54:32 -07:00
Nishant Das
4bc2d628b1
Update Naming to Latest Networking Spec (#3519)
* update naming

* replace with updated version

* more changes

* fixed all tests

* build and lint

* regen protos

* fix test

* remove outdated code

* prestons review

* preston's comments

* preston's review

* preston's review

* lint
2019-09-20 11:57:28 +05:30
terence tsao
798bbbdc82 Cold start for interop (#3437)
* coldstart flags for validator

* WIP beacon node flags

* wip beacon chain, flag fix in validator, arg fix in validator

* checkpoint

* Added interop service

* working on mock chainstart

* save the state lol

* fix tests

* Save genesis validators

* gaz

* fix validator help flags

* WaitForChainStart actually waits for genesis time

* cold start fixes

* cache

* change back

* allow for genesis state too

* remove logs

* increase mmap size

* dont process if head doesn't exist

* add 10ms tolerance

* enable libp2p debug at debug, fix pubsub

* works with checkpt

* initialize justified and finalized in db

* Removed preloadStatePath from blockchain service

* Clean up

* Write to disk for now post state

* revert b466dd536f8eadbdae2264a545a755370223d917

* Builds

* Only RPC test fails now

* use minimal config, no demo config

* clean up branch

* Lint

* resolve lint

* more lint fixes

* lint

* fix viz

* Fixing RPC test

* skip before epoch 2

* RPC time out

* Fixed ordering

* rename

* remove some dbg statements

* ensure index is correct

* fix some panics

* getting closer

* fix tests

* Fix private key

* Fixed RPC test

* Fixed beacon chain build for docker

* Add interop.go to validator go_image

* Fixed docker build

* handle errors

* skip test, skip disconnecting peers

* Fixed docker build

* tolerance for attestation processing

* revert copy

* clearer err message parse

* fix launching from dep contract
2019-09-11 13:38:35 -05:00
shayzluf
1edeb8ec4c Beaconblock over wire (#3436) 2019-09-10 10:24:14 -04:00
Raul Jordan
af07c13730
[Interop] Improve RPC Codebase + Start Beacon Chain With Mock ETH1 Values (#3407)
* add main.go

* interop readme

* proper visibility

* standardize and abstract into simpler funcs

* formatting

* no os pkg

* add test

* no panics anywhere, properly and nicely handle errors

* proper comments

* fix broken test

* readme

* comment

* recommend ssz

* install

* tool now works

* README

* build

* readme

* 64 validators

* rem print

* register the no powchain flag

* work on mock eth1 start

* common interface

* getting closer with the interface defs

* only two uses of powchain

* remove powchain dependency

* remove powchain dependency

* common powchain interface

* proper comment in case of flag

* proper args into rpc services

* rename fields

* pass in mock flag into RPC

* conforms to iface

* use client instead of block fetcher iface

* broken tests

* block fetcher

* finalized

* resolved broken build

* fix build

* comment

* fix tests

* tests pass

* resolved confs

* took them out

* rename into smaller interfaces

* resolve some confs

* ensure tests pass

* properly utilize mock instead of localized mock

* res lint

* lint

* finish test for mock eth1data

* run gazelle

* include flag again

* fix broken build

* disable powchain

* dont dial eth1 nodes

* reenable pow

* use smaller interfaces, standardize naming

* abstract mock into its own package

* faulty mock lint

* fix stutter in lint

* rpc tests all passing

* use mocks for operations

* no more mocks in the entire rpc package

* no  mock

* viz

* testonly
2019-09-09 17:13:50 -04:00
Preston Van Loon
8d234014a4
Fix broadcast ssz (#3423)
* add two types of encoding/decoding ssz

* fix tests

* lint

* lint
2019-09-08 19:34:52 -07:00
Preston Van Loon
399f704bf5
Initial Sync: report healthy before chain started (#3388)
* Return error while syncing

* chainStarted
2019-09-03 11:25:20 -07:00
Preston Van Loon
b1a102fd1d Return error while syncing (#3386) 2019-09-02 14:36:14 -05:00
Preston Van Loon
95c528f0bc First pass: single peer initial sync (#3363)
* lint

* add requests

* add all new stuff

* comment

* preston's review

* initial commit

* reorder sync so it isn't required to wait until start

* checkpoint

* fix

* improved handler API

* Set up prechain start values

* improved handler API

* ooops

* successful peer handshakes

* successful peer handshakes

* successful peer handshakes

* checkpoint

* chpkt

* handle init after chain start

* emit state initialized feed if existing db state

* merge error

* Done

* Test

* Fixed test

* emit state initialized

* force fork choice update

* wait for genesis time

* sync to current slot

* Use saved head in DB

* gaz

* fix tests

* lint

* lint

* lint

* lint

* Revert "Use saved head in DB"

This reverts commit c5f3404fdf333c8aac20bce8c349b1978494616b.

* remove db

* lint

* remove unused interfaces from composite

* resolve comments
2019-08-30 15:15:40 -05:00
Preston Van Loon
856dde497b
Move sync to deprecated- prefix (#3190)
* Move sync to deprecated_ prefix

* do not use underscore

* fix
2019-08-13 12:35:34 -04:00
Preston Van Loon
953c59a302 Wrap errors (#3123) 2019-08-01 19:27:38 -07:00
terence tsao
1b5b8a57e0 Remove unused proto schemas (#3005)
* Update io_kubernetes_build commit hash to 1246899

* Update dependency build_bazel_rules_nodejs to v0.33.1

* Update dependency com_github_hashicorp_golang_lru to v0.5.1

* Update libp2p

* Update io_bazel_rules_k8s commit hash to e68d5d7

* Starting to remove old protos

* Bazel build proto passes

* Fixing pb version

* Cleaned up core package

* Fixing tests

* 6 tests failing

* Update proto bugs

* Fixed incorrect validator ordering proto

* Sync with master

* Update go-ssz commit

* Removed bad copies from v1alpha1 folder

* add json spec json to pb handler

* add nested proto example

* proto/testing test works

* fix refactoring build failures

* use merged ssz

* push latest changes

* used forked json encoding

* used forked json encoding

* fix warning

* fix build issues

* fix test and lint

* fix build

* lint
2019-07-22 10:03:57 -04:00
Preston Van Loon
e744d1a07e Spec freeze updates (#2312)
* Optimize Shuffled Indices Cache (#2728)

* Refactor Deposit Contract Test Setup (#2731)

* add new package

* fix all tests

* lint

* change hash function (#2732)

* Remove Deprecated Validator Protobuf (#2727)

* Remove deprecated validator protos

* Fix to comments

* Fix most of skipped tests (#2735)

* Cache Active Validator Indices, Count, and Balances (#2737)

* Optimize Base Reward Calculation (#2753)

* benchmark process epoch

* revert prof.out

* Add some optimizations

* beware where we use ActiveValidatorIndices...

* revert extra file

* gaz

* quick commit to get feedback

* revert extra file

* started fixing tests

* fixed broken TestProcessCrosslink_NoUpdate

* gaz

* cache randao seed

* fixed all the tests

* fmt and lint

* spacing

* Added todo

* lint

* revert binary file

* started regression test

* basic tests done

* using a fifo for active indices cache

* using a fifo for active count cache

* using a fifo for total balance cache

* using a fifo for active balance cache

* using a fifo for start shard cache

* using a fifo for seed cache

* gaz

* clean up

* fixing tests

* fixed all the core tests

* fixed all the tests!!!

* lint

* comment

* rm'ed commented code

* cache size to 1000 should be good enough

* optimized base reward

* revert binary file

* Added comments to calculate adjusted quotient outside

* removed deprecated configs (#2755)

* Optimize Process Eth1 Data Vote (#2754)

* Cleanup and Docs update (#2756)

* Add graffiti and update generate seed (#2759)

* Benchmark Process Block with Attestations (#2758)

* Tidying up Godoc for Core Package (#2762)

* Clean up Old RPC Endpoints (#2763)

* Update RPC end point for Proposer (#2767)

* add RequestBlock

* run mockgen

* implemented RequestBlock

* updated proto definitions

* updated tests

* updated validator attest tests

* done

* comment

* todo issue

* removed unused proto

* Update attesting indices v0.6 (#2449)

* sort participants slice

* add bitfield functions and tests

* added BitfieldBit test

* add AttestationParticipantsNew

* revert AttestationParticipants to its previous change

* add tests

* remove verifybitfieldnew

* fix tests and remove multiple tests

* remove duplicate test

* change magic number into ceildiv8

* Implement Justification and finalization Processing (#2448)

* Add convert to indexed (#2519)

* sort participants slice

* add bitfield functions and tests

* added BitfieldBit test

* add AttestationParticipantsNew

* revert AttestationParticipants to its previous change

* add tests

* remove verifybitfieldnew

* fix tests and remove multiple tests

* remove duplicate test

* start work

* convert attestation to indexed attestations

* fix test for convert index

* remove calling getter

* add more tests

* remove underscore

* changes name to signature (#2535)

* update registry updates func (#2521)

* update registry updates func

* added tests and moved to epoch processing

* fixed naming issues

* Update Committee Helpers to v0.6.0 (#2398)

* Update Committee Helper Part 2 (#2592)

* Implement Process Slashings for 0.6 (#2523)

* Update Proposer/Attester Slashings and Slashing Helpers (#2603)

* Implement Final Updates 0.6 (#2562)

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* implemented process_final_updates

* move to the top of the file

* add comment back

* gaz

* Test for process final updates

* fixed tests

* fixed all the tests

* Update Reward Helper v0.6 (#2470)

* added BaseReward

* added rewards helper

* added test for BaseReward

* extra space

* move exported function above

* update to new spec (#2614)

* Update Block Processing Voluntary Exits (#2609)

* Update processEth1Data for v0.6 (#2516)

* Clean up Helper Functions Part 1 (#2612)

* Finalize helper functions for 0.6 (#2632)

* Process Beacon Chain Transfers v0.6 (#2642)

* add transfers

* beacon transfer operations complete

* full cov

* transfer testing

* finished tests

* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/core/blocks/block_operations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Implement Process Crosslink From 0.6 (#2460)

* Process Block Eth1 Data v0.6 (#2645)

* Get attestation data slot v0.6 (#2593)

* attestation.go is ready, tests not

* ready for review

* fixing linter issues

* modified Crosslink and AttestationData proto fields to spec 2.0,marked deprecated fields

* gazelle

* fixed tests

* fixed error

* error msg

* Process Block Deposits v0.6 (#2647)

* imports fixes

* deposits tests pass

* wrapped up gazelle

* spacing

* Implement Crosslink Delta Rewards for 0.6 (#2517)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* merged master

* all tests passing

* start testing

* done

* add ProcessBlockHeader v0.6 (#2534)

* add ProcessBlockHeader

* function has all its dependancies in place

* arange the basic ok test

* gazzele and skip test update

* skip wrong sig test

* fmt imports and change requests

* goimports fmt

* map for struct fields to be location independent

* reorder protobuf fields

* added tests

* gazzle fix

* few change requests fixes

* revert changes in types.proto

* revert changes in types

* fix tests

* fix lint

* fmt imports

* fix gazelle

* fix var naming

* pb update

* var naming

* tarance change request fixes

* fix test

* Add Process Registry for Epoch Processing (#2668)

* update update-process-registry

* added back the old tests

* fmt

* gaz

* Follow up on process block header v0.6 (#2666)

* Putting Crosslink Delta Back (#2654)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* merged master

* all tests passing

* start testing

* done

* fixed tests

* addressed shay's feedback

* Implement Attestation Delta for v0.6 (#2646)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* implemented process_attestation_delta

* comments

* comments

* merged master

* all tests passing

* start testing

* done

* merged master

* fixed tests

* tests, more to come

* tests done

* lint

* spaces over tabs

* addressed shay's feedback

* merged master

* Implement process_rewards_and_penalties for 0.6 (#2665)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* implemented process_attestation_delta

* comments

* comments

* merged master

* all tests passing

* start testing

* done

* merged master

* fixed tests

* tests, more to come

* tests done

* lint

* spaces over tabs

* addressed shay's feedback

* starting but need to merge a few things...

* tests

* fmt

* Update Slot Processing and State Transition v0.6 (#2664)

* edit state transition to add slot processing logic, reorder logic

* fix build

* lint

* tests passing

* spacing

* tests pass

* imports

* passing tests

* Implement Process Epoch for 0.6 (#2675)

* can't find process j f functons

* implemented process_epoch

* tests done

* lint

* nishant's feedback

* stupid goland replace

* goimports

* Update CommitteeAssignment (#2693)

* cleaned up skipped tests for core processing (#2697)

* Process Block Attestations v0.6 (#2650)

* attestation.go is ready, tests not

* ready for review

* fixing linter issues

* modified Crosslink and AttestationData proto fields to spec 2.0,marked deprecated fields

* gazelle

* fixed tests

* fixed error

* add att processing:

* process atts

* error msg

* finish process attestations logic

* spacing

* ssz move

* inclusion delay failure passing

* more attestation tests

* more att tests passing

* more tests

* ffg data mismatching test

* ffg tests complete

* gofmt

* fix testing to match attestation updates

* ssz

* lint

* Fixed Skipped Tests for RPC Server (#2712)

* Remove Obsolete Deposit Proto Objects (#2673)

* remove deposit data

* remove deposit input

* fix references

* remove deposit helpers

* fix all refs

* gaz

* rgene proto

* fix all tests

* remove deposit data deprecated field

* fix remaining references

* fix all tests

* fix lint

* regen proto

* fix test

* Remove Deprecated Protobuf State Fields (#2713)

* Remove Deprecated Protobuf Crosslink/Slashing/Block Fields (#2714)

* Remove Deprecated Beacon Block Proto Fields (#2717)

* Remove Deprecated Attestation Proto Fields (#2723)

* Cache Shuffled Validator Indices (#2682)

* YAML shuffle tests for v0.6 (#2667)

* new shuffle tests

* added comment for exported function

* fix format and print

* added config files handling

* gazelle fix

* shuffle test debugging

* added shuffle list and benchmark

* hash function addition from nishant code

* gazelle fix

* remove unused function

* few minor changes

* add test to test protos optimization

* test a bigger list

* remove commented code

* small changes

* fix spec test and test indices to pass

* remove empty line

* abstraction of repeated code and comment arrangement

* terence change requests

* fix new test

* add small comment for better readability

* change from unshuflle to shuffle

* comment

* better comment

* fix all tests

* Remove Latest Block (#2721)

* lint

* remove latest block

* lint

* add proto

* fix build

* Fix Deposit Trie (#2686)

* remove deposit data

* remove deposit input

* fix references

* remove deposit helpers

* fix all refs

* gaz

* rgene proto

* fix all tests

* remove deposit data deprecated field

* fix remaining references

* fix all tests

* fix lint

* new tests with contract

* gaz

* more tests

* fixed bugs

* new test

* finally fixed it

* gaz

* fix test

* Remove Committee Cache (#2729)

* Benchmark Compute Committee (#2698)

* Fixed Skipped Attestation Tests (#2730)

* Optimize Shuffled Indices Cache (#2728)

* Refactor Deposit Contract Test Setup (#2731)

* add new package

* fix all tests

* lint

* change hash function (#2732)

* Remove Deprecated Validator Protobuf (#2727)

* Remove deprecated validator protos

* Fix to comments

* Fix most of skipped tests (#2735)

* Optimize Base Reward Calculation (#2753)

* benchmark process epoch

* revert prof.out

* Add some optimizations

* beware where we use ActiveValidatorIndices...

* revert extra file

* gaz

* quick commit to get feedback

* revert extra file

* started fixing tests

* fixed broken TestProcessCrosslink_NoUpdate

* gaz

* cache randao seed

* fixed all the tests

* fmt and lint

* spacing

* Added todo

* lint

* revert binary file

* started regression test

* basic tests done

* using a fifo for active indices cache

* using a fifo for active count cache

* using a fifo for total balance cache

* using a fifo for active balance cache

* using a fifo for start shard cache

* using a fifo for seed cache

* gaz

* clean up

* fixing tests

* fixed all the core tests

* fixed all the tests!!!

* lint

* comment

* rm'ed commented code

* cache size to 1000 should be good enough

* optimized base reward

* revert binary file

* Added comments to calculate adjusted quotient outside

* removed deprecated configs (#2755)

* Optimize Process Eth1 Data Vote (#2754)

* Cleanup and Docs update (#2756)

* Add graffiti and update generate seed (#2759)

* Benchmark Process Block with Attestations (#2758)

* Tidying up Godoc for Core Package (#2762)

* Clean up Old RPC Endpoints (#2763)

* Update RPC end point for Proposer (#2767)

* add RequestBlock

* run mockgen

* implemented RequestBlock

* updated proto definitions

* updated tests

* updated validator attest tests

* done

* comment

* todo issue

* removed unused proto

* Cache Active Validator Indices, Count, and Balances (#2737)

* Update Deposit Contract (#2648)

* lint

* add new contract

* change version

* remove log

* generating abi and binary files

* fix tests

* update to current version

* new changes

* add new hash function

* save hashed nodes

* add more things

* new method

* add update to trie

* new stuff

* gaz

* more stuff

* finally fixed build

* remove deposit data

* Revert "remove deposit data"

This reverts commit 9085409e91be0c94550af10290eaad72ad40a6de.

* more changes

* lint and gaz

* lint

* Update Shard Helpers for 0.6 (#2497)

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* Update Genesis State Function to v0.6 (#2465)

* add pseudocode

* make changes

* fix all tests

* fix tests

* lint

* regen protos and mocks

* regenerated protos

* started fixing core

* all core tests passing!

* removed shared/forkutils

* started fixing blockchain package

* lint

* updating rpc package

* add back deleted stuff

* add back deleted stuff that was deleted accidentally

* add back protos and mocks

* fix errors

* fix genesis issue

* fix genesis issue for slot ticker

* fix all genesis errors

* fix build files

* temp change for go-ssz

* fix test

* Revert "temp change for go-ssz"

This reverts commit 3411cb9d6d519cb521181486debc3b21603c8873.

* update to latest go-ssz

* unstaged changes

* Update Attester Server RPC Calls (#2773)

* Update config and function parameters to v0.7 (#2791)

* Minor Updates to 0.7 (#2795)

* Refactor Deposit Flow and Cleanup Tests (#2788)

* More WIP on cleaning deposit flow

* Fix tests

* Cleanup and imports

* run gazelle

* Move deposit to block_operations

* gazelle

* Update beacon-chain/core/blocks/block_operations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Fix docs

* Remove unneeded calculations

* Fix tests

* Fix tests finally (?)

* Optimize Committee Assignment RPC (#2787)

* Update BlockRoot to BlockHash (#2816)

* Fix Final Missing Items in Block Processing v0.6 (#2710)

* override config successfully

* passes processing

* add signing root helper

* blockchain tests pass

* tests blocked by signing root

* lint

* fix references

* fix protos

* proper use of signing root

* only few failing tests now

* fix final test

* tests passing

* lint and imports

* rem unused

* Update beacon-chain/core/blocks/block_operations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* lint

* Update beacon-chain/attestation/service_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/db/block_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* rename to hash tree root

* rename decode to unmarshal

* fix

* use latest ssz

* all tests passing

* lint

* fmt

* Add Config YAML for Spec Tests (#2818)

* Align Protobuf Type Names (#2825)

* gofmt

* Revert "Align Protobuf Type Names (#2825)" (#2827)

This reverts commit 882d067144c674bbf4eeee00404acaa0a9b1fd33.

* Update Domain Related Functions (#2832)

* Add Functions for Compressed and Uncompressed HashG2 With Domain (#2833)

* add tests

* gaz

* lint

* Revert "Add Functions for Compressed and Uncompressed HashG2 With Domain (#2833)" (#2835)

This reverts commit 7fb2ebf3f12f043d7bd12f43500dc2bd133df573.

* Add ConvertToPb to package testutil (#2838)

* Block Processing Bug Fixes (#2836)

* Update types PB with Size Tags (#2840)

* Epoch processing spec tests (#2814)

* Remove Deposit Index (#2851)

* Shuffle tests revisited (#2829)

* first commit

* remove old files, add log

* remove duplicate yaml testing code

* reduce visability

* nishant feedback changes

* skip TestFromYaml_Pass

* added tags to bazel build

* gazelle fix

* remove unused vars

* adda back config

* remove config handling

* remove unused var

* gazelle fix

* SSZ compatibility test for protobufs (#2839)

* update workspace spec sha

* remove yamls from branch

*  BLS spec tests (#2826) (#2856)

* bls spec tests

* add more bls tests

* use ioutil instead of bazel runfiles

* dont read bytes

* skip tests that overflow uint64

* manually fix input data

* add tests

* lint and gaz

* add all new changes

* some refactoring, cleanup, remove new API methods that only exist for tests

* gaz

* Remove yamls, skip test

* Slot processing spec test (#2813)

* eth1data rpc endpoint (#2733)

* eth1data rpc endpoint

* first version

* comment added

* gazelle fix

* new function to go once over the deposit array

* fix tests

* export DepositContainer

* terence feedback

* move structure decleration

* binary search

* fix block into Block

* preston feedback

* keep slice sorted to remove overhead in retrival

* merge changes

* feedback

* update to the latest go-ssz

* revert change

* chnages to fit new ssz

* revert merge reversion

* go fmt goimprts duplicate string

* exception for lint unused doesParentExist

* feedback changes

* latesteth1data to eth1data

* goimports and stop exposing Eth1Data

* revert unneeded change

* remove exposure of DepositContainer

* feedback and fixes

* fix workspace duplicate dependancy

* greatest number of deposits at current height

* add count votes function

* change method name

* revert back to latesteth1data

* latesteth1data

* preston feedback

* seperate function add tests fix bug

* stop exposing voteCountMap

* eth1data comment fix

* preston feedback

* fix tests

* new proto files

* workspace to default version of ssz

* new ssz

* chnage test size

* marshalled  marshaled

* Attesting Indices Fix (#2862)

* add change

* fix one test

* fix all tests

* add test

* clear cache

* removed old chaintest, simulated backend and state generator (#2863)

* Block Processing Sanity Spec Tests (#2817)

* update PrevEpoch

* add new changes

* shift to blocks package

* add more changes

* new changes

* updated pb with size tags

* add new changes

* fix errors

* uncomment code

* more changes

* add new changes

* rename and lint

* gaz

* more changes

* proccess slot SigningRoot instead of HashTreeRoot

* ensure yaml generated structs work

* block sanity all passing

* minimal and mainnet all pass

* remove commented code

* fix one test

* fix all tests

* fix again

* no state comparison

* matching spec

* change target viz

* comments gazelle

* clear caches before test cases

* latest attempts

* clean up test format

* remove debugging log, remove yaml

* unskip attestation

* remove skip, check post state, diff state diffs

* handle err

* add bug fixes

* fixed one more bug

* fixed churn limit bug

* change hashProto to HashTreeRoot

* all tests pass :)

* fix all tests

* gaz

* add regression tests

* fix test bug

* Mutation testing fixes for beacon-chain/core/helpers/attestation.go (#2868)

* mutation testing for attestation.go

* new line

* lint

* revert fmt.Errorf deletion

* gofmt

* Add some fixes for mutation testing on blocks.go (#2869)

* Fix sizes

* gaz

* Spec freeze release candidate spectests

* Align Protobuf Type Names  (#2872)

* Removes some deprecated fields from protobuf (#2877)

* search and replace checkpoints

* fix tests, except spec tests

* Update Configs for Freeze (#2876)

* update configs

* updated minimal configs

* almost there

* all tests passing except for spec tests

* better comment for MinGenesisTime

* done, ready for review

* rm seconds per day

* feedback

* Mutation testing fixes for beacon-chain/core/helpers/committee.go (#2870)

* Add some fixes for mutation testing on blocks.go

* working on mutation testing fo committee.go

* gofmt

* goimports

* update readme target

* update latest sha for spec tests

* fix build

* Update State Transition Function (#2867)

* Change Base Reward Factor (#2888)

* Update Freeze Spec Simplification Section - part 1 (#2893)

* finished changes to attesting_indices

* removed index_count <= 2**40 requirement

* lint

* reverted index_count <= 2**40 check

* added short cut len(a) > len(b)

* Update justification bits (#2894)

* updated all the helper pseudocodes (#2895)

* Make Constants Explicit and Minor Cleanups (#2898)

* Rename outdated configs, make constants explicitly delcared

* Remove activate_validator, not needed

* Remove GenesisSlot and GenesisEpoch

* Remove unused import

* Move Block Operation Length Checks to ProcessOperations (#2900)

* Move block operation length checks to ProcessOperations

* Write tests for each length check in ProcessOperations

* Remove unneeded test

* Move checks to a new function

* Move duplicate check back into ProcessOperations

* reorder proto fields (#2902)

* Slashing Penalty Calculation Change (#2889)

* lint

* change config val

* add max helper

* changes to slashing and process slashing, add a min function for integers

* gaz

* fix failing tests

* fix test

* fixed all tests

* Change Yaml tag

* lint

* remove gc hack

* fix test

* gaz

* preston's comments

* change failing field

* fix and regen proto

* lint

* Implement Compact Committee Root (#2897)

* add tags

* add function

* add new code

* add function

* add all new changes

* lint

* add tests

* fix tests

* fix more tests

* fix all outstanding tests

* gaz

* Update beacon-chain/core/helpers/committee.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* comment

* Remove deprecated fields from attestation data (#2892)

* fix broken tests

* remove comment

* fixes

* Update Deposit Contract (#2903)

* update to new contract

* fix references

* fix tests

* fix some more tests

* fix local deposit trie

* gaz

* shays review

* more changes

* update WORKSPACE to use 0.8 spec tests

* Perform Mutesting in Helpers Package (#2912)

* Perform mutesting on validator

* Mutesting in helpers package

* Mutested eth1data

* s/volundary/voluntary (#2914)

* Update BLS Domain (#2916)

* change from integer to byte slice

* add test

* fix func for bytes4

* Fix Spec tests (#2907)

* fix panics

* handle failed transitions

* remove log

* fix to protos

* new changes

* remove i

* change ssz commit

* new changes

* update epoch tests

* fix epoch testing

* fix shuffle tests

* fix test

* Perform Mutesting in Epoch and State Packages (#2913)

* done with updates (#2890)

* Add Max Size Tag for Protobuf Fields (#2908)

* No more space between ssz-size numbers

* Regen pb.go

* Fixed a few incorrect proto fields (#2926)

* Update Validator Workflow (#2906)

* Justification spec tests (#2896)

* update go-ssz

* Fix SSZ Compatibility Test (#2924)

* figuring out how to seqeeze in multiple fields in a tag for pb

* Added max tags and regenerated pb.go

* updated to new standard

* New bitfield types in proto (#2915)

* Cast bytes to correct bitfield, failing tests now though

* Add forked gogo/protobuf until https://github.com/gogo/protobuf/pull/582

* remove newline

* use proper override for gogo-protobuf

* fix a few tags

* forgot to include custody bits and Slashable not used

* Update yaml struct to use pb

* Update workspace to use latest ssz

* All tests fail

* Use the latest go-ssz commit

* All pass except for state (too long to taste)

* Update test.proto

* Added rest of the tests

* use 1 justification bits

* fix tag test, apply @rauljordan's suggestion

* add IsEmpty and use ssz struct

* delete unused file

* Update zero hash to sha256().digest() (#2932)

* update zero hash

* change zero hash to conform with spec

* goimports

* add test for zero hash

* Revert "Update zero hash to sha256().digest() (#2932)" (#2933)

This reverts commit b926ae0667b18aef3f7e0e8ec8a9b3e98b3d2ccc.

* Fix compress validator (#2936)

* fix compress validator

* update go-ssz

* build without the bytes test

* try minimal

* Block operations spec tests (#2828)

* update PrevEpoch

* debugging proposer slashing tests

* fmt

* add deposit tests

* Added skeleton for attestation minimal test

* remove bazel runfiles thing

* add deposits

* proposer slashing test is done

* comment

* complete test, some failing cases

* sig verify on

* refactor slightly to support mainnet and minimal

* included mainnet stuff

* Add block header tests

* volunary exit done

* transfer done

* new changes

* fix all tests

* update domain functions

* fmt

* fixed lint

* fixed all the tests

* fixed a transfer bug

* finished attester slashing tests and fixed a few bugs

* started fixing...

* cleaned up exit and proposr slashing tests

* attester slashing passing

* refactored deposit tests

* remove yamls, update ssz

* Added todo for invalid sig

* gazelle

* deposits test done!

* transfer tests done and pass!

* fix attesting indices bug

* temporarily disabled signature verification

* cleaned up most of the block ops, except for att

* update committee AttestingIndices

* oops, i dont know how or why i changed this file

* fixed all the rpc tests

* 6 more failing packages

* test max transfer in state package

* replace hashproto with treehash in package blockchain

* gazelle

* fix test

* fix test again

* fixed transition test, 2 more left

* expect an error in attestation tests

* Handle panic when no votes in aggregate attestation

* clear cache

* Add differ, add logging, tests pass yay

* remove todo, add tag

* fixed TestReceiveBlock_RemovesPendingDeposits

* TestAttestationMinimal/success_since_max_epochs_per_crosslink fails now...

* handle panics

* Transfer tests were disabled in https://github.com/ethereum/eth2.0-specs/pull/1238

* more fixes after merge, updating block_operations.yaml.go to match yaml

* figuring out how to seqeeze in multiple fields in a tag for pb

* Added max tags and regenerated pb.go

* updated to new standard

* New bitfield types in proto (#2915)

* Cast bytes to correct bitfield, failing tests now though

* Add forked gogo/protobuf until https://github.com/gogo/protobuf/pull/582

* remove newline

* fix references and test panic

* change to proto objects from custom types

* fix panics in tests

* use proper override for gogo-protobuf

* fix a few tags

* fix tests

* forgot to include custody bits and Slashable not used

* fix tests

* sort again

* Update yaml struct to use pb

* Update workspace to use latest ssz

* All tests fail

* Use the latest go-ssz commit

* All pass except for state (too long to taste)

* Update test.proto

* Added rest of the tests

* use 1 justification bits

* minor fixes

* wrong proto.Equal

* fix tag test, apply @rauljordan's suggestion

* add IsEmpty and use ssz struct

* inverted logic

* update zero hash

* change zero hash to conform with spec

* goimports

* add test for zero hash

* Revert "Update zero hash to sha256().digest() (#2932)"

This reverts commit b926ae0667b18aef3f7e0e8ec8a9b3e98b3d2ccc.

* update ssz, fix import, shard big test

* checkpoint

* fix compress validator

* update go-ssz

* missing import

* missing import

* tests now pass

* been a good day

* update test size

* fix lint

* imports and remove unused const

* update bazel jobs flag

* update bazel jobs flag

* satisfy deprecation warning

* Add ssz regression tests for investigation of test failures in PR #2828 (#2935)

* Adding regression tests for investigation

* add another example

* goimports

* add quick comment about test case 0

* Epoch Process Slashings Spec Tests (#2930)

* updated justification bits, tests passing OK

* regen pb.go, clarify bit operations

* justification and finalization tests; failing

* Add wrapper, so we call the correct method

* checkpoint

* Update tar ref

* TestSlashingsMinimal/small_penalty still failing

* Use bigint instead of () and float

* Revert a bad merge from workspace

* Fmt

* add note about https://github.com/ethereum/eth2.0-specs/issues/1284

* improve tests

* gaz

* Perform Mutesting In core/state Package (#2923)

* Perform mutesting on validator

* Mutesting in helpers package

* Mutested eth1data

* Perform mutesting in epoch and state packages

* Fix voluntary exits test

* Fix typo

* Fix comments

* Fix formatting

* Fix error message

* Handle missing errors

* Handle all errors

* Perform Mutesting In State Package

* Fix block roots size

* Remove comment

* Fix error

* add backend service

* Add ssz compatibility tests for signing root (#2931)

* Added tests for signing root

* imports

* fix lint on travis

* fix bes flag

* Final updates spec tests (#2901)

* set up tests

* need to reorder pbs

* figuring out how to seqeeze in multiple fields in a tag for pb

* Added max tags and regenerated pb.go

* updated to new standard

* New bitfield types in proto (#2915)

* Cast bytes to correct bitfield, failing tests now though

* Add forked gogo/protobuf until https://github.com/gogo/protobuf/pull/582

* remove newline

* use proper override for gogo-protobuf

* fix a few tags

* forgot to include custody bits and Slashable not used

* playing with tags idea, can revert this commit later

* fixes after merge

* reset caches before test

* all epoch tests pass

* gazelle

* Genesis trigger (#2905)

* genesis change

* integrate changes

* bodyroot

* remove unused code

* HasChainStarted

* added isValidGenesisState to ProcessLog

* state fix

* fix gazelle

* uint64 timestamp

* SetupInitialDeposits adds proof

* remove unneeded parts of test

* deposithash

* merkleproof from spec utils

* Revert "merkleproof from spec utils"

This reverts commit 1b0a124352e7b62e3c3220fb0d64e295b474b430.

* fix test failures

* chain started and hashtree root in tests

* simple eth2genesistime

* eth2 genesis time

* fix zero time

* add comment

* remove eth1data and feedback

* fix build issues

* main changes: add fields and methods to track active validator
count

* gaz

* fix test

* fix more tests

* improve test utils

* shift spec method to state package, improve test setup

* fixed log processing tests

* remove log

* gaz

* fix invalid metric

* use better tag names, not latest

* Remove Block Signing Root (#2945)

* replace with hash tree root

* Revert "replace with hash tree root"

This reverts commit 77d8f16a160e42f3c3d598df66c30a66657de1bf.

* replace with signing root instead

* remove one more ref

* Create Test Runner for Genesis State Spec Tests (#2940)

* genesis change

* integrate changes

* bodyroot

* remove unused code

* HasChainStarted

* added isValidGenesisState to ProcessLog

* state fix

* fix gazelle

* uint64 timestamp

* SetupInitialDeposits adds proof

* remove unneeded parts of test

* deposithash

* merkleproof from spec utils

* Revert "merkleproof from spec utils"

This reverts commit 1b0a124352e7b62e3c3220fb0d64e295b474b430.

* fix test failures

* chain started and hashtree root in tests

* simple eth2genesistime

* eth2 genesis time

* fix zero time

* add comment

* remove eth1data and feedback

* fix build issues

* main changes: add fields and methods to track active validator
count

* gaz

* fix test

* fix more tests

* improve test utils

* Start genesis spec tests

* shift spec method to state package, improve test setup

* Add Genesis validity spec test

* Bazel

* fixed log processing tests

* remove log

* gaz

* fix invalid metric

* use json tags

* fix up latest changes

* Fix most of test errors

* Attempts to see whats wrong with genesis validity

* Fix merge

* skip minimal

* fix state test

* new commit

* fix nishant comment

* gaz

* Static check on branch spec-v0.6 (#2946)

* Ran staticcheck and fixed the important complains

* commit

* commit

* Create Test Runner for Genesis State Spec Tests (#2940)

* genesis change

* integrate changes

* bodyroot

* remove unused code

* HasChainStarted

* added isValidGenesisState to ProcessLog

* state fix

* fix gazelle

* uint64 timestamp

* SetupInitialDeposits adds proof

* remove unneeded parts of test

* deposithash

* merkleproof from spec utils

* Revert "merkleproof from spec utils"

This reverts commit 1b0a124352e7b62e3c3220fb0d64e295b474b430.

* fix test failures

* chain started and hashtree root in tests

* simple eth2genesistime

* eth2 genesis time

* fix zero time

* add comment

* remove eth1data and feedback

* fix build issues

* main changes: add fields and methods to track active validator
count

* gaz

* fix test

* fix more tests

* improve test utils

* Start genesis spec tests

* shift spec method to state package, improve test setup

* Add Genesis validity spec test

* Bazel

* fixed log processing tests

* remove log

* gaz

* fix invalid metric

* use json tags

* fix up latest changes

* Fix most of test errors

* Attempts to see whats wrong with genesis validity

* Fix merge

* skip minimal

* fix state test

* new commit

* fix nishant comment

* gaz

* Add Back Eth1Data After Bad Merge (#2953)

* eth1data rpc endpoint

* first version

* comment added

* gazelle fix

* new function to go once over the deposit array

* fix tests

* export DepositContainer

* terence feedback

* move structure decleration

* binary search

* fix block into Block

* preston feedback

* keep slice sorted to remove overhead in retrival

* merge changes

* feedback

* update to the latest go-ssz

* revert change

* chnages to fit new ssz

* revert merge reversion

* go fmt goimprts duplicate string

* exception for lint unused doesParentExist

* feedback changes

* latesteth1data to eth1data

* goimports and stop exposing Eth1Data

* revert unneeded change

* remove exposure of DepositContainer

* feedback and fixes

* fix workspace duplicate dependancy

* greatest number of deposits at current height

* add count votes function

* change method name

* revert back to latesteth1data

* latesteth1data

* preston feedback

* seperate function add tests fix bug

* stop exposing voteCountMap

* eth1data comment fix

* preston feedback

* fix tests

* new proto files

* workspace to default version of ssz

* new ssz

* chnage test size

* marshalled  marshaled

* everything passing again

* add skip reason

* cleanup deposit contract slightly

* remove unused chainstart param (#2957)

* fix breakages from #2957 (#2958)

* fix breakages from #2957

* oops

* Fix deposit input data (#2956)

* fix deposit input data

* fix deposit input data

* gaz and build fix

* Add Tests for Genesis Deposits Caching (#2952)

* remove old method and replace with an improved one

* add new files

* gaz

* add test

* added all these tests

* gaz

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fix merkle proof error

* fix config

* Minor fixes for runtime (#2960)

* Minor fixes for runtime

* use comments

* goimports

* revert beacon-chain/core/state/state.go and fix comment for lint

* fix test too

* Minor runtime fixes  (#2961)

* Add support for bundling binaries and fix ARM64 builds (#2970)

* Add support for bundling binaries and fix ARM64 builds

* Fix exports

* ignore manual targets wrt vis check

* fix graknlabs

* update spec tests (#2979)

* hotfix until https://github.com/graknlabs/bazel-distribution/pull/169

* Overflow slashing calculation fix (#2977)

* Runtime Fixes (#2736)

* first batch of fixes

* add log

* more fixes

* another bug fixed

* update deposit contract and other fixes

* remove logs

* new changes

* fixes

* fix build

* remove config

* more fixes

* add more changes

* add back todo

* make compute state root work

* remove commented out and fix condition

* fix commented code

* fix config

* gaz

* remove flag

* remove init

* new fixes

* fix test

* one more fix

* fix all tests

* change back config

* fix one more bug

* remove logging bool

* Only build test targets when running bazel test //...

* Align prysm to spec v0.8.1 (#2978)

* Bazel problem

* Update zero hash representation to be clear (cosmetic)

* Update minor cosmetic fixes

* Fixed lookahead off by 1

* Update randao.go

* update ssz

* test failures fixed

* test fixes

* fix up workspace

* lint

* fixed errs

* Updated pubkey loggings (#2983)

* Fix proposer assignment (#2984)

* add jvm limits

* add jvm limits

* Removed logging from state transition functions (#2980)

* Match spec on proposer index division (#2985)

* Match spec on proposer index division

* gaz

* fixes

* Fix Default Eth1Data (#2982)

* fix bug

* Update beacon-chain/core/state/transition.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* more fixes

* reg test

* Set ejection balance to 1.6 (#2987)

* improve block processing log (#2990)

* Fix HistoricalRootsLimit (#2989)

* fix historical lenght

* fix genesis state initialization and test

* fix genesis state initialization and test

* fix genesis state initialization and test

* fix genesis state initialization and test

* hack config until https://github.com/prysmaticlabs/prysm/issues/2993

* More Runtime Fixes (#2986)

* local changes

* add val sig

* attester fix

* one more fix

* fixed all tests

* rem validator issue

* fix finality issue (#2994)

* Fix validator prev balance calculation (#2992)

* attester fix

* one more fix

* fixed all tests

* Fix validator prev balances calculation

* go fmt

* 48 bytes

* Fix Querier to handle Deposit Logs Race (#2999)

* fix sync issue

* fix build

* add regression test

* add var for magic number

* Fix eth1data and deposits (#2996)

* Work in progress, eth1data works, deposits are included at the appropriate time, and activation happens at the correct time

* revert blockInfo being public

* git tests to build, not yet pass though

* add tests

* some commentary

* fix comment

* goimports

* fmt and remove unused method

* Update rules go (#2975)

* update rules_go

* Fix some cross compile builds stuff

* add missing deps

* update to 0.19.1

* Update Protobufs to Match Ethereum APIs (#2998)

* add beacon block and attestation files

* add all types

* include all new proto type definitions

* add add all proto definitions

* fix all comments to say 48 bytes

* include latest changes

* readd common

* no swag

* add build file

* deps issue

* right package names

* address feedback, maintain parity between upstream ethereumapis

* delete pb

* bad gens

* Update "Testing Prysm" readme section (#3000)

Resolves invalid url link to golangci-lint.

* Update badge to version 0.8.1

* elaborate on test skip

* revert shared/p2p/options.go

* make travis happy with goimports

* Update beacon-chain/core/blocks/block.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
2019-07-19 19:16:10 -05:00
Preston Van Loon
991ee7e81b "Super sync" and naive p2p reputation (#2550)
* checkpoint on super sync with reputation

* ensure handling only expected peers msg

* exclusive of finalized block

* skip block saved already

* clean up struct

* remove 2 more fields

* _

* everything builds, but doesnt test yet

* lint

* fix p2p tests

* space

* space

* space

* fmt

* fmt
2019-05-09 16:02:24 -05:00
terence tsao
104966b63d Sync Responds With Canonical Block Lists (#2539)
* first attempt at canonical blk list

* lint

* condition 1

* ctx w/ time out

* added canonical block list tests

* revert

* add to BeaconChainFlags

* dont use map, use proto

* attempt to use proto, take 1

* add run

* like canonical better than head

* removed unused

* Update proto/beacon/p2p/v1/messages.proto

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* protos
2019-05-08 18:23:06 -05:00
Raul Jordan
76881fd1ae
Do Not Subscribe to Blocks in Initial Sync (#2524)
* only sub to block batches

* batch sub remove

* tests

* fix lint

* gazelle

* delete old im mem blocks code
2019-05-07 21:12:36 -05:00
Raul Jordan
939382ac86 Better Error Message if Failing to Exit Initial Sync (#2483)
* no err throw

* nil errors

* better error on init sync
2019-05-04 10:13:33 +08:00
Raul Jordan
629b76cf44 Only Perform Initial Sync With a Single Peer (#2471)
* fix spacing

* use send instead of broadcast in initial sync
2019-05-02 15:36:23 -04:00
Raul Jordan
81c8b130c6
Only Sync With the Peer With the Highest Observed Slot (#2280)
* only accept the highest finalized slot from peers

* use best peer for state requests

* peer id

* rem old tests

* req peers tests pass

* warn level support for peer not having the same protocol

* pretty

* best peer

* comments

* req peers done

* tests passing

* fully functional

* enforce receiving from the best peer

* comments

* lint fixes
2019-04-26 10:18:43 -05:00
terence tsao
b81eeec374 Optimize block fetch with attestation target fetch (#2373)
* 1st attemp to optimize blk fetch w/ att tgt fetch

* test test test test
2019-04-24 12:21:00 -05:00
Nishant Das
a5def315ae Separate Registry and Balances (#2334)
* all changes

* lint

* terence's review
2019-04-22 10:52:56 -05:00
Raul Jordan
1df15f72c4
Ensure Block Processing Failures Return an Error (#2325)
* ensure block failed processing returns an error

* fixed test

* test assertion corrected

* comments

* fix tests

* imports
2019-04-21 17:38:14 -05:00
terence tsao
1602c5d0c5
Use Withfields for Logging (#2305)
* use with fields

* gaz
2019-04-19 21:09:01 -07:00
Raul Jordan
3e525ebe8b
Block Deletion Upon Block State Transition Error (#2270)
* block deletion upon block state transition error

* include tests

* tests complete

* comment

* imports spacing
2019-04-16 13:06:53 -05:00
Nishant Das
90fed79610 Fix Initial Sync in Restarts (#2197)
* set chainstart value

* revert change in IsSynced and add regression test

* change status

* make a better status query

* allow status check to work on restarts

* spacing
2019-04-09 14:18:23 -04:00
terence tsao
a19f08f7b6 Attestation Using Head State instead of Latest State (#2156)
* headRoot is a better name

* yay merged with master

* fixed attester server returning incorrect attestation field values

* revert stupid mock changes

* fixed and updated all the tests

* uncomment

* head start should handle skip blocks

* Raul's feedback
2019-04-05 09:41:49 -05:00
Preston Van Loon
83130358a9
Rollback state to handle side chain with skip blocks (#2147)
* Add initial test

* chkpt

* add failing test

* add span to historical state lookup

* use db.HighestBlockSlot()

* fix comment

* update comment

* i wrote a test like a good programmer.

* add test back

* add assertion and unskip test, something new failing tho

* trying to fix test

* remove -1, not sure if i need it yet

* Revert "remove -1, not sure if i need it yet"

This reverts commit 2cfcbb8108b28bb3d7135a993d9053150d5f1e6e.

* save historical state on every save state

* fix hsitorical states

* set historical state in initialize state

* change to a bool

* fix error with empty retrieval of states

* Add missing import

* fix test

* lock in receive block

* remove state generator

* Revert "lock in receive block"

This reverts commit 151b10829d70b2dad3055a8db36d0e1269a853f2.

* Fix Initial Sync Not Processing Canonical Block to Produce Canonical State (#2152)

* fix init sync

* fatal if highest observed root does not match

* proto fields

* Update beacon-chain/sync/initial-sync/service.go

* confirm canonical state root

* fix most tests

* failing test

* fix PR tests

* lint

* no simbackend changes

* logf revert

* add todo

* fix off by one

* fix test with deleted property

* merge #2157

* passing tests :)
2019-04-04 22:39:51 -05:00
Raul Jordan
eb900a8193
Refactor Initial Sync, Enable Catching Up to Chain (#2111)
* refactor initial sync to prevent reorg infinite loops

* lint

* fixed build

* passing tests

* tests passing

* terence suggestion

* new attempt

* clean up and refactor sync service

* complete the new initial sync logic

* revert head

* init sync working

* config for blockchain receive block

* all works

* builds

* fix a few more tests

* init sync tests pass

* revert scripts

* revert accounts changes

* lint

* lint2

* travis lint

* fix build

* fix single use argument

* any peer

* imports spacing

* imports

* ready for a rolling restart

* add todo

* fork choice in blocks when exiting sync

* readd finalized state root to requests

* successful build

* revert blockchain config

* old config reversion

* initial sync tests pass

* initial sync full test works

* lint

* use the new block processing api

* new proto defs

* init sync functions again

* remove sync polling

* tests fixed

* fixed catching up with chain

* tests pass

* spacing

* lint

* goimports

* add changes

* add lock and conditional to prevent multiple goroutines

* make reg sync synchronous

* add

* fixed the parent block issue

* fix errors in chain service

* tests pass

* check nil block

* typo

* fix nil state

* merge & conflicts

* revert synchronus reg sync

* add more spans to state db

* fix lint

* lint
2019-04-03 10:13:19 -05:00
Preston Van Loon
76e30200fe Pass ctx to saveState (#2141)
* pass ctx to saveState

* add span
2019-04-02 16:49:45 +08:00
Raul Jordan
7489e52b99
Refactor Chain Service Block Processing API (#2124)
* new block processing api

* fix tests

* lint

* spacing

* Update beacon-chain/blockchain/block_processing.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* fix tests, remove enable powchain
2019-03-31 20:44:16 -05:00
Raul Jordan
5b89da1ce2
Revert "Prevent Reorg Infinite Loops in Initial Sync (#2109)" (#2110)
This reverts commit d0790246a2.
2019-03-29 16:51:58 -05:00
Raul Jordan
d0790246a2
Prevent Reorg Infinite Loops in Initial Sync (#2109)
* refactor initial sync to prevent reorg infinite loops

* lint

* fixed build

* passing tests

* tests passing

* terence suggestion
2019-03-29 13:11:13 -05:00
Nishant Das
7029ed109d
Prune Outdated Deposits after Chainstart (#2090)
* disable pending deposits

* Revert "disable pending deposits"

This reverts commit 1525e01602a70737f8688cdbd63a5ce2ab721d92.

* adding new methods

* gazelle and other fixes

* spacing

* spacing

* Update beacon-chain/sync/initial-sync/service.go

Co-Authored-By: nisdas <nish1993@hotmail.com>

* remove comment
2019-03-28 00:15:29 +08:00
Nishant Das
0809b96c32 Fixes Initial Sync (#2041)
* plug forkchoice to blockchain service's block processing

* fixed tests

* more fixes...

* clean ups

* fixed test

* Update beacon-chain/blockchain/block_processing.go

* merged with 2006 and started fixing tests

* remove prints

* fixed tests

* lint

* include ops service

* if there's a skip slot, slot--

* fixed typo

* started working on test

* no fork choice in propose

* bleh, need to fix state generator first

* state gen takes input slot

* feedback

* fixed tests

* preston's feedback

* fmt

* removed extra logging

* add more logs

* fixed validator attest

* builds

* fixed save block

* children fix

* removed verbose logs

* fix fork choice

* right logs

* Add Prometheus Counter for Reorg (#2051)

* fetch every slot (#2052)

* test Fixes

* lint

* only regenerate state if there was a reorg

* better logging

* fixed seed

* better logging

* process skip slots in assignment requests

* fix lint

* disable state root computation

* filter attestations in regular sync

* log important items

* better info logs

* added spans to stategen

* span in stategen

* set validator deadline

* randao stuff

* disable sig verify

* lint

* lint

* save only using historical states

* use new goroutine for handling sync messages

* change default buffer sizes

* better p2p

* rem some useless logs

* lint

* sync tests complete

* complete tests

* tests fixed

* lint

* fix flakey att service

* bug fixes

* Update beacon-chain/sync/initial-sync/service.go

* Update beacon-chain/sync/initial-sync/service.go

* reenable shuffle

* enable shuffle
2019-03-25 13:23:57 -05:00
Raul Jordan
053038446c
Allow 8 Validator Multinode Cluster to Run Indefinitely (#2050)
* plug forkchoice to blockchain service's block processing

* fixed tests

* more fixes...

* clean ups

* fixed test

* Update beacon-chain/blockchain/block_processing.go

* merged with 2006 and started fixing tests

* remove prints

* fixed tests

* lint

* include ops service

* if there's a skip slot, slot--

* fixed typo

* started working on test

* no fork choice in propose

* bleh, need to fix state generator first

* state gen takes input slot

* feedback

* fixed tests

* preston's feedback

* fmt

* removed extra logging

* add more logs

* fixed validator attest

* builds

* fixed save block

* children fix

* removed verbose logs

* fix fork choice

* right logs

* Add Prometheus Counter for Reorg (#2051)

* fetch every slot (#2052)

* test Fixes

* lint

* only regenerate state if there was a reorg

* better logging

* fixed seed

* better logging

* process skip slots in assignment requests

* fix lint

* disable state root computation

* filter attestations in regular sync

* log important items

* better info logs

* added spans to stategen

* span in stategen

* set validator deadline

* randao stuff

* disable sig verify

* lint

* lint

* save only using historical states

* use new goroutine for handling sync messages

* change default buffer sizes

* better p2p

* rem some useless logs

* lint

* sync tests complete

* complete tests

* tests fixed

* lint

* fix flakey att service

* PR feedback

* undo k8s changes

* Update beacon-chain/blockchain/block_processing.go

* Update beacon-chain/sync/regular_sync.go

* Add feature flag to enable compute state root

* add comment

* gazelle lint fix
2019-03-25 10:21:21 -05:00
Preston Van Loon
7f11cbd5ed
print stack trace on panic recovery (#2042) 2019-03-20 10:17:44 -04:00
terence tsao
4a2ef17be1 fixed typo (#2038) 2019-03-20 01:13:14 -04:00
Preston Van Loon
612bb38077 Cross p2p spans, more spans, synchronous attestations, minor fixes (#2009)
* Fix assignments bug where validators don't retry for assignments on failure

* synch only please

* trying to fix state issues

* trying random stuff

* do not explode

* use ctx

* working build, failing tests

* broadcast local addrs as well as relay addrs

* fixed p2p tests, more tests to fix still

* another test fixed, log warning instead of throw error

* Fix last tests

* godoc

* add test for broadcast in apply fork choiec

* remove unneeded code

* remove tracer adapter, not needed

* remove extra stuff

* remove any

* revert addr_factory

* revert addr_factory

* Revert "revert addr_factory"

This reverts commit e93fb706494a1070158b8db31e67146d6b0648ad.

* Revert "revert addr_factory"

This reverts commit dedaa405559cc818698870c4e4570953367f1e3c.

* revert removal of this code

* unused param
2019-03-17 10:56:05 +08:00
Raul Jordan
5371218b9b
Refactor ChainService for Receiving Blocks Synchronously (#1984)
* refactor chain service

* restructure service and lint

* all calls to receive block in chain service are now blocking

* begin fixing tests

* refactored blockchain tests

* builds correctly

* blockchain tests pass again

* lint

* sync and rpc tests pass again

* done

* add in open tracing

* span in fork choice

* Update beacon-chain/blockchain/block_processing.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>
2019-03-13 17:17:32 -04:00
Nishant Das
000a42deac Make State Requests only Request for Finalized State (#1955)
* remove hash field for state request

* change to finalized state root

* refactor code for bug

* add in saving of finalized state

* add method to db

* adding finalized state root

* add new changes

* add in new changes

* add in return vars

* goimports

* review comments

* error message

* fix broken build

* fix cache state tests

* lint
2019-03-12 16:44:21 -04:00
Nishant Das
7179b6930b Remove Incorrect Error Logs in Sync (#1943) 2019-03-11 15:07:16 -07:00
Nishant Das
028506e7ed Handle Panics in Remaining Services (#1926)
* add recovery for initial sync

* add recovery for rpc

* adding to other services

* remaining services and tests

* fix test

* remove changes to rpc

* handle powchain

* handle in powchain

* abstract to shared package

* gazelle

* lint

* remove for operations

* add in more for operations

* travis

* fix lint
2019-03-10 17:53:28 -05:00
Nishant Das
7c5a7fb044 Handle Skipped Slots in Sync (#1939)
* handle edge cases

* tests passing

* Update beacon-chain/sync/initial-sync/service.go

* Update beacon-chain/sync/initial-sync/service.go

* Update beacon-chain/sync/initial-sync/service_test.go
2019-03-08 10:09:24 -06:00
Raul Jordan
205594c5d6 Fix Travis Failures on Master (#1938)
* resolve travis conflicts

* cp proto

* revert

* remove unused context
2019-03-07 23:54:41 -05:00
Preston Van Loon
2e7b08f97e
p2p fixes / improvements (#1931)
* don't broadcast to yourself

* todo

* Also listen for incoming streams

* send to a specific peer

* add godoc comment

* handle EOF as a normal thing

* gazelle

* add test for subscribing via direct comm
2019-03-07 21:37:15 -05:00
Nishant Das
2f9de6f93c Clean Up Batched Request (#1932)
* goimports

* do not log empty batches

* invalid request
2019-03-07 16:43:48 -06:00
Nishant Das
07a25ca710 Adding Prometheus Metrics and Tracing to Sync (#1858)
* adding tracing and metrics

* clean up main routine

* added tracing to initial sync

* gazelle

* fix tests

* fix naming

* comments

* consistent style in promauto counters

* rename spans

* final comment
2019-03-05 11:31:19 -06:00
Nishant Das
b264cdc557 Allow Node to Request State if it is Behind (#1875)
* fix naming

* comment

* tests
2019-03-05 10:59:49 -06:00
Nishant Das
d162b5603e clean up context (#1850) 2019-03-04 12:57:59 -08:00
Ivan Martinez
5aa7320507 Clear Up Code Inconsistencies (#1836) 2019-03-03 09:31:29 -08:00
Nishant Das
fe2eb5a812 Fixing Sync (#1830)
* fix querier

* new fixes for sync

* reducing batch size and adding equality condition

* remove commented code

* fix test

* fix few more tests
2019-03-03 11:03:47 -06:00
Preston Van Loon
a19cb1864e Tracing additions (#1747) 2019-02-27 19:55:47 -08:00
Nishant Das
a27c52e3a3
Update Initial Sync (#1694)
* adding check for chainHead

* add batch block limit

* adding state root

* removing genesisHash

* removing setBlockForInitialSync

* fixing tests

* remove obsolete check

* changing to check for state initialization

* changing var name

* fix test

* add in tests

* fix all tests

* lint

* reduce test time

* merge fixes

* gazelle
2019-02-26 13:37:28 +08:00
Raul Jordan
20803b4c9d
Revert Usage of ssz.TreeHash Across Repo (#1711)
* revert using ssz.TreeHash

* gazelle
2019-02-25 21:42:31 -06:00
Raul Jordan
e19920aec1
Miscellaneous Runtime Fixes & Improvements - Raul (#1674)
* fatal if impossible to receive chainstart

* fix tests

* fix

* custom delay

* completed custom delay

* errors

* better logs, nothing at genesis

* use demo in val

* add gazelle

* log

* starting to log stuff

* pass in ops

* avoid printing the large #s for debug, still working on tests..

* all around better logging

* fixed build error in epoch process

* fixed state transiton tests

* fixed block tests

* lint

* verify sigs in randao

* ready for inclusion falg

* only print waiting when slot is not valid

* fix build

* mod config

* fixed last justified slot issue

* fix inclusion

* fixed attestation issue

* using zero hash from params instead

* fix tests

* update balance

* removed swp

* more `- genesis_slot` for logs

* rem unused log

* fix broken tests

* account for skip slots in state root computation

* fixes done

* validator guide bug fixes - 671

* epoch boundary at the last slot of the epoch

* fix epoch issue

* more balance cal logs for debugging

* greater balance

* attestaton fixes

* fixes

* addressed testrun

* fixed ejection balance

* fix tests with far future epoch

* revert sync change

* revert initial sync change

* fix changes

* off by one att fix

* revert the att fix

* address comments

* format

* fix build

* rem file
2019-02-24 20:09:45 -06:00
Nishant Das
f5c88e1bcb
Test Syncing with Another Fully Synced Service (#1594)
* clean up test

* adding test setup

* add chainstart check to sync querier

* goimports

* goimports

* backend cleanup

* adding getters

* lint

* moving to db

* gazelle

* adding more services setup

* adding more changes

* adding different test setups

* other merge issues

* imports

* fixing keys

* get intial sync able to be set up

* new changes and gazelle

* use simulated p2p

* everything finally works

* unexport fields

* revert changes from merge

* remove mock server

* add documentation

* gazelle

* add another node

* review comments

* fix tests
2019-02-23 11:36:20 +05:30
terence tsao
8c04ced1a6 Name Changes to Align with V0.3 (#1621)
* fixed epoch_processing

* penalize->slash

* exit -> voluntary_exit

*  SEED_LOOKAHEAD -> MIN_SEED_LOOKAHED

* ENTRY_EXIT_DELAY -> ACTIVATION_EXIT_DELAY

* `INCLUDER_REWARD_QUOTIENT` -> `ATTESTATION_INCLUSION_REWARD_QUOTIEN`

* LatestIndexRoots -> LatestActiveIndexRoots

* `MIN_VALIDATOR_WITHDRAWAL_EPOCHS` -> `MIN_VALIDATOR_WITHDRAWAL_DELAY`

* MAX_WITHDRAWALS_PER_EPOCH -> MAX_EXIT_DEQUEUES_PER_EPOCH

* ETH1_DATA_VOTING_PERIOD -> EPOCHS_PER_ETH1_VOTING_PERIOD

* SLOT_DURATION -> SECONDS_PER_SLOT

* EPOCH_LENGTH -> SLOTS_PER_EPOCH

* SLOT_DURATION -> SECONDS_PER_SLOT take 2

* rest of the misc fixes for config name changes

* remove tools/bootnode/.!74296!bootnode.go

* `current_epoch_start_shard` -> `current_shuffling_start_shard`, `current_shuffling_epoch`, `current_shuffling_see`

* go fmt

* fixed comment

* updated pseudocode comments

* merged master
2019-02-18 10:52:16 -06:00
Raul Jordan
ae13bb5c83
Use Tree Hashing Algorithm Everywhere for Block Roots and State Root (#1577)
* begin using tree hash instead of hash for state and blocks

* replace with ssz

* rem references to vote cache

* replace all instances of state/block hash with root

* goimports

* almost every test in the world fails

* goimports

* all tests pass, tree hash across repo
2019-02-14 14:04:47 -06:00
terence tsao
4cadd534db
Update Genesis Slot and Epoch (#1537) 2019-02-10 23:09:35 +01:00
Nishant Das
9ecef934d8 Update Beacon State to Match the Spec (#1489)
* first line of changes

* fix all tests in core package

* Finished fixing up all tests

* docs

* more docs
2019-02-06 10:44:27 -06:00
Raul Jordan
cec6c8de83 Standardize Naming of Shared Packages, Remove Deprecated Packages (#1435)
* rem txpool

* standardize repo shared utils, remove deprecated items

* complete

* package lvl godoc
2019-01-31 10:53:58 +08:00
Ed Mazurek
10b237e72d db methods: remove Get prefix (#1351)
* db methods: remove Get prefix for getter functions

* db methods: consistent test names by removing Get
2019-01-21 17:34:11 +08:00
Nishant Das
7ff544fddf
Fix travis and Other Linter Errors (#1356)
* fix travis and other linter errors

* remove megacheck
2019-01-21 13:35:34 +08:00
Preston Van Loon
592c5c3d92 Refactor hashing of beaconblock to shared (#1315)
* Add beacon block hash function

* Refactor/remove old hash method

* gazelle
2019-01-15 00:41:20 +08:00
terence tsao
e313e97666
Implement Validator Rotation Logic (#1279) 2019-01-13 12:52:31 -08:00
Andrei Ivasko
b29188128d ToBytes32 (#1306) 2019-01-13 09:04:14 -05:00
Raul Jordan
4bb6196f4f
Wrap Up Data Structure Renaming (#1270)
* spec renames

* fix goimports

* fix items
2019-01-09 03:49:17 -06:00
Raul Jordan
7e1af4fceb
Remove Use of Proto .GetX for Value Getters (#1235) 2019-01-04 21:58:19 -06:00
Raul Jordan
952dbe983e
Integrate Block Operations Into State Transition (#1135) 2018-12-26 21:15:12 -06:00
Nishant Das
5804dcfb67
Add Batch Syncing for Initial Sync (#1119)
* adding proto objects

* simulator changes

* adding stuff for initial sycn

* more changes

* finally fixed test

* fixing tests and sync

* test fix

* adding batched block response

* remove types references

* fix tests

* addressing review

* removing genesis block

* updating protos

* fix merge conflicts

* file permissions
2018-12-26 22:16:46 +08:00
Nishant Das
9db1002d87
Updating Genesis Block to Spec (#1172)
* Updating to spec

* addressing review comments
2018-12-25 14:47:07 +08:00
Raul Jordan
5d26efcfaa
Deprecate types.BeaconState with pb.BeaconState Generated Proto (#1164) 2018-12-23 16:51:04 -06:00
Preston Van Loon
10f45744d6
Use gogo protobuf as the protoc compiler (#1165)
* first commit, remote att types

* no more agg attestation proto

* regen mock

* only attestations

* proto

* att process

* fix att references

* more tests passing

* use att protos

* complete

* change visibility

* use gogoprotobu
2018-12-23 15:34:59 -05:00