Commit Graph

2858 Commits

Author SHA1 Message Date
Nishant Das
a7ccd52a95 Save Deposit Contract Address (#4114)
* save contract address
* Update beacon-chain/node/node.go
* Merge branch 'master' into saveContract
* Merge refs/heads/master into saveContract
2019-11-26 21:01:56 +00:00
terence tsao
1ced4754db Add signatures to logs (#4095)
* Enhance logging with sig
* Fixed
* Merge branch 'master' into add-sig
* Merge branch 'master' into add-sig
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into add-sig
* signature via debug
* Merge branch 'add-sig' of https://github.com/prysmaticlabs/prysm into add-sig
* Merge branch 'master' into add-sig
* Merge branch 'master' into add-sig
* Merge branch 'master' into add-sig
2019-11-26 20:36:18 +00:00
terence tsao
b872f74fd3 Do not save duplicated indices (#4118)
* Added a duplication test

* Refactor

* Updated test

* Do not save dups for indices bucket
2019-11-26 13:26:35 -06:00
Ivan Martinez
c1c48a8af5 Create Benchmarks Package for State Transition (#3688)
* Begin benchmarks file for block processing
* Complete block processing benchmarks
* Begin epoch benchmarks
* Write most of epoch benchmarks
* Start config
* Make cases for max conditions
* Begin work on benchmarking doc
* Update benchmark numbers
* Complete epoch benchmarks
* Minor changes
* Make createFullBlock function
* Clean up block benchmarks
* Begin fixing merge issues
* Start adding 4M benchmarks
* Almost finish epoch benchmarks
* Test blocks under real life conditions
* More progress on benchmarks
* Fixes
* Fix benchmark errors
* Begin fixing benchmarks
* More progress on tests
* Complete epoch benchmarks
* More progress on block benches
* Finish epoch benchmarks, get progress on block benchmarks
* Undo unneeded changes
* Fix
* Fix block benchmarks
* Complete block benchmarks
* Finish block benchmarks
* Complete benchmarks
* Increase block benchmarks to 65536
* Fix everything
* Reset configs after benchmarks
* Fix logging and suggestions
* Fix comments
* Fix benchmarks after merge
* Fix merge issues
* Add sanity tests for benchmark
* Make sanity check simpler
* Begin fixing after merge
* Add log
* Remove extra line
* Remove unneeded change
* Finally get block benchmarks to pass
* Begin fixing epoch test
* Finetuning constants
* Revert "Finetuning constants"

This reverts commit a872790d675154e1ce213285e76fb6e08ef21d79.
* Finetuning
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Add benches for helper functions
* Abstract block generation to testutil
* Create block generation util in testutil
* Gazelle
* Fix deps
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into block-util
* Fix imports
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into block-util
* Merge branch 'master' into block-util
* Change tests to use config and fix integer division
* Merge branch 'block-util' of https://github.com/0xKiwi/prysm into block-util
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into block-util
* Remove logs
* Fix build
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'master' into block-util
* Add test to ensure finalization occurs
* Add check for finalization
* Merge branch 'block-util' of https://github.com/0xKiwi/prysm into block-util
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into block-util
* Add comment for incrementing the state
* Fix test
* Fix test
* Merge branch 'master' into block-util
* Fix testutil use
* Fix tests
* Change var name
* Merge branch 'master' into block-util
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'block-util' of https://github.com/0xKiwi/prysm into new-benchmarks
* Begin cleaning benchmarks
* Get some numbers going
* Use state saved to disk
* Remove cruft
* Cleanup
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Fix merge arrows
* Set up block util and benchmarks for 128 attestations
* Use intended config for benchmark
* Add more benchmark functions
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Add benchmark epoch and modify block gen config to exclude signing
* Cleanup
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Begin unstaleling
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Update block gen util to v0.9 changes
* Prepare benchmarks to use marshalled files
* Cleanup block gen tool some more
* split up into file generation and benchmarking
* Remove logrus
* Merge branch 'master' into new-benchmarks
* Get benchmarks work, start work on process epoch benchmark
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'new-benchmarks' of https://github.com/0xKiwi/prysm into new-benchmarks
* All benchmarks working
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Fix after merge
* Cleanup
* Add bazel target
* Added TestBenchmarkExecuteStateTransition_WithCache
* Change tests to use SSZ and begin making binary
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'new-benchmarks' of https://github.com/0xKiwi/prysm into new-benchmarks
* bazel binary
* Fully change to binary
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Create go_binary to handle benchmark files
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Gofmt
* Remove genesis state from generated files
* Fix tests
* Gazelle
* Fix tests
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Fix block util
* Allow attestations to be in future for block util
* Fix inclusion delay issue
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Finally fix test
* Add README detailing usage and results
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Change test to run with bazel test
* Fix imports
* Merge branch 'master' into new-benchmarks
* Accidentally removed  config change
* Merge branch 'new-benchmarks' of https://github.com/0xKiwi/prysm into new-benchmarks
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Move to core/state/
* Update readme
* Gazelle
* Remove test for cached block
2019-11-26 18:09:57 +00:00
Nishant Das
b88e6dc918 Speed Up Block Processing In Sync (#4075)
* fix proto
* make them non-batched
* gate behind flag
* fix refs
* fix refs
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* use global archiver flags
* lint
* Merge branch 'master' into speedUpProcessing
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* preston's review
* Merge branch 'speedUpProcessing' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* Merge branch 'master' into speedUpProcessing
* Merge branch 'master' into speedUpProcessing
* Merge branch 'master' into speedUpProcessing
2019-11-26 07:15:54 +00:00
terence tsao
3868837471
Tool to dump graphviz data for block tree (#4108) 2019-11-25 21:06:25 -08:00
Preston Van Loon
60b1596c4d Lower conn mgr grace period to 1s (#4109)
* Update options.go
2019-11-26 01:32:10 +00:00
Raul Jordan
4f0dcd5e6e Prevent Requesting Current Epoch for Validator Participation (#4104)
* cannot request current epoch
* test for prev epoch instead
* Merge branch 'master' into no-curr-epoch-participation
* Merge branch 'master' into no-curr-epoch-participation
2019-11-25 23:31:21 +00:00
Preston Van Loon
ac405c714f Enforce --p2p-max-peers (#4106)
* Enforce p2p-max-peers
* high == low
2019-11-25 18:55:20 +00:00
Nishant Das
7d0e5a9dc4 Use Latest Vote Map (#4102)
* add latest vote map
* fix all tests
* remove db crud methods
* Merge branch 'master' into latestVoteMap
* preston's review
* Merge branch 'latestVoteMap' of https://github.com/prysmaticlabs/geth-sharding into latestVoteMap
2019-11-25 16:34:20 +00:00
Raul Jordan
feb1267fee Properly Return Finalized Epoch in GetValidatorParticipation Archival Endpoint (#4091)
* properly handle retrieving archived finalized epochs
* test passes for determining if epoch finalized
* Merge branch 'master' into archive-finality
* Merge refs/heads/master into archive-finality
* Merge refs/heads/master into archive-finality
* Merge refs/heads/master into archive-finality
* Merge refs/heads/master into archive-finality
* Merge refs/heads/master into archive-finality
* Merge branch 'master' into archive-finality
* prevent setup panic
* Merge branch 'archive-finality' of github.com:prysmaticlabs/prysm into archive-finality
* Merge refs/heads/master into archive-finality
2019-11-25 15:26:32 +00:00
Nishant Das
7a9c297206 Revert "Update Pending Queue (#4066)" (#4101)
This reverts commit a264a097cc.
2019-11-24 21:05:51 -08:00
terence tsao
21deed0fb7 Revert "Add Lock When Accessing Checkpoints" (#4094)
* Revert "Add Lock When Accessing Checkpoints (#4086)"

This reverts commit 2f392544a6.
* Merge branch 'master' into revert-4086-checkpointLock
2019-11-24 17:12:56 +00:00
terence tsao
627791c54e Complete finalization metrics (#4096)
* Complete finaliztion metrics

* Fixed test
2019-11-23 17:12:29 -08:00
Jim McDonald
3358bde42d Feedfixes (#4093)
* Migrate ChainStarted and StateInitialized to state notifier
* Provide state notifier to powchain service
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Remove commented line
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Accept err from HeadState() as non-fatal
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Explicit unsubscribes from state channels where required
2019-11-23 11:15:02 +00:00
Jim McDonald
9e45cffabc Move StateInitialized and ChainStarted to state feed (#4084)
* Migrate ChainStarted and StateInitialized to state notifier
* Provide state notifier to powchain service
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Remove commented line
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Accept err from HeadState() as non-fatal
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
2019-11-23 03:35:47 +00:00
terence tsao
2c8ff7b36f Update attester to wait till one-third (#4090)
* Update to 1/3
* Use tag
* Test
* Fixed test
* Merge branch 'master' into update-1/3
* Merge branch 'master' into update-1/3
2019-11-23 02:23:20 +00:00
terence tsao
a7ec0679b5 Implement wait till two thirds (#4089)
* Implement two thirds

* Test
2019-11-22 16:48:40 -06:00
Preston Van Loon
f717c5d852 Release --prune-finalized-states to all (#4082)
* Release --prune-finalized-states to all
* Merge branch 'master' into deprecate-ff-prune-finalized-states
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-prune-finalized-states
2019-11-22 20:29:46 +00:00
Preston Van Loon
0cec0ee6c3 Release --optimize-process-epoch to all (#4080)
* Release optimize-proces-epoch to all
* Merge branch 'master' into deprecate-ff-optimize-process-epoch
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
2019-11-22 09:40:53 +00:00
Nishant Das
2f392544a6 Add Lock When Accessing Checkpoints (#4086)
* fix data races
* Merge branch 'master' into checkpointLock
* Merge branch 'master' into checkpointLock
* Merge refs/heads/master into checkpointLock
2019-11-22 06:34:42 +00:00
Preston Van Loon
75ce8359eb Buildkite: Disable failing BES (#4087)
* Disable failing BES
2019-11-22 06:11:41 +00:00
terence tsao
f5cb04012e Aggregator selection from RPC to validator client (#4071)
* Config
* Updated proto
* Updated pool
* Updated RPC
* Updated validator client
* run time works
* Clean ups
* Fix tests
* Visibility
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into aggregator
* Raul's feedback
* Tests for RPC server
* Tests for validator client
* Span
* More tests
* Use go routine for SubmitAggregateAndProof
* Go routines
* Updated comments
* Use array of roles
* Fixed tests
* Build
* Update validator/client/runner.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update validator/client/runner.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* If
* Merge branch 'refactor-validator-roles' of https://github.com/prysmaticlabs/prysm into refactor-validator-roles
* Empty
* Feedback
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into aggregator
* Removed proto/eth/v1alpha1/shard_chain.pb.go?
* Cleaned up
* Revert
* Comments
* Lint
* Comment
* Merge branch 'master' into aggregator
2019-11-22 05:11:38 +00:00
Raul Jordan
f461d1e024 Resolve Panic in ListValidatorBalances RPC (#4051)
* adding default response
* regression test for archival
* include full regression test
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* listbal
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* Merge refs/heads/master into resolve-panic-rpc
2019-11-22 04:39:28 +00:00
Preston Van Loon
bdbd0aaeb8 Deprecate feature flag --scatter (#4079)
* deprecate --scatter. issue #4031
* forgot one for #4061
* use deprecatedUsage
* hidden
* Merge branch 'master' into deprecate-ff-scatter
* Merge branch 'master' into deprecate-ff-scatter
2019-11-22 04:08:49 +00:00
Preston Van Loon
715d06a215 gRPC Gateway: Emit JSON empty fields by default (#4085)
* Emit JSON empty fields by default
* Merge branch 'master' into emit-json-empty-fields
2019-11-22 03:36:47 +00:00
terence tsao
976a3af637
Refactor validator roles into an array (#4081) 2019-11-21 14:35:20 -08:00
Raul Jordan
8f8d2d36c0
Filter ListValidators by Active in RPC (#4061)
* update workspace

* include active filter

* fix up latest changes to match naming

* better comments, fix evaluators

* latest master

* filter items

* filter only active validators
2019-11-21 14:29:24 -06:00
Nishant Das
a264a097cc
Update Pending Queue (#4066)
* update queue

* fix test

* put this all in validate method

* remove ancestors too

* not needed

* terence's review

* period

* preston's review
2019-11-21 21:24:50 +08:00
shayzluf
4330839bc1
Add surround check to endpoint (#4065)
* first version of the watchtower api

* service files

* Begin work on grpc server

* More changes to server

* REnames and mock setup

* working test

* merge

* double propose detection test

* nishant review

* todo change

* gaz

* fix service

* gaz

* remove unused import

* gaz

* resolve circular dependency

* resolve circular dependency 2nd try

* remove package

* fix package

* fix test

* added tests

* gaz

* remove status check

* gaz

* remove context

* remove context

* change var name

* moved to rpc dir

* gaz

* remove server code

* gaz

* slasher server

* visibility change

* pb

* service update

* gaz

* slasher grpc server

* making it work

* setup db and start

* gaz

* service flags fixes

* grpc service running

* go imports

* remove new initializer

* gaz

* remove feature flags

* change back SetupSlasherDB

* fix SetupSlasherDB calls

* define err

* fix bad merge

* fix test

* fix imports

* fix imports

* fix imports

* add cancel

* comment stop

* fix cancel issue

* remove unneeded code

* bring back bad merge that removed TODO

* remove use of epoch as am input

* fixed slasher to be runable again

* wait for channel close

* gaz

* small test

* flags fix

* fix flag order

* double vote detection

* remove source epoch from indexed attestation indices

* change server method to receive indexed attestation

* start implementation

* double vote detection

* proto

* pb

* fir comment

* add surround detection and retrieval to endpoint

* nishant review

* import fix

* fix miss order

* fix detection 0 case
added tests

* terence review
2019-11-21 12:41:23 +05:30
terence tsao
835418d1e3
Fix UpdateCommitteeCache slot (#4074) 2019-11-20 21:29:43 -08:00
Raul Jordan
ae07dc7962
Archive Data Even Through Skip Slots (#4054)
* red test first

* does not archive through skip slot

* test out at runtime

* underflow check

* fix tests

* rem info log
2019-11-19 23:53:28 -06:00
shayzluf
d071a0a90a
Double vote detection (#4049)
* first version of the watchtower api

* service files

* Begin work on grpc server

* More changes to server

* REnames and mock setup

* working test

* merge

* double propose detection test

* nishant review

* todo change

* gaz

* fix service

* gaz

* remove unused import

* gaz

* resolve circular dependency

* resolve circular dependency 2nd try

* remove package

* fix package

* fix test

* added tests

* gaz

* remove status check

* gaz

* remove context

* remove context

* change var name

* moved to rpc dir

* gaz

* remove server code

* gaz

* slasher server

* visibility change

* pb

* service update

* gaz

* slasher grpc server

* making it work

* setup db and start

* gaz

* service flags fixes

* grpc service running

* go imports

* remove new initializer

* gaz

* remove feature flags

* change back SetupSlasherDB

* fix SetupSlasherDB calls

* define err

* fix bad merge

* fix test

* fix imports

* fix imports

* fix imports

* add cancel

* comment stop

* fix cancel issue

* remove unneeded code

* bring back bad merge that removed TODO

* remove use of epoch as am input

* fixed slasher to be runable again

* wait for channel close

* gaz

* small test

* flags fix

* fix flag order

* double vote detection

* remove source epoch from indexed attestation indices

* change server method to receive indexed attestation

* start implementation

* double vote detection

* proto

* pb

* fir comment

* nishant review

* import fix

* Update slasher/db/indexed_attestations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence feedback
2019-11-20 10:44:50 +05:30
Ivan Martinez
2d7802c637 Rename featureconfig.Flag to Flags (#4063) 2019-11-19 21:03:00 -06:00
terence tsao
fcb663acde Implement aggregation helpers (#4062)
* Aggregation helpers

* Tests

* Config

* Faulty test cases

* Err
2019-11-19 20:24:39 -06:00
Raul Jordan
858dbbf038
Update Ethereum APIs and Match Schemas (#4059)
* update workspace

* include active filter

* fix up latest changes to match naming

* better comments, fix evaluators

* latest master

* Update proto/eth/v1alpha1/beacon_chain.proto
2019-11-19 18:36:45 -06:00
Jim McDonald
49c2dd2cfc Move the state notifier to a different module (#4058)
* Move state notifier to statefeed

* Updates to state notifier

* Create state feed in beacon node

* Formatting
2019-11-19 16:15:48 -06:00
terence tsao
7a22e98c0f Update ChainHead (#4053)
* Can build

* All tests pass

* Update beacon-chain/blockchain/chain_info.go

* Fix context

* Update chainhead

* Tests

* Tests

* e2e

* Update ordering

* Typo

* Use root to get slot

* Division
2019-11-19 13:33:13 -06:00
terence tsao
26da7c4114 Nil state fallback in Blockchain.HeadState() (#4042)
* Can build

* All tests pass

* Update beacon-chain/blockchain/chain_info.go

* Fix context
2019-11-19 10:12:50 -06:00
Nishant Das
7acb45d186 add one more return (#4050) 2019-11-19 09:40:15 -06:00
Preston Van Loon
24a5000e47 return to exit select loop rather than break (#4040) 2019-11-19 21:22:45 +08:00
Jim McDonald
65d920e13a Add generic state feed (#4004)
* Initial implementation of state feed

* Add instructions on adding new events

* Tidy up log messages

* Tidy up mock

* Update beacon-chain/core/statefeed/events.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/core/statefeed/events.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Remove unused BlockReceivedData

* Rename BlockHash to BlockRoot in BlockProcessedData

* Punctuation

* Use correct root for block processed event

* StateFeeder -> StateNotifier; fix up tests.

* Add Verified flag to BlockProcessed event

* Fix visibility in Bazel
2019-11-19 17:17:41 +08:00
Preston Van Loon
d27d18b192
Update protobuf to v3.10.1 and rules_docker to v0.12.1 (#4048)
* update protobuf v3.10.1

* update rules_docker
2019-11-18 23:02:25 -08:00
Nishant Das
0e88085661
add step fix (#4047) 2019-11-19 12:51:40 +08:00
Preston Van Loon
3f6435ac80
Fix server side beacon blocks by range (#4046) 2019-11-18 19:56:37 -08:00
Preston Van Loon
64b69d9216 remove fully async from bes upload (#4044) 2019-11-19 10:57:34 +08:00
Preston Van Loon
13207a9de5
Improve validator status method (#4032)
* Cleanup validatorStatus

* gaz

* fix tests

* fix tests
2019-11-18 16:47:02 -08:00
Raul Jordan
ab756ec094
Return Empty Results Instead of Pagination Error in RPC + Prevent Future Epoch Requests (#4030)
* return empty if no attestations

* list balances proper response

* standardize epoch error

* future epoch error test

* no results test

* no results in list attestations

* test for list blocks no results

* cannot request future epoch for balances rpc

* test for no results in balances

* adding tests for get validator

* cannot request future in participation

* useless conditional

* resolve old epoch test

* completed failing tests

* fix request bug
2019-11-18 17:24:33 -06:00
terence tsao
499f05f34b Return error instead of logging (#4039) 2019-11-18 14:09:26 -08:00
Preston Van Loon
0077654fb5
Fix deleted branch from ethereumapis (#4034) 2019-11-18 13:14:49 -08:00