Commit Graph

1304 Commits

Author SHA1 Message Date
Jim McDonald
0e99e4af4f Update lastUpdated on peerstatus.Set() (#4152)
* Update lastUpdated on Set()
* Merge branch 'master' into lastupdated
* gazelle
* Merge branch 'master' into lastupdated
* Merge branch 'master' into lastupdated
2019-11-30 19:47:08 +00:00
terence tsao
e87337a97a Update forkchoice spec link to v0.9.0 (#4147)
* Update forkchoice doc link to v0.9.0
* Merge refs/heads/master into update-link
2019-11-30 05:48:18 +00:00
Jim McDonald
53523b3eef Implement ListPeers API call (#4151)
* update ethereumapis from https://github.com/prysmaticlabs/ethereumapis/pull/55
* add stub for https://github.com/prysmaticlabs/prysm/issues/4141
* Add ListPeers API call
* Merge
* Add comment for exported method
* Fix visibility of new peers package.
* Merge branch 'master' into peersapi
2019-11-30 05:36:02 +00:00
terence tsao
5ec02b28a5 Remove pruned states check (#4153)
* Removed already pruned check
* Tested run time
2019-11-30 05:07:13 +00:00
Preston Van Loon
b08f3f760d Update ethereumapis (#4142)
* update ethereumapis from https://github.com/prysmaticlabs/ethereumapis/pull/55
* add stub for https://github.com/prysmaticlabs/prysm/issues/4141
2019-11-29 16:44:51 +00:00
terence tsao
7495961d6b Prune boundary state (#4139)
* Delete epoch boundary slot of last finalized epoch
* Case to cover start slot is skipped
* Test
* Feature flag
* feature gate the new functionality only
* Update DB for migration
* Test
* Fmt
* Fixed test
* Gazelle
2019-11-28 23:05:47 +00:00
Preston Van Loon
4dbf68b50c Fix log message from PR #4130 (#4136)
* fix log message from PR #4130
* Merge refs/heads/master into fix-log-2
2019-11-27 23:49:47 +00:00
terence tsao
e90358cd8e Removed unused mocks (#4135)
* Removed unused mocks

* Lint

* Gaz
2019-11-27 14:52:24 -06:00
Nishant Das
80865ff3f2 Account for Skipped Slots When Requesting for Blocks (#4130)
* add check

* Update beacon-chain/sync/initial-sync/round_robin.go
2019-11-27 11:18:18 -06:00
Jim McDonald
60469ec7ee Avoid crash if peer goes missing (#4115)
* Migrate ChainStarted and StateInitialized to state notifier
* Provide state notifier to powchain service
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Remove commented line
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Accept err from HeadState() as non-fatal
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Do not crash if peer goes missing
* Additional catches
* Merge branch 'master' into rrfix
* Use single refresh time
* Merge branch 'master' into rrfix
2019-11-27 16:00:59 +00:00
Nishant Das
3682bf1cda Check Best Peer Before Syncing (#4128)
* add check
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into checkBestPeer
* Merge refs/heads/master into checkBestPeer
2019-11-27 06:56:02 +00:00
Preston Van Loon
e203f66fe0 DB Improvements: Snappy compression, remove some unnecessary batch / goroutines (#4125)
* do not use batch for SaveAttestations
* use snappy compression
* Encode / decode everything with snappy
* Add snappy migration path
* batch is probably fine...
* fix test
* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into remove-batch-attestations
* add sanity check
* remove that thing
* gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into remove-batch-attestations
2019-11-27 06:32:56 +00:00
terence tsao
04df922ac9 Add votes to tree graph (#4127)
* Added block tree tool
* Gaz
* Updated workspace
* Playing around
* Adding votes
* Votes work
* Comments
* Gaz
* Add tools to subpackage
* Merge branch 'master' into block-tree-tool-1
2019-11-27 06:07:52 +00:00
Raul Jordan
0326be86b5 Apply Patch Rules to Use EthereumAPIs Generated Protos in Prysm (#4112)
* starting on patch
* finish determining all required patches
* properly redefine the patch rules
* new patch
* rem double semicolon
* fix patch file
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-eth-protos
* building the deps
* test target passes using ethereumapis
* compile gateway
* attempting to build everything
* e2e use ethereumapis
* more fixes for slasher
* other item
* getting closer to compiling slasher
* build slasher package
* Merge branch 'master' into deprecate-eth-protos
* Merge branch 'master' into deprecate-eth-protos
* fix benches
* lint gazelle
* Merge branch 'deprecate-eth-protos' of github.com:prysmaticlabs/prysm into deprecate-eth-protos
* proper gateway
* lint
* Merge branch 'master' into deprecate-eth-protos
* fix build
* Merge branch 'deprecate-eth-protos' of github.com:prysmaticlabs/prysm into deprecate-eth-protos
* use swag
* resolve
* ignore change
* include new patch changes
* fix test
* builds
* fix e2e
* gaz
2019-11-27 05:08:18 +00:00
Nishant Das
a7ccd52a95 Save Deposit Contract Address (#4114)
* save contract address
* Update beacon-chain/node/node.go
* Merge branch 'master' into saveContract
* Merge refs/heads/master into saveContract
2019-11-26 21:01:56 +00:00
terence tsao
1ced4754db Add signatures to logs (#4095)
* Enhance logging with sig
* Fixed
* Merge branch 'master' into add-sig
* Merge branch 'master' into add-sig
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into add-sig
* signature via debug
* Merge branch 'add-sig' of https://github.com/prysmaticlabs/prysm into add-sig
* Merge branch 'master' into add-sig
* Merge branch 'master' into add-sig
* Merge branch 'master' into add-sig
2019-11-26 20:36:18 +00:00
terence tsao
b872f74fd3 Do not save duplicated indices (#4118)
* Added a duplication test

* Refactor

* Updated test

* Do not save dups for indices bucket
2019-11-26 13:26:35 -06:00
Ivan Martinez
c1c48a8af5 Create Benchmarks Package for State Transition (#3688)
* Begin benchmarks file for block processing
* Complete block processing benchmarks
* Begin epoch benchmarks
* Write most of epoch benchmarks
* Start config
* Make cases for max conditions
* Begin work on benchmarking doc
* Update benchmark numbers
* Complete epoch benchmarks
* Minor changes
* Make createFullBlock function
* Clean up block benchmarks
* Begin fixing merge issues
* Start adding 4M benchmarks
* Almost finish epoch benchmarks
* Test blocks under real life conditions
* More progress on benchmarks
* Fixes
* Fix benchmark errors
* Begin fixing benchmarks
* More progress on tests
* Complete epoch benchmarks
* More progress on block benches
* Finish epoch benchmarks, get progress on block benchmarks
* Undo unneeded changes
* Fix
* Fix block benchmarks
* Complete block benchmarks
* Finish block benchmarks
* Complete benchmarks
* Increase block benchmarks to 65536
* Fix everything
* Reset configs after benchmarks
* Fix logging and suggestions
* Fix comments
* Fix benchmarks after merge
* Fix merge issues
* Add sanity tests for benchmark
* Make sanity check simpler
* Begin fixing after merge
* Add log
* Remove extra line
* Remove unneeded change
* Finally get block benchmarks to pass
* Begin fixing epoch test
* Finetuning constants
* Revert "Finetuning constants"

This reverts commit a872790d675154e1ce213285e76fb6e08ef21d79.
* Finetuning
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Add benches for helper functions
* Abstract block generation to testutil
* Create block generation util in testutil
* Gazelle
* Fix deps
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into block-util
* Fix imports
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into block-util
* Merge branch 'master' into block-util
* Change tests to use config and fix integer division
* Merge branch 'block-util' of https://github.com/0xKiwi/prysm into block-util
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into block-util
* Remove logs
* Fix build
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'master' into block-util
* Add test to ensure finalization occurs
* Add check for finalization
* Merge branch 'block-util' of https://github.com/0xKiwi/prysm into block-util
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into block-util
* Add comment for incrementing the state
* Fix test
* Fix test
* Merge branch 'master' into block-util
* Fix testutil use
* Fix tests
* Change var name
* Merge branch 'master' into block-util
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'block-util' of https://github.com/0xKiwi/prysm into new-benchmarks
* Begin cleaning benchmarks
* Get some numbers going
* Use state saved to disk
* Remove cruft
* Cleanup
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Fix merge arrows
* Set up block util and benchmarks for 128 attestations
* Use intended config for benchmark
* Add more benchmark functions
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Add benchmark epoch and modify block gen config to exclude signing
* Cleanup
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Begin unstaleling
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Update block gen util to v0.9 changes
* Prepare benchmarks to use marshalled files
* Cleanup block gen tool some more
* split up into file generation and benchmarking
* Remove logrus
* Merge branch 'master' into new-benchmarks
* Get benchmarks work, start work on process epoch benchmark
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'new-benchmarks' of https://github.com/0xKiwi/prysm into new-benchmarks
* All benchmarks working
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Fix after merge
* Cleanup
* Add bazel target
* Added TestBenchmarkExecuteStateTransition_WithCache
* Change tests to use SSZ and begin making binary
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Merge branch 'new-benchmarks' of https://github.com/0xKiwi/prysm into new-benchmarks
* bazel binary
* Fully change to binary
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Create go_binary to handle benchmark files
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Gofmt
* Remove genesis state from generated files
* Fix tests
* Gazelle
* Fix tests
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Fix block util
* Allow attestations to be in future for block util
* Fix inclusion delay issue
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Finally fix test
* Add README detailing usage and results
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Change test to run with bazel test
* Fix imports
* Merge branch 'master' into new-benchmarks
* Accidentally removed  config change
* Merge branch 'new-benchmarks' of https://github.com/0xKiwi/prysm into new-benchmarks
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into new-benchmarks
* Move to core/state/
* Update readme
* Gazelle
* Remove test for cached block
2019-11-26 18:09:57 +00:00
Nishant Das
b88e6dc918 Speed Up Block Processing In Sync (#4075)
* fix proto
* make them non-batched
* gate behind flag
* fix refs
* fix refs
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* use global archiver flags
* lint
* Merge branch 'master' into speedUpProcessing
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* preston's review
* Merge branch 'speedUpProcessing' of https://github.com/prysmaticlabs/geth-sharding into speedUpProcessing
* Merge branch 'master' into speedUpProcessing
* Merge branch 'master' into speedUpProcessing
* Merge branch 'master' into speedUpProcessing
2019-11-26 07:15:54 +00:00
Preston Van Loon
60b1596c4d Lower conn mgr grace period to 1s (#4109)
* Update options.go
2019-11-26 01:32:10 +00:00
Raul Jordan
4f0dcd5e6e Prevent Requesting Current Epoch for Validator Participation (#4104)
* cannot request current epoch
* test for prev epoch instead
* Merge branch 'master' into no-curr-epoch-participation
* Merge branch 'master' into no-curr-epoch-participation
2019-11-25 23:31:21 +00:00
Preston Van Loon
ac405c714f Enforce --p2p-max-peers (#4106)
* Enforce p2p-max-peers
* high == low
2019-11-25 18:55:20 +00:00
Nishant Das
7d0e5a9dc4 Use Latest Vote Map (#4102)
* add latest vote map
* fix all tests
* remove db crud methods
* Merge branch 'master' into latestVoteMap
* preston's review
* Merge branch 'latestVoteMap' of https://github.com/prysmaticlabs/geth-sharding into latestVoteMap
2019-11-25 16:34:20 +00:00
Raul Jordan
feb1267fee Properly Return Finalized Epoch in GetValidatorParticipation Archival Endpoint (#4091)
* properly handle retrieving archived finalized epochs
* test passes for determining if epoch finalized
* Merge branch 'master' into archive-finality
* Merge refs/heads/master into archive-finality
* Merge refs/heads/master into archive-finality
* Merge refs/heads/master into archive-finality
* Merge refs/heads/master into archive-finality
* Merge refs/heads/master into archive-finality
* Merge branch 'master' into archive-finality
* prevent setup panic
* Merge branch 'archive-finality' of github.com:prysmaticlabs/prysm into archive-finality
* Merge refs/heads/master into archive-finality
2019-11-25 15:26:32 +00:00
Nishant Das
7a9c297206 Revert "Update Pending Queue (#4066)" (#4101)
This reverts commit a264a097cc.
2019-11-24 21:05:51 -08:00
terence tsao
21deed0fb7 Revert "Add Lock When Accessing Checkpoints" (#4094)
* Revert "Add Lock When Accessing Checkpoints (#4086)"

This reverts commit 2f392544a6.
* Merge branch 'master' into revert-4086-checkpointLock
2019-11-24 17:12:56 +00:00
terence tsao
627791c54e Complete finalization metrics (#4096)
* Complete finaliztion metrics

* Fixed test
2019-11-23 17:12:29 -08:00
Jim McDonald
3358bde42d Feedfixes (#4093)
* Migrate ChainStarted and StateInitialized to state notifier
* Provide state notifier to powchain service
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Remove commented line
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Accept err from HeadState() as non-fatal
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Explicit unsubscribes from state channels where required
2019-11-23 11:15:02 +00:00
Jim McDonald
9e45cffabc Move StateInitialized and ChainStarted to state feed (#4084)
* Migrate ChainStarted and StateInitialized to state notifier
* Provide state notifier to powchain service
* Merge remote-tracking branch 'upstream/master'
* Merge remote-tracking branch 'upstream/master'
* Remove commented line
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge remote-tracking branch 'upstream/master'
* Merge branch 'master' of github.com:mcdee/prysm
* Accept err from HeadState() as non-fatal
* Merge branch 'master' into master
* Merge branch 'master' into master
* Merge branch 'master' into master
2019-11-23 03:35:47 +00:00
Preston Van Loon
f717c5d852 Release --prune-finalized-states to all (#4082)
* Release --prune-finalized-states to all
* Merge branch 'master' into deprecate-ff-prune-finalized-states
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge refs/heads/master into deprecate-ff-prune-finalized-states
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-prune-finalized-states
2019-11-22 20:29:46 +00:00
Preston Van Loon
0cec0ee6c3 Release --optimize-process-epoch to all (#4080)
* Release optimize-proces-epoch to all
* Merge branch 'master' into deprecate-ff-optimize-process-epoch
* Merge branch 'master' of github.com:prysmaticlabs/prysm into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
* Merge refs/heads/master into deprecate-ff-optimize-process-epoch
2019-11-22 09:40:53 +00:00
Nishant Das
2f392544a6 Add Lock When Accessing Checkpoints (#4086)
* fix data races
* Merge branch 'master' into checkpointLock
* Merge branch 'master' into checkpointLock
* Merge refs/heads/master into checkpointLock
2019-11-22 06:34:42 +00:00
terence tsao
f5cb04012e Aggregator selection from RPC to validator client (#4071)
* Config
* Updated proto
* Updated pool
* Updated RPC
* Updated validator client
* run time works
* Clean ups
* Fix tests
* Visibility
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into aggregator
* Raul's feedback
* Tests for RPC server
* Tests for validator client
* Span
* More tests
* Use go routine for SubmitAggregateAndProof
* Go routines
* Updated comments
* Use array of roles
* Fixed tests
* Build
* Update validator/client/runner.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update validator/client/runner.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* If
* Merge branch 'refactor-validator-roles' of https://github.com/prysmaticlabs/prysm into refactor-validator-roles
* Empty
* Feedback
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into aggregator
* Removed proto/eth/v1alpha1/shard_chain.pb.go?
* Cleaned up
* Revert
* Comments
* Lint
* Comment
* Merge branch 'master' into aggregator
2019-11-22 05:11:38 +00:00
Raul Jordan
f461d1e024 Resolve Panic in ListValidatorBalances RPC (#4051)
* adding default response
* regression test for archival
* include full regression test
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* listbal
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* Merge branch 'master' into resolve-panic-rpc
* Merge refs/heads/master into resolve-panic-rpc
2019-11-22 04:39:28 +00:00
Preston Van Loon
715d06a215 gRPC Gateway: Emit JSON empty fields by default (#4085)
* Emit JSON empty fields by default
* Merge branch 'master' into emit-json-empty-fields
2019-11-22 03:36:47 +00:00
Raul Jordan
8f8d2d36c0
Filter ListValidators by Active in RPC (#4061)
* update workspace

* include active filter

* fix up latest changes to match naming

* better comments, fix evaluators

* latest master

* filter items

* filter only active validators
2019-11-21 14:29:24 -06:00
Nishant Das
a264a097cc
Update Pending Queue (#4066)
* update queue

* fix test

* put this all in validate method

* remove ancestors too

* not needed

* terence's review

* period

* preston's review
2019-11-21 21:24:50 +08:00
terence tsao
835418d1e3
Fix UpdateCommitteeCache slot (#4074) 2019-11-20 21:29:43 -08:00
Raul Jordan
ae07dc7962
Archive Data Even Through Skip Slots (#4054)
* red test first

* does not archive through skip slot

* test out at runtime

* underflow check

* fix tests

* rem info log
2019-11-19 23:53:28 -06:00
Ivan Martinez
2d7802c637 Rename featureconfig.Flag to Flags (#4063) 2019-11-19 21:03:00 -06:00
terence tsao
fcb663acde Implement aggregation helpers (#4062)
* Aggregation helpers

* Tests

* Config

* Faulty test cases

* Err
2019-11-19 20:24:39 -06:00
Raul Jordan
858dbbf038
Update Ethereum APIs and Match Schemas (#4059)
* update workspace

* include active filter

* fix up latest changes to match naming

* better comments, fix evaluators

* latest master

* Update proto/eth/v1alpha1/beacon_chain.proto
2019-11-19 18:36:45 -06:00
Jim McDonald
49c2dd2cfc Move the state notifier to a different module (#4058)
* Move state notifier to statefeed

* Updates to state notifier

* Create state feed in beacon node

* Formatting
2019-11-19 16:15:48 -06:00
terence tsao
7a22e98c0f Update ChainHead (#4053)
* Can build

* All tests pass

* Update beacon-chain/blockchain/chain_info.go

* Fix context

* Update chainhead

* Tests

* Tests

* e2e

* Update ordering

* Typo

* Use root to get slot

* Division
2019-11-19 13:33:13 -06:00
terence tsao
26da7c4114 Nil state fallback in Blockchain.HeadState() (#4042)
* Can build

* All tests pass

* Update beacon-chain/blockchain/chain_info.go

* Fix context
2019-11-19 10:12:50 -06:00
Nishant Das
7acb45d186 add one more return (#4050) 2019-11-19 09:40:15 -06:00
Preston Van Loon
24a5000e47 return to exit select loop rather than break (#4040) 2019-11-19 21:22:45 +08:00
Jim McDonald
65d920e13a Add generic state feed (#4004)
* Initial implementation of state feed

* Add instructions on adding new events

* Tidy up log messages

* Tidy up mock

* Update beacon-chain/core/statefeed/events.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/core/statefeed/events.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Remove unused BlockReceivedData

* Rename BlockHash to BlockRoot in BlockProcessedData

* Punctuation

* Use correct root for block processed event

* StateFeeder -> StateNotifier; fix up tests.

* Add Verified flag to BlockProcessed event

* Fix visibility in Bazel
2019-11-19 17:17:41 +08:00
Nishant Das
0e88085661
add step fix (#4047) 2019-11-19 12:51:40 +08:00
Preston Van Loon
3f6435ac80
Fix server side beacon blocks by range (#4046) 2019-11-18 19:56:37 -08:00