Commit Graph

868 Commits

Author SHA1 Message Date
Preston Van Loon
78a25f99c3
Update fastssz (#6760)
* Update fastssz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fmt
* gaz
* Merge refs/heads/master into update-fssz
* goimports
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Fix
* fix ethereumapis
* fix again
* kafka
* fix gen file
* fix compute signing root
* gofmt
* checkpoint progress
* progress
* checkpoint
* progress
* Fix build
* checkpoint
* helpers
* Another test fixed
* gaz
* another test fix
* gofmt
* some fixes
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fix one test
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fill empty checkpoint roots
* more padding
* more padding
* Fix //beacon-chain/rpc/debug:go_default_test
* fix //beacon-chain/core/state:go_default_test
* fix //beacon-chain/core/state:go_default_test
* fix some htr errors
* fix //slasher/rpc:go_default_test
* Progress on //beacon-chain/core/blocks:go_default_test
* Progress on //beacon-chain/core/blocks:go_default_test
* Progress on //beacon-chain/core/blocks:go_default_test
* fix //slasher/db/kv:go_default_test
* progress
* fix //beacon-chain/sync/initial-sync:go_raceon_test
* gofmt and gaz
* fix one more test, taking a break
* Fix //beacon-chain/core/blocks:go_default_test
* Complete beacon-chain/powchain
* Do most of beacon-chain/rpc/beacon/
* Do most of beacon-chain/blockchain
* fix //beacon-chain/operations/attestations/kv:go_default_test
* Fix //beacon-chain/cache/depositcache:go_default_test
* Fix //slasher/detection:go_default_test
* Progress
* fix //beacon-chain/rpc/validator:go_default_test
* gofmt
* fix //validator/client:go_default_test
* fix
* fix //beacon-chain/blockchain:go_raceoff_test
* fix //beacon-chain/rpc/beacon:go_default_test
* fix 1 of 4 shards in //beacon-chain/sync:go_default_test
* Fix //beacon-chain/sync:go_default_test and gofmt
* prevent panic
* fix //beacon-chain/state/stategen:go_default_test
* fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fix most tests
* Self review, go mod tidy, run regen scripts
* fix slasher
* Update ethereumapis
* disable spawn strategy override
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Remove extra line in imports
* Remove extra line in imports
* Gofmt
* PR feedback from @nisdas
2020-08-24 01:46:17 +00:00
terence tsao
f2afeed9da
Pool: Use a TTL cache for seen bits (#7015) 2020-08-21 16:27:51 -07:00
Raul Jordan
c9caf5dfc5
Set Healthz to Return Error if Chain Has Not Yet Started (#7077)
* healthz in sync when chain not started
* Merge refs/heads/master into fix-healthz-when-process-deposits
* Merge refs/heads/master into fix-healthz-when-process-deposits
* Merge refs/heads/master into fix-healthz-when-process-deposits
2020-08-21 19:09:52 +00:00
Raul Jordan
b4c7a14759
Do Not Unset Rate Limiter Before Node Shutdown (#7078)
* reset rate limiter before shutdown
* Merge refs/heads/master into empty-rate-limit
* Merge refs/heads/master into empty-rate-limit
* Update beacon-chain/sync/service.go
2020-08-21 18:48:43 +00:00
Victor Farazdagi
05d6dc7a10
Improves block deduplication on batch processing (#7075)
* fixes block deduplication on batch processing
* Nishant's suggestions
2020-08-21 10:51:27 +00:00
terence tsao
55074bcc6c
Tidying up a few core functions (#6995)
* DB: add block roots test
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Pool: add seen atts map
* Pool: use seen atts map
* Pool: clear seen map
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Core: clean up unused noverify and tidy up namings
* Gaz
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Core: Add no verify back. Better namings
* Merge branch 'rm-no-verify' of github.com:prysmaticlabs/prysm into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Core: fixed a test
* Merge branch 'rm-no-verify' of github.com:prysmaticlabs/prysm into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
* Merge refs/heads/master into rm-no-verify
2020-08-20 19:53:22 +00:00
Nishant Das
7744c3ae47
Batch Block Roots Requesting (#7027)
* checkpoint
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into batchRootReq
* test
* gaz
* fix test
* comment
* Merge refs/heads/master into multipleBranchProcessing
* Merge refs/heads/master into multipleBranchProcessing
* Merge refs/heads/master into multipleBranchProcessing
* Merge refs/heads/master into multipleBranchProcessing
* Merge refs/heads/master into multipleBranchProcessing
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into batchRootReq
* terence's review
* Merge branch 'multipleBranchProcessing' of https://github.com/prysmaticlabs/geth-sharding into batchRootReq
2020-08-20 04:50:14 +00:00
Nishant Das
ba5da21026
Shift Sync Warn and Error Messages To Debug (#7059)
* change

* change
2020-08-20 11:30:41 +08:00
Preston Van Loon
3aa95b9c16
Use correct context for IsFinalizedBlock call (#7053)
* Use correct context for IsFinalizedBlock call
* Merge branch 'master' into use-correct-ctx
2020-08-19 19:55:39 +00:00
terence tsao
399dc85737
Use map to reject seen pending block (#7045) 2020-08-18 09:21:25 -07:00
Nishant Das
d81c9ffcfc
Reject Duplicate Blocks From Being Saved (#7044)
* check for duplicates

* add test
2020-08-18 18:47:44 +08:00
Victor Farazdagi
0a5ec502b0
Updates resyncIfBehind() functionality (#7039)
* defines BestNonFinalized
* updates blocks_fetcher
* updates blocks_queue
* Merge branch 'master' into fix-resync
* Nishant's suggestion
* Merge branch 'fix-resync' of github.com:prysmaticlabs/prysm into fix-resync
* unit test
* Merge branch 'master' into fix-resync
* fixes test
* Merge refs/heads/master into fix-resync
2020-08-18 02:10:45 +00:00
Preston Van Loon
4d463c4a85
Don't ban blocks for context deadlines (#7040)
* Don't ban blocks for context deadlines
* Don't ban blocks for context deadlines
* Add test
* Merge branch 'master' into dont-ban-ctx-deadline2
* fmt
2020-08-18 01:21:10 +00:00
terence tsao
9caa92cae4
Pending queue to handle same slot blocks (#7036)
* Pending queue: rough draft
* Removed extra RUnlock
* apply diff
* Update pending queue tests
* Merge branch 'fix-pending-blocks' of github.com:prysmaticlabs/prysm into fix-pending-blocks
* Merge branch 'master' into fix-pending-blocks
* Comments
* Merge branch 'fix-pending-blocks' of github.com:prysmaticlabs/prysm into fix-pending-blocks
2020-08-18 00:01:32 +00:00
Victor Farazdagi
bde3073bd6
Wait interval, before finalized peers re-checked (#7032)
* wait on no finalized peers
* Merge branch 'master' into better-no-finalized-peers-handling
2020-08-17 16:01:16 +00:00
Nishant Das
f193dc2f54
Do Not Retrieve From DB For Block Roots (#7028)
* don't use db
2020-08-17 08:41:45 +00:00
Preston Van Loon
d66f72939e
Ignore subscription cancelled warning (#7020)
* Resolve TODO #6449
* Merge refs/heads/master into fix-6449
2020-08-17 01:58:30 +00:00
Preston Van Loon
c3725d11f2
Revert "Sync: Dont ban block roots for context.DeadlineExceeded" (#7017)
* Revert "Sync: Dont ban block roots for context.DeadlineExceeded (#7016)"

This reverts commit a4bbd82a45.
* Merge refs/heads/master into revert-7016-dont-ban-ctx-deadline
2020-08-17 01:18:38 +00:00
terence tsao
715cd40f56
Sync: check blocks and roots lengths (#7019) 2020-08-16 16:48:39 -07:00
Preston Van Loon
a4bbd82a45
Sync: Dont ban block roots for context.DeadlineExceeded (#7016)
* Dont set bad block root if it was context deadline with processing
2020-08-16 19:24:34 +00:00
Nishant Das
0be1957c28
Use Stage 1 For Non Finalized Sync (#7012)
* checkpoint
* dont cancel
* remove
* sync mode
* fixes build
* cap max retries when no finalized peers are found
* use max peers
* change it
* fixes TestService_roundRobinSync/Multiple_peers_with_different_finalized_epoch
* fixes blocks fetcher tests
* fixes blocks queue tests
* fixes TestService_blockProviderScoring test
* gofmt
* Merge branch 'master' into useStage1
* Update round_robin.go

preston's review
* Preston's suggestions
* Merge branch 'useStage1' of github.com:prysmaticlabs/prysm into useStage1
* fixes test
2020-08-16 17:51:14 +00:00
Nishant Das
17b810f24e
Fix 2nd Stage of Sync (#7006)
* handle 2nd stage better
* 2 stage sort
* add warning
* better processing
* victor's review
* fix one test
* Update beacon-chain/blockchain/receive_block.go
* fixes TestService_roundRobinSync/Multiple_peers_with_many_skipped_slots
* Merge branch 'fixSync' of github.com:prysmaticlabs/prysm into fixSync
* fixes TestService_process
* fixes TestService_blockProviderScoring
* Preston's suggestion
* increase number of available slots, allowing 2nd phase to wrap up
* more concise limits
2020-08-15 15:14:59 +00:00
Victor Farazdagi
4de0b9fe69
Peer scoring: init sync (#6709)
* refactors redundant part of varname
* introduces score_block_providers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* gazelle
* adds comment
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* removes redundant checks
* add block provider decay test
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* adds case
* penalize inactive peers
* adds scorebadresponses test
* inroduces no-activity penalty
* gazelle
* gofmt
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* expanded tests
* implement SortBlockProviders
* change -> update
* updates block fetcher peer filter
* fixes test
* allows to keep track of peer id
* updates scoring coefficients
* fixes test
* block fetcher update
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* disables empty batch penalty
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* removes outdated code
* update filterPeers
* gazelle
* updates var
* revert changes to var name
* updates blocks_fetcher
* minor fix to import name
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* add max processed blocks cap
* impoves scoring of stale peers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* fixes test
* adds weight sorting to scored peers
* return pid when fetching batches
* updates round robin
* gazelle
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* updates block provider decay count
* go tidy
* cherry pick
* fixes test
* go tidy
* Merge branch 'peer-scorer-weighted-sorter' into init-sync-peer-scoring-slow-peers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* refactors blocks fetcher: repackage methods
* Merge branch 'refactor-blocks-fetcher' into init-sync-peer-scoring-slow-peers
* minor fixes
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* allow scores in range (0;1) in weighted filter
* filterScoredPeers improve test suite
* puts feature behind the flag
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* fixes tests
* Merge branch 'init-sync-peer-scoring-slow-peers' of github.com:prysmaticlabs/prysm into init-sync-peer-scoring-slow-peers
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* Merge branch 'master' into init-sync-peer-scoring-slow-peers
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* Update beacon-chain/sync/initial-sync/blocks_fetcher_test.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>
* Nishant's suggestion on peer limit variable
* better explanation of non-blocking peer scoring
* Shay's sugession on peer naming
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* Update beacon-chain/sync/initial-sync/blocks_fetcher.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* gofmt
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
* Merge refs/heads/master into init-sync-peer-scoring-slow-peers
2020-08-13 17:33:57 +00:00
Victor Farazdagi
74101612ce
Moves AssertLogsContains to assertions package (#6985)
* first working implementation
* assertions tests
* adds to requires
* merges assert and require tests into a single suite
* gazelle
* Merge branch 'merge-assert-require-tests' into assert-logs-contains-move-to-assertions
* gazelle
* updates references
* fixes build issue
* Merge branch 'master' into assert-logs-contains-move-to-assertions
* Merge refs/heads/master into assert-logs-contains-move-to-assertions
* Merge branch 'master' into assert-logs-contains-move-to-assertions
* fixes build issue
* Merge branch 'assert-logs-contains-move-to-assertions' of github.com:prysmaticlabs/prysm into assert-logs-contains-move-to-assertions
* Merge refs/heads/master into assert-logs-contains-move-to-assertions
* Merge refs/heads/master into assert-logs-contains-move-to-assertions
2020-08-13 16:22:25 +00:00
Victor Farazdagi
7611f6d84a
injects peer.ID into requested/queued/fetched batches (#6979)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-13 16:42:21 +08:00
terence tsao
46cb3abe2e
Add and use genesis time in config (#6966)
* Config: add genesis time and update medalla

* Helper: add GenesisTime getter

* Rest: use the helper
2020-08-11 14:36:32 -05:00
Victor Farazdagi
c24666e02e
refactors blocks fetcher: repackage methods (#6962) 2020-08-11 10:15:18 -05:00
rkapka
a5f0fd29fc
Fix antipatterns (#6961)
* Merge variable declaration and assignment
* Use result of type assertion to simplify cases
* Replace call to bytes.Compare with bytes.Equal
* Drop unnecessary use of the blank identifier
* Replace x.Sub(time.Now()) with time.Until(x)
* Function literal can be simplified
* Use a single append to concatenate two slices
* Replace time.Now().Sub(x) with time.Since(x)
* Omit comparison with boolean constant
* Omit redundant nil check on slices
* Nested if can be replaced with else-if
* Function call can be replaced with helper function
* Omit redundant control flow
* Use plain channel send or receive
* Simplify returning boolean expression
* Merge branch 'origin-master' into fix-antipatterns
* Merge branch 'master' into fix-antipatterns
* Merge branch 'origin-master' into revert-single-channel-antipattern
* Revert "Use plain channel send or receive"

This reverts commit b3108fdc3a656a9c9b1823409dc6a263e69f9e92.
2020-08-11 12:14:28 +00:00
rkapka
1f21e196b6
Fix multiple anti-patterns from DeepSource analysis (#6951)
* Merge variable declaration and assignment
* Use result of type assertion to simplify cases
* Replace call to bytes.Compare with bytes.Equal
* Drop unnecessary use of the blank identifier
* Replace x.Sub(time.Now()) with time.Until(x)
* Function literal can be simplified
* Use a single append to concatenate two slices
* Replace time.Now().Sub(x) with time.Since(x)
* Omit comparison with boolean constant
* Omit redundant nil check on slices
* Nested if can be replaced with else-if
* Function call can be replaced with helper function
* Omit redundant control flow
* Use plain channel send or receive
* Simplify returning boolean expression
* Merge branch 'origin-master' into fix-antipatterns
* Merge branch 'master' into fix-antipatterns
2020-08-10 16:16:45 +00:00
Preston Van Loon
18b3203f31
Wait for subnet peers before broadcasting onto attestation subnet topic (#6893)
* Initial pass

* Add metric to measure success

* Use a subnet RWLock to prevent duplicate requests, give up after 3 attempts

* push latest commented code

* try with non-blocking broadcast

* Add feature flag, ignore parent deadline if any

* Add slot as metadata

* add tests

* gaz

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-10 10:27:50 -05:00
Nishant Das
b1946ead23
Make Initial Sync Failures Debug (#6930)
* shift to debug logs
* Merge branch 'master' into makeFailuresDebug
* go sum
* Merge branch 'master' into makeFailuresDebug
* Merge branch 'master' into makeFailuresDebug
* Merge refs/heads/master into makeFailuresDebug
2020-08-09 19:47:52 +00:00
terence tsao
7591240366
Revert "Subscriber: use target epoch for hasSeen cache" (#6944)
* Revert "Subscriber: use target epoch for hasSeen cache (#6908)"

This reverts commit 3275a86ece.
2020-08-09 18:52:37 +00:00
terence tsao
3275a86ece
Subscriber: use target epoch for hasSeen cache (#6908)
* Subscriber: use target epoch for hasSeen cache
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
* Merge refs/heads/master into use-target
2020-08-07 20:55:42 +00:00
rkapka
e492343ffd
Code quality issues - Remove unused code (#6864)
* remove unused code

* gofmt

* remove junk file

* bring back metrics-related functions

* bring back keymanager's code
2020-08-05 13:59:46 +02:00
Nishant Das
859e59b6f7
Change Sync Error Logs To Debug Logs (#6865)
* switch over to debug logs
* change back
* Change e2e to debug logging
* Merge refs/heads/master into changeToDebug-2
* Merge refs/heads/master into changeToDebug-2
2020-08-04 20:30:40 +00:00
terence tsao
ec4de10001
Add helper ComputeDomainAndSign (#6773)
* Starting
* Merge branch 'master' of github.com:prysmaticlabs/prysm into domain-and-sign
* core blocks tests: updated
* core state tests: updated
* rpc validator tests updated
* sync tests updated
* testutil updated
* Beacon-chain: update tests
* Shared: update tests
* Rest of the tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into domain-and-sign
* Merge refs/heads/master into domain-and-sign
* Merge refs/heads/master into domain-and-sign
* Merge refs/heads/master into domain-and-sign
* Merge refs/heads/master into domain-and-sign
* Sync with master
* Merge branch 'domain-and-sign' of github.com:prysmaticlabs/prysm into domain-and-sign
* Merge refs/heads/master into domain-and-sign
2020-07-30 19:11:22 +00:00
terence tsao
bcea2c1b22
Config/flag: medalla as default (#6770)
* config: use medalla as default

* Tests: align with medalla spec

* Config/flag: deprecate medalla feature flag

* Update shared/featureconfig/flags.go

* Spectest: set genesis forkversion to 0,0,0,0

* Config: set medalla as defat in Configure

* Extra space

* Shard: use copy

* Revert a p2p fork test

* Update default medalla for tools

* Use medalla config for e2e

* Fix E2E

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-07-30 08:45:00 -05:00
Raul Jordan
62c92af247
Remove Deprecated SSZ Decode Edge Case for BeaconBlocksByRoots (#6771)
* remove temp unmarshal edge case
* removed deprecated func
* final err fix
* Merge refs/heads/master into fix-exploit
2020-07-29 23:15:14 +00:00
Victor Farazdagi
84697a0a59
Adds peer scorer manager (#6752)
* prepares peer scorer collection
* decouples scoring service into separate object
* updates references
* renames package
* removes redundant test init
* gazelle
* gofmt
* Merge branch 'master' into peer-scorer-manager
* updates comment
* fix build
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
* Merge refs/heads/master into peer-scorer-manager
2020-07-29 09:26:46 +00:00
terence tsao
d6eb6d3f41
Move ValidateAttestationTime earlier for sync (#6755)
* Move time checks to the start
* Merge branch 'master' of github.com:prysmaticlabs/prysm into check-time-first
* Merge refs/heads/master into check-time-first
* Merge refs/heads/master into check-time-first
* Merge refs/heads/master into check-time-first
* Merge refs/heads/master into check-time-first
* Merge refs/heads/master into check-time-first
2020-07-29 05:31:50 +00:00
terence tsao
027ee4c18c
Add a ComputeDomainVerifySigningRoot helper (#6743)
* Add ComputeDomainVerifySigningRoot
* Update validate aggregate proof signatures to use it
* Update verify proposer slashing to use it
* Update test to use it
* Merge branch 'master' into domain-and-signingroot
* Quick fix on test
* Merge branch 'domain-and-signingroot' of github.com:prysmaticlabs/prysm into domain-and-signingroot
* Merge branch 'master' of github.com:prysmaticlabs/prysm into domain-and-signingroot
* Merge refs/heads/master into domain-and-signingroot
* Merge refs/heads/master into domain-and-signingroot
2020-07-28 16:49:47 +00:00
Nishant Das
664349e553
Account for Bad Blocks in Gossip (#6726)
* add bad block cache

* terence's review

* terence's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-27 19:57:19 +08:00
Nishant Das
7a12fc6d75
Fix Metadata Sequence Checks (#6711)
* checkpoint

* Update beacon-chain/sync/rpc_test.go

* change conditional

* fix test

* fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-25 11:53:41 -05:00
Victor Farazdagi
4400321081
Handles case when no peers with finalized blocks are found (#6679)
* Handles case when no peers with finalized blocks are found
* more tests
* Merge branch 'master' into handle-zero-block-init-sync
* Merge branch 'master' into handle-zero-block-init-sync
2020-07-22 18:14:24 +00:00
Victor Farazdagi
f0ffd5af03
Introduces peer scorer service (#6579)
* introduces peer scorer service
* gazelle
* Merge branch 'master' into peer-scoring-service
* fixes comment
* fix build error
* linter suggestions
* Merge branch 'master' into peer-scoring-service
* updates tests
* Merge branch 'master' into peer-scoring-service
* badresponses scorer tests
* gazelle
* Merge branch 'master' into peer-scoring-service
* Merge branch 'master' into peer-scoring-service
* adds scorer_test
* gazelle
* updates bad response default penalty
* more comments
* Merge branch 'master' into peer-scoring-service
* Merge branch 'master' into peer-scoring-service
* Merge branch 'master' into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* introduces peerdatastore into status
* makes sure that commong peer data store is used
* Merge branch 'peer-scoring-service' of github.com:prysmaticlabs/prysm into peer-scoring-service
* linter
* gazelle
* updates tests
* Merge branch 'master' into peer-scoring-service
* fixes tests
* Nishant's suggestions
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge refs/heads/master into peer-scoring-service
* Merge branch 'master' into peer-scoring-service
* gofmt
* Nishant's suggestion to use isBadPeer
* Merge refs/heads/master into peer-scoring-service
2020-07-20 15:12:59 +00:00
Nishant Das
cc3c3a0c54
QSP 29: Add Rate Limiter For All Topics (#6606)
* initial commit
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into rateLimiter
* finally
* add mutex
* rate limiter for all rpc reqs
* remove recursive readlocks
* spelling
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* fix all tests
* fix all tests
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
* Merge refs/heads/master into rateLimiter
2020-07-17 08:58:51 +00:00
Victor Farazdagi
c2615168d9
Applies assertion funcs to sync tests (#6603)
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* gazelle
* Merge branch 'master' into sync-apply-testutils-assertions
* gazelle
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* Merge branch 'master' into sync-apply-testutils-assertions
2020-07-15 04:41:11 +00:00
terence tsao
8fda48409c
Remove batch blocks save at getAttPreState (#6584)
* Check if block exists in cache
* Consider init sync blocks
* Fixed test
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
* Merge refs/heads/master into get-block
2020-07-14 15:46:00 +00:00
Nishant Das
27577bc324
QSP 32: Add Appropriate Stream Deadlines for RPC Requests (#6583)
* add no deadlines
* Merge branch 'master' into fixDeadlines
* nogo
* Merge refs/heads/master into fixDeadlines
* Merge refs/heads/master into fixDeadlines
2020-07-13 19:40:12 +00:00
Victor Farazdagi
c35bdf2649
Refactors calls to deprecated libp2p methods (#6554)
* refactors calls to deprecated IDB58Decode()
* Merge branch 'master' into p2p-refactor-deprecated
* updated packages
* gazelle
* mod tidy
* refactors publish()/subscribe() deprecated methods
* gofmt
* test update join/leave topic methods
* re-arrange imports
* Merge branch 'master' into p2p-refactor-deprecated
* Merge refs/heads/master into p2p-refactor-deprecated
2020-07-13 02:28:40 +00:00
Nishant Das
d4c3546434
Validate RPC Topics Before Sending Requests (#6558)
* clean up

* fix panic

* add test mapping

* add schema version change

* fix test

* fix another test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-13 09:20:53 +08:00
Preston Van Loon
1f35384578
Reject a block which its parent is not part of the finalized chain (#6549)
* Reject a block which its parent is not part of the finalized chain
* Refactor verifyBlkDescendant to be public and omit the slot argument, which was only used for error messages
* Merge branch 'refactor-verifyBlkDescendant-signature' into reject-bad-block
* fix
* impl in mock
* impl in mock
* gofmt
* Merge refs/heads/master into refactor-verifyBlkDescendant-signature
* Merge branch 'refactor-verifyBlkDescendant-signature' into reject-bad-block
* fix test
* add test
* Merge branch 'refactor-verifyBlkDescendant-signature' into reject-bad-block
* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into reject-bad-block
* move up in the validation pipeline, @terencechain offline feedback
* Merge refs/heads/master into reject-bad-block
2020-07-11 03:57:42 +00:00
terence tsao
d54cefbe42
Part 1 of blockchain pkg clean up (#6537)
* Fix span names

* Clean up on block func

* Add updateFinalized helper

* Deprecate ReceiveBlockNoPubsub for ReceiveBlock

* Proposer to broadcast block

* Update migrate interface

* Replace ReceiveBlock for all

* Go fmt
2020-07-09 18:50:48 -05:00
Ivan Martinez
322998f7f1
Move subnet topics to global vars (#6525)
* Deduplicate subnet topic definitions
* Move topics to global file
* Gaz
* Merge branch 'master' into move-topics
* Fix
* Merge refs/heads/master into move-topics
* Bazel
* Merge branch 'move-topics' of github.com:prysmaticlabs/prysm into move-topics
* Fix tests
* Fix
* Undo e2e changes
* Revert "Undo e2e changes"

This reverts commit 3037bb35904ee3a2f54050e920ee8c75a4cb8036.
* Fix
* Fix
* Merge refs/heads/master into move-topics
* Merge refs/heads/master into move-topics
* Merge refs/heads/master into move-topics
* Comments
* Merge refs/heads/master into move-topics
* Merge refs/heads/master into move-topics
2020-07-09 17:38:15 +00:00
Preston Van Loon
fd80f73286
Improve make() capacity allocations (#6517)
* attestationutil.AttestingIndices: Minor improvement on indices array allocation
* progress
* more progress on makes
* progress
* more progress
* Merge branch 'master' of github.com:prysmaticlabs/prysm into memory1
* gaz
* fmt and imports
* Merge branch 'master' into memory2
* Min()
* remove spaces
* Merge branch 'master' of github.com:prysmaticlabs/prysm into memory2
* revert beacon-chain/operations/attestations/kv/block.go
* partially revert beacon-chain/operations/attestations/kv/aggregated.go
* Merge branch 'master' into memory2
* Merge branch 'master' into memory2
2020-07-09 15:50:58 +00:00
Nishant Das
c804347fc4
add fix and test (#6533)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-09 09:46:35 -05:00
terence tsao
fe13f1f856
Add init-sync-verbose feature flag (#6515)
* Add flag
* Use flag
* Merge refs/heads/master into init-sync-verbose
* Merge refs/heads/master into init-sync-verbose
* Merge refs/heads/master into init-sync-verbose
* Merge refs/heads/master into init-sync-verbose
* Merge refs/heads/master into init-sync-verbose
2020-07-08 04:19:58 +00:00
Preston Van Loon
1067800430
p2p.AddConnectionHandler: Remove unused arg for goodbyeFunc (#6511)
* p2p.AddConnectionHandler: Remove unused arg for goodbyeFunc
* accidental rename reqFunc
* update test
* Merge branch 'master' into rm-unused-param
* Merge branch 'master' into rm-unused-param
2020-07-08 02:04:15 +00:00
Nishant Das
64fa474434
Batch Verify Blocks (#6469)
* add everything so far
* checkpoint progress
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* fix
* checkpoint
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* checkpoint again
* checkpoint again
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* commenting
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* bls cleanup
* revert this back
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* revert core changes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* add flag
* add test
* add one more test
* clean up
* comment
* lint
* terence's review
* Merge refs/heads/master into fastBLS
* Merge refs/heads/master into fastBLS
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* Merge refs/heads/master into fastBLS
* remove additional method
* Merge branch 'fastBLS' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* fix
* Merge refs/heads/master into fastBLS
* copy
* Merge branch 'fastBLS' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
2020-07-07 04:16:12 +00:00
Preston Van Loon
e96e1f0569
Add a placeholder constant for ignoring cancelled subscription errors from libp2p pubsub. (#6496)
* Add a placeholder constant for ignoring cancelled subscription errors
from libp2p pubsub.

This change temporarily resolves
https://github.com/prysmaticlabs/prysm/issues/6449.

This change should be revisited after
https://github.com/libp2p/go-libp2p-pubsub/pull/356 merges and the prysm
libp2p pubsub dependency is updated.
* Merge branch 'master' into issue-6449
* Merge branch 'master' into issue-6449
* Merge branch 'master' into issue-6449
2020-07-06 23:55:55 +00:00
terence tsao
def7b602e3
Hot states use no DB (#6488)
* Add cache to service struct
* Update hot getters/setters to use cache
* Update migration
* Update other services to adapt
* Fix initial sync get state
* Update getter related tests
* Update hot related tests
* Update migrate related tests
* New awesome tests for migration
* Clean up rest of the tests
* Merge refs/heads/master into hot-state-no-db
* Fix block chain head tests
* Fix block chain processor tests
* Fixed RPC tests
* Update cold getter and test
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
* Fix sync tests
* Short cut if state is already in DB
* Remove uneeded saves
* Update beacon-chain/state/stategen/hot_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/getter_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/getter_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/setter_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Preston's feedback
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
* Return a copy of cache states
* Remove hot state caches check in StateByRoot
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
* Merge refs/heads/master into hot-state-no-db
* Raul's feedback
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
2020-07-06 17:22:12 +00:00
Ivan Martinez
c69b3f568e
Move hard-coded constants to global consts (#6467)
* Move hard-coded constants to global consts
* Fixes
* Merge branch 'master' into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Comments
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge refs/heads/master into remove-hard-constants
* Merge branch 'master' of github.com:prysmaticlabs/prysm into remove-hard-constants
2020-07-03 15:25:32 +00:00
Nishant Das
7896e087e5
Add in Better Logging for Requests (#6483)
* add in better logging
* gaz
* last one
* mess about
* check strings
* fix
2020-07-03 10:08:45 +00:00
Nishant Das
ac9bc5d98b
Reset Streams at the End of the Request (#6481)
* close all incoming streams

* use proper helper

* shift to debug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-03 16:21:18 +08:00
Raul Jordan
5b708b54d9
QSP-13 Only Allow Snappy P2P Encoding (#6415)
* enforce only snappy

* fix up tests

* replace with ssz snappy in tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-03 11:24:30 +08:00
Preston Van Loon
aa52e693ce
Sync: do not remove attestations from pool on failed block processing (#6474)
* Add test and return error on failed block processing
* use atts in body
* Merge branch 'master' into p2p-potential-error-block
2020-07-02 20:24:30 +00:00
Victor Farazdagi
f279cf09b8
Removes excessive concurrency from init-sync queue (#6471)
* Removes excessive concurrency from init-sync queue
* update buffer size
2020-07-02 13:06:12 +00:00
Victor Farazdagi
92c5c651f7
Refactors init-sync: queue streams blocks in batches (#6470)
* init-sync: queue streams blocks in batches
2020-07-02 11:19:07 +00:00
Victor Farazdagi
1b0cfb408e
QSP-BestPractice: Refactors inline ttfb/resp time constants (#6463)
* refactors inline ttfb/resp time constants
2020-07-01 09:47:59 +00:00
Raul Jordan
a0c38c8d0a
QSP-33 Check Max Response For All Topics (#6424)
* remove max len funcs

* fix up tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-30 09:50:28 -05:00
Nishant Das
21ead0a025
Record Peer Latency (#6456)
* add latency
* Merge branch 'master' into addLatency
* Merge refs/heads/master into addLatency
2020-06-30 03:53:14 +00:00
Victor Farazdagi
196f4c6222
QSP-BestPractice Refactors failover peer selection in init-sync (#6442)
* refactors failover peer selection in init-sync
* Merge refs/heads/master into init-sync-more-effective-failover
2020-06-29 15:23:04 +00:00
Preston Van Loon
bfb6e5e4a0
Use spec attestation time verification in gRPC requests (#6429)
* Re-use attestation time verification

* lint

* fix imports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-28 14:37:17 -07:00
Preston Van Loon
3ba3133657
Cleanup validateAggregateAttTime with a better error message, more tests, and more clear inline comments (#6423)
* Add span for updateRecentCanonicalBlocks
* Cleanup validateAggregateAttTime with a better error message, more tests, and more clear inline comments
* Merge branch 'master' into cleanup-validateAggregateAttTime
* revert beacon-chain/blockchain/head.go
* Merge branch 'cleanup-validateAggregateAttTime' of github.com:prysmaticlabs/prysm into cleanup-validateAggregateAttTime
* Merge refs/heads/master into cleanup-validateAggregateAttTime
* Merge refs/heads/master into cleanup-validateAggregateAttTime
2020-06-26 19:36:44 +00:00
Nishant Das
f1c0b2e0da
Fix Invalid SSZ in Block Roots RPC Request (#6405)
* fix block roots error
* remove comment
* fix
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fixBlockRootsError
* change back
* add stream deadlines
* Merge branch 'master' into fixBlockRootsError
* add todos
* Merge branch 'fixBlockRootsError' of https://github.com/prysmaticlabs/geth-sharding into fixBlockRootsError
* Update beacon-chain/sync/rpc_beacon_blocks_by_root.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* gofmt
* fix test
* fix failing test
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fixBlockRootsError
* fix all
* Merge branch 'master' into fixBlockRootsError
* Merge branch 'master' into fixBlockRootsError
* Merge refs/heads/master into fixBlockRootsError
2020-06-26 17:05:04 +00:00
Raul Jordan
ab89053597
QSP-45 Add Check for No Connected Peers BestFinalized (#6402)
* no error returned
* no target root
* fix build
* Merge refs/heads/master into best-finalized
2020-06-26 16:35:12 +00:00
Raul Jordan
252f758baa
QSP-9 Prevent Casting to Int if Possible (#6349)
* no cast to int

* fix up significant casting issues

* more casting

* even more casting fixes

* more casts

* fix subnets

* back to ints

* final touches

* broken test fix

* add in blocks test fix

* unskip

* revert bytes fixes

* casting fixes

* Update beacon-chain/db/kv/state.go

* Update beacon-chain/db/kv/blocks.go

* fmt

* slash:

* fix val tests

* fix up conf

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-06-26 11:07:00 -05:00
Victor Farazdagi
78465e2549
QSP-6: Enforces crypto-secure PRNGs (#6401)
* adds cryptorand analyzer

* better naming

* rely on suffix

* sync/pending_* use crypto/rand

* define shared/rand

* updates fetcher

* fixes rand issue in sync package

* gofmt

* shared/rand: more docs + add exclusion nogo_config.json

* updates validator/assignments

* updates comment

* fixes remaning cases

* re-arranges comments

* fixes tests

* renames in shared/rand API

* adds simple no-panic test

* gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-26 09:58:47 -05:00
Nishant Das
73f3a653bf
Clean Up Dead Peers (#6413)
* clean up dead peers
* Merge refs/heads/master into clearDeadPeers
* comment
* Merge branch 'clearDeadPeers' of https://github.com/prysmaticlabs/geth-sharding into clearDeadPeers
2020-06-26 12:24:55 +00:00
Raul Jordan
1b430e0c17
QSP-62 Corrupted Clock In Stream Deadlines (#6404)
* deadline warn
* add proper godoc information
* clarify
* Merge refs/heads/master into corrupted-clock
* fix broken build
* Merge refs/heads/master into corrupted-clock
* Merge refs/heads/master into corrupted-clock
2020-06-25 22:36:18 +00:00
Victor Farazdagi
00f24f5729
Deprecates --disable-init-sync-wrr flag (#6400)
* deprecates wrr-related flag

* gofmt + gazelle

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-25 10:26:30 -05:00
Nishant Das
5fdf6310f7
add back handler (#6398)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-25 13:07:31 +03:00
terence tsao
9a156bd73a
Best practice follow up - remove outdated todos (#6392)
* Addressed 30.

* Remove outdated todos

* Go fmt

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-24 23:14:11 -05:00
Preston Van Loon
fb8be4d555
Refactor BLS (#6395)
* refactor BLS to expose an interface rather than a single implementation
* Split up tests, gofmt, goimports, docs
* godoc for signature.Copy()
* more godocs
* revert gomod / deps changes
* rm newline
* revert proto changes
* rename bls12 to herumi for clarity
2020-06-25 00:47:51 +00:00
terence tsao
a9c1d25a35
Best practice feedback - part 2 (#6389)
* Feedback 10. Rename to highest epoch
* Feedback 11. Rename to pidepoch
* Feedback 39. Swap length
* Typo
* Merge refs/heads/master into best-practice-pt2
2020-06-24 18:06:19 +00:00
Shay Zluf
b8e1bf0445
Fix proposer slashing context timeout test (#6379)
* Fix proposer slashing context timeout test
* goimports
* revert slashing change
* Merge branch 'master' into fix_proposer_slashing_test
2020-06-24 12:10:40 +00:00
Shay Zluf
edb08bd209
QPS-20 implement removeDisconnectedPeerStatus (#6370)
* QPS-20 implement removeDisconnectedPeerStatus
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* remove function
* Merge branch 'remove_disconnected_peer_status' of github.com:prysmaticlabs/prysm into remove_disconnected_peer_status
* Merge refs/heads/master into remove_disconnected_peer_status
* Merge branch 'remove_disconnected_peer_status' of github.com:prysmaticlabs/prysm into remove_disconnected_peer_status
2020-06-24 10:40:33 +00:00
Nishant Das
e45a06cbf1
Revert "QSP-42 Remove Double Unsubscribe in Initial Sync" (#6376)
* Revert "QSP-42 Remove Double Unsubscribe in Initial Sync (#6368)"

This reverts commit 637354f037.

* comment

* add go fmt
2020-06-24 17:01:48 +08:00
Ivan Martinez
e9a597b43c
[QSP-6] Change usages of math/rand to crypto/rand (#6362)
* Change math/rand to crypto/rand
* Change length
* Merge branch 'master' of github.com:prysmaticlabs/prysm into change-to-crypto-rand
* Change assignments to true random
* Merge branch 'master' into change-to-crypto-rand
* Change randomness
* Merge branch 'change-to-crypto-rand' of github.com:prysmaticlabs/prysm into change-to-crypto-rand
* Remove comments
* Test
* Fix test
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
* Merge refs/heads/master into change-to-crypto-rand
2020-06-24 05:47:51 +00:00
terence tsao
a4cfd147ff
Address spec adherence feedbacks (#6365)
* Addressed 1
* Address 2.
* Addressed 3. and 4.
* Addressed 6.
* Addressed 7
* Addressed 8
* Addressed 9.
* Addressed 10.
* Addressed 11.
* Addressed 12.
* Addressed 13.
* Delete old test
* Merge refs/heads/master into spec-feedbacks
* Change comment "pure" to "stateless"
* Fmt
* Typo
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
2020-06-24 03:14:44 +00:00
Raul Jordan
637354f037
QSP-42 Remove Double Unsubscribe in Initial Sync (#6368)
* rem double unsub
* Update beacon-chain/sync/initial-sync/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/master into qsp-unsub
* Merge refs/heads/master into qsp-unsub
* Merge refs/heads/master into qsp-unsub
* Merge refs/heads/master into qsp-unsub
* Merge refs/heads/master into qsp-unsub
2020-06-23 23:50:52 +00:00
Raul Jordan
2bdb825c04
QSP-61 Clarify Why Process Pending Attestations May Exit Early (#6371)
* clarify with comment
* Merge refs/heads/master into qsp-61
* Merge refs/heads/master into qsp-61
* Merge refs/heads/master into qsp-61
2020-06-23 22:34:31 +00:00
Raul Jordan
e1310adb4c
QSP-Docs All Documentation Improvements from Quantstamp Audit Report (#6367)
* docs improvements so far
* all misc docs items
* finalized
* Merge branch 'master' into qsp-docs
* fix broken test
* Merge branch 'master' into qsp-docs
2020-06-23 21:11:20 +00:00
Preston Van Loon
f9038674d1
Refactor setupDB to return the state summary cache used by the database (#6369) 2020-06-23 13:40:55 -07:00
Victor Farazdagi
22b3123ebe
Resolves issues from the best practices in sync package (#6357)
* removes redundant locks
* fixes 37
* fixes 38
* reverts lock removal
* Merge branch 'master' into sync-best-practices-updates
* Merge refs/heads/master into sync-best-practices-updates
2020-06-23 12:00:29 +00:00
Nishant Das
36c82b26e4
Minor Improvements (#6353)
* minor improvements
* Update beacon-chain/sync/deadlines.go
* go fmt
2020-06-23 09:00:09 +00:00
terence tsao
368af7e53f
Clean up old block processing paths (#6311)
* Remove old block processing paths

* Go fmt

* Go fmt again

* Fix tests

* Using the wright receiver

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-06-22 17:19:33 -07:00
Victor Farazdagi
41c8ff4c1b
Renames receiver in sync service (#6348)
* updates coinciding vars
* renames receiver in sync service
* Merge branch 'master' into sync-rename-receiver
2020-06-22 20:37:48 +00:00
Victor Farazdagi
6f8349cdb4
Reclaims leakybucket resources in sync service (#6339)
* Reclaims leakybucket resources
* move calls to defer
* do not reuse queue, after stopping
2020-06-22 11:23:23 +00:00
Jim McDonald
86e9b0ae51
Ping handler timeout (#6302)
* Fix goroutine using cancelled context.

* Only close stream when finished

* Simplify function structure
2020-06-18 19:59:18 +08:00
Nishant Das
340ddf20cb
Update Node RPC Endpoints (#6289)
* checkpoint progress
* some more stuff
* fix tests
* fix tests
* lint
* fix test and initialization
* gaz
* Merge branch 'master' into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
* Merge refs/heads/master into updateP2PRPC
2020-06-18 03:53:46 +00:00
terence tsao
6781bd643b
Fix getting att pre state in sync (#6295) 2020-06-17 16:44:43 -07:00
terence tsao
523fc621d9
First part of block chain service cleanup (#6286)
* Add helper to prevent zero hashes
* Test
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Remove ReceiveBlockNoPubsubForkchoice and rename ReceiveBlockInitialSync
* Apply feature flag to process_block.go
* Update mock and tests
* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Merge refs/heads/master into bc-pkg-cleanup
* Merge refs/heads/master into bc-pkg-cleanup
* Fix mock ReceiveBlockInitialSync. Thanks Victor!
* Merge branch 'bc-pkg-cleanup' of github.com:prysmaticlabs/prysm into bc-pkg-cleanup
* Merge refs/heads/master into bc-pkg-cleanup
* Merge refs/heads/master into bc-pkg-cleanup
2020-06-17 17:11:21 +00:00
Shay Zluf
472fbd3949
Less restrictive block stream for slashing detection (#6269)
* verify sig in stream

* state cache used

* fix import

* fix import

* fix tests

* add log for proposer slashing

* fix e2e

* use chain head to verify sig

* revert fix

* remove extra line

* add line

* revert extra log

* fix tests

* terence feedback

* fix

* fix test

* remove extra setup

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-17 12:38:07 -04:00
terence tsao
4c143f4ed4
Fix underflow in resyncIfBehind (#6274) 2020-06-15 16:46:36 -07:00
Nishant Das
933ab6808b
mark peer as bad (#6250) 2020-06-14 07:10:18 -07:00
Nishant Das
c725a9c8f2
Sort Duplicate Blocks In Response (#6222)
* sort out new blocks
* add unit test
* Merge refs/heads/master into sortDuplicates
* victor's review
* Merge branch 'sortDuplicates' of https://github.com/prysmaticlabs/geth-sharding into sortDuplicates
* Merge refs/heads/master into sortDuplicates
2020-06-12 13:50:07 +00:00
Nishant Das
412da8e7cc
Follow Up to #6225 (#6226)
* don't wait
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into dontWait
* revert some changes
* bring back
2020-06-12 13:23:29 +00:00
Nishant Das
82f4d6a14a
Do Not Wait Till Genesis (#6225)
* don't wait
2020-06-12 11:44:04 +00:00
terence tsao
cf3260b948
Revert "Send blocks that fails validation to slasher" (#6217)
* Revert "Send blocks that fail p2p validation to slasher (#6164)"

This reverts commit f40a7575de.
* Merge branch 'master' into revert-6164-slasherP2P
2020-06-11 17:11:28 +00:00
terence tsao
83242466f4
Fix proposer calculation in sync (#6214) 2020-06-11 09:31:01 -07:00
Shay Zluf
f40a7575de
Send blocks that fail p2p validation to slasher (#6164)
* send blocks that fails validation to block feed for slasher. in order to enable slashing proposers

* add sig validation

* fix imports

* Update shared/featureconfig/config.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update shared/featureconfig/config.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update shared/featureconfig/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* remove extra validation path for slasher

* ivan feedback

* notify all blocks that pass minimal validation to block stream

* notify all blocks that pass minimal validation to block stream

* add to e2e flags

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-10 13:24:05 -04:00
Nishant Das
d1a1043ef1
Update to v0.12 (#5614) 2020-06-09 15:40:48 -07:00
Victor Farazdagi
ba63186c92
Init-sync queue assert order and uniqueness of blocks emitted (#6146)
* removes duplicate code
* Merge branch 'master' into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge refs/heads/master into init-sync-queue-dedup-blocks
* Merge branch 'master' into init-sync-queue-dedup-blocks
* removes notification
* Merge branch 'init-sync-queue-dedup-blocks' of github.com:prysmaticlabs/prysm into init-sync-queue-dedup-blocks
* adds test
* updates service
* get rid of deepequal for err check
* Merge refs/heads/master into init-sync-queue-dedup-blocks
2020-06-08 14:52:43 +00:00
Nishant Das
a1e3fc9500
Fix Gossip Clock Disparity (#6141)
* fix disparity
* Merge refs/heads/master into fixGossipDisparity
* Merge refs/heads/master into fixGossipDisparity
* Merge refs/heads/master into fixGossipDisparity
* Merge refs/heads/master into fixGossipDisparity
* Merge refs/heads/master into fixGossipDisparity
* Merge refs/heads/master into fixGossipDisparity
2020-06-06 01:54:44 +00:00
Nishant Das
c06d61c3d0
fix issue (#6152) 2020-06-06 09:16:46 +08:00
terence tsao
6af7664a56
Check attestation has pre state before retrieving it (#6149)
* Use state util to get block root

* Move prestate to below

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-05 18:23:35 -05:00
Victor Farazdagi
fd3cb0be55
Re-enables multi-failure test case in round robin (#6143)
* re-enables multi-failure test case in round robin
2020-06-05 11:26:10 +00:00
Victor Farazdagi
84ae840b5d
Re-enable previous fetching algorithm testing (#6125)
* re-enable previous fetching algorithm testing
2020-06-04 11:26:02 +00:00
Ivan Martinez
24d315c6d1
Add public keys to response for GetValidatorPerformance (#6115)
* Add public keys to response for GetValidatorPerformance
* Merge branch 'master' into add-pubkey-resp
* Merge branch 'master' into add-pubkey-resp
* Fixed all the broken tests (#6117)
* Merge branch 'add-pubkey-resp' of github.com:prysmaticlabs/prysm into add-pubkey-resp
* Fix skip
* Merge branch 'master' into add-pubkey-resp
* Fix ethapis
* Merge branch 'master' into add-pubkey-resp
2020-06-04 04:55:29 +00:00
terence tsao
c4afb8e2e7
Save unaggregated att in beacon_aggregate_and_proof subscriber (#6109)
* Use state util to get block root
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Also save unaggregated att
* Merge refs/heads/master into fix-save-agg
* Handle error
* Merge branch 'fix-save-agg' of github.com:prysmaticlabs/prysm into fix-save-agg
* Test
* Merge refs/heads/master into fix-save-agg
2020-06-03 19:49:43 +00:00
Preston Van Loon
d2337d0ec1
Disable some static analysis checks for coverage builds (#6105)
* Disable some static analysis checks for coverage builds
* Merge branch 'master' into fix-coverage-builds
* disable lostcancel for third_party code
* Merge branch 'fix-coverage-builds' of github.com:prysmaticlabs/prysm into fix-coverage-builds
* Merge refs/heads/master into fix-coverage-builds
* A few lostcancel fixes
* Merge branch 'fix-coverage-builds' of github.com:prysmaticlabs/prysm into fix-coverage-builds
* Merge refs/heads/master into fix-coverage-builds
2020-06-03 19:22:48 +00:00
Nishant Das
e715339aab
Fix Status Messages During Genesis (#6046)
* fix status messages
* fix finalized checkpoint
* remove code
* Merge refs/heads/master into fixStatusMessages
* Update beacon-chain/blockchain/service.go
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* fix
* fix test
* fix again
* fix
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
* Merge refs/heads/master into fixStatusMessages
2020-06-01 20:26:42 +00:00
Nishant Das
ceaceeb33c
Reset Streams Properly (#6066)
* reset streams

* fix build
2020-06-01 16:14:47 +08:00
terence tsao
6e3b78b99e
Ran code cleanup from goland (#6064)
* Ran code cleanup from goland

* Typo
2020-05-31 15:08:36 -05:00
Preston Van Loon
49a0d3caf0
Refactor dependencies, make Prysm "go gettable" (#6053)
* Fix a few deps to work with go.mod, check in generated files

* Update Gossipsub to 1.1 (#5998)

* update libs

* add new validators

* add new deps

* new set of deps

* tls

* further fix gossip update

* get everything to build

* clean up

* gaz

* fix build

* fix all tests

* add deps to images

* imports

Co-authored-by: rauljordan <raul@prysmaticlabs.com>

* Beacon chain builds with go build

* fix bazel

* fix dep

* lint

* Add github action for testing go

* on PR for any branch

* fix libp2p test failure

* Fix TestProcessBlock_PassesProcessingConditions by updating the proposer index in test

* Revert "Fix TestProcessBlock_PassesProcessingConditions by updating the proposer index in test"

This reverts commit 43676894ab01f03fe90a9b8ee3ecfbc2ec1ec4e4.

* Compute and set proposer index instead of hard code

* Add back go mod/sum, fix deps

* go build ./...

* Temporarily skip two tests

* Fix kafka confluent patch

* Fix kafka confluent patch

* fix kafka build

* fix kafka

* Add info in DEPENDENCIES. Added a stub link for Why Bazel? until https://github.com/prysmaticlabs/documentation/issues/138

* Update fuzz ssz files as well

* Update fuzz ssz files as well

* getting closer

* rollback rules_go and gazelle

* fix gogo protobuf

* install librdkafka-dev as part of github actions

* Update kafka to a recent version where librkafkfa is not required for go modules

* clarify comment

* fix kafka build

* disable go tests

* comment

* Fix geth dependencies for end to end

* rename word

* lint

* fix docker

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-05-31 14:44:34 +08:00
Victor Farazdagi
0243bdcf37
Fixes overflow issue in sync/pending_blocks_queue (#6039)
* fixes overflow issue
* Merge refs/heads/master into fix-overflow
* Merge refs/heads/master into fix-overflow
2020-05-29 13:50:41 +00:00
Victor Farazdagi
a0bf8cba52
Removes redundant type conversions (#6041)
* removes redundant type conversions
* remove redundant type in struct decl
* removes redundant parens
* Merge refs/heads/master into redundant-type-conversions
2020-05-29 12:42:39 +00:00
Nishant Das
979c0074c7
Register RPC And Pubsub Handlers After Genesis is Determined (#6020)
* fix
* fix test
2020-05-28 05:39:40 +00:00
Victor Farazdagi
f4778048ac Init sync arbitrary batch length + full bandwidth utilization (#5978)
* allow to jump over 50K skipped slots
* more tests
* removes debug logs
* minor fixes
* re-arrage pid updates
* optimizes queue and fsm
* Merge branch 'master' into init-sync-arbitrary-batch-len
* fixes benchmark test
* Merge branch 'master' into init-sync-arbitrary-batch-len
* queue cleanup
* updates polling interval
* fsm test: allMachinesInState
* fsm test: highestStartBlock
* rest of fsm tests
* Merge branch 'master' into init-sync-arbitrary-batch-len
* Merge refs/heads/master into init-sync-arbitrary-batch-len
* Merge refs/heads/master into init-sync-arbitrary-batch-len
* Merge refs/heads/master into init-sync-arbitrary-batch-len
* Merge refs/heads/master into init-sync-arbitrary-batch-len
* Merge refs/heads/master into init-sync-arbitrary-batch-len
* parse machines in order
* Merge branch 'init-sync-arbitrary-batch-len' of github.com:prysmaticlabs/prysm into init-sync-arbitrary-batch-len
2020-05-27 09:13:41 +00:00
terence tsao
3b65cb6d0b
Consistent BLS function inputs (#5969)
* Make function input consistent

* Change back

* Typo

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-25 16:08:12 -05:00
Victor Farazdagi
f81c8c6825
Allow to jump over 50K+ skipped slots in init-sync (#5975)
* allow to jump over 50K skipped slots
* more tests
* removes debug logs
* minor fixes
* re-arrage pid updates
* Merge refs/heads/master into init-sync-50K-skipped-slots
2020-05-25 16:43:59 +00:00
Shay Zluf
17169e5a2d
External slashing protection (#5895)
* slasher grpc client

* do not export

* slasher on a different package

* fix featureconfig

* change to rough time

* revert roughtime

* remove extra comma

* revert order change

* goimports

* fix comments and tests

* fix package name

* revert reorder

* comment for start

* service

* fix visibility

* external slasher validator protection implementation

* gaz

* fix comment

* add comments

* nishant feedback

* raul feedback

* preston feedback

* fix flags

* fix imports

* fix imports

* port 4002

* added tests

* fix log

* fix imports

* fix imports name

* raul feedback

* gaz

* terence comment

* change name

* runtime fixes

* add flag check

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-20 10:23:22 -05:00
Victor Farazdagi
8241b7e20d
Re-arranges fetcher locks -> per peer level (#5906)
* fetcher locks -> per peer level
* typo
* removes redundant setting to nil
* Merge refs/heads/master into init-sync-rearrange-locks
2020-05-19 17:48:32 +00:00
Victor Farazdagi
ca26745720
Implements WRR in init-sync, full bandwidth utilization (#5887)
* implements weighted round robin in init-sync
* protection against evil peer
* reshuffle
* shorten flag
* deterministic order of wrr
* Merge branch 'master' into init-sync-wrr
* Merge refs/heads/master into init-sync-wrr
* Merge refs/heads/master into init-sync-wrr
* allow peers with the same capacity to pe selected at random
* Merge branch 'init-sync-wrr' of github.com:prysmaticlabs/prysm into init-sync-wrr
* adds wrr to e2e tests
* Merge refs/heads/master into init-sync-wrr
* Merge refs/heads/master into init-sync-wrr
* Merge refs/heads/master into init-sync-wrr
2020-05-18 18:59:03 +00:00
Nishant Das
eb77f56d8a
Sort Blocks Before Returning them To Peer (#5894)
* add tests and check
* lint
* Merge refs/heads/master into sortBlocks
2020-05-18 17:58:20 +00:00
Ivan Martinez
5d8b5a3c30
Remove DB check for receiving p2p slashings (#5892)
* Removes DB check for receiving p2p slashings
* Fix bad logic
* Merge refs/heads/master into p2p-remove-db-check
* Whoops
* Merge branch 'p2p-remove-db-check' of github.com:prysmaticlabs/prysm into p2p-remove-db-check
2020-05-18 07:36:37 +00:00
Nishant Das
5bb9278a88
make them debug (#5891) 2020-05-18 02:27:18 -04:00
Victor Farazdagi
d79950ab3b
Optimizes FSM event triggering (#5881)
* Optimizes machine triggering
* updates queue
* Merge branch 'master' into init-sync-fsm-optimizations
* gazelle
* Merge branch 'init-sync-fsm-optimizations' of github.com:prysmaticlabs/prysm into init-sync-fsm-optimizations
2020-05-16 20:26:00 +00:00
Victor Farazdagi
1cc7d67025
Adds rate limiting tests to beaconBlocksByRangeRPCHandler (#5866)
* overflow/nooverflow tests
* formatting
* check peer disconnecting
* more tests
* Merge branch 'master' into sync-rate-overflow-tests
* formatting
* Merge branch 'sync-rate-overflow-tests' of github.com:prysmaticlabs/prysm into sync-rate-overflow-tests
2020-05-15 20:51:23 +00:00
Nishant Das
320bb46f35
More Handshake Cleanup (#5863)
* change to debug log

* disconnect on wrong fork

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-15 11:08:42 -05:00
Raul Jordan
007a98b711
Fix Stream Duties Server + Get Duties Before ChainStart (#5849)
* beacon node stream duties fixes

* comment

* fix critical seconds bug

* back to 12

* config changes

* Update beacon-chain/sync/initial-sync/service.go

* revert config

* Update beacon-chain/rpc/validator/assignments.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-05-15 10:25:43 -05:00
Victor Farazdagi
f461f586a1
Refactors redundant global variables from sync (#5860)
* refactors global variables from sync
* gazelle
* Merge branch 'master' into sync-remove-globals
* better spacing
* Merge branch 'sync-remove-globals' of github.com:prysmaticlabs/prysm into sync-remove-globals
* Merge branch 'master' into sync-remove-globals
2020-05-15 08:53:19 +00:00
Ivan Martinez
6e9839f987
Validate slashings upon pool exit (#5848)
* Validate slashings upon pool exit
* Change in implementation
* Fix test
* Merge branch 'master' into validate-slashing
* Merge refs/heads/master into validate-slashing
* Merge refs/heads/master into validate-slashing
* Merge refs/heads/master into validate-slashing
* Undo e2e changes
* Merge branch 'validate-slashing' of github.com:prysmaticlabs/prysm into validate-slashing
* Merge refs/heads/master into validate-slashing
2020-05-14 17:11:28 +00:00
Nishant Das
20c2d78731
add new methods and tests (#5852)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-14 22:44:03 +08:00
Victor Farazdagi
471779e4fc
Fixes bug with excessive rate limiting (#5853)
* fixes bug with excessive rate limiting
2020-05-14 13:43:36 +00:00
Nishant Das
34c02ffd34
Allow a Larger Allowance for Peer Faults (#5847)
* allow larger allowance
* change to 5
* Merge branch 'master' into allowLargerAllowance
* test
2020-05-14 03:08:58 +00:00
Shay Zluf
ffa08f5a85
Stream p2p agg attestation (#5809)
* stream aggreagted attestations from p2p network to indexed attestation stream

* remove excessive log

* fix test

* handle nil attestation as well

* Update beacon-chain/sync/subscriber_beacon_aggregate_proof.go

* Update beacon-chain/sync/subscriber_beacon_aggregate_proof.go

* Update beacon-chain/sync/subscriber_beacon_aggregate_proof_test.go

* terence feedback

* sort imports

* sort imports

* Change to received buffer

* preston feedback

* error log

* raul feedback

* more logging changes

* fix duplicate package name

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
2020-05-13 12:21:53 -05:00
Raul Jordan
578703fa05
Properly Divide SecondsPerSlot Into Milliseconds (#5821)
* use util for dividing slot durations

* fix places with dividing slot duration

* more dividers fixed

* format import

* fix formatting in validator test

* fix slot test tolerance

* nishant suggestions

* imports

* resolve e2e

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-13 11:28:17 +08:00
Nishant Das
ab23fadc5e
Revert Change Added in #5811 (#5828)
* revert

* remove
2020-05-12 21:39:36 +08:00
Victor Farazdagi
baf9ec60eb
Adds BlockBatchLimitBurstFactor flag (#5825)
* adds BlockBatchLimitBurstFactor flag
* Merge branch 'master' into burst-factor-flag
2020-05-12 11:31:10 +00:00
Nishant Das
edfb32021b
Validate Finalized Root in Status RPC Requests (#5811)
* additional status validation

* changes

* clean up

* Update beacon-chain/db/kv/finalized_block_roots.go

* Update beacon-chain/sync/rpc_status.go

* Update beacon-chain/sync/rpc_status.go

* Update beacon-chain/sync/rpc_status.go

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-12 11:22:01 +08:00
terence tsao
b5e7f9dd6d
Run go fmt in prysm (#5815)
* go fmt
2020-05-11 22:03:15 +00:00
Nishant Das
6840afb45e
Fix Issues With Peer Handshakes (#5799)
* add check

* make handshake better

* fix a lot of things

* Update beacon-chain/p2p/handshake.go

* Update beacon-chain/p2p/handshake.go

* fix things

* fix

* fix handshake

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-11 12:29:23 +08:00
Victor Farazdagi
e53452a437
Moves connectPeers to common test setup file (#5800)
* moves connectPeers
2020-05-10 14:47:05 +00:00
Preston Van Loon
c6cac96a3b
Duplicate init sync test target for better coverage reporting (#5789)
* duplicate target for better coverage testing
* Merge refs/heads/master into init-sync-raceoff
2020-05-09 01:40:14 +00:00
Preston Van Loon
87e0fe5928
Refactor ReceiveBlock to receive the block root. (#5785)
* refactor ReceiveBlockNoPubsub to receive block root
* Refactor ReceiveBlock to receive block root
* A few other minor refactoring to reduce block HTR
* use arg, remove HTR
* more slight refactoring, comments
* fix test build
* Merge refs/heads/master into receive-with-root
* Merge refs/heads/master into receive-with-root
2020-05-09 01:22:59 +00:00
Shay Zluf
c1a8b41347
Is slashable block rpc endpoint implementation (#5765)
* is slashable block rpc endpoint implementation

* add span

* raul feedback

* fix error

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-08 08:51:56 +03:00
Ivan Martinez
9a1157465e
Change non-mutating validator accesses to ReadOnly (#5776)
* Change instances of ValidatorAtIndex to ReadOnly where possible

* Use ReadOnly for VerifyExit and Slashings

* Move length check to before lock

* Improve readonly tests

* undo process attester changes

* Fix test
2020-05-07 14:15:51 -04:00
Victor Farazdagi
40c29744e3
Init-sync discard debug logs in tests (#5774)
* refactors block fetching function to fetcher
* more comments fixed
* Merge branch 'master' into init-sync-optimizations
* moves helper code into separate file
* move test setup code to testmain
* removes bench
2020-05-07 13:13:45 +00:00
Victor Farazdagi
c4eb8c7a16
Refactors block fetching function to fetcher (#5766)
* refactors block fetching function to fetcher
* more comments fixed
* Merge refs/heads/master into init-sync-upd
* Merge refs/heads/master into init-sync-upd
* Merge refs/heads/master into init-sync-upd
2020-05-06 21:29:50 +00:00
Nishant Das
ae66f63ec2
Memory Improvements (#5760)
* fix methods
* fix path for only initial sync
* update comment
* Merge refs/heads/master into memImprovements
2020-05-06 05:04:02 +00:00
Nishant Das
bf2a37b27f
Remove Old Sync Routine (#5761)
* remove old sync
2020-05-06 04:36:34 +00:00
Nishant Das
3a677ef342
Add Persistent Subnets (#5734)
* add params

* add changes

* bug fixes

* fix method

* get new assignments

* add test and comments

* change to slice of uint64

* add test

* lint

* Update beacon-chain/rpc/validator/assignments_test.go

* Update beacon-chain/cache/committee_ids.go

* Update beacon-chain/rpc/validator/assignments.go

* add comment

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-05 14:26:20 -05:00
Victor Farazdagi
bbde2a6820
Assert no side effects occur on test (#5714)
* add reset funcs
* init -> TestMain for beacon config params
* fixes build file
* core/epoch spectests updated
* core/blocks spectests updated
* fix the rest of spectests
* SetupTestConfigCleanup
* revert copy() api
* rever all cases of copy()
* fixes spectests
* updates fork_test
* config_test update
* fixes spectest/config_test
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge branch 'master' into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge branch 'master' into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
2020-05-05 08:39:38 +00:00
Preston Van Loon
eb2abbdd8b
libfuzz based tests (#5095)
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* use opt for fuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* use ubuntu as the default build image
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Move fuzz tests
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* making progress on p2p fuzz
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* use a single stream repeatedly
* use a single stream repeatedly
* use many streams
* fixes
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Add basic test for ssz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Use a custom go_library to add libfuzzer
* delete old corpus, fix rpc_status_fuzz
* move fuzz.bzl
* minor fixes
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* add c-shared library (doesnt work)
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* fix build
* fixes
* Merge refs/heads/master into go-1.14-libfuzz
* lint
* more lint
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* more lint
* allow environment variable
* Merge refs/heads/master into go-1.14-libfuzz
* Add a fuzzit image with the beacon_states
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* lint
* readme
* lint
* fix lint again i think
* gaz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* breakup deps
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* don't panic on failure
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* panic if no beacon states to read
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* update rules_foreign_cc
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* remove c_shared target, manually tag tests
* Add readme to fuzz package
* delete a few comments
* delete a few comments
* lint
* gaz
* gaz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
2020-05-05 07:22:26 +00:00
Preston Van Loon
eb27f35982
Use the custom HTR in a few more places (#5743) 2020-05-05 14:42:53 +08:00
Preston Van Loon
9be8a456eb
Use stateutil.BlockRoot everywhere (#5739)
* Sweeping replace of ssz.HashTreeRoot(block) to stateutil.BlockRoot, minor tweaks to critical path of proposeBlock
* imports
* Merge branch 'master' into better-block-htr
* Add --enable-custom-block-htr to e2e flags
* Merge branch 'better-block-htr' of github.com:prysmaticlabs/prysm into better-block-htr
2020-05-05 04:30:24 +00:00
terence tsao
e8da03a6b2
Use AttestationPreState (#5737) 2020-05-04 18:48:14 -05:00
Nishant Das
1432332923
Return Genesis Block (#5709)
* add check
* lint
* lint
* Merge branch 'master' into returnGenesis
* Merge refs/heads/master into returnGenesis
* Merge refs/heads/master into returnGenesis
* Merge refs/heads/master into returnGenesis
* Merge refs/heads/master into returnGenesis
* Merge refs/heads/master into returnGenesis
* terence's review
* Merge branch 'returnGenesis' of https://github.com/prysmaticlabs/geth-sharding into returnGenesis
* Merge refs/heads/master into returnGenesis
* Merge refs/heads/master into returnGenesis
* fix failing test
2020-05-04 07:54:31 +00:00
Victor Farazdagi
b5c4dc2a75
Refactor db teardown to testing.TB.Cleanup (#5725)
* init-sync updates
* slasher/db/kv tests
* beacon-chain/rpc/beacon tests
* update kv_test
* beacon-chain/rpc-validator tests updated
* slasher/db/kv - remove teardown method
* beacon-chain/sync tests updated
* beacon-chain/db/kv tests updated
* beacon-chain/blockchain tests updated
* beacon-chain/state/stategen tests updated
* beacon-chain/powchain updates
* updates rest of slasher tests
* validator/db tests
* rest of the tests
* minor comments update
* gazelle
* Merge refs/heads/master into teardowndb-to-cleanup
2020-05-04 01:14:34 +00:00
terence tsao
42961277e1
Add histogram metrics for block arrival time (#5708)
* Add metrics to track block propogation delay

* Grammar

* Grammar

* Metrics for block sent

* Refactor into own function

* Removed sent

* Lint

* Use histogram. Thanks Preston for the suggestion!

* s/total/histogram

* Update name to `block_arrival_latency_milliseconds`

* Use milliseconds

* Use unix nano

* Use SlotToTime

* Preston's feedback

* Handle err
2020-05-01 19:35:15 -07:00
terence tsao
fdad7e67b0
Update pubsub seen cache to be per epoch instead of per slot (#5698)
* hasSeenAggregatorIndexSlot -> hasSeenAggregatorIndexEpoch
* Fix test
* Update subscriber
* setSeenCommitteeIndicesSlot -> setSeenCommitteeIndicesEpoch
* Fix test
* Revert "setSeenCommitteeIndicesSlot -> setSeenCommitteeIndicesEpoch"

This reverts commit bd638ae55675d711f67f98afc8a348803e72cd3a.
* Fixed unaggregated att seen cache to use per slot
2020-04-30 22:16:14 +00:00
Nishant Das
5f53f9fde1
Clean Up Of Dynamic Subscriptions (#5690)
* fix subscriptions

* add reg test
2020-04-30 09:14:59 -05:00
Nishant Das
4d30519dda
fix bugs (#5683) 2020-04-29 14:28:22 -05:00
Raul Jordan
3416962fc2
All Beacon Node Package-Level Godocs (#5677)
* package level godocs fixed

* all beacon node godocs

* comment and gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-29 12:40:33 -05:00
Raw Pong Ghmoa
3fc6afa02e
beacon/log: augment logs with block roots (#5675)
* beacon/log: augment synced log with block root and finalized root
* beacon/log: augment initial sync log with block root
* beacon/sync: degrade pending block log to DEBUG level
* beacon/log: restore info level for state transition
* beacon/log: improve info logging
* Merge branch 'master' into q9-log-block
* Merge branch 'master' into q9-log-block
2020-04-29 16:57:04 +00:00
terence tsao
05fcb1073a
Remove duplicated sig verification (#5676)
* Remove duplicated sig verification
2020-04-29 15:57:58 +00:00
Victor Farazdagi
44b93d9df9
Fix tests depending on configuration flag (#5650)
* fixes segfault
* Merge branch 'master' into fix-nil-segfaults
* upd state
* Merge branch 'master' into fix-nil-segfaults
* updates shard=1
* Merge refs/heads/master into fix-nil-segfaults
* Merge refs/heads/master into fix-nil-segfaults
* Merge refs/heads/master into fix-nil-segfaults
* fixes tests
* Merge branch 'fix-nil-segfaults' of github.com:prysmaticlabs/prysm into fix-nil-segfaults
* reset config to nil
* add flag assertion to other beacon-chain/rpc files
* Merge refs/heads/master into fix-nil-segfaults
* fix the rest featureconfig.init calls
* Merge branch 'fix-nil-segfaults' of github.com:prysmaticlabs/prysm into fix-nil-segfaults
* remove config setting from init
* gazelle
* reset shard back to 4
2020-04-27 19:44:35 +00:00
Nishant Das
b6353da793
Add Block Batch Limit Flag (#5646)
* add flag

* add flag

* gaz

* fix lint

* fix build issues

* revert initial sync changes

* fix tests

* Update beacon-chain/sync/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-27 09:35:40 -05:00
Nishant Das
07e8609b69
Exit Early from Responding (#5637)
* exit early
2020-04-27 03:18:02 +00:00
Nishant Das
aca9691a0d
Stagger Block By Range Responses (#5618)
* refactor method

* fix method

* comments

* Update beacon-chain/sync/rpc_beacon_blocks_by_range.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/sync/rpc_beacon_blocks_by_range.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

* Apply suggestions from code review

Co-Authored-By: Victor Farazdagi <simple.square@gmail.com>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-26 13:49:47 +08:00
terence tsao
3ac15e36a1
Use correct domain to verify aggregate selection (#5604)
* Use domain selection proof
* Merge refs/heads/master into selection-domain
* Merge branch 'master' of github.com:prysmaticlabs/prysm into selection-domain
* Fixed all the tests
* Merge branch 'selection-domain' of github.com:prysmaticlabs/prysm into selection-domain
2020-04-24 17:26:10 +00:00
Nishant Das
f9eb54661a
Less Noisy P2P Logs (#5607)
* fix logging issues
* change again
* fix errors
* Merge refs/heads/master into lessNoisy
* gaz
* Merge branch 'lessNoisy' of https://github.com/prysmaticlabs/geth-sharding into lessNoisy
2020-04-24 04:10:53 +00:00
Nishant Das
2ebd684924
Add Goodbye Message When Disconnecting With Peers (#5589)
* add goodbye message
* Merge branch 'master' into addGoodbye
* fix interface
* Merge refs/heads/master into addGoodbye
* use wrapper func
* Merge branch 'addGoodbye' of https://github.com/prysmaticlabs/geth-sharding into addGoodbye
* Merge refs/heads/master into addGoodbye
* Merge refs/heads/master into addGoodbye
* Merge refs/heads/master into addGoodbye
* Merge refs/heads/master into addGoodbye
2020-04-23 19:46:24 +00:00
terence tsao
7b38e97e83
Invert --disable-new-state-mgmt to --enable-new-state-mgmt (#5567)
* Invert the flag
* Merge branch 'master' into invert-disable-new-state-mgmt
* Merge refs/heads/master into invert-disable-new-state-mgmt
* Add new-state-mgmt to e2e flag
* Merge branch 'invert-disable-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-disable-new-state-mgmt
* Merge refs/heads/master into invert-disable-new-state-mgmt
2020-04-21 22:30:22 +00:00
Nishant Das
639e3072fe
Cancel Initial Sync Properly (#5529)
* fix cancelling
* Merge branch 'master' into cancelServices
* Merge refs/heads/master into cancelServices
* Merge refs/heads/master into cancelServices
2020-04-20 04:04:45 +00:00
Preston Van Loon
33774721bb
p2p/sync: Wrong genesis digest is used to register pubsub topic subscriptions (race condition) (#5500)
* Return error when genesis time or genesis root is not set
* Merge refs/heads/master into fail-on-unset-fork-digest
* move to own helpers
* make it non public
* Merge branch 'fail-on-unset-fork-digest' of https://github.com/prysmaticlabs/geth-sharding into fail-on-unset-fork-digest
* lint
* fix
* return error
* fix tests and error
* Merge refs/heads/master into fail-on-unset-fork-digest
* first round of test fixes
* second round of fixes
* Merge branch 'fail-on-unset-fork-digest' of https://github.com/prysmaticlabs/geth-sharding into fail-on-unset-fork-digest
* lint
* Merge refs/heads/master into fail-on-unset-fork-digest
* gaz
* Merge branch 'fail-on-unset-fork-digest' of https://github.com/prysmaticlabs/geth-sharding into fail-on-unset-fork-digest
2020-04-19 02:32:53 +00:00
terence tsao
0078f42554
Disable some block checks without state mgmt (#5498)
* Revert "Add WaitForSynced to beacon node for validator startup (#5366)"

This reverts commit 1224e75ba1.
* Disable various checks in block validator sync without new state mgmt
* Revert "Revert "Add WaitForSynced to beacon node for validator startup (#5366)""

This reverts commit 927d8d9e06ab700e10c5848eb46def6b40f1ecfa.
* Invert
2020-04-18 20:08:55 +00:00
Ivan Martinez
1224e75ba1
Add WaitForSynced to beacon node for validator startup (#5366)
* Add WaitForSynced to beacon-chain
* Fix build issues
* Fix comment
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-validator
* Fix tests
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-validator
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into simplify-validator
* Merge branch 'master' into simplify-validator
* Merge branch 'master' into simplify-validator
* Merge branch 'master' into simplify-validator
2020-04-18 10:45:20 +00:00
Nishant Das
28733f2c9e
Add Separate Network Config (#5454)
* clean up config
* add discv5 config
* add gaz
* Merge branch 'master' into configCleanup
* Merge refs/heads/master into configCleanup
* Merge refs/heads/master into configCleanup
* Merge refs/heads/master into configCleanup
* Merge refs/heads/master into configCleanup
* Merge refs/heads/master into configCleanup
* Merge refs/heads/master into configCleanup
2020-04-16 22:21:44 +00:00
Nishant Das
92131695b1
Fix Network Encoder (#5435)
* add new block encoding for snappy
* Merge branch 'master' into fixEncoder
* flip flag
* Merge branch 'fixEncoder' of https://github.com/prysmaticlabs/geth-sharding into fixEncoder
* not an underscore
2020-04-15 03:03:44 +00:00
terence tsao
cb045dd0e3
Align code base to v0.11 (#5127)
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* add in new patch and workspace
* update cloners
* Handle rewards overflow (#5122)

* Refactoring of initial sync (#5096)

* implements blocks queue

* refactors updateCounter method

* fixes deadlock on stop w/o start

* refactors updateSchedulerState

* more tests on schduler

* parseFetchResponse tests

* wraps up tests for blocks queue

* eod commit

* fixes data race in round robin

* revamps fetcher

* fixes race conditions + livelocks + deadlocks

* less verbose output

* fixes data race, by isolating critical sections

* minor refactoring: resolves blocking calls

* implements init-sync queue

* udpate fetch/send buffers in blocks fetcher

* blockState enum-like type alias

* refactors common code into releaseTicket()

* better gc

* linter

* minor fix to round robin

* moves original round robin into its own package

* adds enableInitSyncQueue flag

* fixes issue with init-sync service selection

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* initsyncv1 -> initsyncold

* adds span

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Handle rewards overflow

* Revert "Refactoring of initial sync (#5096)"

This reverts commit 3ec2a0f9e0.

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* updated block operations
* updated validator client
* Merge refs/heads/master into v0.10.1
* updated block operations test
* skip benchmark test
* updated transition test
* updated db kv tests
* updated ops tests
* updated ops tests
* updated slashing tests
* updated rpc tests
* updated state utils
* updated test utils and miscs
* Temp skips minimal spec tests
* Fixed proposer slashing test
* Gaz
* Skip 2 more minimal tests
* Skip 2 more minimal tests
* Update readme
* gaz
* Conflict
* Fix import and not use
* Update workspace for new spec test
* Fix workspace
* Merge refs/heads/master into v0.10.1
* Update workspace with new ethapi commit
* Unblock a few tests
* Merge refs/heads/master into v0.10.1
* fixed block op test
* gaz
* Merge refs/heads/master into v0.10.1
* Skip gen state test (test setup issue
* Updated hysteresis config
* Updated epoch processing for new hyteresis
* Updated tests
* regen proto beacon
* update state util for state root
* update state types
* update getter and setters
* update compute domain and get domain and tests
* update validators
* Add forkdata proto
* Updated compute domain api, moved it to helper pkg
* Merge refs/heads/master into v0.10.1
* Fixed all core tests
* Fixed all the sync tests
* Fixed all the rpc tests
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Fixed conflict
* Fixed conflict
* Conflict fix
* visibility
* Fixed validator tests
* Fixing test util
* Fixed rest of non spec tests
* Fixed a bug proposer index wasn't included
* gaz
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Updated eth1 data voting period to epoch based
* Fixed failed tests
* fix bug
* fix error
* Fixed more misc tests
* Add new SignedAggregateAndProof to pass spec test
* Update minimalConfig.PersistentCommitteePeriod
* allow to rebuild trie
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Skip e2e tests
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Align aggregator action with v0.11 (#5146)
* Remove Head Root from Beacon Block by Range Request (#5165)

* make proto changes
* remove head root
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* add back herumi's library
* Update ethapi in workspace, started fixing test. Hand off to Nishant
* fix build
* All tests passing
* Align finalized slot check with v0.11 (#5166)
* Merge branch 'master' into v0.11
* Add DoS resistance for v0.11 (#5158)
* Add Fork Digest Helper (#5173)
* Extend DoS prevention to rest of operation objects (#5174)

* Update mapping

* Add caches

* Update seen block in validation pipeline

* Update seen att in validation pipeline

* Update seen att in validation pipeline

* Fixed rest of tests

* Gazelle

* Better writes

* Lint

* Preston's feedback

* Switched to LRU cache and fixed tests

* Gazelle

* Fix test

* Update proposer slashing

* Update proposer slashing

* Fixed a block test

* Update exit

* Update atteser slashing

* Raul's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add remote keymanager (#5133)

* Add remote keymanager

* Add generic signRoot() helper

* Add tests for remote keymanager

* NewRemote -> NewRemoteWallet

* signRoot -> signOject, to increase reuse

* Fix end-to-end compile error

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Add Snappy Framing to the Encoder (#5172)

* change to framing

* more fixes

* fix everything

* add stricter limits

* preston feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Move Subnet Functionality to its Own File (#5179)

* move subnets to their own file

* fix build fail

* build

* Update beacon-chain/p2p/discovery_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Sync with master
* Verify proposer signature in sync (#5206)
* Fix Signed Attestation In Sync (#5207)
* Add Eth2 Fork ENR Functionality (#5181)

* add fork entry enr

* add in fork

* add the required fork entry to node

* add and retrieve fork entry

* await state initialized

* utilize new structure

* more progress, utilizing a config map instead

* send the genesis validators root via the event feed

* struct method for discovery

* fix broken builds

* fixed up more tsts using state feed initializer

* fix up most tests

* only one more failing test

* almost done with tests

* p2p tests all pass

* config fix

* fix blockchain test

* gaz

* add in todo

* lint

* add compare func

* ensure fork ENR versions match between peers

* add in test for discovery

* test name

* tests complete

* tests done

* done

* comments

* fix all flakes

* addressed comments

* build using ssz gen

* marshal record

* use custom ssz

* deduplicate import

* fix build

* add enr proto

* p2p tests done

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Verify aggregator signature in sync (#5208)
* Add Fork Digest For Gossip Topics (#5191)

* update for the day

* fix remaining failing test

* fix one more test

* change message

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

* implement fork digest'

* align digest to interface'

* passed all tests

* spawn in goroutine

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fix Incorrect Attester Slashing Method (#5229)
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Remove keystore keymanager from validator (#5236)

* Remove keystore keymanager from validator

* Update dependency

* Update validator/flags/flags.go

* Update validator/flags/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* fix broadcaster
* update metrics with fork digest for p2p (#5251)

* update metrics with fork digest for p2p

* update p2p metrics

* update metrics using att values

* wrapped up

* fix bug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix incorrect domain type comments (#5250)

* Fix incorrect domain type comments
* resolve conflicts
* fix broken broadcast test
* fix tests
* include protocol suffix
* fix confs
* lint
* fix test
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* resolve broken slasher test'
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' into v0.11
* fix config override
* Remove deprecated parameters (#5249)
* Avoid div by zero in extreme balance case (#5273)

* Return effective balance increment instead of 1

* Update to new spec tests v0.11.1

* Revert "Regen historical states for `new-state-mgmt` compatibility (#5261)"

This reverts commit df9a534826.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Remove deprecated parameters (#5249)" (#5276)

This reverts commit 7d17c9ac3455ee15c67b3645485693309216bc97.
* Verify block proposer index before gossip  (#5274)

* Update pipeline

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add in Proposer Index to Custom HTR (#5269)

* fix test

* Update beacon-chain/state/stateutil/blocks_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Resolve Flakey P2P Tests (#5285)

* double time for flakey test

* fix test flakeyness in p2p:

* flakey

* time tolerance

* greater tolerance
* Merge branch 'master' into v0.11
* release resources correctly (#5287)
* Merge refs/heads/master into v0.11
* Enable NOISE Handshake by Default v0.11 (#5272)

* noise handshakes by default

* fix build

* noisy noise everywhere

* deprecated noisy noise flag with more noise

* add secio as fallback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
* Merge refs/heads/master into v0.11
* new ports
* fix broken build
* Make `new-state-mgmt` canonical  (#5289)

* Invert the flags
* Update checking messages
* Fixed all db tests
* Fixed rest of the block chain tests
* Fix chain race tests
* Fixed rpc tests
* Disable soudns better...
* Merge branch 'v0.11' into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Fix export
* Merge branch 'invert-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-new-state-mgmt
* Fix conflict tests
* Gazelle
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge branch 'master' into v0.11
* resolve flakeyness
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Detect Proposer Slashing Implementation (#5139)

* detect blocks

* detect blocks

* use stub

* use stub

* use stub

* todo

* fix test

* add tests and utils

* fix imports

* fix imports

* fix comment

* todo

* proposerIndex

* fix broken test

* formatting and simplified if

* Update slasher/detection/service.go

* Update slasher/detection/testing/utils.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fixed up final comments

* better naming

* Update slasher/detection/service.go

* Update slasher/detection/service.go

* Update slasher/detection/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* no more named args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Add Metadata And Ping RPC methods (#5271)

* add new proto files

* add flag and helper

* add initializer

* imports

* add ping method

* add receive/send ping request

* add ping test

* refactor rpc methods and add ping test

* finish adding all tests

* fix up tests

* Apply suggestions from code review

* lint

* imports

* lint

* Update beacon-chain/p2p/service.go

* Update shared/cmd/flags.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Updates for remote keymanager (#5260)
* Merge branch 'spec-v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge remote-tracking branch 'origin' into v0.11
* Update to slash by slot instead of epoch (#5297)

* change to slash by slot instead of epoch

* gaz

* fix test

* fix test

* fix infinite loop on error parse
* Sync with master
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Update proposer protection to v0.11 (#5292)

* Complete most of changes

* Fix other tests

* Test progress

* Tests

* Finish tests

* update pbs

* Fix mocked tests

* Gazelle

* pt 2

* Fix

* Fixes

* Fix tests wit hwrong copying
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Implement `SubscribeCommitteeSubnet` method (#5299)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append
* Merge refs/heads/master into v0.11
* resolve confs
* Validator subscribe subnet to next epoch (#5312)

* Alert to subscribe to next epoch

* Fixed tests

* Comments

* Fixed tests

* Update validator/client/validator.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)

This reverts commit 47e5a2cf96f5add151bf135a5352c2dad7922615.
* Aggregate on demand for v0.11 (#5302)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Wait 1/3 on validator side

* Lint

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Fixed duplication

* Tests

* Skip e2e

* Update beacon-chain/rpc/validator/aggregator.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Refactor Dynamic Subscriptions (#5318)

* clean up

* comment

* metrics

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Fix listindexed attestations and detect historic attestations (#5321)

* fix list indexed attestations

* fix tests

* goimports

* names
* Add check for slot == 0 (#5322)
* Change attester protection to return default if DB is empty (#5323)

* Change how default values are set

* Remove unused imports

* Remove wasteful db call

* Fix db tests

* Fix db test
* Merge refs/heads/master into v0.11
* fix it (#5326)
* V0.11 run time fixes to use interop config (#5324)

* Started testing
* Bunch of fixes
* use-interop
* Sync with v0.11
* Conflict
* Uncomment wait for activation
* Move pending block queue from subscriber to validator pipeline
* Merge branch 'v0.11' into use-interop-config
* passing tests
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Nil Checks in Process Attestation v0.11 (#5331)

* Started testing

* Bunch of fixes

* use-interop

* Sync with v0.11

* Uncomment wait for activation

* Move pending block queue from subscriber to validator pipeline

* passing tests

* nil checks to prevent panics

* lint

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Validator batch subscribe subnets (#5332)

* Update both beacon node and validator

* Comments

* Tests

* Lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Validator smarter subscribe (#5334)
* Fix incorrect proposer index calculation (#5336)

* Use correct parent state

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* enhance error
* enhance error
* Update P2P Service to Handle Local Metadata (#5319)

* add metadata to ENR

* add new methods

* glue everything

* fix all tests and refs

* add tests

* add more tests

* Apply suggestions from code review

* fix method

* raul's review

* gaz

* fix test setup

* fix all tests

* better naming

* fix broken test

* validate nil

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Revert "Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)" (#5343)

This reverts commit e5aef1686e582fc2077767c42187c8527f3a742f.
* Wait for Genesis Event to Start P2P (#5303)

* use event feed for state initialized events

* add in handler for tests

* wait till genesis for p2p

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Avoid duplicated aggregation request (#5346)

* Avoid duplicated aggregation request

* Test and lock

* Gaz
* Fix Validate For Metadata (#5348)

* return true

* shay's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Multiple Proposer Slots Allowed Per Epoch for Validators (#5344)

* allow multiple proposer slots

* multi propose

* proposer indices to slots map

* remove deprecated comm assign

* Apply suggestions from code review

* resolve broken tests, add logic in validator client

* fix val tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Networking Fixes (#5349)

* close stream later

* add ping method

* add method

* lint
* More efficient aggregation on demand (#5354)
* Return Nil Error if Pre-Genesis in P2P Service Healthz Check (#5355)

* pregenesis healthz check:

* optimal

* right order

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* no comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Release DiscoveryV5 for Testnet Restart (#5357)

* release discv5

* fix build
* Fix Overflow in Status Check (#5361)

* fix overflow

* Apply suggestions from code review
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* fix after merge
* Merge refs/heads/master into v0.11
* Make Mainnet Config Default, No More Demo Config  (#5367)

* bye bye demo config

* gaz

* fix usage

* fix dep

* gaz

* Update default balance for sendDeposits

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Use FastSSZ Marshal/Unmarshal for DB Encodings in v0.11.1 (#5351)

* try

* use marshaler structure for db instead of proto

* white list types

* attempt

* revert

* testutil.NewBeaconState()

* Fully populate fields for round trip ssz marshal

* fix //beacon-chain/db/kv:go_default_test

* more passing tests

* another test target passed

* fixed stategen

* blockchain tests green

* passing sync

* more targets fixed

* more test fixes in rpc/validator

* most rpc val

* validators test fixes

* skip round robin old

* aggregate test

* whitelist done

* Update beacon-chain/rpc/validator/attester_test.go

* edit baz

* Fixed tests

* Fixed getblock test

* Add back init

* reduce test size

* fix broken build

* tests pass

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Reconnect slasher streams on beacon node shutdown (#5376)

* restart streams on beacon node shutdown

* fix comment

* remove export

* ivan feedback

* ivan feedback

* case insensitive

* Update slasher/beaconclient/receivers.go

* raul feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' into v0.11
* Merge refs/heads/master into v0.11
* Amend Faucet to Offer 32.5 ETH for v0.11 (#5378)

* deposit amount in faucet

* fix eth amount

* gas cost
* unskip exec transition test
* Revert "Enable NOISE Handshake by Default v0.11 (#5272)" (#5381)

This reverts commit a8d32d504a8f923cdf7fa9dfc2684f8804fbab92.
* Merge refs/heads/master into v0.11
* use string for deposit flag
* Update Bootnode to v0.11 (#5387)

* fix bootnode

* add changes

* gaz

* fix docker
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* build fix
* fix flaky test
* Merge refs/heads/master into v0.11
* Unskip E2E for V0.11 (#5386)

* Begin work on fixing e2e for v0.11

* Start bootnode work

* Begin implementing bootnode into e2e

* Fix E2E for v0.11

* Remove extra

* gaz

* Remove unused key gen code

* Remove trailing multiaddr code

* add skip for slashing

* Fix slashing e2e

* Fix docker image build
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.11
* Merge refs/heads/master into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* Update beacon-chain/p2p/broadcaster_test.go
* Merge refs/heads/master into v0.11
* Pass E2E Tests for v0.11 and Enable Attestation Subnets By Default (#5407)
* Update README.md

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/p2p/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/keystore/deposit_input.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update tools/faucet/server.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update proto/beacon/p2p/v1/BUILD.bazel

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/bls/spectest/aggregate_verify_test.go
* Addressed feedback. All test passing
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Update beacon-chain/core/blocks/block_operations_fuzz_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update shared/testutil/helpers.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/helpers/signing_root.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Resolve Misc v0.11 Items (Raul) (#5414)

* address all comments

* set faucet

* nishant feedback

* Update beacon-chain/p2p/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert keymanager changes (#5416)

* Revert "Updates for remote keymanager (#5260)"

This reverts commit bbcd895db50ce5e7c0ecb64210471cf56f63b373.

* Revert "Remove keystore keymanager from validator (#5236)"

This reverts commit 46008770c162e741251e13772fd7356b43a9af87.

* Revert "Update eth2 wallet keymanager (#4984)"

This reverts commit 7f7ef43f218598a671aaeb327342d7e5130fe8b1.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update BLS and limit visibility (#5415)

* remove duplicated BLS, add golang.org/x/mod

* Update BLS and restrict visibility

* fix build
* Fix eth1data test and fix order of ops (#5413)
* use multiaddr builder (#5419)
* Unskip benchutil and minor v0.11 fixes (#5417)

* Unskip benchutil tests

* Remove protos and gaz

* Fixes
* Networking Fixes (#5421)

* check

* fix test

* fix size

* fix test

* more fixes

* fix test again
* Update ethereum APIs with latest master
* Error handling for v0.11 tests (#5428)

* Proper err handling for tests

* Lint

* Fixed rest of the tests

* Gaz

* Fixed old master tests
* Sync with master
* Rm old aggregate_test.go
2020-04-14 20:27:03 +00:00
Victor Farazdagi
748d513c62
proper error checking and type assertions (#5424)
* proper error checking and type assertions
2020-04-14 16:41:09 +00:00
Preston Van Loon
d5ddd012bc
Enforce error handling and checking type assertions (#5403)
* Enforce error handling and checking type assertions
* Reference issue #5404 in the TODO message
* doc description
* Merge branch 'master' into errcheck
* fix tests and address @nisdas feedbacK
* gaz
* fix docker image
2020-04-13 04:11:09 +00:00
Preston Van Loon
7a5010ecea
Add roughtime static code analysis and fix all violations (#5370)
* Add roughtime static code analysis and fix all violations
* Merge branch 'master' into enforce-roughtime
2020-04-09 23:35:42 +00:00
Chris Hobcroft
b7afc90266
Fixed typo in stdout log (#5317)
"Round robin**g** sync request failed"

changed to

"Round robin sync request failed"
2020-04-06 20:34:24 +08:00
Victor Farazdagi
f440c815f9
Init sync update highest slot (#5298)
* updates highest slot before wrapping up
* more verbose error message
* error w/o stack
* revert back
2020-04-04 17:11:38 +03:00
Preston Van Loon
7bdd1355b8
Add maligned struct static check (#5296)
* Add maligned static check
* Add file, oops
* lint
2020-04-03 05:09:15 +00:00
terence tsao
3544ed2818
Invert init-sync-batch-save-blocks flag for v0.11 (#5293) 2020-04-02 14:46:14 -07:00
Victor Farazdagi
c26a492225
Init sync optimizations (#5284)
* fix naming slot -> epoch
* better handling of long periods w/o finality
* fixes issue with pointer going too far ahead
2020-04-02 06:54:05 +03:00
Raul Jordan
f385a1dea6
Release Skip Slot Cache to All (#5280)
* no more skip slot cache

* imports

* deprecated

* fix flakeyness

* disable in e2e

* build

* fix viz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-01 19:09:54 -07:00
terence tsao
7f7866ff2a
Micro optimizations on new-state-mgmt service for initial syncing (#5241)
* Starting a quick PoC

* Rate limit to one epoch worth of blocks in memory

* Proof of concept working

* Quick comment out

* Save previous finalized checkpoint

* Test

* Minor fixes

* More run time fixes

* Remove panic

* Feature flag

* Removed unused methods

* Fixed tests

* E2e test

* comment

* Compatible with current initial sync

* Starting

* New cache

* Cache getters and setters

* It should be part of state gen

* Need to use cache for DB

* Don't have to use finalized state

* Rm unused file

* some changes to memory mgmt when using mempool

* More run time fixes

* Can sync to head

* Feedback

* Revert "some changes to memory mgmt when using mempool"

This reverts commit f5b3e7ff4714fef9f0397007f519a45fa259ad24.

* Fixed sync tests

* Fixed existing tests

* Test for state summary getter

* Gaz

* Fix kafka passthrough

* Fixed inputs

* Gaz

* Fixed build

* Fixed visibility

* Trying without the ignore

* Didn't work..

* Fix kafka

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-03-30 17:10:45 -05:00
terence tsao
c5f186d56f
Batch save blocks for initial sync. 80% faster BPS (#5215)
* Starting a quick PoC
* Rate limit to one epoch worth of blocks in memory
* Proof of concept working
* Quick comment out
* Save previous finalized checkpoint
* Merge branch 'master' of github.com:prysmaticlabs/prysm into batch-save
* Test
* Merge branch 'prev-finalized-getter' into batch-save
* Minor fixes
* Use a map
* More run time fixes
* Remove panic
* Feature flag
* Removed unused methods
* Fixed tests
* E2e test
* Merge branch 'master' into batch-save
* comment
* Merge branch 'master' into batch-save
* Compatible with current initial sync
* Merge branch 'batch-save' of github.com:prysmaticlabs/prysm into batch-save
* Merge refs/heads/master into batch-save
* Merge refs/heads/master into batch-save
* Merge refs/heads/master into batch-save
* Merge branch 'master' of github.com:prysmaticlabs/prysm into batch-save
* Feedback
* Merge branch 'batch-save' of github.com:prysmaticlabs/prysm into batch-save
* Merge refs/heads/master into batch-save
2020-03-30 18:04:10 +00:00
Preston Van Loon
a24546152b
HashProto: Use fastssz when available (#5218)
* Use fastssz when available
* fix tests
* fix most tests
* Merge branch 'master' into faster-hash-proto
* Merge refs/heads/master into faster-hash-proto
* Merge refs/heads/master into faster-hash-proto
* Merge refs/heads/master into faster-hash-proto
* fix last test
* Merge branch 'faster-hash-proto' of github.com:prysmaticlabs/prysm into faster-hash-proto-2
* lint
* fix last test
* fix again
* Update beacon-chain/cache/checkpoint_state_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Merge refs/heads/master into faster-hash-proto
2020-03-28 18:32:11 +00:00
Victor Farazdagi
7ebb3c1784
init-sync revamp (#5148)
* fix issue with rate limiting
* force fetcher to wait for minimum peers
* adds backoff interval
* cap the max blocks requested from a peer
* queue rewritten
* adds docs to fsm
* fix visibility
* updates fsm
* fsm tests added
* optimizes queue resource allocations
* removes debug log
* replace auto-fixed comment
* fixes typo
* better handling of evil peers
* fixes test
* minor fixes to fsm
* better interface for findEpochState func
2020-03-27 09:54:57 +03:00
Preston Van Loon
c4c9a8465a
Faster hashing for attestation pool (#5217)
* use faster hash proto
* Merge branch 'master' into faster-att-pool
* gaz
* Merge branch 'faster-att-pool' of github.com:prysmaticlabs/prysm into faster-att-pool
* nil checks and failing tests
* Merge refs/heads/master into faster-att-pool
* Merge refs/heads/master into faster-att-pool
* Merge refs/heads/master into faster-att-pool
* Merge refs/heads/master into faster-att-pool
* Merge refs/heads/master into faster-att-pool
* Fix
* Merge branch 'faster-att-pool' of github.com:prysmaticlabs/prysm into faster-att-pool
* Fix tests
2020-03-26 23:55:25 +00:00
Preston Van Loon
35d74981a0
Correctly return attestation data for late requests (#5183)
* Add functionality to support attestation requests that are older than the head state

* lint

* gaz

* Handle nil state case

* handle underflow of first epoch

* Remove redundant and possibly wrong genesisTime struct field

* fix remaining tests

* gofmt

* remove debug comment

* use stategen.StateByRoot interchangably with beaconDB.State

* gofmt

* goimports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-23 21:30:28 -07:00
terence tsao
2e0158d7c5
Add HighestSlotBlock Getter for db (#5182)
* Starting

* Update block getters in db

* New test

* New test for save blocks as well

* Delete blocks can clear bits tests

* Fmt
2020-03-23 18:42:41 -05:00
Nishant Das
b0128ad894
Add Attestation Subnet Bitfield (#4989)
* bump bitfield dep

* add new methods

* get it working

* add nil check

* add check

* one more check

* add flag

* everything works local run

* add debug log

* more changes

* ensuring p2p interface works enough for tests to pass

* all tests pass

* include proper naming and comments to fix lint

* Apply suggestions from code review

* discover by peers

* cannot figure out why 0 peers

* remove keys

* fix test

* fix it

* fix again

* remove log

* change back

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-23 09:41:47 -05:00
Preston Van Loon
d06b0e8a86
Refactor attestationutil.AttestingIndices (#5159)
* Refactor AttestingIndices to not return any error. Add tests. Add shortcut for fully attested attestation
* attestationutil.ConvertToIndexed never returned error either
* fix test
* remove shortcut
* revert test...
2020-03-22 00:23:37 +00:00
terence tsao
dc3fb018fe
Fix new state mgmt sync stuck in a loop (#5142) 2020-03-19 18:46:35 -07:00
Preston Van Loon
7c110e54f0
Add ssz marshal and unmarshal for most data structures (#5121)
* Add ssz marshal and unmarshal for most data structures
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Update ferran SSZ
* Update ferran's SSZ
* Merge refs/heads/master into ssz-stuff
* fix tests
* Merge branch 'ssz-stuff' of github.com:prysmaticlabs/prysm into ssz-stuff
* gaz
2020-03-19 02:39:23 +00:00
Raul Jordan
3043d4722f
Attestation Dynamic Committee Subnets (#5123)
* initiate cache
* imports fix
* add in feature config flag
* utilize a dynamic set of subnets
* Merge branch 'master' into att-subnets
* add in feature config flag
* Merge branch 'att-subnets' of github.com:prysmaticlabs/prysm into att-subnets
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into att-subnets
* shift
* more changes
* gaz
* Update beacon-chain/rpc/validator/assignments.go
* Update beacon-chain/rpc/validator/assignments.go
* add flag
* Merge branch 'att-subnets' of https://github.com/prysmaticlabs/geth-sharding into att-subnets
* Merge branch 'master' into att-subnets
* Merge refs/heads/master into att-subnets
* no double flag
* Merge branch 'att-subnets' of github.com:prysmaticlabs/prysm into att-subnets
* amend committee ids to better name
* gaz
2020-03-18 23:13:37 +00:00
Nishant Das
9f46000dba
change to batch size (#5128)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-18 17:57:20 +08:00
Victor Farazdagi
3ec2a0f9e0
Refactoring of initial sync (#5096)
* implements blocks queue

* refactors updateCounter method

* fixes deadlock on stop w/o start

* refactors updateSchedulerState

* more tests on schduler

* parseFetchResponse tests

* wraps up tests for blocks queue

* eod commit

* fixes data race in round robin

* revamps fetcher

* fixes race conditions + livelocks + deadlocks

* less verbose output

* fixes data race, by isolating critical sections

* minor refactoring: resolves blocking calls

* implements init-sync queue

* udpate fetch/send buffers in blocks fetcher

* blockState enum-like type alias

* refactors common code into releaseTicket()

* better gc

* linter

* minor fix to round robin

* moves original round robin into its own package

* adds enableInitSyncQueue flag

* fixes issue with init-sync service selection

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* initsyncv1 -> initsyncold

* adds span

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-03-17 12:27:18 -05:00
terence tsao
e529f5b1d6
Part 1 of integrating new state mgmt to run time (#5108) 2020-03-16 12:07:07 -07:00
Victor Farazdagi
f18bada8c9
Init sync blocks queue (#5064)
* fixes data race, by isolating critical sections

* minor refactoring: resolves blocking calls

* implements init-sync queue

* udpate fetch/send buffers in blocks fetcher

* blockState enum-like type alias

* refactors common code into releaseTicket()

* better gc

* linter

* Update beacon-chain/sync/initial-sync/blocks_queue.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Update beacon-chain/sync/initial-sync/blocks_queue_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-16 18:21:36 +03:00
terence tsao
3660732f44
Resume new state mgmt (#5102) 2020-03-15 09:47:49 -07:00
Victor Farazdagi
1137403e4b
Init sync pre queue (#5098)
* fixes data race, by isolating critical sections

* minor refactoring: resolves blocking calls

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-14 13:21:07 -05:00
shayzluf
035eaffd9d
handle slashing from p2p (#5047)
* handle slashing from p2p

* gaz

* remove unneeded check

* add tests

* gaz  goimports

* text update

* Apply suggestions from code review

* add proto.equal

* fix test

* add context to call

* fix state bug found by terence

* fix tests add error type handling

* nil checks

* nil head state check

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-13 16:47:27 -05:00
terence tsao
43722e45f4
Save cold state (#5077) 2020-03-12 05:58:06 -07:00
Victor Farazdagi
7b30845c01
fixes races in blocks fetcher (#5068) 2020-03-11 14:21:41 +03:00
prylabs-bulldozer[bot]
e635e5b205
Feature flag to gate prune state upon start up (#5011)
* Added feature flag to gate prune state upon start up
2020-03-05 06:24:59 +00:00
Victor Farazdagi
239efe7410
init sync: adds blocks fetcher service (#4978)
* init sync: adds blocks fetcher service

* init-sync: rework ctx handling

* init-sync: fix long lines

* removes redundant method

* adds buffer to requests channel

* adds jaeger spans

* fixes overly long comment line
2020-03-04 20:19:09 +03:00
shayzluf
59575bcac9
fuzz core/blocks package (#4907)
* fuzz core/blocks package

* gaz goimports

* fix test

* terence feedback

* terence feedback

* add error to domain. halfway through

* adding error to domain

* goimports

* added error handling to test

* error instead of continue

* terence and nishant feedback

* domain error handling

* domain error handling

* handle nil validator in ReadOnlyValidator creation

* goinmports

* [4]byte domain type

* [4]byte domain type

* [4]byte domain type fix tests

* fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-03-03 19:02:14 +05:30
Raul Jordan
22bbed0059
Stream Indexed Attestations RPC Implementation (#4941)
* stream indexed attestations impl
* mock regen
* test for stream indexed
* atts test
* no bls
* gaz
* Merge refs/heads/master into implement-stream-indexed
* use feed for atts instead
* remove unused imports
* Merge refs/heads/master into implement-stream-indexed
* fix tests in beacon
* properly use pointers
* imports
* import
2020-02-26 20:14:22 +00:00
Jim McDonald
deb76f1c15
Fix double period in span name (#4910) 2020-02-19 14:52:44 -06:00
terence tsao
731cc0bd44
Handing pending atts if they dont have the state (#4904) 2020-02-19 06:46:30 -08:00
Nishant Das
44856f9500
Add Unsafe Sync (#4906)
* add unsafe flag

* imports

* Use finalized epoch

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-02-19 00:25:07 -08:00
Aranha
8885d715f2
fixed panic: runtime error: integer divide by zero #4777 (#4823) 2020-02-16 15:13:03 -07:00
Preston Van Loon
f06dfd6108
Secure lock when accessing the map only (#4849)
* Secure lock when accessing the map

* wrong lock

* Remove some deadlocks
2020-02-12 20:33:14 -06:00
Nishant Das
c7698cda1c
Add Pending Attestation Lock fix (#4840)
* add pending lock fix

* use easier locking
2020-02-12 08:58:40 -08:00
Nishant Das
a11f1804a2
lock fix (#4839) 2020-02-11 23:33:15 -08:00
Raul Jordan
c3785e03ba
Add Lock to Processing Pending Attestations (#4833)
* add lock to pending atts
* Merge branch 'master' into add-att-lock
* changed to rlock
* broken build
* Merge refs/heads/master into add-att-lock
* refactor to prevent deadlock
2020-02-11 20:01:33 +00:00
Nishant Das
8c90e38770
Fix Goodbye RPC handler (#4831)
* fix goodbye messages

* fix test

* fix test
2020-02-11 09:08:01 -06:00
Preston Van Loon
4598344918
Clear initial sync state caches after round robin sync (#4817)
* Clear initial sync state caches after round robin sync

* fix test mock

* lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-10 10:39:18 -08:00
terence tsao
8a02003d4b Feature flag to disable head update on attestation basis (#4802) 2020-02-09 11:44:17 -08:00
Preston Van Loon
7e0d0502aa Prevent panic on wrong interface conversion (#4803)
* Prevent panic on wrong interface conversion
* remove import
2020-02-09 08:41:50 +00:00
Preston Van Loon
70cb06d50f
Report unhealthy if we think we are out of sync (#4796)
* Report unhealthy if we think we are out of sync
* gofmt
* Merge refs/heads/master into out-of-sync-unhealthy
2020-02-08 18:50:14 +00:00
Nishant Das
efd27c7b2b
Fix Dynamic Topic Subscriptions (#4767)
* check for sync status

* check for chainstart

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-02-08 06:49:17 +08:00
Nishant Das
f16a71f178
Revert "Update Slices More Efficiently" (#4790)
* Revert "Update Slices More Efficiently (#4789)"

This reverts commit 669e1ea787.
2020-02-07 16:58:36 +00:00
Nishant Das
669e1ea787
Update Slices More Efficiently (#4789)
* better cached states

* lint

* jim's review
2020-02-07 09:06:39 -06:00
Preston Van Loon
34178aff2a
Strict verify attestations in pubsub (#4782)
* Verify attestations before putting them into the pool
* use existing method
* Validate aggregated ones too
* Revert "Validate aggregated ones too"

This reverts commit a55646d131813c3e65e0539ef5b2e5bda19a5e91.
* Merge branch 'master' of github.com:prysmaticlabs/prysm into verify-all-atts
* Add feature flag
* The remaining shared reference fields with conditional copy on write
* Merge branch 'master' into better-copy-2
* Merge branch 'better-copy-2' of github.com:prysmaticlabs/prysm into verify-all-atts
* gaz
* fix build, put into validate
* lint
* Merge branch 'master' of github.com:prysmaticlabs/prysm into verify-all-atts
* why does goland do this to me
* revert unrelated change
* fix tests
* Update shared/featureconfig/config.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Merge refs/heads/master into verify-all-atts
* Update beacon-chain/blockchain/testing/mock.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* gofmt
2020-02-07 03:21:55 +00:00
Raul Jordan
a9d144ad1f
Stream Blocks Functionality for RPC (#4771)
* stream blocks functionality included
* necessary tests for stream blocks and notifier
* gazelle and tests passing
* gazelle and tests passing
* Merge branch 'master' into stream-block
* Update beacon-chain/core/feed/block/events.go
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* naming
* build
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* Merge refs/heads/master into stream-block
* fix up tests
* Merge branch 'stream-block' of github.com:prysmaticlabs/prysm into stream-block
* Merge refs/heads/master into stream-block
* shay comment
* Merge refs/heads/master into stream-block
* Merge branch 'stream-block' of github.com:prysmaticlabs/prysm into stream-block
* Merge refs/heads/master into stream-block
2020-02-06 20:14:38 +00:00
Preston Van Loon
27e7be6279
Add & use FinalizedCheckpointEpoch() to state (#4766)
* Add and use FinalizedCheckpointEpoch() to state
* comment
2020-02-06 03:26:15 +00:00
Jim McDonald
945edb6c8f
Sync to highest possible head given the peers available (#4570)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-02-05 12:44:32 -06:00
Raul Jordan
061960c9e2
Resolve Miscellaneous Bugs in Beacon Node (#4743)
* add in nil check for head block

* fix logic

* unused import
2020-02-04 11:21:02 -06:00
Nishant Das
95b6cca399
add lock (#4739) 2020-02-04 16:31:31 +08:00
terence tsao
8492273fa7
Better log for Requesting block for pending attestation... (#4731)
* Better log
* Use debug
* Merge branch 'master' into req-blk-log
* Merge branch 'master' into req-blk-log
2020-02-03 21:11:21 +00:00