Commit Graph

769 Commits

Author SHA1 Message Date
Victor Farazdagi
095c4d5dd5
Peer status peer scorer (#7480)
* define and enforce minimum scorer interface

* better decoupling of multiple scorers in service

* removes redundant weight

* adds peer_status scorer

* minir re-arrangement

* rely on scorer in peer status service

* gazelle

* updates rpc_status

* fix build

* better interface verifying

* remove unnecessary locks

* mark todo

* simplify service

* remove redundant references

* avoid passing contexts

* remove unused context

* refactor errors to p2p package

* refactor goodbye codes into p2p

* simplify status api

* remove isbad method from peers

* update scoring service

* introduce validation error

* gazelle

* add score

* restore isbad method

* resolve dep cycle

* gazelle

* peer status scorer: test score calculation

* bad responses scorer: bad peer score

* remove redundant type checks

* pass nil config

* add rounding

* test IsBadPeer

* test bad peers list

* more tests

* check validation error on non-existent peer

* max peer slot -> highest peer slot

* remove redundant comment

* combine

* combine

* introduce var

* fix tests

* remove redundant update

* minor fix

* Nishant's suggestion

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-18 15:51:42 +00:00
Victor Farazdagi
7735a083b2
Extract common types from sync (#7843)
* extract common types from sync

* fix tests

* simplify

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-18 04:17:42 +00:00
terence tsao
925fba0570
Validate beacon block in pending queue (#7847) 2020-11-17 13:50:51 -08:00
terence tsao
7c54cfea3f
Hardening unaggregated attestation queue check (#7834)
* Add more checks and tests

* Move VerifyLmdFfgConsistency

* Move VerifyFinalizedConsistency

* Move VerifyFinalizedConsistency higher

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-17 16:31:43 +00:00
terence tsao
ad5151f25d
Hardening aggregated attestation queue check (#7826) 2020-11-17 07:25:18 +00:00
Nishant Das
39817c0586
Add Back Flag to Subscribe to All Subnets (#7836) 2020-11-17 05:25:35 +00:00
terence tsao
71c6164c42
Remove a few old metrics (#7825) 2020-11-16 18:27:41 +00:00
james-rms
758ec96d6d
beacon-chain: fix segfault (#7822)
Observed this segfault running all tests on mater, occurring
in around 2-3 out of 10 test runs.

```
FAIL: //beacon-chain/sync:go_default_test (shard 3 of 4, run 1 of 10) (see /home/j/.cache/bazel/_bazel_j/1ba834ca9d49f27aeb8f0bbb6f28fdf3/execroot/prysm/bazel-out/k8-fastbuild/testlogs/beacon-chain/sync/go_default_test/shard_3_of_4_run_1_of_10/test.log)
INFO: From Testing //beacon-chain/sync:go_default_test (shard 3 of 4, run 1 of 10):
==================== Test output for //beacon-chain/sync:go_default_test (shard 3 of 4, run 1 of 10):
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x138eea6]

goroutine 1660 [running]:
github.com/prysmaticlabs/prysm/shared/abool.(*AtomicBool).IsSet(...)
	shared/abool/abool.go:39
github.com/prysmaticlabs/prysm/beacon-chain/sync.(*Service).subscribeStaticWithSubnets.func1(0xc002dd4400, 0xc002990940, 0x17bca26, 0x1e)
	beacon-chain/sync/subscriber.go:207 +0xe6
created by github.com/prysmaticlabs/prysm/beacon-chain/sync.(*Service).subscribeStaticWithSubnets
	beacon-chain/sync/subscriber.go:200 +0x172
================================================================================
```

TestStaticSubnets was testing a Service with an uninitialized
chainStarted value. This commit initializes chainStarted explicitly
in all tests that construct a Service. This reduces the observed flake
rate to 0/10 runs. This was verified with:

```
./bazel.sh test //beacon-chain/sync:go_default_test --runs_per_test 10
```
2020-11-16 12:10:34 +01:00
Victor Farazdagi
f361450e8d
Update TestMain() to use os.Exit() (#7814)
* update TestMain

* fix sync/initial-sync test

* restore code in rate limiter

* fix rate_limiter tests
2020-11-13 18:28:14 -08:00
Nishant Das
d281ef9c56
Clean Up GoodByes (#7790)
* clean up

* cleanup

* fix

* fix tests

* change

* deepsource

* fix test
2020-11-13 12:58:13 +00:00
Raul Jordan
4290ba416c
Fix Prysm Runtime Data Races (#7770)
* handle state trie data races

* race fixes

* added proper locks

* fix gaz

* use thread-safe refs() function
2020-11-10 20:57:07 -06:00
Victor Farazdagi
09e3f0360e
Remove redundant calls to os.exit() in TestMain (#7761)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-10 14:56:47 +00:00
terence tsao
1b9911ccc3
Batch verify aggregated attestation signatures (#7744)
* First take. Got benchmark numbers

* Remove benchmark test

* Final clean up

* Failing to verify aggregator index should be reject
2020-11-10 00:54:44 +00:00
Victor Farazdagi
51796d77f6
Init-sync: backtracking to alternative forks (#7564)
* cherry-pick from previous commits

* use finalized checpoint instead of head slot

* add helpers

* add dedicated error for missing blocks

* add backtracking options to fetcher

* TestBlocksFetcher_alternativeSlotBefore regression test

* update peer block provider test

* update injected deps

* update round robing deps

* update fetcher

* patch status and stream handlers

* test template

* gazelle

* revert BestNonFinalized changes

* extract waitForBandwidth method

* extend test

* remove redundant code

* add flag

* update queue

* update thresholds

* upd inequality

* minor cleanup

* check for mode

* add backtracking queue test

* merge from master

* remove redundant test

* correct logging level

* fix tests

* remove test template

* Nishant's suggestion to parametrize magic number
2020-11-06 15:54:20 +00:00
Victor Farazdagi
6e21b7a623
Add one more assertion to fetcher test (#7733) 2020-11-05 19:48:49 +00:00
Victor Farazdagi
f6cbfd5e27
Init-sync fetcher: backtracking (#7704)
* fetcher: backtracking

* gazelle

* amend comments

* cleanup

* cleanup

* minor fixes

* simplify

* remove unrelevant test

* add TestBlocksFetcher_findAncestor

* more tests

* Address Nishant's feedback

* optimize backtracking

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-05 07:27:46 +00:00
Nishant Das
bafc7479b0
Add Backoff For Peer Dials (#7726)
* add tests

* Update beacon-chain/p2p/peers/status_test.go

* fix conditional

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-11-05 05:52:49 +00:00
Victor Farazdagi
c5a8363998
Remove redundant error return from filterPeers() (#7730) 2020-11-04 23:29:50 +00:00
Victor Farazdagi
7acd73e1fe
Refactor p2p requests: extract BeaconBlocksByRangeRequest and BeaconBlocksByRootRequest (#7718)
* extract SendBeaconBlocksByRangeRequest

* adds extra processing

* extract SendBeaconBlocksByRootRequest

* extract into rpc_send_request.go

* add tests

* more tests

* gazelle

* remove redundant import

* for consistency

* for consistency

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-04 20:09:19 +00:00
Nishant Das
090b71bec5
clean up better (#7723) 2020-11-04 15:24:29 +08:00
Raul Jordan
5d93ee1843
Use Atomic Booleans in Sync Service (#7682)
* use atomic bool in tests as well

* gaz

* wrapped up tests

* fix test

* shadowing fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-03 19:12:24 +00:00
Victor Farazdagi
926d3b9b34
Init-sync: more tests + minor refactoring (#7692)
* cherry-pick commits

* re-arrange calls
2020-10-31 21:33:57 +00:00
Nishant Das
df762bbfee
cleaner logging (#7689) 2020-10-31 19:10:08 +08:00
Nishant Das
211d9bc0b9
Update BLST And Herumi (#7632)
* fix build from source

* clean up

* update again

* change everything

* workaround for now

* fix versioning

* all passing now

* fix build issues

* clean up

* revert use of MulVerify

* gaz

* stub

* Apply suggestions from code review

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* fix all

* fix test

* todo

* fix stub

* revert back

* make deep source happy

* Update shared/bls/herumi/public_key.go

* Update shared/bls/blst/signature.go

* Update shared/bls/blst/signature_test.go

* imports

* move iface to common, export errors

* rm iface build

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-10-30 19:06:33 +00:00
Victor Farazdagi
4fc0a50569
Turn init-sync FSM logs to trace level (#7670)
* trace

* go fmt

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-29 04:55:07 +00:00
Preston Van Loon
c1c0b53c25
validateCommitteeIndexBeaconAttestation: Reorder boolean logic to reduce db calls (#7671)
* reorder boolean logic to shortcut

* same issue, in another place

* refactor for the greater good
2020-10-29 04:14:16 +00:00
Preston Van Loon
149d3b84fa
Check attestation target checkpoint epoch matches attestation slot (#7667)
* Check attestation target checkpoint is within current or previous epoch

* reject bad att where slot does not match target

* Add test, reduce redundant check
2020-10-29 00:52:18 +00:00
Preston Van Loon
fcbb168c76
Code health: review map usage (#7635)
* remove unused cache states map

* correct typo

* Remove unused array

* Add lock around deposits cache chainstart pubkeys

* Copy attestation before grabbing lock. This may reduce lock contention time as other callers wanting the lock do not need to wait as long for the lock to become available.

* Copy attestation before grabbing lock. This may reduce lock contention time as other callers wanting the lock do not need to wait as long for the lock to become available.

* Set capacity to 1 since it is known that the slice will be 1 after insertion

* require validatorSlashingPreconditionCheck caller to hold lock

* Add lock for voluntary exits pool HasBeenIncluded

* Require rate limiter retrieveCollector to hold lock

* Add lock requirement assertions in sync

* Remove unused struct

* remove ClearCachedStates API

* field initSyncState is unused

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-26 21:17:07 +00:00
terence tsao
ec77196197
Remove checkpoint info cache and usages (#7642)
* Remove checkpoint info cache and usages

* Gazelle
2020-10-26 14:09:19 -05:00
Victor Farazdagi
e6d688f6d5
Init-sync: re-arrange tests (#7641)
* move tests

* formatting

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-26 08:54:06 +00:00
Victor Farazdagi
8a3b75e9e3
Update init-sync FSM (#7640)
* update fsm

* more tests
2020-10-26 05:26:35 +00:00
Radosław Kapka
7e2112b4ba
Fix panic issues on beacon node shutdown (#7625)
* allow service cleanup on node shutdown

* revert error to fatal

* remove unused struct field
2020-10-23 12:43:48 +00:00
Nishant Das
6eb022ffa1
Clean Up Block Retrieval Methods (#7593)
* fix and add tests

* return for 0

* add test case

* fix

* fix fail

* add comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-23 04:23:15 +00:00
Raul Jordan
13af8a7a37
Remove Unusued Base Parameter in SubscribeWithBase (#7608)
* remove base in sub with base param

* refactor subscribeWithBase signature
2020-10-22 16:56:24 +00:00
Nishant Das
7131cd991c
Ignore PreGenesis Messages Instead Of Rejecting Them (#7550)
* ignorePreGenesisMessages

* fix test + handelaar's review

* fix issues

* change back

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 15:43:35 +00:00
Nishant Das
ddbece5988
Fix Range Request Responses (#7531)
* checkpoint

* fix tests

* fix all tests

* imports

* gazelle

* reviews

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-22 09:33:01 -05:00
Preston Van Loon
b4c1c1db9b
Add lighthouse p2p Good Bye codes (#7601)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-22 00:35:10 +00:00
Victor Farazdagi
0b53a89d00
More verbose output on init-sync (#7578)
* more verbose output on init-sync

* log slot

* no error stack

* remove redundant data

* less verbose
2020-10-20 14:59:38 +00:00
terence tsao
9db6c0042b
Add LMD FFG consistency check to aggregated attestation (#7573)
* Add lmd and ffg check

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-19 23:19:44 +00:00
terence tsao
3d70d757a1
Add checking for finalized consistency (#7568)
* Add checking for finalized consistency

* Add tests

* @prestonvanloon's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-19 21:52:18 +00:00
Preston Van Loon
ab40a112c5
Print the underlying error when attestation fails to get pre state (#7567)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-19 17:03:27 +00:00
Nishant Das
b502876e98
Fix Mismatched BlockRoots In State Replay (#7559)
* fix mismatched roots

* alternate approach

* preston's review

* comment
2020-10-18 09:39:27 -07:00
Preston Van Loon
07e7e030d9
Update PubSub and include topic filter (#7496)
* Update pubsub and fix topicIDs

* WIP filter

* Add suggested code from @bidlocode

* add tests and fix bugs

* more tests

* Wait until state initialized to accept pubsub filtering

* rename for clarity and clarify comment

* fix test builds

* Autofix issues in 2 files

Resolved issues in the following files via DeepSource Autofix:
1. beacon-chain/p2p/pubsub_filter.go
2. beacon-chain/p2p/pubsub_filter_test.go

* @nisdas pr feedback

* pr feedback and fuzz fix

* Update beacon-chain/p2p/pubsub_filter.go

* Must have protocol suffix

* Must have protocol suffix

* gofmt

* rm test, fix panic

* Fix tests

* Add isInitialized check

* Add a few more tests for better coverage

* cache fork digest, make pubsub filter part of the p2p service

* rename service

* gofmt

* Add comment

* fix

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-16 07:05:40 +00:00
Nishant Das
0b64a335c8
check chainstart (#7494)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-15 22:03:48 +08:00
terence tsao
7076a1ec4a
More feature flag deletions (#7533)
* Delete disable state lock and init sync verbose flags

* Delete disable slashing broadcast

* Remove disable wait for sync, noise, eth1 cache, static subnet

* Remove enable broadcast recovery attemp and make it as default

* Remove disable head update on per attestation

* Revert disable att braodcast discovery attempt

* gazelle

* Fixed an anti pattern

* Add enableAttBroadcastDiscoveryAttempts back

* Add back WaitForSync

* Remove extra lines

* Use DisableDynamicCommitteeSubnets path per @prestonvanloon feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 23:28:49 +00:00
Ivan Martinez
803d7c9bd2
Add functionality to retrieve all pending items from pools (#7530)
* Allow slashing pools to retrieve all items

* Add functionality to exits too to retrieve all exits

* Rename to noLimit

* ndo err

* Fix tests

* Fix test

* Fix test again

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-14 21:08:24 +00:00
Victor Farazdagi
a005c77b3f
Update error usage patterns to go1.13+ (#7529)
* rely on errors.Is

* wrap errors
2020-10-14 17:39:52 +00:00
terence tsao
8f04c555fc
More feature flags removal (#7526)
* Remove disable domain cache

* Remove don't verify att sig flag

* Remove batch verify and attester copies

* Remove batch verify usage

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 16:18:40 +00:00
Nishant Das
022b6667e5
Use Custom SSZ for P2P Types (#7436)
* checkpoint progress

* add roundtrip tests

* change all

* remove error response

* clean up

* Update beacon-chain/sync/error_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* gaz

* fix tests

* fmt

* gaz

* change back

* fix again

* clean up

* deep source

* fix all tests

* add gaz

* fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-14 07:55:28 +00:00
Victor Farazdagi
b9844024b4
Update comments describing init-sync process (#7521)
* updates comments

* fetcher mode from config

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 01:44:46 +00:00