Commit Graph

804 Commits

Author SHA1 Message Date
Radosław Kapka
9b3e1eb643
Implement GetHealth in the node API (#8217)
* Implement GetHealth in the node API

* repair fuzz mock

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-06 20:11:20 +00:00
Radosław Kapka
392e61fbee
Revert changes to sync service status check (#8215) 2021-01-06 11:19:53 +00:00
Radosław Kapka
7135a8542f
Check if initial sync service has been initialized (#8214) 2021-01-06 10:45:22 +00:00
Victor Farazdagi
9d737d60f4
Declare err in loop to limit its scope (#8200)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-05 13:55:23 +00:00
Victor Farazdagi
e772e8c8c2
Init sync: minor style fixes (#8156)
* simplify FSM calls

* improve calculateHeadAndTargetEpochs API

* simplify

* simplify nil assignment
2020-12-18 14:31:15 +00:00
terence tsao
20dede7532
Move state summary cache to DB (#8101) 2020-12-16 08:56:21 -08:00
Nishant Das
508c5fcf2f
More Efficient Validation of Proposer Index (#8107)
* metric

* make it better

* make it better

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-15 16:27:05 +00:00
Victor Farazdagi
0f18867f08
Remove duplicate package imports (#8104)
* remove duplicate imports

* remove unused lookupLimit

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-14 22:22:55 +00:00
Steven Allen
2428880058
Update go-libp2p to 0.12.0 (#8015)
* Update go-libp2p to 0.12.0

go-libp2p 0.12.0 made some significant changes to the stream interfaces around
stream closing:

* Close now closes in both directions and frees the stream. However, unlike
FullClose did, it doesn't _wait_ for the remote peer to respond with an EOF.
* To close for writing, call CloseWrite (like one would on a TCP connection, etc.).

This patch:

* Replaces calls to FullClose with Close where appropriate.
* Replaces calls to Close with CloseWrite where appropriate.
* Removes redundant Close calls.
* Calls Reset to where appropriate to indicate that the request/response was
  aborted. Unlike Close, this will not flush and will not cause the remote peer
  to read an EOF. Instead, the remote peer will read an ErrReset error.
* Ensures we always either close or reset streams. Send wasn't closing the
  stream on some error paths.
* Now that stream closing is async, we explicitly wait for a response when
  "hanging up" on a peer (so we don't hang up before they receive our
  response/goodbye message).

* update bazel

* Gazelle

* revert unintentional bazel workspace change

* appease an overzealous linter

* update to latest

* Refactor encoder

* gazelle

* Gazelle

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-14 17:22:25 +00:00
Potuz
4d1f01aacc
Change block_arrival_latency buckets to exponential format (#8065)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-12-14 16:00:07 +00:00
Victor Farazdagi
b9848dc94f
Remove unused vars in tests (#8103)
* update process_block_test

* service_attester_test

* update service and rpc_status tests

* go fmt
2020-12-13 05:23:13 +00:00
Nishant Das
29804fa572
Refactor Subnet Search (#8048)
* checkpoint progress

* checkpoint

* clean up

* do better

* fix test

* fix

* fix

* preston's review

* fix

* fix

* fix

* add iterator

* go doc

* make it a config parameter

* Apply suggestions from code review

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-11 18:40:56 -08:00
Victor Farazdagi
b51aec6981
Init sync: minor fixes (#8060)
* fixes typo

* simplify code
2020-12-07 09:33:06 +00:00
Victor Farazdagi
be078d6a16
Update TestMain(): do not call os.Exit() explicitly (#8046)
* update workspace

* update testmain
2020-12-04 16:10:07 +00:00
Victor Farazdagi
afbfaedea4
Unskip fixed init-sync service test (#8042) 2020-12-04 11:57:46 +00:00
Victor Farazdagi
647b4cf108
Round robin: half open interval in syncToFinalizedEpoch (#8039) 2020-12-03 21:55:42 +00:00
Victor Farazdagi
7dd0c24fea
Skip bogus init-sync test (#8038) 2020-12-03 20:20:04 +00:00
Victor Farazdagi
3092f75ec2
Init sync: conditional syncing to finalized slot (#7999)
* extract sync methods

* add unit test

* gazelle

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* better set back step

Co-authored-by: Shay Zluf <thezluf@gmail.com>
2020-12-03 14:10:51 +00:00
Victor Farazdagi
5417e8cf31
Init-sync: enable peer scorer by default (#7974)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 00:45:20 +00:00
Victor Farazdagi
c7f7a29d7e
remove redundant start slot (#7991)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-02 20:47:39 +03:00
Shay Zluf
f9c696ed54
Pending block queue lock fixes (#8002)
* Unlock in case of error

* fix comment

* fix comment

* revert one defer

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-12-02 11:50:37 +02:00
Potuz
9a1423d62d
Log with field error instead of err (#7998) 2020-12-01 05:38:42 -08:00
terence tsao
b150acffca
Recover state summary for sync validation (#7994)
* Recover state summary

* Add test

* Fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-29 20:42:45 -06:00
Victor Farazdagi
54a42ce4a8
more robust processing of invalid head slot (#7990) 2020-11-29 15:03:25 -06:00
terence tsao
622ab94465
Move Verified and saved pending attestations to pool to debug (#7928)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-25 22:52:57 +00:00
Nishant Das
6f766ed583
Exit Pending Queue Properly (#7927)
* exit properly

* terence's review

* Update beacon-chain/sync/pending_blocks_queue.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/sync/pending_blocks_queue.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-23 19:34:40 +00:00
Nishant Das
8c3faaa4c7
Add Back Error/Debug Logs (#7922)
* add back logs

* add back string

* Reformat loggings

Co-authored-by: Terence Tsao <terence@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-23 18:29:27 +00:00
terence tsao
2172cd60a6
Update sync loggings (#7914) 2020-11-22 14:31:55 -06:00
Nishant Das
60d99c83eb
fix pending queue (#7899) 2020-11-22 08:42:27 -08:00
Preston Van Loon
1ba747b3c9
Change log from ERROR to DEBUG (#7892) 2020-11-21 23:14:19 -08:00
Fabrice Cheng
34a26740ff
Change attestation signature error to DEBUG level (#7891) 2020-11-22 00:08:51 +00:00
terence tsao
f63e89813d
Remove chain not started error (#7879)
* Remove chain not started error

* Add genesis state not created error
2020-11-21 01:28:55 +00:00
Nishant Das
8a256de2dd
Check Target Root Better (#7837)
* check better

* bring it down

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-19 11:13:54 +00:00
terence tsao
4b6441f626
Pending block queue caching with TTL (#7816)
* Update pending blks queue to ttl one

* Update tests

* Comment

* Gazelle

* Fix fuzz

* More comments

* Fix fuxx import

* Nishant's feedback

* Happy lint

* Return error for len(blks) >= maxBlocksPerSlot

* Ensure proposer time conv

* don't use gcache's default exp time it's 0

* fix TestService_AddPeningBlockToQueueOverMax

* Update beacon-chain/sync/pending_blocks_queue.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* Fix time conversion

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-11-19 05:15:58 +00:00
Nishant Das
eb7ab16f92
Change Back Metadata Error Check (#7852)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-19 04:17:26 +00:00
Victor Farazdagi
095c4d5dd5
Peer status peer scorer (#7480)
* define and enforce minimum scorer interface

* better decoupling of multiple scorers in service

* removes redundant weight

* adds peer_status scorer

* minir re-arrangement

* rely on scorer in peer status service

* gazelle

* updates rpc_status

* fix build

* better interface verifying

* remove unnecessary locks

* mark todo

* simplify service

* remove redundant references

* avoid passing contexts

* remove unused context

* refactor errors to p2p package

* refactor goodbye codes into p2p

* simplify status api

* remove isbad method from peers

* update scoring service

* introduce validation error

* gazelle

* add score

* restore isbad method

* resolve dep cycle

* gazelle

* peer status scorer: test score calculation

* bad responses scorer: bad peer score

* remove redundant type checks

* pass nil config

* add rounding

* test IsBadPeer

* test bad peers list

* more tests

* check validation error on non-existent peer

* max peer slot -> highest peer slot

* remove redundant comment

* combine

* combine

* introduce var

* fix tests

* remove redundant update

* minor fix

* Nishant's suggestion

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-18 15:51:42 +00:00
Victor Farazdagi
7735a083b2
Extract common types from sync (#7843)
* extract common types from sync

* fix tests

* simplify

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-18 04:17:42 +00:00
terence tsao
925fba0570
Validate beacon block in pending queue (#7847) 2020-11-17 13:50:51 -08:00
terence tsao
7c54cfea3f
Hardening unaggregated attestation queue check (#7834)
* Add more checks and tests

* Move VerifyLmdFfgConsistency

* Move VerifyFinalizedConsistency

* Move VerifyFinalizedConsistency higher

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-17 16:31:43 +00:00
terence tsao
ad5151f25d
Hardening aggregated attestation queue check (#7826) 2020-11-17 07:25:18 +00:00
Nishant Das
39817c0586
Add Back Flag to Subscribe to All Subnets (#7836) 2020-11-17 05:25:35 +00:00
terence tsao
71c6164c42
Remove a few old metrics (#7825) 2020-11-16 18:27:41 +00:00
james-rms
758ec96d6d
beacon-chain: fix segfault (#7822)
Observed this segfault running all tests on mater, occurring
in around 2-3 out of 10 test runs.

```
FAIL: //beacon-chain/sync:go_default_test (shard 3 of 4, run 1 of 10) (see /home/j/.cache/bazel/_bazel_j/1ba834ca9d49f27aeb8f0bbb6f28fdf3/execroot/prysm/bazel-out/k8-fastbuild/testlogs/beacon-chain/sync/go_default_test/shard_3_of_4_run_1_of_10/test.log)
INFO: From Testing //beacon-chain/sync:go_default_test (shard 3 of 4, run 1 of 10):
==================== Test output for //beacon-chain/sync:go_default_test (shard 3 of 4, run 1 of 10):
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x138eea6]

goroutine 1660 [running]:
github.com/prysmaticlabs/prysm/shared/abool.(*AtomicBool).IsSet(...)
	shared/abool/abool.go:39
github.com/prysmaticlabs/prysm/beacon-chain/sync.(*Service).subscribeStaticWithSubnets.func1(0xc002dd4400, 0xc002990940, 0x17bca26, 0x1e)
	beacon-chain/sync/subscriber.go:207 +0xe6
created by github.com/prysmaticlabs/prysm/beacon-chain/sync.(*Service).subscribeStaticWithSubnets
	beacon-chain/sync/subscriber.go:200 +0x172
================================================================================
```

TestStaticSubnets was testing a Service with an uninitialized
chainStarted value. This commit initializes chainStarted explicitly
in all tests that construct a Service. This reduces the observed flake
rate to 0/10 runs. This was verified with:

```
./bazel.sh test //beacon-chain/sync:go_default_test --runs_per_test 10
```
2020-11-16 12:10:34 +01:00
Victor Farazdagi
f361450e8d
Update TestMain() to use os.Exit() (#7814)
* update TestMain

* fix sync/initial-sync test

* restore code in rate limiter

* fix rate_limiter tests
2020-11-13 18:28:14 -08:00
Nishant Das
d281ef9c56
Clean Up GoodByes (#7790)
* clean up

* cleanup

* fix

* fix tests

* change

* deepsource

* fix test
2020-11-13 12:58:13 +00:00
Raul Jordan
4290ba416c
Fix Prysm Runtime Data Races (#7770)
* handle state trie data races

* race fixes

* added proper locks

* fix gaz

* use thread-safe refs() function
2020-11-10 20:57:07 -06:00
Victor Farazdagi
09e3f0360e
Remove redundant calls to os.exit() in TestMain (#7761)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-10 14:56:47 +00:00
terence tsao
1b9911ccc3
Batch verify aggregated attestation signatures (#7744)
* First take. Got benchmark numbers

* Remove benchmark test

* Final clean up

* Failing to verify aggregator index should be reject
2020-11-10 00:54:44 +00:00
Victor Farazdagi
51796d77f6
Init-sync: backtracking to alternative forks (#7564)
* cherry-pick from previous commits

* use finalized checpoint instead of head slot

* add helpers

* add dedicated error for missing blocks

* add backtracking options to fetcher

* TestBlocksFetcher_alternativeSlotBefore regression test

* update peer block provider test

* update injected deps

* update round robing deps

* update fetcher

* patch status and stream handlers

* test template

* gazelle

* revert BestNonFinalized changes

* extract waitForBandwidth method

* extend test

* remove redundant code

* add flag

* update queue

* update thresholds

* upd inequality

* minor cleanup

* check for mode

* add backtracking queue test

* merge from master

* remove redundant test

* correct logging level

* fix tests

* remove test template

* Nishant's suggestion to parametrize magic number
2020-11-06 15:54:20 +00:00
Victor Farazdagi
6e21b7a623
Add one more assertion to fetcher test (#7733) 2020-11-05 19:48:49 +00:00