Commit Graph

59 Commits

Author SHA1 Message Date
Radosław Kapka
2b6e86ec1b
Some test improvements (#10928)
* extract DeterministicGenesisStateWithGenesisBlock

(cherry picked from commit a5e3a9c9bbbacb23a644f5c68c92839a315f66a1)

# Conflicts:
#	testing/util/state.go

* part 1

* part 2

* part 3

* fix errors

* db interface public visibility

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-24 17:22:39 +00:00
terencechain
61cbe3709b
Ignore subset aggregates (#10674)
* Ignore subset aggregates

* Add test

* Update BUILD.bazel

* Update validate_sync_contribution_proof_test.go

* Update validate_sync_contribution_proof_test.go

* Don't utilize pooled objects. Use direct caches

* Handle mainnet/minimal better

* Revert att changes

* Check overlaps before set

* Feedbacks

* Fixed a copy bug

* Fixed the same copy bug in committee indices cache

* Use SafeCopyBytes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-25 05:40:06 +00:00
terencechain
03ae8672b6
Refactor and use has_block getter (#10592)
* Refactor and use has block getter

* Update blocks_fetcher_utils.go

* Fix tests

* Fix tests

* Fix test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-04 00:17:40 +00:00
Raul Jordan
16bbf5602f
Move Consensus Type Wrappers Into Consensus Types Package (#10598)
* builds

* move block to consensus-types

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-02 15:43:40 +00:00
Nishant Das
7c3d89b25f
Graduate Batch Gossip Verification Feature (#10553)
* grad feat

* fix stuck tests
2022-04-26 08:28:35 -04:00
Preston Van Loon
b697463da9
wrapper: Remove deprecated fork specific wrapper methods for signed beacon blocks (#10369)
* Remove deprecated wrappers, lots of refactoring

* Revert proto/prysm/v1alpha1/validator.proto

* fix tests

* fix test

* fix conversion in e2e

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-25 23:00:44 +00:00
Mohamed Zahoor
a103dd91c0
cherry picked PR for #10274 (#10300)
* ignore topic messages (except block topic) during optimistic sync

* address review comments

* nit pick fix

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-03-06 08:09:12 +00:00
Radosław Kapka
c5c039fd6b
Unify GenesisValidator(s)Root throughout the codebase (#10230)
* Unify `GenesisValidator(s)Root` throughout the codebase

* comments and literals

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-02-14 13:34:38 +00:00
terence tsao
4bf935928b
Use field params root length (#10012)
* Use config length 32

* Add root length to minimal

* Fix build

* Gaz
2021-12-14 18:42:05 +00:00
Raul Jordan
233171d17c
[Service Config Revamp] - Sync Service With Functional Options (#9859)
* sync config refactor

* rem

* rem

* testing

* gaz

* next

* fuzz

* build

* fuzz

* rev

* log

* cfg
2021-11-05 19:08:58 +00:00
Raul Jordan
df33ce3309
Remaining Slasher Beacon Node Changes (#9701)
* slasher beacon node changes

* remaining beacon node items

* moar changes

* gaz

* flag fix

* rem slashable

* builds

* imports

* fix up

* pruning faster test

* deepsource

* fix wrong item

* node node feature flags

* broken test

* preston review

* more preston comments

* comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-29 18:17:37 +00:00
Preston Van Loon
ee0a453b7b
core: refactor signing and domain methods from helper to core/signing pkg (#9520)
* Move domain function and all signing root functions from beacon-chain/core/helpers to beacon-chain/core

* @terencechain suggestion to put these methods under core/signing
2021-09-27 16:19:20 +00:00
terence tsao
376d248c22
Add in progress handler to committee cache (#9664)
* Add in progress handler for committee cache

* Remove debug print

* Update validators.go

* Fix all the tests

* More tests

* Update committee_disabled.go

* Update committee_disabled.go

* Update testing util

* Update main.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-09-26 23:27:57 +08:00
Nishant Das
6e4c2b4b20
Wrap Gossip Validators With Error (#9660) 2021-09-25 18:06:48 -07:00
Raul Jordan
a9a4bb9163
Move Shared/Testutil into Testing (#9659)
* move testutil

* util pkg

* build

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 18:53:46 +00:00
Raul Jordan
29513c804c
Create Encoding Bytesutil (#9658)
* bytesutil

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 15:23:37 +00:00
Raul Jordan
f3d6dbcc1e
Move Shared/Params Into Config/Params (#9642)
* config params into pkg

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-21 19:59:25 +00:00
terence tsao
77de467250
add lru (#9592) 2021-09-14 18:11:25 -05:00
Nishant Das
ee28dc3d4f
Add in Fork Digest For Subscribers (#9538)
* add in digest

* radek's review

* raul's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-09-07 16:58:24 +00:00
Marcin Górzyński
02966e64d8
Feature lru cache wrapper 2 (#9511)
* Add Wrapper to LRU Cache to handle Invalid Parameters #9461

* Regenerate BUILD.bazel and simplify tests using lru.Cache

* Fix: fuzz_exports.go build error

* Fix: block_fuzz.go

* Revert lru.Cache interface

* Remove redundant err check in pending_attestations_queue_test.go

* Add tests for lru wrapper

* Use lru package in prysm/shared instead of lruwrpr

* Fix: goimports

* Fix: BUILD.bazel

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-09-02 18:36:54 +08:00
terence tsao
1284496648
Increase seen attestation cache sizes (#9476)
* Increase attestation cache sizes

* Update WORKSPACE

* Go fmt

* Fix tests

* Go fmt

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-26 21:37:35 +00:00
Raul Jordan
6dadb80cc4
Move v1alpha1 into Prysm API namespace (#9245) 2021-07-21 21:34:07 +00:00
terence tsao
b22935a724
Rename AttestationNotifier (#9197) 2021-07-14 18:32:03 -07:00
terence tsao
1d3a9983cc
Move block interface next to generated pb (#9146)
* Move block interface next to pb

* Update fuzz build bazel

* Move interface to /proto/interface and wrapper next to generated pb

* Fix fuzz build bazel

* Add //proto/eth/v1alpha1/wrapper

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-06 15:34:05 +00:00
Raul Jordan
5aac06f04e
Move EthereumAPIs Into Prysm (#8968)
* begin move

* use same import path

* imports

* regen protos

* regen

* no rename

* generate ssz

* gaz

* fmt

* edit build file

* imports

* modify

* remove generated files

* remove protos

* edit imports in prysm

* beacon chain all builds

* edit script

* add generated pbs

* add replace rules

* license for ethereumapis protos

* change visibility

* fmt

* update build files to gaz ignore

* use proper form

* edit imports

* wrap block

* revert scripts

* revert go mod
2021-06-02 18:49:52 -05:00
terence tsao
638e76cbd7
Sort imports (#8959) 2021-05-31 20:47:30 +02:00
terence tsao
276d03553c
Block interface clean up (#8947) 2021-05-26 11:33:46 -07:00
Nishant Das
caf9bdbc6f
Use Block Interface Across Prysm (#8918)
* commit initial work

* checkpoint current work

* gaz

* checkpoint

* req/resp changes

* initial-sync

* finally works

* fix error

* fix bugs

* fix issue

* fix issues

* fix refs

* tests

* more text fixes

* more text fixes

* more text fixes

* fix tests

* fix tests

* tests

* finally fix builds

* finally

* comments

* fix fuzz

* share common library

* fix

* fix

* add in more defensive nil checks

* add in more defensive nil checks

* imports

* Apply suggestions from code review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Apply suggestions from code review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_interface.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_wrapper.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_interface.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* imports

* fix bad changes

* fix

* terence's review

* terence's review

* fmt

* Update beacon-chain/rpc/beacon/blocks.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fix tests

* fix

* fix all tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-05-26 16:19:54 +00:00
kevlu93
eca67cec4c
Embed Config Pattern for Sync Services (#8636)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-03-21 19:07:42 +00:00
terence tsao
aa389c82a1
Send attestations over feed at right place (#8605) 2021-03-15 07:47:10 -07:00
terence tsao
20dede7532
Move state summary cache to DB (#8101) 2020-12-16 08:56:21 -08:00
terence tsao
ec77196197
Remove checkpoint info cache and usages (#7642)
* Remove checkpoint info cache and usages

* Gazelle
2020-10-26 14:09:19 -05:00
terence tsao
3d70d757a1
Add checking for finalized consistency (#7568)
* Add checking for finalized consistency

* Add tests

* @prestonvanloon's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-19 21:52:18 +00:00
Preston Van Loon
07e7e030d9
Update PubSub and include topic filter (#7496)
* Update pubsub and fix topicIDs

* WIP filter

* Add suggested code from @bidlocode

* add tests and fix bugs

* more tests

* Wait until state initialized to accept pubsub filtering

* rename for clarity and clarify comment

* fix test builds

* Autofix issues in 2 files

Resolved issues in the following files via DeepSource Autofix:
1. beacon-chain/p2p/pubsub_filter.go
2. beacon-chain/p2p/pubsub_filter_test.go

* @nisdas pr feedback

* pr feedback and fuzz fix

* Update beacon-chain/p2p/pubsub_filter.go

* Must have protocol suffix

* Must have protocol suffix

* gofmt

* rm test, fix panic

* Fix tests

* Add isInitialized check

* Add a few more tests for better coverage

* cache fork digest, make pubsub filter part of the p2p service

* rename service

* gofmt

* Add comment

* fix

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-16 07:05:40 +00:00
terence tsao
3734bfacce
Add target epoch and slot consistency check (#7288)
* Add target epoch and slot consistency
* Remove debug logs
* Merge refs/heads/master into att-epoch-consistency
* Merge refs/heads/master into att-epoch-consistency
* Merge refs/heads/master into att-epoch-consistency
2020-09-22 05:13:12 +00:00
terence tsao
b928e9531c
Two more gossip validations for unaggregated atts subnet (#7278)
* Add two more gossip validations for unaggregated attestations

* Removed a typo log

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-20 08:31:10 +08:00
Preston Van Loon
b7175b3482
Update fastssz: Attempt 2 (#7115)
* Revert "Revert "Update fastssz" (#7100)"

This reverts commit b954db9704.
* Preston's patch
* Merge branch 'master' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* Update fssz, add regression test case
* more HTR with fssz
* fix some tests
* only one test left
* Make it so that HTR will work
* gofmt, imports
* gofmt, imports
* fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* fix
* Merge branch 'master' into revert-7100-revert-6760-update-fssz
* Merge refs/heads/master into revert-7100-revert-6760-update-fssz
* gaz
* Merge branch 'revert-7100-revert-6760-update-fssz' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* Merge refs/heads/master into revert-7100-revert-6760-update-fssz
* fix test
* Merge branch 'revert-7100-revert-6760-update-fssz' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* Merge refs/heads/master into revert-7100-revert-6760-update-fssz
2020-08-27 18:13:32 +00:00
terence tsao
c764099231
Use checkpoint info cache (#7098)
* Use checkpoint cache

* Add it to dev

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-24 17:11:45 -05:00
terence tsao
b954db9704
Revert "Update fastssz" (#7100)
* Revert "Update fastssz (#6760)"

This reverts commit 78a25f99c3.
* Merge refs/heads/master into revert-6760-update-fssz
2020-08-24 20:06:28 +00:00
Preston Van Loon
78a25f99c3
Update fastssz (#6760)
* Update fastssz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fmt
* gaz
* Merge refs/heads/master into update-fssz
* goimports
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Fix
* fix ethereumapis
* fix again
* kafka
* fix gen file
* fix compute signing root
* gofmt
* checkpoint progress
* progress
* checkpoint
* progress
* Fix build
* checkpoint
* helpers
* Another test fixed
* gaz
* another test fix
* gofmt
* some fixes
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fix one test
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fill empty checkpoint roots
* more padding
* more padding
* Fix //beacon-chain/rpc/debug:go_default_test
* fix //beacon-chain/core/state:go_default_test
* fix //beacon-chain/core/state:go_default_test
* fix some htr errors
* fix //slasher/rpc:go_default_test
* Progress on //beacon-chain/core/blocks:go_default_test
* Progress on //beacon-chain/core/blocks:go_default_test
* Progress on //beacon-chain/core/blocks:go_default_test
* fix //slasher/db/kv:go_default_test
* progress
* fix //beacon-chain/sync/initial-sync:go_raceon_test
* gofmt and gaz
* fix one more test, taking a break
* Fix //beacon-chain/core/blocks:go_default_test
* Complete beacon-chain/powchain
* Do most of beacon-chain/rpc/beacon/
* Do most of beacon-chain/blockchain
* fix //beacon-chain/operations/attestations/kv:go_default_test
* Fix //beacon-chain/cache/depositcache:go_default_test
* Fix //slasher/detection:go_default_test
* Progress
* fix //beacon-chain/rpc/validator:go_default_test
* gofmt
* fix //validator/client:go_default_test
* fix
* fix //beacon-chain/blockchain:go_raceoff_test
* fix //beacon-chain/rpc/beacon:go_default_test
* fix 1 of 4 shards in //beacon-chain/sync:go_default_test
* Fix //beacon-chain/sync:go_default_test and gofmt
* prevent panic
* fix //beacon-chain/state/stategen:go_default_test
* fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fix most tests
* Self review, go mod tidy, run regen scripts
* fix slasher
* Update ethereumapis
* disable spawn strategy override
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Remove extra line in imports
* Remove extra line in imports
* Gofmt
* PR feedback from @nisdas
2020-08-24 01:46:17 +00:00
Preston Van Loon
4d463c4a85
Don't ban blocks for context deadlines (#7040)
* Don't ban blocks for context deadlines
* Don't ban blocks for context deadlines
* Add test
* Merge branch 'master' into dont-ban-ctx-deadline2
* fmt
2020-08-18 01:21:10 +00:00
Nishant Das
664349e553
Account for Bad Blocks in Gossip (#6726)
* add bad block cache

* terence's review

* terence's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-27 19:57:19 +08:00
Victor Farazdagi
c2615168d9
Applies assertion funcs to sync tests (#6603)
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* gazelle
* Merge branch 'master' into sync-apply-testutils-assertions
* gazelle
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* applies assertion funcs to sync/initial-sync tests
* Merge branch 'master' into sync-apply-testutils-assertions
2020-07-15 04:41:11 +00:00
terence tsao
a4cfd147ff
Address spec adherence feedbacks (#6365)
* Addressed 1
* Address 2.
* Addressed 3. and 4.
* Addressed 6.
* Addressed 7
* Addressed 8
* Addressed 9.
* Addressed 10.
* Addressed 11.
* Addressed 12.
* Addressed 13.
* Delete old test
* Merge refs/heads/master into spec-feedbacks
* Change comment "pure" to "stateless"
* Fmt
* Typo
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
* Merge refs/heads/master into spec-feedbacks
2020-06-24 03:14:44 +00:00
Preston Van Loon
f9038674d1
Refactor setupDB to return the state summary cache used by the database (#6369) 2020-06-23 13:40:55 -07:00
Nishant Das
d1a1043ef1
Update to v0.12 (#5614) 2020-06-09 15:40:48 -07:00
terence tsao
0b2b77c5b0
Remove validate_beacon_attestation (#4700) 2020-01-31 15:35:13 -08:00
terence tsao
6d2a2ebadf Update run time to v0.9.3 (#4154)
* Remove custody (#3986)

* Update proto fields

* Updated block operations

* Fixed all block operation tests

* Fixed tests part 1

* Fixed tests part 1

* All tests pass

* Clean up

* Skip spec test

* Fixed ssz test

* Skip ssz test

* Skip mainnet tests

* Update beacon-chain/operations/attestation.go

* Update beacon-chain/operations/attestation.go
* Decoy flip flop check (#3987)
* Bounce attack check (#3989)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Update beacon-chain/blockchain/forkchoice/process_block.go
* Conflict
* Unskip mainnet spec tests (#3998)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update beacon-chain/core/blocks/block_operations.go
* Unskip minimal spec tests (#3999)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update workspace

* Unskip all minimal spec tests

* Update workspace for general test
* Unskip test (#4001)
* Update minimal seconds per slot to 6 (#3978)
* Bounce attack tests (#3993)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Slots since epoch starts tests

* Update justified checkpt tests

* Conflict

* Fixed logic

* Update process_block.go

* Use helper
* Conflict
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.1
* Conflict
* Fixed failed tests
* Lower MinGenesisActiveValidatorCount to 16384 (#4100)
* Fork choice beacon block checks (#4107)

* Prevent future blocks check and test

* Removed old code
* Update aggregation proto (#4121)

* Update def
* Update spec test
* Conflict
* Update workspace
* patch
* Resolve conflict
* Patch
* Change workspace
* Update ethereumapis to a forked branch at commit 6eb1193e47
* Fixed all the tests
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into conflict
* fix patch
* Need to regenerate test data
* Merge branch 'master' into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Enable snappy compression for all (#4157)

* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* Validate aggregate and proof subscriber (#4159)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Conflict
* Update workspace
* Conflict
* Conflict
* Conflict
* Merge branch 'master' into v0.9.2
* Merge branch 'master' into v0.9.2
* Conflict
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Remove migrate to snappy  (#4205)
* Feature flag: Deprecate --prune-states, release to all (#4204)

* Deprecated prune-states, release to all

* imports

* remote unused import

* remove unused import

* Rm prune state test

* gaz
* Refactoring for dynamic pubsub subscriptions for non-aggregated attestations (#4189)

* checkpoint progress

* chkpt

* checkpoint progress

* put pipeline in its own file

* remove unused imports

* add test, it's failing though

* fix test

* remove head state issue

* add clear db flag to e2e

* add some more error handling, debug logging

* skip processing if chain has not started

* fix test

* wrap in go routine to see if anything breaks

* remove duplicated topic

* Add a regression test. Thanks @nisdas for finding the original problem. May it never happen again *fingers crossed*

* Comments

* gofmt

* comment out with TODO
* Sync with master
* Sync with master
* RPC servers use attestation pool (#4223)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Refactor RPC to Fully Utilize Ethereum APIs (#4243)

* include attester as a file in the validator server

* remove old proposer server impl

* include new patch and properly sync changes

* align with public pbs

* ensure matches rpc def

* fix up status tests

* resolve all broken test files in the validator rpc package

* gazelle include

* fix up the duties implementation

* fixed up all get duties functions

* all tests pass

* utilize new ethereum apis

* amend validator client to use the new beacon node validator rpc client

* fix up most of validator items

* added in mock

* fix up test

* readd test

* add chain serv mock

* fix a few more validator methods

* all validator tests passingggg

* fix broken test

* resolve even more broken tests

* all tests passsssss

* fix lint

* try PR

* fix up test

* resolve broken other tests
* Sync with master
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Aggregate and proof subscriber (#4240)

* Added subscribers

* Fixed conflict

* Tests

* fix up patch

* Use upstream pb

* include latest patch

* Fmt

* Save state before head block
* skip tests (#4275)
* Delete block attestations from the pool (#4241)

* Added subscribers
* Clean up
* Fixed conflict
* Delete atts in pool in validate pipeline
* Moved it to subscriber
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into use-att-pool-3
* Test
* Fixed test
* Initial work on voluntary exit (#4207)

* Initial implementation of voluntary exit: RPC call

* Update for recent merges

* Break out validation logic for voluntary exits to core module

* RequestExit -> ProposeExit

* Decrease exit package visibility

* Move to operation feed

* Wrap errors
* Fix critical proposer selection bug #4259 (#4265)

* fix critical proposer selection bug #4259

* gofmt

* add 1 more validator to make it 5

* more tests

* Fixed archivedProposerIndex

* Fixed TestFilterAttestation_OK

* Refactor ComputeProposerIndex, add regression test for potential out of range panic

* handle case of nil validator

* Update validators_test.go
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Leftover merge files, oops
* gaz
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* Fixes Duplicate Validator Bug (#4322)

* Update dict

* Test helper

* Regression test

* Comment

* Reset test cache
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fixes after PR #4328
* Complete attestation pool for run time (#4286)

* Added subscribers

* Fixed conflict

* Delete atts in pool in validate pipeline

* Moved it to subscriber

* Test

* Fixed test

* New curl for forkchoice attestations

* Starting att pool service for fork choice

* Update pool interface

* Update pool interface

* Update sync and node

* Lint

* Gazelle

* Updated servers, filled in missing functionalities

* RPC working with 1 beacon node 64 validators

* Started writing tests. Yay

* Test to aggregate and save multiple fork choice atts

* Tests for BatchAttestations for fork choice

* Fixed exisiting tests

* Minor fixes

* Fmt

* Added batch saves

* Lint

* Mo tests yay

* Delete test

* Fmt

* Update interval

* Fixed aggregation broadcast

* Clean up based on design review comment

* Fixed setupBeaconChain

* Raul's feedback. s/error/err
* resolve conflicts
* Merge branch 'v0.9.2' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Removed old protos and fixed tests (#4336)
* Merge refs/heads/master into v0.9.2
* Disallow duplicated indices and test (#4339)
* Explicit use of GENESIS_SLOT in fork choice (#4343)
* Update from 2 to 3 (#4345)
* Remove verify unaggregated attestation when aggregating (#4347)
* use slot ticker instead of run every (#4348)
* Add context check for unbounded loop work (#4346)
* Revert "Explicit use of GENESIS_SLOT in fork choice (#4343)" (#4349)

This reverts commit d3f6753c77f8f733563d00ab649c5159b2c2926f.
* Refactor Powchain Service (#4306)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* fix lint

* raul's comments

* use one method

* fix test

* raul's comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Ensure best better-justification is stored for fork choice (#4342)

* Ensure best better-justification is stored. Minor refactor
* Tests
* Merge refs/heads/v0.9.2 into better-best-justified
* Merge refs/heads/v0.9.2 into better-best-justified
* Ensure that epoch of attestation slot matches the target epoch (#4341)

* Disallow duplicated indices and test
* Add slot to target epoch check to on_attestation
* Add slot to target epoch check to process_attestation
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into no-dup-att-indices
* Fixed TestProcessAttestations_PrevEpochFFGDataMismatches
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Update beacon-chain/blockchain/forkchoice/process_attestation_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Filter viable branches in fork choice (#4355)
* Only activate upon finality (#4359)

* Updated functions
* Tests
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into queue-fix-on-finality
* Comment
* Merge refs/heads/v0.9.2 into queue-fix-on-finality
* Fixed failing test from 4359 (#4360)

* Fixed
* Skip registry spec tests
* Wait for state to be initialized at least once before running slot ticker based on genesis time (#4364)
* Sync with master
* Fix checkpoint root to  use genesis block root (#4368)
* Return an error on nil head state in fork choice (#4369)

* Return error if nil head state

* Fixed tests. Saved childen blocks state

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Update metrics every epoch (#4367)
* return empty slice if state is nil (#4365)
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Pubsub: Broadcast attestations to committee based subnets (#4316)

* Working on un-aggregated pubsub topics

* update subscriber to call pool

* checkpointing

* fix

* untested message validation

* minor fixes

* rename slotsSinceGenesis to slotsSince

* some progress on a unit test, subscribe is not being called still...

* dont change topic

* need to set the data on the message

* restore topic

* fixes

* some helpful parameter changes for mainnet operations

* lint

* Terence feedback

* unskip e2e

* Unit test for validate committee index beacon attestation

* PR feedbacK

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into resolveConflicts
* remove condition
* Remove unused operation pool (#4361)
* Merge refs/heads/master into v0.9.2
* Aggregate attestations periodically  (#4376)
* Persist ETH1 Data to Disk (#4329)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* adding in new proto message

* remove outdated vars

* add new changes

* remove latest eth1data

* continue refactoring

* finally works

* lint

* fix test

* fix all tests

* fix all tests again

* fix build

* change back

* add full eth1 test

* fix logs and test

* add constant

* changes

* fix bug

* lint

* fix another bug

* change back

* Apply suggestions from code review

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fixed VerifyIndexedAttestation (#4382)
* rm signing root (#4381)

* rm signing root

* Fixed VerifyIndexedAttestation

* Check proposer slashed status inside ProcessBlockHeaderNoVerify

* Fixed TestUpdateJustified_CouldUpdateBest

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove Redundant Trie Generation (#4383)

* remove trie generation
* remove deposit hashes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* fix build
* Conflict
* Implement StreamAttestations RPC Endpoint (#4390)

* started attestation stream

* stream attestations test

* on slot tick test passing

* imports

* gaz

* Update beacon-chain/rpc/beacon/attestations_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

Co-authored-by: shayzluf <thezluf@gmail.com>
* Fixed goimport (#4394)
* Use custom stateutil ssz for ssz HTR spec tests (#4396)

* Use custom stateutil ssz for ssz HTR spec tests

* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* set mainnet to be the default for build and run (#4398)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* gracefully handle deduplicated registration of topic validators (#4399)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* SSZ: temporarily disable roots cache until cache issues can be resolved (#4407)

* temporarily disable roots cache until cache issues can be resolved

* Also use custom ssz for spectests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove process block attestations as separate routine (#4408)

* Removed old save/process block atts

* Fixed tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save Deposit Cache to Disk (#4384)

* change to protos

* fix build

* glue everything together

* fix test

* raul's review

* preston's comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix activation queue sorting (#4409)

* Removed old save/process block atts

* Fixed tests

* Proper sorting by eligibility epoch then by indices

* Deleted old colde
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' into v0.9.2
* Merge refs/heads/master into v0.9.2
* stop recursive lookup if context is cancelled (#4420)
* Fix proposal bug (#4419)
* Add Pending Deposits Safely (#4422)

* safely prune cache

* use proper method

* preston's,terence's reviews and comments

* revert change to build files

* use as feature config instead
* Release custom state ssz (#4421)

* Release custom state ssz, change all HTR of beacon state to use custom method

* typo

* use mainnet config

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Update initial sync save justified to align with v0.9.3 (#4432)
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fix build
* don't blacklist on pubsub (#4435)
* Fix Flakey Slot Ticker Test (#4434)

* use interface instead for the slot ticker

* fixed up flakey tests

* add gen time

* get duties comment

* fix lifecycle test

* more fixes
* Configurable min genesis delay (#4437)

* Configurable min genesis delay based on https://github.com/ethereum/eth2.0-specs/pull/1557

* remove feature flag for genesis delay

* fix

* demo config feedback
* patch readme
* save keys unencrypted for validators (#4439)
* Add new demo configuration targeting mainnet scale (#4397)

* Add new demo configuration targeting mainnet, with 1/10th of the deposit value

* reduce quotant by 1/10th. Use 1/10th mainnet values

* only change the inactivity quotant

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save justified checkpoint state (#4433)

* Save justified checkpoint state

* Lint

* Feedback

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update shared/testutil/deposits.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update proto/testing/ssz_regression_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/epoch/epoch_processing.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/kv/forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/proposer.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/prepare_forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/aggregator/server.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/cache/depositcache/pending_deposits.go
* Update beacon-chain/cache/depositcache/pending_deposits_test.go
* Update beacon-chain/rpc/validator/proposer.go
* Merge refs/heads/master into v0.9.2
* Fix e2e genesis delay issues (#4442)

* fix e2e genesis delay issues

* register flag

* typo

* Update shared/featureconfig/config.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* skip demo e2e

* fix validator

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Batch Eth1 RPC Calls (#4392)

* add new methods

* get it working

* optimize past deposit logs processing

* revert change

* fix all tests

* use mock

* lint

* lint

* check for nil

* stop panics

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Terence's Review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-01-07 18:47:39 +00:00
Preston Van Loon
961dd21554 Use libp2p gossipsub upstream validator framework (#4318)
* add reject all pubsub validator to stop automatic propagation of messages
* gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into pubsub-validator
* refactor p2p validator pipeline
* add sanity check
* Merge branch 'pubsub-validator' of github.com:prysmaticlabs/prysm into pubsub-validator
* fixed up test
* rem
* gaz
* Merge refs/heads/master into pubsub-validator
* fix from self test
* ensure validator data is set
* resolve todo
* Merge refs/heads/master into pubsub-validator
* gaz
* Merge refs/heads/master into pubsub-validator
* Merge branch 'pubsub-validator' of github.com:prysmaticlabs/prysm into pubsub-validator
* Merge refs/heads/master into pubsub-validator
* remove all of the 'from self' logic. filed https://github.com/libp2p/go-libp2p-pubsub/issues/250
* Merge branch 'pubsub-validator' of github.com:prysmaticlabs/prysm into pubsub-validator
* gaz
* update comment
* Merge refs/heads/master into pubsub-validator
* rename "VaidatorData"
* Merge branch 'pubsub-validator' of github.com:prysmaticlabs/prysm into pubsub-validator
* refactor
* one more bit of refactoring
* Update beacon-chain/sync/validate_beacon_attestation.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* skip validation on self messages, add @nisdas feedback to increment failure counter
* Merge branch 'pubsub-validator' of github.com:prysmaticlabs/prysm into pubsub-validator
* remove flakey
2019-12-20 03:18:08 +00:00
terence tsao
23a6c20dd4 Service as proper names (#4293) 2019-12-16 19:53:55 -06:00