Commit Graph

108 Commits

Author SHA1 Message Date
terence tsao
f5cb04012e Aggregator selection from RPC to validator client (#4071)
* Config
* Updated proto
* Updated pool
* Updated RPC
* Updated validator client
* run time works
* Clean ups
* Fix tests
* Visibility
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into aggregator
* Raul's feedback
* Tests for RPC server
* Tests for validator client
* Span
* More tests
* Use go routine for SubmitAggregateAndProof
* Go routines
* Updated comments
* Use array of roles
* Fixed tests
* Build
* Update validator/client/runner.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update validator/client/runner.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* If
* Merge branch 'refactor-validator-roles' of https://github.com/prysmaticlabs/prysm into refactor-validator-roles
* Empty
* Feedback
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into aggregator
* Removed proto/eth/v1alpha1/shard_chain.pb.go?
* Cleaned up
* Revert
* Comments
* Lint
* Comment
* Merge branch 'master' into aggregator
2019-11-22 05:11:38 +00:00
terence tsao
835418d1e3
Fix UpdateCommitteeCache slot (#4074) 2019-11-20 21:29:43 -08:00
terence tsao
fcb663acde Implement aggregation helpers (#4062)
* Aggregation helpers

* Tests

* Config

* Faulty test cases

* Err
2019-11-19 20:24:39 -06:00
Nishant Das
023dfebc73 Revert "Reverts the Revert (#4011)" (#4026)
This reverts commit c4ca8a47b3.
2019-11-17 22:48:22 -08:00
Raul Jordan
5169209360
Properly Archive Active Set Changes (#4007)
* archiving information properly

* tests passing

* broken test fix
2019-11-15 10:45:02 -06:00
Nishant Das
c4ca8a47b3 Reverts the Revert (#4011)
* Revert "Revert "Change BLS Library to Herumi (#3752)" (#4006)"

This reverts commit 904898e405.

* turn it on

* make all docker images with cgo deps static

* change back

* fix build

* switch back

* address gateway

* fix library again
2019-11-15 10:27:23 -06:00
Nishant Das
904898e405 Revert "Change BLS Library to Herumi (#3752)" (#4006)
This reverts commit 24583864b4.
2019-11-14 13:00:50 -05:00
Nishant Das
24583864b4 Change BLS Library to Herumi (#3752)
* change to herumi's bls

* change alias

* change to better

* add benchmark

* build

* change to bazel fork

* fix prefix

* make it work with library

* update to latest

* change again

* add import

* update to latest

* add sha commit

* new static lib with groups swapped

* using herumis new lib

* fix dep paths in c headers

* update again

* new changes

* fix commit

* fix serialization

* comment

* fix test

* fix to herumis latest version

* fix test

* fix benchmarks

* add new workspace

* change commit and remove init

* get test to pass

* remove parameter

* remove reverse byte order

* make gazelle happy

* set pure to off

* fix failing tests

* remove old ref

* use HashWithDomain functions

* update to latest version

* clean up

* gaz

* add back removed code

* switch off pure
2019-11-14 09:51:42 -06:00
terence tsao
81c53c26fb Update run time to v0.9 (#3935)
* Update seed domains (#3872)

* Remove Transfers (#3870)

* Remove active index roots and compact committee roots (#3869)

* Update inclusion reward (#3886)

* Alter proposer selection logic (#3884)

* Fix early committee bias (#3888)

* Remove shards and committees (#3896)

* Epoch spec tests v0.9 (#3907)

* Block spec test v0.9 (#3905)

* rm'ed in protobuf

* build proto

* build proto

* build proto

* fix core package

* Gazelle

* Fixed all the tests

* Fixed static test

* Comment out spec test for now

* One more skip

* fix-roundRobinSync (#3862)

* Starting but need new seed function

* Revert initial sync

* Updated Proposer Slashing

* Fixed all tests

* Lint

* Update inclusion reward

* Fill randao mixes with eth1 data hash

* Test

* Fixing test part1

* All tests passing

* One last test

* Updated config

* Build proto

* Proper skip message

* Conflict and fmt

* Removed crosslinks and shards. Built

* Format and gazelle

* Fixed all the block package tests

* Fixed all the helper tests

* All epoch package tests pass

* All core package tests pass

* Fixed operation tests

* Started fixing rpc test

* RPC tests passed!

* Fixed all init sync tests

* All tests pass

* Fixed blockchain tests

* Lint

* Lint

* Preston's feedback

* Starting

* Remove container

* Fixed block spec tests

* All passing except for block_processing test

* Failing block processing test

* Starting

* Add AggregateAndProof

* All mainnet test passes

* Update deposit contract (#3906)

* Proto spec tests v0.9 (#3908)

* Starting

* Add AggregateAndProof

* Unskip block util tests (#3910)

* rm'ed in protobuf

* build proto

* build proto

* build proto

* fix core package

* Gazelle

* Fixed all the tests

* Fixed static test

* Comment out spec test for now

* One more skip

* fix-roundRobinSync (#3862)

* Starting but need new seed function

* Revert initial sync

* Updated Proposer Slashing

* Fixed all tests

* Lint

* Update inclusion reward

* Fill randao mixes with eth1 data hash

* Test

* Fixing test part1

* All tests passing

* One last test

* Updated config

* Build proto

* Proper skip message

* Conflict and fmt

* Removed crosslinks and shards. Built

* Format and gazelle

* Fixed all the block package tests

* Fixed all the helper tests

* All epoch package tests pass

* All core package tests pass

* Fixed operation tests

* Started fixing rpc test

* RPC tests passed!

* Fixed all init sync tests

* All tests pass

* Fixed blockchain tests

* Lint

* Lint

* Preston's feedback

* Starting

* Remove container

* Fixed block spec tests

* All passing except for block_processing test

* Failing block processing test

* Starting

* Add AggregateAndProof

* All mainnet test passes

* Unskip block util tests

* Slot processing spec test V0.9 (#3912)

* Starting

* Add AggregateAndProof

* Unskip slot processing mainnet test

* Unskip minimal spec test for finalization (#3920)

* Remove outdated interop tests (#3922)

* Rm outdated interop tests

* Rm test runner

* Gazelle

* Update validator to use proposer slot (#3919)

* Fix committee assignment (#3931)

* Replace shard with committee index (#3930)

* Conflict

* Clean up (#3933)

* Remove shard filter in db (#3936)

* Remove lightouse compatibility test (#3939)

* Update Committee Cache for v0.9 (#3948)

* Updated committee cache

* Removed shuffled indices cache

* Started testing run time

* Lint

* Fixed test

* Safeguard against nil head state

* address edge case

* add test

* Fixed TestRoundRobinSync by doubling the epochs

* Unskip TestProtoCompatability (#3958)

* Unskip TestProtoCompatability

* Update WORKSPACE

* Fix minimal config (#3959)

* fix minimal configs

* fix hardcoded value in test

* Simplify verify att time (#3961)

* update readme for deposit contract, regen bindings for vyper 0.1.0b12 (#3963)

* update readme for deposit contract, regen bindings

* medium

* Check nil base state (#3964)

* Copy Block When Receiving it From Sync (#3966)

* copy block

* clone for other service methods too

* Change logging of Bitfield  (#3956)

* change logging of bits

* preston's review

* Unskip Beacon Server Test (#3962)

* run test till the end

* fix up proto message types

* fmt

* resolve broken tests

* better error handling

* fixing new logic to use archived proposer info

* fix up logic

* clip using the max effective balance

* broken build fix with num arg mismatch

* amend archive

* archival logic changed

* rename test

* archive both proposer and attester seeds

* page size 100

* further experiments

* further experimentation, archivedProposerIndex seems wrong

* test passes

* rem log

* fix broken test

* fix test

* gaz

* fix imports

* ethapis
2019-11-11 17:03:43 -05:00
Preston Van Loon
7d1633230d Feature gate all caches (#3894) 2019-11-03 17:20:08 -08:00
Preston Van Loon
1b1e994a80
Set committee assignment deadline to 1 epoch (#3902)
* set deadline to epoch end

* fix vis
2019-10-30 18:35:02 -04:00
Preston Van Loon
2b444ea954 Add feature flag to enable shuffled index cache (#3893)
* add feature flag to enable shuffled index cache

* gaz'
2019-10-30 10:26:05 +08:00
terence tsao
e42af4f11d
Check attestations bitfield overlaps (#3804) 2019-10-18 17:41:27 -07:00
Preston Van Loon
86a8ec035c
Aggregation helper improvements (#3789)
* aggregation helper

* lint
2019-10-16 11:11:58 -07:00
Preston Van Loon
23764c4640
Abstract verifySlot to helper package (#3731)
* Abstract verifySlot to helper package

* blocks -> slots

* fix test
2019-10-07 16:15:50 +09:00
terence tsao
750bc83369
Clean up feature flag namings (#3715) 2019-10-07 14:11:49 +09:00
Ivan Martinez
14d9a83cda Add Block Generation Util to testutil package (#3674) (#3709) 2019-10-04 16:07:46 -07:00
terence tsao
273871940c Test code in different packages (#3710)
* Moved a few packagesXXX to test_XXX

* Gaz
2019-10-04 15:41:09 +08:00
Nishant Das
4daf62fc28 Fix Validator Activation (#3684) 2019-10-02 13:26:11 -07:00
Ivan Martinez
8bab55d88e Add Block Generation Util to testutil package (#3674)
* Create block generation util in testutil

* Gazelle

* Fix deps

* Fix imports

* Change tests to use config and fix integer division

* Remove logs

* Fix build

* Add test to ensure finalization occurs

* Add check for finalization

* Add comment for incrementing the state

* Fix test

* Fix tes

* Fix testutil us

* Fix tests

* Change var name

* Add regression test for bug with large validator count

* Fix bazel test
2019-10-02 08:15:40 -07:00
Jim McDonald
323ee8dfac SetupInitialDeposits() now returns deposit data roots (#3683) 2019-10-02 10:50:34 +08:00
Raul Jordan
3d37a4e038
Optimize Domain Data RPC Request (#3671)
* gaz

* fix broken build

* fix broken test

* fix broken test
2019-10-01 09:36:36 -05:00
Preston Van Loon
2dd71c076e Bulk update renovate (#3659)
* Update libp2p

* Update com_google_protobuf commit hash to 97b1802

* Update graknlabs_bazel_distribution commit hash to 962f3a7

* Update io_kubernetes_build commit hash to b6d1648

* Update dependency build_bazel_rules_nodejs to v0.38.0

* Update dependency com_github_paulbellamy_ratecounter to v0.2.0

* Update libp2p

* Update dependency com_github_go_stack_stack to v1

* Update dependency com_github_karlseguin_ccache to v2

* Update dependency com_github_rs_cors to v1

* Update dependency io_k8s_client_go to v12

* Update dependency io_k8s_klog to v1

* Update dependency io_k8s_sigs_yaml to v1

* minor build fixes
2019-09-29 21:54:54 -05:00
terence tsao
5675038e5d
Add active indices functionality to cache (#3629) 2019-09-29 12:10:11 -07:00
terence tsao
4c8269aca3
Part 4 of caching improvement - Use Cache (#3625) 2019-09-27 15:56:08 -07:00
terence tsao
14bc8d7637 Part 3 of caching improvement - Update cache (#3617)
* Implemented UpdateCommitteeCache in committee.go

* Implemented test for UpdateCommitteeCache

* Updated test to use mainnet config
2019-09-27 10:43:09 -05:00
terence tsao
d7891fca88
Part 1 of caching improvement - ShuffledIndices function (#3605) 2019-09-26 16:45:40 -07:00
terence tsao
b8bd28cca2
Move SplitOffset to Sliceutil (#3606) 2019-09-26 16:25:55 -07:00
shayzluf
8f0b131631 Slasher grpc service (#3271) 2019-09-26 09:29:10 -07:00
terence tsao
fb74dae835
Remove dead cache (#3584) 2019-09-25 18:23:15 -07:00
Nishant Das
8ce96428b1 Fixes our Attestation Aggregation Issues (#3579) 2019-09-25 07:39:16 -07:00
Nishant Das
0e329fc115 Attestation Server Fix (#3570) 2019-09-24 08:19:37 -07:00
Raul Jordan
b8bdf71d5b
Allow for Fetching RPC Historical Assignments from Archived Data (#3559)
* archive participation begin implementation

* validator participation compute

* comments

* compute participation common func

* full test for archiving data

* gazelle

* complete tests

* gaz

* define archived validator assignment method

* archived assign logic

* need to use compute committee next

* compute archival assignments helper func

* properly compute committee using current shard

* modify the assignments request to take in a query filter item

* more intuitive implementation of list assignments

* utilize the query filter

* complete implementation

* revamp tests

* fixing current tests before adding archive tests

* test now passes using len filtered indices for total size

* include prop index in test

* revert bad change

* use ethapis

* add necessary tests

* comments
2019-09-23 16:04:59 -05:00
terence tsao
2b2ef4f37c
Added AggregatedAttestation helper (#3539) 2019-09-21 09:43:18 -07:00
Raul Jordan
49a529388b
Resolve Miscellaneous Prysm TODOs (#3465)
* resolve

* resolve

* return

* remove deprecated protos

* rem deprecated pbs

* resolve cache

* resolve md TODO

* node server

* resolve config todo

* resolve even more

* broken build
2019-09-16 15:45:03 -05:00
terence tsao
798bbbdc82 Cold start for interop (#3437)
* coldstart flags for validator

* WIP beacon node flags

* wip beacon chain, flag fix in validator, arg fix in validator

* checkpoint

* Added interop service

* working on mock chainstart

* save the state lol

* fix tests

* Save genesis validators

* gaz

* fix validator help flags

* WaitForChainStart actually waits for genesis time

* cold start fixes

* cache

* change back

* allow for genesis state too

* remove logs

* increase mmap size

* dont process if head doesn't exist

* add 10ms tolerance

* enable libp2p debug at debug, fix pubsub

* works with checkpt

* initialize justified and finalized in db

* Removed preloadStatePath from blockchain service

* Clean up

* Write to disk for now post state

* revert b466dd536f8eadbdae2264a545a755370223d917

* Builds

* Only RPC test fails now

* use minimal config, no demo config

* clean up branch

* Lint

* resolve lint

* more lint fixes

* lint

* fix viz

* Fixing RPC test

* skip before epoch 2

* RPC time out

* Fixed ordering

* rename

* remove some dbg statements

* ensure index is correct

* fix some panics

* getting closer

* fix tests

* Fix private key

* Fixed RPC test

* Fixed beacon chain build for docker

* Add interop.go to validator go_image

* Fixed docker build

* handle errors

* skip test, skip disconnecting peers

* Fixed docker build

* tolerance for attestation processing

* revert copy

* clearer err message parse

* fix launching from dep contract
2019-09-11 13:38:35 -05:00
Ivan Martinez
5e939378d0 Update to spec v0.8.3 (#3355)
* Ignore latest messages in fork choice prior to latest justified

* Make sure Compact Committee Roots isn't changed by process_final_updates

* WIP add attestation bitfields length to match committee length

* Begin work on updating spec tests to 0.8.2

* WIP set up for new spec test structure

* Fix slashings

* Get mainnet tests mostly passing for attestations and attester slashings

* Fix process attestation test

* Undo change

* Complete spec tests for all operations
Still need sanity block tests

* Fix BLS sigs

* Reduce amount of reused code in core/blocks/spectests/

* Fix tests

* Update block sanity tests to 0.8.2

* Update epoch spec tests to 0.8.2

* Clean up all tests and fix shuffling/epoch tests

* WIP update bls tests to 0.8.2

* WIP update bls tests to 0.8.3

* Finish BLS spectest update to 0.8.3

* Fix shuffling spec tests

* Fix more tests

* Update proto ssz spec tests to 0.8.3

* Attempt to fix PrevEpochFFGDataMismatches test

* Goimports

* Fix documentation

* fix test

* Use custom general spec tests

* Reduce code footprint

* Remove unneeded minimal skip

* Fix for comments

* Fix for comments

* Fix test

* Small fixes

* Cleanup block spec tests a bit

* Undo change

* fix validator

* Fix validator tests

* Run gazelle

* Fix error output for epoch spec tests
2019-09-08 12:41:52 -07:00
terence tsao
4235980511 Clean up post --next (#3411)
* Delete old code

* RPC mock testing

* Fixed BUILD

* Conflict

* Lint

* More lint
2019-09-06 22:39:14 -04:00
terence tsao
14c59b2ff9 Remove deprecated services and --next (#3371)
* Save new validators in DB

* Use info

* Add total validator count

* Fixed tests

* Add new test

* Revert light client config

* Add state metrics back

* Gaz

* Mark old ones as deprecated

* Deprecate not --next services

* Fixed all operation tests

* Fixed node test

* All tests passing locally

* Add deprecated-p2p back, blocked by boostrap-query

* Revert message proto

* delete deprecated DB items

* delete all other instances of old db

* gaz

* cycle rem

* clear db
2019-09-05 11:04:06 -05:00
terence tsao
11ac9585ad
Deprecate the old, and add new DB setup util for tests (#3208) 2019-08-14 11:48:28 -07:00
terence tsao
32c426ed1b Replaced block and state roots construction to SlotsPerHistoricalRoot (#3160) 2019-08-08 13:12:35 -05:00
terence tsao
bccd2f95cc
Finish error wrapping (#3135) 2019-08-04 15:45:03 -07:00
Preston Van Loon
953c59a302 Wrap errors (#3123) 2019-08-01 19:27:38 -07:00
Preston Van Loon
63cf0f07a2 Disable caches, allow toggle via feature flag (#3107) 2019-07-29 20:38:05 -07:00
terence tsao
956b07f5c1 Implement state transition no verify (#3048)
* Implemented state transition w/o sig verification

* ExecuteStateTransitionNoValidateStateRoot -> ExecuteStateTransitionNoVerify

* Fixed all the tests

* Extra spaces

* Gazelle

* Conflict

* Added start shard cache back

* typos
2019-07-27 00:24:42 +05:30
Raul Jordan
40fca7bb2c
Update SSZ Dependency With Tree Hash Cache Enabled (#3076)
* utilize new SSZ

* workspace
2019-07-25 23:49:49 -05:00
Ivan Martinez
e452b46873
Remove Optional Signature Verification for VoluntaryExit and BlockHeader (#3053)
* Remove verifySignatures from ProposerSlashings

* Remove flag from process transfers

* resolve all conflicts

* fix more references to old pbs

* Fix merge conflicts

* Remove verifySignature flag from ProcessBlockHeader

* fx spectest

* Fix test errors

* Fix tests

* Fix tests

* Goimports

* Fix test finally

* Move test helpers to testutil

* Goimports

* Fix imports

* Add tests for new helpers

* Run gazelle

* Fix tests
2019-07-25 13:53:46 -04:00
Preston Van Loon
be514076c1 Remove optional verify signatures argument when verifying deposits (#3052)
* remove optional verification of deposit signatures

* use minimal config for easier setup

* progress

* progress

* Fix a few test errors

* Fix more of tests

* fix imports, gazelle

* fix rpc package

* fix blocks package

* fixed state test

* fixed powchain tests

* add comments

* remove todo

* Update beacon-chain/rpc/validator_server_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
2019-07-23 22:19:14 +05:30
Raul Jordan
5374350a1c
Standardize Flags at Top Level and Remove Deprecated Utils (#3046)
* move shuffling to core

* remove old utils

* move flags to top level

* package lvl comment removal

* fix up references to flags

* revert node.go

* revert p2p_config.go

* revert main.go

* revert validator node.go

* revert validator main.go

* add flags pkg

* viz

* goimports
2019-07-23 08:58:20 -05:00
Preston Van Loon
6a930ba175 Remove optional verifyTree argument (#3047)
* remove optional verifyTree argument

* remove fmt

* do not provide a default eth1data, return an error instead

* add a test for this new logic

* gaz
2019-07-22 16:47:11 -05:00