Commit Graph

259 Commits

Author SHA1 Message Date
terence tsao
976a3af637
Refactor validator roles into an array (#4081) 2019-11-21 14:35:20 -08:00
Nishant Das
023dfebc73 Revert "Reverts the Revert (#4011)" (#4026)
This reverts commit c4ca8a47b3.
2019-11-17 22:48:22 -08:00
Ivan Martinez
68edad13bc End To End Tests for Demo and Minimal config (#3932)
* Begin working on end to end tests using geth dev chain

* Start on beacon node set up

* More progress on bnode setup

* Complete flow until chainstart, begin work on evaluators

* More progress on evaluators

* Start changing bazel run to direct binary

* Move endtoend to inside beacon-chain

* use bazel provided geth, use bazel test

* tempdir

* use fork rules_go

* Change to use UUID dir and bazel binaries

* Truncate UUID a bit

* Get full run from chainstart to evaluating

* Rewrite to react to logs rather than arbitrarily wait

* Fix export

* Move evaluators to evaluators.go

* Add peer check test

* Add more comments

* Remove unneeded exports

* Check all nodes have the correct amount of peers

* Change name to onGenesisEpoch

* Remove extra wait times where not needed

* Cleanup

* Add log for beacon start

* Fix deposit amount

* Make room for eth1follow distnce

* Cleanup and fix minimal test

* Goimports

* Fix imports

* gazelle and minimal

* manual

* Fix for comments

* Make timing rely on reading logs, and cleanup

* Fix for comments

* Fix workspace

* Cleanup

* Fix visibility

* Cleanup and some comments

* Address comments

* Fix for v0.9

* Modify for v0.9

* Move to own package outside of beacon-chain

* Gazelle

* Polishing, logging

* Fix filenames

* Add more logs

* Add flag logging

* Cover for page not having libp2p info

* Improve multiAddr detection

* Add more logs

* Add missing flags

* Add log printing to defer

* Get multiAddr from logs

* Fix logging and detection

* Change evaluators to rely on EpochTimer

* Add evaluator for ValidatorParticipation

* Fix validator participation evaluator

* Cleanup, comments and fix participation calculation

* Cleanup

* Let the file searcher search for longer

* Change participation to check for full

* Log out file contents if text isnt found

* Split into different files

* Disable IPC and use RPC instead, change tmp dir to bazel dir

* Change visibility

* Gazelle

* Add e2e tag

* new line
2019-11-15 13:56:26 -05:00
Nishant Das
c4ca8a47b3 Reverts the Revert (#4011)
* Revert "Revert "Change BLS Library to Herumi (#3752)" (#4006)"

This reverts commit 904898e405.

* turn it on

* make all docker images with cgo deps static

* change back

* fix build

* switch back

* address gateway

* fix library again
2019-11-15 10:27:23 -06:00
Nishant Das
904898e405 Revert "Change BLS Library to Herumi (#3752)" (#4006)
This reverts commit 24583864b4.
2019-11-14 13:00:50 -05:00
Nishant Das
24583864b4 Change BLS Library to Herumi (#3752)
* change to herumi's bls

* change alias

* change to better

* add benchmark

* build

* change to bazel fork

* fix prefix

* make it work with library

* update to latest

* change again

* add import

* update to latest

* add sha commit

* new static lib with groups swapped

* using herumis new lib

* fix dep paths in c headers

* update again

* new changes

* fix commit

* fix serialization

* comment

* fix test

* fix to herumis latest version

* fix test

* fix benchmarks

* add new workspace

* change commit and remove init

* get test to pass

* remove parameter

* remove reverse byte order

* make gazelle happy

* set pure to off

* fix failing tests

* remove old ref

* use HashWithDomain functions

* update to latest version

* clean up

* gaz

* add back removed code

* switch off pure
2019-11-14 09:51:42 -06:00
terence tsao
89f4053c33
Validator clean up part 1 (#3976) 2019-11-12 09:14:03 -08:00
terence tsao
81c53c26fb Update run time to v0.9 (#3935)
* Update seed domains (#3872)

* Remove Transfers (#3870)

* Remove active index roots and compact committee roots (#3869)

* Update inclusion reward (#3886)

* Alter proposer selection logic (#3884)

* Fix early committee bias (#3888)

* Remove shards and committees (#3896)

* Epoch spec tests v0.9 (#3907)

* Block spec test v0.9 (#3905)

* rm'ed in protobuf

* build proto

* build proto

* build proto

* fix core package

* Gazelle

* Fixed all the tests

* Fixed static test

* Comment out spec test for now

* One more skip

* fix-roundRobinSync (#3862)

* Starting but need new seed function

* Revert initial sync

* Updated Proposer Slashing

* Fixed all tests

* Lint

* Update inclusion reward

* Fill randao mixes with eth1 data hash

* Test

* Fixing test part1

* All tests passing

* One last test

* Updated config

* Build proto

* Proper skip message

* Conflict and fmt

* Removed crosslinks and shards. Built

* Format and gazelle

* Fixed all the block package tests

* Fixed all the helper tests

* All epoch package tests pass

* All core package tests pass

* Fixed operation tests

* Started fixing rpc test

* RPC tests passed!

* Fixed all init sync tests

* All tests pass

* Fixed blockchain tests

* Lint

* Lint

* Preston's feedback

* Starting

* Remove container

* Fixed block spec tests

* All passing except for block_processing test

* Failing block processing test

* Starting

* Add AggregateAndProof

* All mainnet test passes

* Update deposit contract (#3906)

* Proto spec tests v0.9 (#3908)

* Starting

* Add AggregateAndProof

* Unskip block util tests (#3910)

* rm'ed in protobuf

* build proto

* build proto

* build proto

* fix core package

* Gazelle

* Fixed all the tests

* Fixed static test

* Comment out spec test for now

* One more skip

* fix-roundRobinSync (#3862)

* Starting but need new seed function

* Revert initial sync

* Updated Proposer Slashing

* Fixed all tests

* Lint

* Update inclusion reward

* Fill randao mixes with eth1 data hash

* Test

* Fixing test part1

* All tests passing

* One last test

* Updated config

* Build proto

* Proper skip message

* Conflict and fmt

* Removed crosslinks and shards. Built

* Format and gazelle

* Fixed all the block package tests

* Fixed all the helper tests

* All epoch package tests pass

* All core package tests pass

* Fixed operation tests

* Started fixing rpc test

* RPC tests passed!

* Fixed all init sync tests

* All tests pass

* Fixed blockchain tests

* Lint

* Lint

* Preston's feedback

* Starting

* Remove container

* Fixed block spec tests

* All passing except for block_processing test

* Failing block processing test

* Starting

* Add AggregateAndProof

* All mainnet test passes

* Unskip block util tests

* Slot processing spec test V0.9 (#3912)

* Starting

* Add AggregateAndProof

* Unskip slot processing mainnet test

* Unskip minimal spec test for finalization (#3920)

* Remove outdated interop tests (#3922)

* Rm outdated interop tests

* Rm test runner

* Gazelle

* Update validator to use proposer slot (#3919)

* Fix committee assignment (#3931)

* Replace shard with committee index (#3930)

* Conflict

* Clean up (#3933)

* Remove shard filter in db (#3936)

* Remove lightouse compatibility test (#3939)

* Update Committee Cache for v0.9 (#3948)

* Updated committee cache

* Removed shuffled indices cache

* Started testing run time

* Lint

* Fixed test

* Safeguard against nil head state

* address edge case

* add test

* Fixed TestRoundRobinSync by doubling the epochs

* Unskip TestProtoCompatability (#3958)

* Unskip TestProtoCompatability

* Update WORKSPACE

* Fix minimal config (#3959)

* fix minimal configs

* fix hardcoded value in test

* Simplify verify att time (#3961)

* update readme for deposit contract, regen bindings for vyper 0.1.0b12 (#3963)

* update readme for deposit contract, regen bindings

* medium

* Check nil base state (#3964)

* Copy Block When Receiving it From Sync (#3966)

* copy block

* clone for other service methods too

* Change logging of Bitfield  (#3956)

* change logging of bits

* preston's review

* Unskip Beacon Server Test (#3962)

* run test till the end

* fix up proto message types

* fmt

* resolve broken tests

* better error handling

* fixing new logic to use archived proposer info

* fix up logic

* clip using the max effective balance

* broken build fix with num arg mismatch

* amend archive

* archival logic changed

* rename test

* archive both proposer and attester seeds

* page size 100

* further experiments

* further experimentation, archivedProposerIndex seems wrong

* test passes

* rem log

* fix broken test

* fix test

* gaz

* fix imports

* ethapis
2019-11-11 17:03:43 -05:00
Nishant Das
a938c305b4 Do Not Reject Attestations in Genesis Epoch (#3955) 2019-11-08 08:09:07 -08:00
terence tsao
768c2bd812
Block validator from getting assignment until beacon node is sync'ed (#3953) 2019-11-07 17:59:11 -08:00
Alex Nebotov
5a789fca4a Added the panic capture so that information about panic gets into the log file (#3877)
* Added the panic capture so that information about it gets into the log file

* Printf -> Errorf

* Update validator/main.go  "run time" -> "Runtime"

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/main.go    "run time" -> "Runtime"

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>
2019-11-06 10:18:51 -06:00
Preston Van Loon
1b1e994a80
Set committee assignment deadline to 1 epoch (#3902)
* set deadline to epoch end

* fix vis
2019-10-30 18:35:02 -04:00
Preston Van Loon
fbc31dc99b validator client side grpc metrics (#3903) 2019-10-30 15:30:10 -07:00
Nishant Das
c9a7a9c709
Attestation is Decoupled From Proposal (#3800)
* fix

* add correct test

* add helper

* gaz

* add helper

* gaz
2019-10-22 07:15:32 +08:00
Marius Kjærstad
635e20529a Updated why-combining-sharding-and-casper link in validator/README.md (#3810)
Updated why-combining-sharding-and-casper link in validator/README.md
2019-10-20 19:43:30 -07:00
Jim McDonald
b0eccd24a2 Tidy up logging (#3784) 2019-10-16 06:37:43 -07:00
Jim McDonald
9d441011d7 Remove key duplication (#3763)
* Remove key duplication

* Break out function to allow testing
2019-10-15 11:20:23 -07:00
Jim McDonald
aa07843157 Change public key map identifier to byte array (#3716) 2019-10-12 11:22:51 +09:00
terence tsao
750bc83369
Clean up feature flag namings (#3715) 2019-10-07 14:11:49 +09:00
Jim McDonald
73ffde869f Tidy up logging (#3711) 2019-10-04 14:11:59 -07:00
Nishant Das
24cbcc552f change to hex (#3712) 2019-10-04 04:36:57 -07:00
Preston Van Loon
20e97bc6c3 remove fluentd timestamp (#3708) 2019-10-03 20:56:10 -07:00
Nishant Das
4daf62fc28 Fix Validator Activation (#3684) 2019-10-02 13:26:11 -07:00
terence tsao
c632b96454 Validator logging improvements (#3661)
* Starting

* Update logging for service

* Update logging for assignment

* Update logging for attest

* Update logging for propose

* Update logging for balance update

* Final touchup

* Fixed test

* Fixed test

* Feedback

* Fix

* Fix all the tests
2019-10-02 12:18:01 +08:00
Raul Jordan
8d215feb25
Large Prysm Performance Improvements (#3622)
* skip bls verification with a feature flag at runtime

* gazelle

* more bls mocks

* block roots efficiency

* db block roots now does not show up on the flame graphs

* save validator latest votes batch

* batch save att

* misc improvements to pprof

* import

* include validator index cache

* error if no filter criteria

* resolved comments

* build fix

* lint

* remove delay global

* attestation and block test fixes

* preston suggestions

* fix db tests

* fix missing broken tests

* tests pass
2019-09-30 15:45:53 -05:00
Preston Van Loon
571efc11d1
add error spans, interceptrs (#3641) 2019-09-29 11:48:55 -07:00
Preston Van Loon
508fac65be make the print the same number of characters (#3626) 2019-09-28 09:41:02 +08:00
Preston Van Loon
00e68c6cc7 Use demo config for accounts create (#3627) 2019-09-27 15:48:49 -07:00
Jim McDonald
6a4b46ab0e Tidy up logging in the validator (#3597)
* Tidy up logging in the validator

* Log full public key when validator first initialised

* Use 'validator' rather than 'pubKey' for traces; use full public key
2019-09-27 09:47:03 +08:00
Preston Van Loon
af1301ddcb
update rules go to support go 1.13.1 (#3599)
* update rules go to support go 1.13.1

* gazelle update
2019-09-26 18:06:59 -07:00
Nishant Das
9b008522b8 Refactor Validator Start Routine (#3594)
* make demo default

* make minimal config a flag

* lint

* initialize config at the start

* gaz

* make main method cleaner

* remove interop.go

* fix test

* lint

* gaz

* Update validator/accounts/interop.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fix docker build

* fix docker build
2019-09-26 13:23:25 -05:00
Nishant Das
3dcaeabb3e Fix Validator Account Creation (#3577)
* print raw tx data

* revert change
2019-09-24 20:16:08 -07:00
Nishant Das
26af4496c0 Batch Validator Performance Requests (#3520)
* change proto msg types

* change server and client

* regen protos
2019-09-18 11:47:14 -05:00
terence tsao
9ab08e6998
Remove beacon rpc service (#3515)
* first version of the watchtower api

* first version

* delete watchtower

* move to message loop

* roughtime

* one time

* fix test

* Fixed test

* Fixed proposer server

* Gaz

* gaz

* Stuck

* Tests passing

* Fixed all the tests
2019-09-18 06:34:50 -07:00
Preston Van Loon
a2685245f2 demo-config flag (#3473)
* democonfig

* 3.2
2019-09-15 14:24:08 -05:00
Nishant Das
d597410d9b MultiLocking in Operations Service (#3470)
* add lock

* add cache

* modify lock retrieval
2019-09-15 13:51:37 -05:00
terence tsao
9d62e542e5
Clean up validator server (#3466) 2019-09-14 13:31:38 -07:00
terence tsao
798bbbdc82 Cold start for interop (#3437)
* coldstart flags for validator

* WIP beacon node flags

* wip beacon chain, flag fix in validator, arg fix in validator

* checkpoint

* Added interop service

* working on mock chainstart

* save the state lol

* fix tests

* Save genesis validators

* gaz

* fix validator help flags

* WaitForChainStart actually waits for genesis time

* cold start fixes

* cache

* change back

* allow for genesis state too

* remove logs

* increase mmap size

* dont process if head doesn't exist

* add 10ms tolerance

* enable libp2p debug at debug, fix pubsub

* works with checkpt

* initialize justified and finalized in db

* Removed preloadStatePath from blockchain service

* Clean up

* Write to disk for now post state

* revert b466dd536f8eadbdae2264a545a755370223d917

* Builds

* Only RPC test fails now

* use minimal config, no demo config

* clean up branch

* Lint

* resolve lint

* more lint fixes

* lint

* fix viz

* Fixing RPC test

* skip before epoch 2

* RPC time out

* Fixed ordering

* rename

* remove some dbg statements

* ensure index is correct

* fix some panics

* getting closer

* fix tests

* Fix private key

* Fixed RPC test

* Fixed beacon chain build for docker

* Add interop.go to validator go_image

* Fixed docker build

* handle errors

* skip test, skip disconnecting peers

* Fixed docker build

* tolerance for attestation processing

* revert copy

* clearer err message parse

* fix launching from dep contract
2019-09-11 13:38:35 -05:00
Ivan Martinez
5e939378d0 Update to spec v0.8.3 (#3355)
* Ignore latest messages in fork choice prior to latest justified

* Make sure Compact Committee Roots isn't changed by process_final_updates

* WIP add attestation bitfields length to match committee length

* Begin work on updating spec tests to 0.8.2

* WIP set up for new spec test structure

* Fix slashings

* Get mainnet tests mostly passing for attestations and attester slashings

* Fix process attestation test

* Undo change

* Complete spec tests for all operations
Still need sanity block tests

* Fix BLS sigs

* Reduce amount of reused code in core/blocks/spectests/

* Fix tests

* Update block sanity tests to 0.8.2

* Update epoch spec tests to 0.8.2

* Clean up all tests and fix shuffling/epoch tests

* WIP update bls tests to 0.8.2

* WIP update bls tests to 0.8.3

* Finish BLS spectest update to 0.8.3

* Fix shuffling spec tests

* Fix more tests

* Update proto ssz spec tests to 0.8.3

* Attempt to fix PrevEpochFFGDataMismatches test

* Goimports

* Fix documentation

* fix test

* Use custom general spec tests

* Reduce code footprint

* Remove unneeded minimal skip

* Fix for comments

* Fix for comments

* Fix test

* Small fixes

* Cleanup block spec tests a bit

* Undo change

* fix validator

* Fix validator tests

* Run gazelle

* Fix error output for epoch spec tests
2019-09-08 12:41:52 -07:00
Nishant Das
fb20fc7881 change to wrap (#3413) 2019-09-06 14:01:59 -07:00
skillful-alex
9e5864fc61 Added roughtime to validator waitToSlotMidpoint (#3344)
* add roughtime to validator waitToSlotMidpoint

* gazelle
2019-08-28 14:59:30 -05:00
Raul Jordan
14f77449ce
Include Prysm Tool to Generate Unencrypted Keys (#3324)
* next compatible, tests pass

* terence feedback

* skip comment

* fixes

* misc fix

* on block

* parse from unencrypted keys json

* mod val client

* launching unencrypted workssss

* fix broken build

* fix up build

* rem prints

* unencrypted keys file generator

* generate json

* unencrypted keys gen files

* tool done

* function abstractions

* removed docker img stuff

* lint
2019-08-28 11:07:31 -05:00
Raul Jordan
7beafa159d
Support Starting Validator Binary from Unencrypted Keys JSON (#3308)
* next compatible, tests pass

* terence feedback

* skip comment

* fixes

* misc fix

* on block

* parse from unencrypted keys json

* mod val client

* launching unencrypted workssss

* fix broken build

* fix up build

* rem prints

* resolve lint

* bls comment

* fix docker deps

* gaz
2019-08-26 16:07:09 -05:00
terence tsao
32c426ed1b Replaced block and state roots construction to SlotsPerHistoricalRoot (#3160) 2019-08-08 13:12:35 -05:00
terence tsao
2ec8a46cb2
forgot to wrap these two errors (#3156) 2019-08-08 06:55:25 -07:00
Preston Van Loon
953c59a302 Wrap errors (#3123) 2019-08-01 19:27:38 -07:00
terence tsao
08ac1c3c35 Remove nil for assignment when slot is 0 (#3117) 2019-07-31 17:46:11 -05:00
Nishant Das
8cbd1097d7 Fix Signature Verification for Attestations (#3080)
* fix bug and test

* terence's review
2019-07-26 23:18:44 -05:00
Preston Van Loon
f58afa62af
fix docker image builds (#3068) 2019-07-24 15:19:44 -04:00
Nishant Das
590aaaf370 Fix Batched Block Response (#3012) 2019-07-24 07:53:38 -07:00