Commit Graph

2766 Commits

Author SHA1 Message Date
Preston Van Loon
b5984af17c update bazel-go-ethereum, delete unused deps (#3580) 2019-09-25 10:30:25 -07:00
Preston Van Loon
5345ddf686 Initial Sync: Round robin (#3538)
* first pass, step 1 works

* naive from finalized to head

* delete commented code

* checkpoint progress on tests

* passing test

* abstract code slightly

* failure cases

* chkpt

* mostly working, missing a single block and having timeout

* passing tests

* comments

* comments

* gaz

* clarify comments

* progress on a few new cases

* add back bootnode query tool

* bootstrap from DHT

* chunked responses in round robin

* fix tests and deadlines

* add basic counter, time estimation

* hello -> handshakes

* show peers in use during sync

* just one last test failure

* only request blocks starting in the finalized epoch for step 1

* revert that

* comment out test and add better commentary

* move requestBlocks out to pointer receiver

* mathutil

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* PR feedback

* PR feedback
2019-09-25 12:00:04 -05:00
Nishant Das
8ce96428b1 Fixes our Attestation Aggregation Issues (#3579) 2019-09-25 07:39:16 -07:00
Raul Jordan
5398faea44 Do Not Archive Active Indices (#3573)
* dont archive active indices

* eliminate logs
2019-09-25 17:18:37 +08:00
terence tsao
6c892dc376 General indices helpers (#3575)
* Implemented Power of 2 helpers

* Test for power of 2 helpers

* Gazelle

* Fmt

* Implemented MerkleTree

* Test for MerkleTree

* Fixed tests

* Implemented ConcatGeneralizedIndices and GeneralizedIndexLength

* Tests for the above

* Benchmarked copy, it's faster

* Implemented rest of the indices helpers

* Tests for indices helpers

* Delete
2019-09-25 17:05:35 +08:00
Preston Van Loon
7c9ddfeb58
delete ssz server which is no longer needed (#3578) 2019-09-24 22:01:27 -07:00
Nishant Das
3dcaeabb3e Fix Validator Account Creation (#3577)
* print raw tx data

* revert change
2019-09-24 20:16:08 -07:00
terence tsao
2335b5eae7
Merkle tree implementation (#3572) 2019-09-24 17:37:45 -07:00
terence tsao
e64287773c
Merkle tree math helpers (#3571) 2019-09-24 09:58:18 -07:00
Nishant Das
0e329fc115 Attestation Server Fix (#3570) 2019-09-24 08:19:37 -07:00
Nishant Das
0db690df75 Chunked Responses (#3528)
* update naming

* replace with updated version

* more changes

* fixed all tests

* build and lint

* regen protos

* fix test

* remove outdated code

* prestons review

* add chunk size

* more fixes to chunked responses

* handle eof

* fix all tests

* abstract into common method

* add comment

* preston's comments

* preston's review

* preston's review

* lint

* add encoding methods

* gaz

* simplify

* simplify

* lint

* change naming

* update

* handle eof separately

* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* remove def

* preston's review

* preston's review

* add unit tests

* add delay to fix test
2019-09-24 07:56:50 -07:00
Jim McDonald
41631c2e3a Fix incorrect value in comment (#3558) 2019-09-24 06:52:55 -07:00
Preston Van Loon
07360bcc07
Write failed blocks to disk (#3569)
* write a failed requested block to disk

* write faile blocks from pubsub too

* gaz
2019-09-23 21:46:40 -07:00
terence tsao
cbcbb487ac
Lock around validateBeaconBlockPubSub (#3567) 2019-09-23 18:43:07 -07:00
Preston Van Loon
ad47817bcd Only write interop ssz states to disk with flag ON (#3566)
* only write SSZ states to disk with flag on

* lint

* also write blocks
2019-09-23 18:36:12 -05:00
Raul Jordan
305d0299dd
Revamp GetValidators to Retrieve Historical Validators By Epoch (#3563)
* archive participation begin implementation

* validator participation compute

* comments

* compute participation common func

* full test for archiving data

* gazelle

* complete tests

* gaz

* properly retrieve the validators

* revert weird change

* historical validator fetching

* resolves issues with current tests

* adding test for old epoch validators

* tests in
2019-09-23 18:00:38 -05:00
Preston Van Loon
5d33514001 Restore bootnode query tool for kaddht and fix bootstrapping (#3565)
* add back bootnode query tool

* bootstrap from DHT
2019-09-23 17:18:01 -05:00
Preston Van Loon
f5aa25821d
p2p: Relay support, CIDR whitelist, connection maintenance (#3564)
* add relay dial

* add relay support advertisement, add peer watching to maintain connections to bootstrap nodes and relay nodes

* gofmt

* double space
2019-09-23 14:43:53 -07:00
Raul Jordan
b8bdf71d5b
Allow for Fetching RPC Historical Assignments from Archived Data (#3559)
* archive participation begin implementation

* validator participation compute

* comments

* compute participation common func

* full test for archiving data

* gazelle

* complete tests

* gaz

* define archived validator assignment method

* archived assign logic

* need to use compute committee next

* compute archival assignments helper func

* properly compute committee using current shard

* modify the assignments request to take in a query filter item

* more intuitive implementation of list assignments

* utilize the query filter

* complete implementation

* revamp tests

* fixing current tests before adding archive tests

* test now passes using len filtered indices for total size

* include prop index in test

* revert bad change

* use ethapis

* add necessary tests

* comments
2019-09-23 16:04:59 -05:00
terence tsao
9577e2c123 Removed attestation check for votes (#3562) 2019-09-23 15:44:13 -05:00
skillful-alex
b015dc793a HeadFetcher data race fix (#3460)
* HeadFetcher data race fix

* bazel run //:gazelle -- fix

* add the db teardown to test

* add TestChainService_SaveHead_DataRace test

* split race and norace tests

* change testset name

* test CI with 'spectest' tag instead of 'raceon'

* one more test CI with 'spectest' tag instead of 'raceon'

* bazel run //:gazelle -- fix

* set test tag to 'race_on'

* not clone the state
2019-09-23 14:24:42 -05:00
terence tsao
4432c88f73 Update beacon chain server AttestationPool (#3560)
* Run time bug

* Still failing

* Run time working

* Run time working

* Gazelle

* Fixed all the tests

* Revert config

* Revert back test configs

* Revert config

* Tested run time again, everything is good

* Implemented AttestationPoolNoVerify
2019-09-23 13:01:57 -05:00
Nishant Das
b5b10a8d35 Add Back Kademlia DHT to Prysm (#3557)
* serve nodes

* remove testing flag

* add back bootnode

* add dht

* add back dht

* gaz

* fix build

* bootnode works in runtime

* fix all references

* all tests pass

* remove feature flag

* separate out ports

* lint

* fix docker build

* use one error package
2019-09-23 10:24:16 -07:00
Raul Jordan
5294a6c5af Use Archive in Retrieving Validator Balances (#3536) 2019-09-23 09:19:18 -07:00
terence tsao
ab2d4e8ad6 Implement attestation pool in memory (#3542)
* Run time bug

* Still failing

* Run time working

* Run time working

* Gazelle

* Fixed all the tests

* Revert config

* Revert back test configs

* Revert config

* Tested run time again, everything is good
2019-09-23 11:11:44 -05:00
Raul Jordan
64795bd231
Utilize Archived Data in GetValidatorParticipation RPC Server (#3526)
* archive participation begin implementation

* validator participation compute

* comments

* compute participation common func

* full test for archiving data

* gazelle

* complete tests

* gaz

* remove double negative grammar in comment

* use archive in rpc

* uses the archive!

* error if nothing found in archive

* comment

* use head fetcher and root

* tests pass

* archive active set changes appropriately

* archive committees

* archive info

* done with committee info archiving

* archived set changes stored

* fix build

* test for archive balances and active indices

* further abstractions

* only archive epoch end

* tests all pass

* tests pass now

* archive done

* test for activated validators

* tests for exited

* amend message

* use different proto

* finalization fetcher

* gaz

* use root

* use ctx

* use new ethapis

* use proper hash

* match apis compatibility

* match apis

* properly use participation

* fix tests

* use right commit
2019-09-23 10:47:34 -05:00
terence tsao
4e6ed2744d Fixed 2 cosmetic errors (#3543)
* Fixed

* Treehash

* Updated error msg
2019-09-23 08:30:39 -07:00
Nishant Das
41ea8a18a0 Expose Nodes (#3556)
* serve nodes

* remove testing flag
2019-09-23 08:18:58 -07:00
Nishant Das
71098b6ed8
Lookup Bootnode instead of Random Nodes (#3555)
* add workspace change

* change to lookup bootnode
2019-09-23 10:17:11 +05:30
terence tsao
b7853f1fa8 Batch renovate updates (#3554)
* Update graknlabs_bazel_distribution commit hash to d4a7864

* Update io_kubernetes_build commit hash to 24cc8eb

* Update dependency build_bazel_rules_nodejs to v0.37.1

* Update dependency com_google_cloud_go to v0.46.3

* Update libp2p

* Update dependency com_github_spf13_pflag to v1
2019-09-23 08:16:56 +05:30
Nishant Das
762f108ea5 add workspace change (#3545) 2019-09-22 11:14:04 -07:00
terence tsao
041735ef54
Removed dead code (#3544) 2019-09-21 22:04:20 -07:00
terence tsao
315d4f0549 Save aggregated attestations in DB after processing (#3541)
* Added AggregatedAttestation helper

* Implemented `saveNewAttestation` for processing attestation

* Implemented `savesNewBlockAttestations` for processing block

* Tests

* Fix name

* Raul's feedback
2019-09-21 13:57:32 -05:00
terence tsao
2b2ef4f37c
Added AggregatedAttestation helper (#3539) 2019-09-21 09:43:18 -07:00
Preston Van Loon
fb8d6a4046 infrequently ping bootnode (#3540) 2019-09-21 09:21:44 -07:00
Preston Van Loon
37596ac188 Is len(map) threadsafe? (#3535) 2019-09-20 13:05:08 -05:00
Preston Van Loon
9fcc6fc201
Wait until fully synced to process pubsub messages (#3514)
* context timeout for pubsub message processing

* add syncing check

* gofmt

* use a global cache

* lint

* fmt

* fix conflicts

* revert change

* gaz
2019-09-20 10:54:32 -07:00
terence tsao
c29a7be0ec Fix justified check point mutation (#3534)
* first version of the watchtower api

* first version

* delete watchtower

* move to message loop

* roughtime

* one time

* fix test

* Fixed

* Fixed

* Revert unused lock
2019-09-20 10:44:28 -07:00
terence tsao
cf2ad1f21c Parent blocks fetching/processing (#3459)
* first version of the watchtower api

* Initial prototype of sync parent fetching/processing

* Another map to track seen block root

* Fixed fmt

* Ready to live test

* Ready to live test

* Seperate pending block queue into its own

* first version

* delete watchtower

* move to message loop

* roughtime

* one time

* fix test

* Started testing but peer list empty

* Comment

* Loggins

* Stuck at decoding non proto type

* Revert

* First take, need feedback

* Run time panics at hello

* Revert

* use reflect properly

* Fixed subscriber

* instantiate helper

* More reverts

* Revert back tests

* Cont when EOF

* Working

* Clean hello tracker on peer disconnection

* Clean hello tracker on peer disconnection

* Move to validation

* Propoer locking

* Propoer locking

* Fmt

* Nishant's feedbacke

* More feedback

* All tests passing

* fix build

* remove log

* gaz

* Added the todo
2019-09-20 10:08:32 -07:00
Raul Jordan
a2aa142b90
Archive Remaining Data at Epoch End in Archiver Service (#3531)
* archive participation begin implementation

* validator participation compute

* comments

* compute participation common func

* full test for archiving data

* gazelle

* complete tests

* gaz

* remove double negative grammar in comment

* use head fetcher and root

* tests pass

* archive active set changes appropriately

* archive committees

* archive info

* done with committee info archiving

* archived set changes stored

* fix build

* test for archive balances and active indices

* further abstractions

* only archive epoch end

* tests all pass

* tests pass now

* archive done

* test for activated validators

* tests for exited

* use ctx
2019-09-20 11:51:06 -05:00
Nishant Das
44e5e5de65 Remove Error Message Type (#3533)
* proto change

* fix test

* fix error resp
2019-09-20 09:13:38 -05:00
Nishant Das
4bc2d628b1
Update Naming to Latest Networking Spec (#3519)
* update naming

* replace with updated version

* more changes

* fixed all tests

* build and lint

* regen protos

* fix test

* remove outdated code

* prestons review

* preston's comments

* preston's review

* preston's review

* lint
2019-09-20 11:57:28 +05:30
terence tsao
ac176a5078 Update validators votes from incoming block (#3530)
* first version of the watchtower api

* first version

* delete watchtower

* move to message loop

* roughtime

* one time

* fix test

* Update block attestation votes

* Clean up

* Found a bug

* Confirmed to be working in run time

* Confirmed to be working run time

* Raul's feedback

* Tests
2019-09-19 20:34:57 -05:00
Raul Jordan
4ffef61e1d
Archive Validator Participation on End of Epoch Event (#3524)
* archive participation begin implementation

* validator participation compute

* comments

* compute participation common func

* full test for archiving data

* gazelle

* complete tests

* gaz

* remove double negative grammar in comment

* use head fetcher and root

* tests pass
2019-09-19 15:59:23 -05:00
Preston Van Loon
cba44e5151 log a warning on unhealthy healthz (#3529) 2019-09-19 20:39:55 +05:30
Nishant Das
8179ed57b9
fix validator (#3527) 2019-09-19 13:30:47 +05:30
Raul Jordan
e8b6951591
Complete ListAttestations EthereumAPIs v1alpha (#3452)
* retrieve attestations by block root as well

* add beacon block root filter

* rem err unimpl

* add changes to list atts filter proto

* utilize the new filter attributes

* add filter types

* utilize filters in the api server impl

* tests for filter

* tests pass

* filter test done

* fix test by using head fetcher instead

* gaz

* no panic

* use new ethapis commit

* elim panic

* res panic

* ensure proto compatibility

* fixed broken test
2019-09-18 20:14:26 -05:00
Preston Van Loon
33ef5f9150 Use hex string private keys for enr calculator (#3525)
* use hex string private keys for enr calculator

* use hex string private keys for enr calculator
2019-09-18 16:27:34 -05:00
terence tsao
495621e99b
Sync RPC to support none proto message (#3512) 2019-09-18 13:48:16 -07:00
Preston Van Loon
e1861bdb31 Clean hello tracker on peer disconnection (#3522)
* Clean hello tracker on peer disconnection

* Clean hello tracker on peer disconnection
2019-09-18 15:02:34 -05:00