Commit Graph

328 Commits

Author SHA1 Message Date
Raul Jordan
2690c2080d
Implement GetValidatorQueue RPC (#3574)
* changes

* active set change

* helpers for active set changes

* include the helpers in archive service

* table driven tests for helpers

* use from archive

* remove item

* properly use the keys in the response

* test for active set changes

* test passing

* test passing no archive

* archive tests completed

* add ethapis latest commit

* begin implementation get validator queue

* include queue

* finish the queue implementation

* consolidated churn  limit

* pending active testing

* pending active t est

* tests below churn in response

* pubkey

* below  churn  test

* only fetches below the churn  limit

* exit queue churn clip as needed

* full test for  pending active below churn limits

* pending exit test

* pending exit logic

* pending exit below churn test

* all tests done for queue impl

* revert some bad changes

* bug
2019-09-26 16:46:06 -05:00
Raul Jordan
3a890e70f7
Implement Active Set Changes RPC Method (#3568)
* changes

* active set change

* helpers for active set changes

* include the helpers in archive service

* table driven tests for helpers

* use from archive

* remove item

* properly use the keys in the response

* test for active set changes

* test passing

* test passing no archive

* archive tests completed

* add ethapis latest commit

* fix test
2019-09-25 18:06:02 -05:00
Nishant Das
8ce96428b1 Fixes our Attestation Aggregation Issues (#3579) 2019-09-25 07:39:16 -07:00
Nishant Das
0e329fc115 Attestation Server Fix (#3570) 2019-09-24 08:19:37 -07:00
Preston Van Loon
07360bcc07
Write failed blocks to disk (#3569)
* write a failed requested block to disk

* write faile blocks from pubsub too

* gaz
2019-09-23 21:46:40 -07:00
Raul Jordan
305d0299dd
Revamp GetValidators to Retrieve Historical Validators By Epoch (#3563)
* archive participation begin implementation

* validator participation compute

* comments

* compute participation common func

* full test for archiving data

* gazelle

* complete tests

* gaz

* properly retrieve the validators

* revert weird change

* historical validator fetching

* resolves issues with current tests

* adding test for old epoch validators

* tests in
2019-09-23 18:00:38 -05:00
Raul Jordan
b8bdf71d5b
Allow for Fetching RPC Historical Assignments from Archived Data (#3559)
* archive participation begin implementation

* validator participation compute

* comments

* compute participation common func

* full test for archiving data

* gazelle

* complete tests

* gaz

* define archived validator assignment method

* archived assign logic

* need to use compute committee next

* compute archival assignments helper func

* properly compute committee using current shard

* modify the assignments request to take in a query filter item

* more intuitive implementation of list assignments

* utilize the query filter

* complete implementation

* revamp tests

* fixing current tests before adding archive tests

* test now passes using len filtered indices for total size

* include prop index in test

* revert bad change

* use ethapis

* add necessary tests

* comments
2019-09-23 16:04:59 -05:00
terence tsao
4432c88f73 Update beacon chain server AttestationPool (#3560)
* Run time bug

* Still failing

* Run time working

* Run time working

* Gazelle

* Fixed all the tests

* Revert config

* Revert back test configs

* Revert config

* Tested run time again, everything is good

* Implemented AttestationPoolNoVerify
2019-09-23 13:01:57 -05:00
Raul Jordan
5294a6c5af Use Archive in Retrieving Validator Balances (#3536) 2019-09-23 09:19:18 -07:00
terence tsao
ab2d4e8ad6 Implement attestation pool in memory (#3542)
* Run time bug

* Still failing

* Run time working

* Run time working

* Gazelle

* Fixed all the tests

* Revert config

* Revert back test configs

* Revert config

* Tested run time again, everything is good
2019-09-23 11:11:44 -05:00
Raul Jordan
64795bd231
Utilize Archived Data in GetValidatorParticipation RPC Server (#3526)
* archive participation begin implementation

* validator participation compute

* comments

* compute participation common func

* full test for archiving data

* gazelle

* complete tests

* gaz

* remove double negative grammar in comment

* use archive in rpc

* uses the archive!

* error if nothing found in archive

* comment

* use head fetcher and root

* tests pass

* archive active set changes appropriately

* archive committees

* archive info

* done with committee info archiving

* archived set changes stored

* fix build

* test for archive balances and active indices

* further abstractions

* only archive epoch end

* tests all pass

* tests pass now

* archive done

* test for activated validators

* tests for exited

* amend message

* use different proto

* finalization fetcher

* gaz

* use root

* use ctx

* use new ethapis

* use proper hash

* match apis compatibility

* match apis

* properly use participation

* fix tests

* use right commit
2019-09-23 10:47:34 -05:00
terence tsao
4e6ed2744d Fixed 2 cosmetic errors (#3543)
* Fixed

* Treehash

* Updated error msg
2019-09-23 08:30:39 -07:00
Preston Van Loon
9fcc6fc201
Wait until fully synced to process pubsub messages (#3514)
* context timeout for pubsub message processing

* add syncing check

* gofmt

* use a global cache

* lint

* fmt

* fix conflicts

* revert change

* gaz
2019-09-20 10:54:32 -07:00
Raul Jordan
4ffef61e1d
Archive Validator Participation on End of Epoch Event (#3524)
* archive participation begin implementation

* validator participation compute

* comments

* compute participation common func

* full test for archiving data

* gazelle

* complete tests

* gaz

* remove double negative grammar in comment

* use head fetcher and root

* tests pass
2019-09-19 15:59:23 -05:00
Nishant Das
8179ed57b9
fix validator (#3527) 2019-09-19 13:30:47 +05:30
Raul Jordan
e8b6951591
Complete ListAttestations EthereumAPIs v1alpha (#3452)
* retrieve attestations by block root as well

* add beacon block root filter

* rem err unimpl

* add changes to list atts filter proto

* utilize the new filter attributes

* add filter types

* utilize filters in the api server impl

* tests for filter

* tests pass

* filter test done

* fix test by using head fetcher instead

* gaz

* no panic

* use new ethapis commit

* elim panic

* res panic

* ensure proto compatibility

* fixed broken test
2019-09-18 20:14:26 -05:00
Nishant Das
26af4496c0 Batch Validator Performance Requests (#3520)
* change proto msg types

* change server and client

* regen protos
2019-09-18 11:47:14 -05:00
terence tsao
9ab08e6998
Remove beacon rpc service (#3515)
* first version of the watchtower api

* first version

* delete watchtower

* move to message loop

* roughtime

* one time

* fix test

* Fixed test

* Fixed proposer server

* Gaz

* gaz

* Stuck

* Tests passing

* Fixed all the tests
2019-09-18 06:34:50 -07:00
Nishant Das
b1b76ac87c Handle Attestations in a Separate Goroutine (#3487)
* move into separate routine

* preston's review

* use opencensus
2019-09-17 11:17:21 -05:00
Raul Jordan
49a529388b
Resolve Miscellaneous Prysm TODOs (#3465)
* resolve

* resolve

* return

* remove deprecated protos

* rem deprecated pbs

* resolve cache

* resolve md TODO

* node server

* resolve config todo

* resolve even more

* broken build
2019-09-16 15:45:03 -05:00
terence tsao
bf07cfcdab Clean up operation service (#3468)
* Cleaned up operation service

* Fixed all the tests

* Fixed node.go

* Review feedback

* Todo
2019-09-16 12:05:30 -05:00
terence tsao
9d62e542e5
Clean up validator server (#3466) 2019-09-14 13:31:38 -07:00
Raul Jordan
e0e3dada7b
Abstract Usage of Deposit Cache Into Interface (#3443)
* deposit cache refactor begin

* use interface for has chain  started

* use deposit fetcher interface instead

* use moar interfaces

* comment

* gaz

* fix breaking build

* lint

* implement chainstart fetcher

* allow start to work

* fix broken tests
2019-09-11 23:30:04 -05:00
terence tsao
798bbbdc82 Cold start for interop (#3437)
* coldstart flags for validator

* WIP beacon node flags

* wip beacon chain, flag fix in validator, arg fix in validator

* checkpoint

* Added interop service

* working on mock chainstart

* save the state lol

* fix tests

* Save genesis validators

* gaz

* fix validator help flags

* WaitForChainStart actually waits for genesis time

* cold start fixes

* cache

* change back

* allow for genesis state too

* remove logs

* increase mmap size

* dont process if head doesn't exist

* add 10ms tolerance

* enable libp2p debug at debug, fix pubsub

* works with checkpt

* initialize justified and finalized in db

* Removed preloadStatePath from blockchain service

* Clean up

* Write to disk for now post state

* revert b466dd536f8eadbdae2264a545a755370223d917

* Builds

* Only RPC test fails now

* use minimal config, no demo config

* clean up branch

* Lint

* resolve lint

* more lint fixes

* lint

* fix viz

* Fixing RPC test

* skip before epoch 2

* RPC time out

* Fixed ordering

* rename

* remove some dbg statements

* ensure index is correct

* fix some panics

* getting closer

* fix tests

* Fix private key

* Fixed RPC test

* Fixed beacon chain build for docker

* Add interop.go to validator go_image

* Fixed docker build

* handle errors

* skip test, skip disconnecting peers

* Fixed docker build

* tolerance for attestation processing

* revert copy

* clearer err message parse

* fix launching from dep contract
2019-09-11 13:38:35 -05:00
Raul Jordan
af07c13730
[Interop] Improve RPC Codebase + Start Beacon Chain With Mock ETH1 Values (#3407)
* add main.go

* interop readme

* proper visibility

* standardize and abstract into simpler funcs

* formatting

* no os pkg

* add test

* no panics anywhere, properly and nicely handle errors

* proper comments

* fix broken test

* readme

* comment

* recommend ssz

* install

* tool now works

* README

* build

* readme

* 64 validators

* rem print

* register the no powchain flag

* work on mock eth1 start

* common interface

* getting closer with the interface defs

* only two uses of powchain

* remove powchain dependency

* remove powchain dependency

* common powchain interface

* proper comment in case of flag

* proper args into rpc services

* rename fields

* pass in mock flag into RPC

* conforms to iface

* use client instead of block fetcher iface

* broken tests

* block fetcher

* finalized

* resolved broken build

* fix build

* comment

* fix tests

* tests pass

* resolved confs

* took them out

* rename into smaller interfaces

* resolve some confs

* ensure tests pass

* properly utilize mock instead of localized mock

* res lint

* lint

* finish test for mock eth1data

* run gazelle

* include flag again

* fix broken build

* disable powchain

* dont dial eth1 nodes

* reenable pow

* use smaller interfaces, standardize naming

* abstract mock into its own package

* faulty mock lint

* fix stutter in lint

* rpc tests all passing

* use mocks for operations

* no more mocks in the entire rpc package

* no  mock

* viz

* testonly
2019-09-09 17:13:50 -04:00
Ivan Martinez
5e939378d0 Update to spec v0.8.3 (#3355)
* Ignore latest messages in fork choice prior to latest justified

* Make sure Compact Committee Roots isn't changed by process_final_updates

* WIP add attestation bitfields length to match committee length

* Begin work on updating spec tests to 0.8.2

* WIP set up for new spec test structure

* Fix slashings

* Get mainnet tests mostly passing for attestations and attester slashings

* Fix process attestation test

* Undo change

* Complete spec tests for all operations
Still need sanity block tests

* Fix BLS sigs

* Reduce amount of reused code in core/blocks/spectests/

* Fix tests

* Update block sanity tests to 0.8.2

* Update epoch spec tests to 0.8.2

* Clean up all tests and fix shuffling/epoch tests

* WIP update bls tests to 0.8.2

* WIP update bls tests to 0.8.3

* Finish BLS spectest update to 0.8.3

* Fix shuffling spec tests

* Fix more tests

* Update proto ssz spec tests to 0.8.3

* Attempt to fix PrevEpochFFGDataMismatches test

* Goimports

* Fix documentation

* fix test

* Use custom general spec tests

* Reduce code footprint

* Remove unneeded minimal skip

* Fix for comments

* Fix for comments

* Fix test

* Small fixes

* Cleanup block spec tests a bit

* Undo change

* fix validator

* Fix validator tests

* Run gazelle

* Fix error output for epoch spec tests
2019-09-08 12:41:52 -07:00
terence tsao
4235980511 Clean up post --next (#3411)
* Delete old code

* RPC mock testing

* Fixed BUILD

* Conflict

* Lint

* More lint
2019-09-06 22:39:14 -04:00
Nishant Das
fb20fc7881 change to wrap (#3413) 2019-09-06 14:01:59 -07:00
terence tsao
14c59b2ff9 Remove deprecated services and --next (#3371)
* Save new validators in DB

* Use info

* Add total validator count

* Fixed tests

* Add new test

* Revert light client config

* Add state metrics back

* Gaz

* Mark old ones as deprecated

* Deprecate not --next services

* Fixed all operation tests

* Fixed node test

* All tests passing locally

* Add deprecated-p2p back, blocked by boostrap-query

* Revert message proto

* delete deprecated DB items

* delete all other instances of old db

* gaz

* cycle rem

* clear db
2019-09-05 11:04:06 -05:00
Preston Van Loon
b0e6d7215c
Tracing: Add additional attributes (#3404)
* Add some attributes for tracing

* gaz
2019-09-03 20:03:09 -07:00
terence tsao
314bc513af
Fixed validator pubkey -> index getter (#3361) 2019-08-30 13:50:21 -07:00
terence tsao
d8fd7e502a Fix GetChainHead for RPC (#3352)
* Fix ChainHeadQuery

* Fixed test
2019-08-29 10:17:21 -05:00
Raul Jordan
e8f030977a
wait for chainstart (#3343) 2019-08-28 12:07:58 -05:00
terence tsao
9f2c2f0197 Minor runtime fixes (#3335) 2019-08-27 22:19:47 -05:00
Nishant Das
701c70ae3b add better logging (#3329) 2019-08-27 06:27:04 -07:00
terence tsao
f188609137 Implement GetHead for RPC (#3326)
* next compatible, tests pass

* terence feedback

* skip comment

* fixes

* misc fix

* on block

* Update RPC service to use chain info

* OOps

* All tests pass, run time with 8 validators work!

* Remove saving genesis validator

* Revert gensis count

* Move redundant headstate

* Comments

* Implemented GetChainHead

* Test works

* Moved mock package

* Fixed visibility for BUILD file

* Conflict
2019-08-26 15:59:17 -05:00
terence tsao
64d0826469
Update RPC service to use chain info (#3309) 2019-08-26 13:06:16 -07:00
Raul Jordan
9ad00ffafb
Use New Attestation Receiver Method in RPC (#3287)
* next compatible, tests pass

* terence feedback

* skip comment

* fixes

* misc fix

* on block

* resolved err
2019-08-25 15:45:55 -05:00
terence tsao
045badc5f3
Fix competing attestation check (#3305) 2019-08-24 16:56:40 -07:00
Nishant Das
111f225177 Remove IsAttCanonical From Operations Service (#3298) 2019-08-24 06:50:43 -06:00
Raul Jordan
2e8a06d6d4
Use New Blockchain Service in RPC Package (#3286)
* new chain service usage via interface

* put in the new chain service in propose blk

* deprecate with new service for canonical block roots

* remove old chain serv absolutely in validator server

* full legacy code compatible in beacon server

* fully compliant

* full deprecation at service level

* no more mock chain serv

* fix beacon server tests

* add changes to prop server

* broken build

* --next compatible

* conditional register of chain service

* proper conversion

* nil deref
2019-08-23 13:53:07 -05:00
Nishant Das
ebb0e398d3 Deposit Cache Fix (#3280)
* fix cache

* fix spacing
2019-08-22 22:49:03 -04:00
Raul Jordan
f342224410
Full RPC Package Compliance With New DB Interface (#3275)
* deprecate db

* fix build

* begin integrating new db

* gaz

* use more of the new db

* newest implementation uses head state

* remove more deprecated items

* setup validators in state helper

* fix up some tests with the new db

* resolve broken build

* gaz

* begin ensuring tests pass

* optional idx

* list validator balances passing

* default page size passing

* only two failing

* fixed most tests, found edge case

* allow nil return and add proper tests

* pass tests

* fix head block root problem

* working with the new db

* every ethereumapis method now compliant with both dbs

* pass in db into server

* proposer server all compliant

* validator service fully compliant

* fix broken build, tests pass

* spacing

* compute state root and propose block tests passing with new db

* complete proposer server tests revamp

* validator tests halfway through passing with new db

* more validator server tests

* more than halfway there

* so so close

* all validators tests done

* attester server tests fixing

* use new api

* attester server complete

* complete
2019-08-22 20:39:06 -05:00
Raul Jordan
8f01b76366
Integrate DB Refactor Into Ethereum APIs Beacon Chain Server (#3245)
* deprecate db

* fix build

* begin integrating new db

* gaz

* use more of the new db

* newest implementation uses head state

* remove more deprecated items

* setup validators in state helper

* fix up some tests with the new db

* resolve broken build

* gaz

* begin ensuring tests pass

* optional idx

* list validator balances passing

* default page size passing

* only two failing

* fixed most tests, found edge case

* allow nil return and add proper tests

* pass tests

* fix head block root problem

* working with the new db

* every ethereumapis method now compliant with both dbs

* pass in db into server
2019-08-22 15:28:53 -05:00
Nishant Das
83083b9c65
Fix BLS Aggregation Method (#3269)
* lint

* update to new method

* fix all tests
2019-08-22 11:45:02 +05:30
terence tsao
b91639a32e
Deprecate old block chain service (#3268)
* seperate out block chain services

* Fix blockchain service config

* Gazelle

* Fixed tests
2019-08-21 19:14:24 -06:00
Preston Van Loon
01de412956
Minor runtime fixes for --next (#3265)
* some runtime fixes

* fixes

* fixes

* fixes

* fixes

* fixes
2019-08-21 16:58:38 -04:00
Preston Van Loon
acb20e269c Add flags to support new database, new sync (#3252) 2019-08-21 10:04:00 -06:00
Preston Van Loon
0b8cbd06b6
Add flag for testing new p2p (#3243)
* refactor a bit to select p2p

* lint

* fix build

* fix build

* fix build

* fix build

* fix build
2019-08-19 17:20:56 -04:00
terence tsao
6ac0d12f5b Part 4 of update fork choice - mark old functions deprecated (#3215)
* Mark these soon to be deprecated functions as "deprecated"

* Deprecate all

* Marked fork choice reorg test as deprecated

* Marked fork choice reorg test as deprecated

* Gaz
2019-08-19 11:34:25 -04:00