Commit Graph

628 Commits

Author SHA1 Message Date
terence tsao
f10f87725e
Avoid increment reorg counter twice (#2082) 2019-03-26 17:01:43 -07:00
Preston Van Loon
ac7d5a7780
Update state metrics on save state (#2083)
* Update state metrics on save state

* fix tests
2019-03-26 18:29:45 -05:00
Preston Van Loon
ec41d37cf4
Revert grpc error in db package (#2076)
* Revert "status err code"

This reverts commit 27112ff2841ae7ccfb7d7eca665e06bbad81755d.

* imports
2019-03-26 16:11:21 -05:00
Preston Van Loon
699a8e9087 use a mutex/lock on the attestation store so that it is not being mutated during reads (#2079) 2019-03-26 14:00:49 -07:00
Preston Van Loon
7420fac5fc
Add gRPC prometheus middleware (#2077) 2019-03-26 11:26:47 -05:00
terence tsao
724383751e Helpful Logging for UpdateLatestAttestation (#2078)
* helpful logging for updating latest attestation

* debug over info

* imports
2019-03-26 11:16:36 -05:00
Raul Jordan
a4ea0a1eac
Disable Crosslinks for Testnet (#2075)
* disable crosslinks for testnet

* feature config

* add missing flag

* flag in tests

* fix tests
2019-03-26 10:40:55 -05:00
Raul Jordan
fcc18fac02
Treat Post & Pre-ChainStart Validators Equally in Assignments Fetching (#2074)
* treat post/prechainstart validators equally in assignment fetching

* Update validator/client/fake_validator_test.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* status err code

* imports

* beyond activation epoch

* assigned to slot log

* gazelle
2019-03-26 09:03:05 -05:00
Nishant Das
0809b96c32 Fixes Initial Sync (#2041)
* plug forkchoice to blockchain service's block processing

* fixed tests

* more fixes...

* clean ups

* fixed test

* Update beacon-chain/blockchain/block_processing.go

* merged with 2006 and started fixing tests

* remove prints

* fixed tests

* lint

* include ops service

* if there's a skip slot, slot--

* fixed typo

* started working on test

* no fork choice in propose

* bleh, need to fix state generator first

* state gen takes input slot

* feedback

* fixed tests

* preston's feedback

* fmt

* removed extra logging

* add more logs

* fixed validator attest

* builds

* fixed save block

* children fix

* removed verbose logs

* fix fork choice

* right logs

* Add Prometheus Counter for Reorg (#2051)

* fetch every slot (#2052)

* test Fixes

* lint

* only regenerate state if there was a reorg

* better logging

* fixed seed

* better logging

* process skip slots in assignment requests

* fix lint

* disable state root computation

* filter attestations in regular sync

* log important items

* better info logs

* added spans to stategen

* span in stategen

* set validator deadline

* randao stuff

* disable sig verify

* lint

* lint

* save only using historical states

* use new goroutine for handling sync messages

* change default buffer sizes

* better p2p

* rem some useless logs

* lint

* sync tests complete

* complete tests

* tests fixed

* lint

* fix flakey att service

* bug fixes

* Update beacon-chain/sync/initial-sync/service.go

* Update beacon-chain/sync/initial-sync/service.go

* reenable shuffle

* enable shuffle
2019-03-25 13:23:57 -05:00
Raul Jordan
053038446c
Allow 8 Validator Multinode Cluster to Run Indefinitely (#2050)
* plug forkchoice to blockchain service's block processing

* fixed tests

* more fixes...

* clean ups

* fixed test

* Update beacon-chain/blockchain/block_processing.go

* merged with 2006 and started fixing tests

* remove prints

* fixed tests

* lint

* include ops service

* if there's a skip slot, slot--

* fixed typo

* started working on test

* no fork choice in propose

* bleh, need to fix state generator first

* state gen takes input slot

* feedback

* fixed tests

* preston's feedback

* fmt

* removed extra logging

* add more logs

* fixed validator attest

* builds

* fixed save block

* children fix

* removed verbose logs

* fix fork choice

* right logs

* Add Prometheus Counter for Reorg (#2051)

* fetch every slot (#2052)

* test Fixes

* lint

* only regenerate state if there was a reorg

* better logging

* fixed seed

* better logging

* process skip slots in assignment requests

* fix lint

* disable state root computation

* filter attestations in regular sync

* log important items

* better info logs

* added spans to stategen

* span in stategen

* set validator deadline

* randao stuff

* disable sig verify

* lint

* lint

* save only using historical states

* use new goroutine for handling sync messages

* change default buffer sizes

* better p2p

* rem some useless logs

* lint

* sync tests complete

* complete tests

* tests fixed

* lint

* fix flakey att service

* PR feedback

* undo k8s changes

* Update beacon-chain/blockchain/block_processing.go

* Update beacon-chain/sync/regular_sync.go

* Add feature flag to enable compute state root

* add comment

* gazelle lint fix
2019-03-25 10:21:21 -05:00
Preston Van Loon
5a1cda482b
add error reporting to spans, remove a few unnecessary spans, add some attribute data to spans (#2056) 2019-03-23 20:54:02 -05:00
shayzluf
6335833c94 Validator: performing for multiple keys - update rpc proto (#2040)
* first version - broken

* Scripts to update mockgen

* working proto changes

* resolve review remarks

* fix goimport issues
2019-03-23 19:46:25 -05:00
Ivan Martinez
ca90acca77 Create feature config for selective testing (#1997)
* Create a feature flagging system

* Refactor how featureflags works and improve docs

* Gazelle and final touches

* Refactor, new flags in the package

* Remove old flags since they have been moved

* Fix bazel

* Update to comments

* Full coverage

* Fix formatting

* Align code with numbers

* Run bazel and fix docs

* Fix test for flag
2019-03-20 22:57:25 -04:00
terence tsao
18bdcedd26
Test on Skip Slot for FFG Check Point (#2049) 2019-03-20 18:57:34 -07:00
Preston Van Loon
b806cd7cf9
State size metric (#1780)
* add metric for state-size

* update comment
2019-03-20 21:29:24 -04:00
terence tsao
92640f9182
Refactor State Gen To Take Input Slot (#2045) 2019-03-20 15:33:31 -07:00
Preston Van Loon
83b00369b4 add regression test (#2046) 2019-03-20 15:19:48 -07:00
terence tsao
0aae75a4e7 Handle skip slots for FFG check points (#2044)
* if there's a skip slot, slot--

* fixed typo
2019-03-20 11:18:55 -06:00
Preston Van Loon
d7291ed123
fix this to the proper message type (#2043) 2019-03-20 11:13:52 -04:00
Preston Van Loon
7f11cbd5ed
print stack trace on panic recovery (#2042) 2019-03-20 10:17:44 -04:00
terence tsao
4a2ef17be1 fixed typo (#2038) 2019-03-20 01:13:14 -04:00
Nishant Das
bb20b3eaad Fix State Pruning (#2034)
* fix pruning

* Update beacon-chain/db/state.go

Co-Authored-By: nisdas <nish1993@hotmail.com>

* fix tests

* fix config params
2019-03-20 00:03:45 -04:00
Preston Van Loon
03dfafcb43
Broadcast attestations (#2035)
* broadcast attestations

* register p2p topics

* handle p2p messages for attestation announcements

* p2p -> P2P

* gazelle

* Remove unused var
2019-03-19 23:54:16 -04:00
Raul Jordan
78acb82182 Broadcast Blocks Before Processing (#2033)
* broadcast before processing

* fix sync
2019-03-19 23:41:38 -04:00
Preston Van Loon
58a13a2eb7 Use imported interfaces in sync package (#2032)
* Fix spans in regular_sync

* more regular sync ctx fixes

* rename

* use more imported interfaces

* add godoc comments
2019-03-19 17:07:49 -06:00
Preston Van Loon
eb080799ec
Fix spans in regular_sync (#2031)
* Fix spans in regular_sync

* more regular sync ctx fixes

* rename
2019-03-19 17:35:01 -04:00
terence tsao
7f5b6eb3fc
Update FFG Check Points During Block Processing (#2006) 2019-03-19 10:16:17 -07:00
terence tsao
fe7be56bec
log db path (#2029) 2019-03-19 10:53:17 -06:00
Preston Van Loon
f44c7082d8
Prevent invalid pending attestations (#1994)
* prevent invalid pending attestations

* no past block

* regress test

* +1

* fix lint

* fix buld

* revert unrelated

* passing tests, still need regression tst
2019-03-19 12:25:34 -04:00
Nishant Das
f801bfedcf Fix State Generator (#2010) 2019-03-18 20:15:50 -06:00
Raul Jordan
7a83acf46d
Improve Validator Logging and Default Flags (#2013)
* improve validator logging significantly

* warn validator not yet assigned to epoch

* build added

* report validator logs

* validator performance logging

* all logging improvements

* validator attest better logs

* improved attester logging

* average balances

* dont hash useless hashes

* Update validator/client/validator_propose.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* address some comments

* standardize with block root hash32, renamings

* gazelle

* builds

* improve average eth balance

* eth net gains/losses

* fix tests

* spacing

* goimports

* avg balance

* update pbs

* addressed preston comments

* imports

* gazelle
2019-03-18 09:45:28 -06:00
Raul Jordan
a98912ea58
State Transition Configuration Options (#2023)
* logging config options

* tests pass

* better naming

* log proposer idx instead of sig

* log avg balance
2019-03-18 00:19:44 -06:00
Nishant Das
78ff565429
Update Aggregate Attestation (#2004)
* add fixes

* Apply suggestions from code review

Co-Authored-By: nisdas <nish1993@hotmail.com>

* review comments
2019-03-18 11:57:12 +08:00
gzuhlwang
bb1389e362 fix typo (#2005) 2019-03-17 12:35:58 -04:00
Preston Van Loon
612bb38077 Cross p2p spans, more spans, synchronous attestations, minor fixes (#2009)
* Fix assignments bug where validators don't retry for assignments on failure

* synch only please

* trying to fix state issues

* trying random stuff

* do not explode

* use ctx

* working build, failing tests

* broadcast local addrs as well as relay addrs

* fixed p2p tests, more tests to fix still

* another test fixed, log warning instead of throw error

* Fix last tests

* godoc

* add test for broadcast in apply fork choiec

* remove unneeded code

* remove tracer adapter, not needed

* remove extra stuff

* remove any

* revert addr_factory

* revert addr_factory

* Revert "revert addr_factory"

This reverts commit e93fb706494a1070158b8db31e67146d6b0648ad.

* Revert "revert addr_factory"

This reverts commit dedaa405559cc818698870c4e4570953367f1e3c.

* revert removal of this code

* unused param
2019-03-17 10:56:05 +08:00
Nishant Das
04f4a7f156
Add Clear DB Flag (#2002)
* clearDB

* lint
2019-03-15 11:20:49 +08:00
Raul Jordan
f1d77a816e Revert "Update FFG Check Points During Block Processing (#1991)" (#1998)
This reverts commit b1ffc693e0.
2019-03-14 16:05:31 -07:00
Nishant Das
00ca1d7991 Fix Errors with Attestation to Current Chainhead (#1990)
* fix errors

* fix bit check to little endian

* fix all tests
2019-03-14 12:29:44 -04:00
terence tsao
b1ffc693e0 Update FFG Check Points During Block Processing (#1991)
* update FFG check points in service

* where do stategenerators go?

* commented out state_generator_test for now

* moved state generator back to appropiate location

* fixed block processing with state generator

* fmt

* import circle bad commit

* fix

* taking preston's fix

* gaz

* save justified/finalized objs during beacon chain init
2019-03-14 11:22:05 -04:00
Nishant Das
d5c05f083b
add to cache (#1988) 2019-03-14 14:43:39 +08:00
terence tsao
e5f66d0b21
Revert "Update FFG Check Points During Block Processing (#1982)" (#1987)
This reverts commit 2c805b731d.
2019-03-13 21:48:54 -07:00
terence tsao
2c805b731d Update FFG Check Points During Block Processing (#1982)
* update FFG check points in service

* where do stategenerators go?

* commented out state_generator_test for now

* moved state generator back to appropiate location

* fixed block processing with state generator

* fmt

* import circle bad commit

* fix

* taking preston's fix

* gaz
2019-03-14 00:22:47 -04:00
Nishant Das
a2e3ae95ae revert change (#1985) 2019-03-13 20:31:03 -04:00
Raul Jordan
5371218b9b
Refactor ChainService for Receiving Blocks Synchronously (#1984)
* refactor chain service

* restructure service and lint

* all calls to receive block in chain service are now blocking

* begin fixing tests

* refactored blockchain tests

* builds correctly

* blockchain tests pass again

* lint

* sync and rpc tests pass again

* done

* add in open tracing

* span in fork choice

* Update beacon-chain/blockchain/block_processing.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>
2019-03-13 17:17:32 -04:00
Raul Jordan
544c040fcb
restructure powchain (#1983) 2019-03-13 16:47:00 -04:00
terence tsao
b89391fcad
DB wrappers for justified and finalized objects (#1978) 2019-03-13 08:12:42 -07:00
Ivan Martinez
84b7c3654b Rename forkutils -> forkutil (#1976) 2019-03-12 16:39:13 -07:00
Nishant Das
000a42deac Make State Requests only Request for Finalized State (#1955)
* remove hash field for state request

* change to finalized state root

* refactor code for bug

* add in saving of finalized state

* add method to db

* adding finalized state root

* add new changes

* add in new changes

* add in return vars

* goimports

* review comments

* error message

* fix broken build

* fix cache state tests

* lint
2019-03-12 16:44:21 -04:00
Ivan Martinez
700264b3c7 Add caching to beacon state DB (#1775)
* Initial addition to cache beacon state

* Add benchmark to tests

* Fix benchmarks

* Fix benchmarks

* Fix caching to requests

* Update to master

* Remove b.N assignments

* Init cache on state init
2019-03-12 15:30:10 -04:00
terence tsao
c320877dea
Part 1 of X - LMD Ghost Benchmark Tests (#1950) 2019-03-12 07:08:13 -07:00
Raul Jordan
c1a9937760
Post ChainStart Validator Activation (#1934)
* implemented all the merkle func signatures

* branch indices complete

* check for index out of range in generating merkle proof

* completed full tests for sparse merkle trie impl

* lint

* formatting

* gazelle

* commentary

* ivan comments

* fmt

* get rid of the deposit trie

* recalculate trie when eth1 data changes

* default data response recalculates tree

* update merkle trie to use raw bytes

* use the new verify merkle root func

* builds

* if default data response historical deposits are empty, return the deposit root at the contract at the time

* work on trie

* trying again with more logging

* keep track of merkle trie indices, use correct big int ops

* use uint for merkle idx

* add todo

* update ticker correctly

* fix config and remove unnecessary logs

* readd plus one fix

* clarify some details

* weird imports spacing

* gazelle, lint

* fix tests using the new deposit trie

* builds but tests still fail

* rpc tests

* lint

* tests pass

* bazel lint

* rem commented block

* revert att slot fix

* preston comments

* comments

* fix build

* address last comment

* use counter

* imports
2019-03-12 00:05:55 -04:00
Preston Van Loon
4266757511
typo (#1968) 2019-03-11 21:15:50 -04:00
Nishant Das
7179b6930b Remove Incorrect Error Logs in Sync (#1943) 2019-03-11 15:07:16 -07:00
terence tsao
64913d66f4
Fix off by 1 for attestation justified epoch checking (#1963) 2019-03-10 21:07:05 -07:00
Preston Van Loon
bdbc572f3f Add grpc middleware to recover from panics (#1956) 2019-03-10 18:19:52 -07:00
Preston Van Loon
e4588d9be0
Revert "Create and Verify Signatures for Attestations (#1908)" (#1958)
This reverts commit 23072983ff.
2019-03-10 20:02:47 -04:00
Nishant Das
028506e7ed Handle Panics in Remaining Services (#1926)
* add recovery for initial sync

* add recovery for rpc

* adding to other services

* remaining services and tests

* fix test

* remove changes to rpc

* handle powchain

* handle in powchain

* abstract to shared package

* gazelle

* lint

* remove for operations

* add in more for operations

* travis

* fix lint
2019-03-10 17:53:28 -05:00
terence tsao
19977afa96
Misc updates for epoch processing (#1957) 2019-03-10 12:21:40 -07:00
terence tsao
2c14c3e109
Clean up Atts Older Than Epoch Length (#1942) 2019-03-09 15:40:51 -06:00
Preston Van Loon
509c973936 minor fixes (#1949) 2019-03-08 21:12:38 -06:00
Nishant Das
c989021393
Fix Hashing Errors in Sync (#1948)
* fix incorrect hashing

* fix hashing of blocks
2019-03-09 10:36:44 +08:00
Nishant Das
7c5a7fb044 Handle Skipped Slots in Sync (#1939)
* handle edge cases

* tests passing

* Update beacon-chain/sync/initial-sync/service.go

* Update beacon-chain/sync/initial-sync/service.go

* Update beacon-chain/sync/initial-sync/service_test.go
2019-03-08 10:09:24 -06:00
terence tsao
40d082bdc5
LMD GHOST fork choice implementation and tests (#1933) 2019-03-08 07:55:55 -06:00
Ivan Martinez
23072983ff Create and Verify Signatures for Attestations (#1908)
* Verify signatures of attestations

* Implement BLS Signing for attestations

* Remove custody bit 0 from the attestation for now

* Fixes tests for attestations

* Fix tests to ensure they use proper attester indice

* Run gazelle

* Goimports

* Test attestation sigs in block operations

* Change formatting and make sure signatures are actually verified

* Fix duplicate import

* Fix duplicate import from merge

* Organize attestation sig to be conssitent with codebase

* Update to comments

* Change signatures to use aggregation

* Run gazelle

* Change function to return err instead of bool
Also gofmt

* Fix for comments

* Move createAggregationSignature to a function in attestations.go
2019-03-08 15:30:01 +08:00
Raul Jordan
205594c5d6 Fix Travis Failures on Master (#1938)
* resolve travis conflicts

* cp proto

* revert

* remove unused context
2019-03-07 23:54:41 -05:00
Preston Van Loon
2e7b08f97e
p2p fixes / improvements (#1931)
* don't broadcast to yourself

* todo

* Also listen for incoming streams

* send to a specific peer

* add godoc comment

* handle EOF as a normal thing

* gazelle

* add test for subscribing via direct comm
2019-03-07 21:37:15 -05:00
Nishant Das
2f9de6f93c Clean Up Batched Request (#1932)
* goimports

* do not log empty batches

* invalid request
2019-03-07 16:43:48 -06:00
terence tsao
1b995d1527
Fix Current Block Children Implementation (#1923) 2019-03-07 11:03:16 -06:00
Preston Van Loon
90225aac31 PendingAttestations: allow for request to specify a slot (#1922)
* PendingAttestations: allow for request to specify a slot

* enhance the test
2019-03-07 09:56:22 -06:00
Nishant Das
de22ef2bbd add tests and fix (#1925) 2019-03-07 09:39:29 -05:00
Nishant Das
6d69ac1abd
Add State Generator (#1718)
* add finalized function

* add functions

* gazelle

* add separate package for import cycles

* main tests are passing

* add finalized state test

* add block test

* fix tests and gazelle

* lint

* build file

* remove package

* remove unecessary method

* visibility

* remove package

* comments

* final comments

* imports spacing

* goimports
2019-03-07 11:02:47 +08:00
terence tsao
23e2dd65a3
Sync Drops Attestations < 1 Epoch Length (#1917) 2019-03-06 18:48:32 -06:00
Preston Van Loon
018f200ce9
Fix attestations again (#1916) 2019-03-06 18:59:47 -05:00
Raul Jordan
728c9fbfdc
fix the wait for activation condition (#1915) 2019-03-06 17:16:06 -06:00
Preston Van Loon
dbb519880f
Fix off by one edge case for pending attestations (#1914)
* fix off by one error

* better plus one

* lint

* fix off by one
2019-03-06 17:03:31 -06:00
terence tsao
936c4dde77
Implement AttestationTargets for ForkChoice (#1912) 2019-03-06 15:03:52 -06:00
Preston Van Loon
34f0241f32
Fix wait for activation (#1913)
* fix but no test

* ensure a canonical state is sent
2019-03-06 15:13:09 -05:00
Preston Van Loon
869f08ad26
Proposer Service: Only return attestations within the last epoch (#1911)
* Only return attestations within the last epoch

* make array

* rename for clarity
2019-03-06 11:54:02 -05:00
terence tsao
ddcb92c549
Discard Attestations Older Than Prev Epoch Start Slot (#1905) 2019-03-05 21:27:28 -08:00
Preston Van Loon
eda03ff767
Recover from a panic in p2p (#1906)
* recover from a panic in p2p

* Add safely process function

* print proto message

* add status to the span, if available

* add status to the span, if available
2019-03-05 23:57:44 -05:00
terence tsao
b6ad702f3f update beacon chain spec link (#1897) 2019-03-05 18:54:58 -05:00
Raul Jordan
c22e4dc2ae
Wait for Validator Activation Stream Server (#1896)
* create wait for activation stream

* wait for activation server stream open

* complete server side logic

* formatting

* first test passing

* context closed test

* lint

* fix build failure

* imports

* eliminate unused cancel var
2019-03-05 17:06:50 -06:00
Raul Jordan
aebb523bca
Request Block Parent If Not Found During Regular Sync (#1892)
* logic for requesting missing block parent

* rearrange block parent fetch logic

* reorder logic

* tests pass

* gazelle

* typo

* weird spacing

* imports
2019-03-05 16:39:44 -06:00
terence tsao
d8da9757d8
Add Spans for Epoch Processing (#1895) 2019-03-05 14:18:07 -08:00
terence tsao
cf54bc794b Add Spans for Processing Block (#1894)
* add spans for processing slot

* gaz

* rm main

* added spans for block processing
2019-03-05 15:09:50 -06:00
terence tsao
ef54152071 Add Spans for Processing Slots (#1890)
* add spans for processing slot

* gaz

* rm main
2019-03-05 14:22:09 -06:00
Nishant Das
07a25ca710 Adding Prometheus Metrics and Tracing to Sync (#1858)
* adding tracing and metrics

* clean up main routine

* added tracing to initial sync

* gazelle

* fix tests

* fix naming

* comments

* consistent style in promauto counters

* rename spans

* final comment
2019-03-05 11:31:19 -06:00
Nishant Das
b264cdc557 Allow Node to Request State if it is Behind (#1875)
* fix naming

* comment

* tests
2019-03-05 10:59:49 -06:00
terence tsao
3f9124b30c
minus genesis epoch (#1886) 2019-03-05 07:51:13 -08:00
terence tsao
dd40320a1c Save and Delete Validators Pubkey - Idx Post Chainstart (#1873)
* starting tests

* added tests

* fixed comments

* gaz

* make sure key is using EntryExitEffectEpoch
2019-03-05 15:22:15 +08:00
Preston Van Loon
9999b3280a
Use promauto to automatically register metrics (#1872)
* Use promauto to automatically register metrics

* sub genesis slot
2019-03-04 21:38:03 -05:00
Nishant Das
d162b5603e clean up context (#1850) 2019-03-04 12:57:59 -08:00
skillful-alex
4194d78bed Add Web3Service Health Checks. (Issue #1204) (#1581)
* described the Status () function

* web3 service health checks

* remove method LatestBlockTime and add tests

* remove lastHeadInteractTime

* corrected conflicts of different branches of development

* goimports -w beacon-chain/powchain/service_test.go
2019-03-04 15:37:24 -05:00
Preston Van Loon
6af522986e
Enable POW chain by default (#1861)
* enable POW chain by default

* fix tests in node.go

* ignore ctx
2019-03-04 15:10:03 -05:00
terence tsao
eb9d4a630b add metrics for justification and finalization (#1854) 2019-03-03 22:53:20 -08:00
terence tsao
b00891880e
Implement Validator Status RPC Methods (#1839) 2019-03-03 21:28:00 -08:00
Raul Jordan
0ce7c8c599
remove old timestamp error log (#1807) 2019-03-03 15:26:59 -08:00
terence tsao
17682321fe
Clean Up Validator RPC Server (#1798) 2019-03-03 14:55:12 -08:00
Ivan Martinez
5aa7320507 Clear Up Code Inconsistencies (#1836) 2019-03-03 09:31:29 -08:00
Nishant Das
fe2eb5a812 Fixing Sync (#1830)
* fix querier

* new fixes for sync

* reducing batch size and adding equality condition

* remove commented code

* fix test

* fix few more tests
2019-03-03 11:03:47 -06:00