Commit Graph

168 Commits

Author SHA1 Message Date
Raul Jordan
3e525ebe8b
Block Deletion Upon Block State Transition Error (#2270)
* block deletion upon block state transition error

* include tests

* tests complete

* comment

* imports spacing
2019-04-16 13:06:53 -05:00
Nishant Das
90fed79610 Fix Initial Sync in Restarts (#2197)
* set chainstart value

* revert change in IsSynced and add regression test

* change status

* make a better status query

* allow status check to work on restarts

* spacing
2019-04-09 14:18:23 -04:00
terence tsao
a19f08f7b6 Attestation Using Head State instead of Latest State (#2156)
* headRoot is a better name

* yay merged with master

* fixed attester server returning incorrect attestation field values

* revert stupid mock changes

* fixed and updated all the tests

* uncomment

* head start should handle skip blocks

* Raul's feedback
2019-04-05 09:41:49 -05:00
Preston Van Loon
83130358a9
Rollback state to handle side chain with skip blocks (#2147)
* Add initial test

* chkpt

* add failing test

* add span to historical state lookup

* use db.HighestBlockSlot()

* fix comment

* update comment

* i wrote a test like a good programmer.

* add test back

* add assertion and unskip test, something new failing tho

* trying to fix test

* remove -1, not sure if i need it yet

* Revert "remove -1, not sure if i need it yet"

This reverts commit 2cfcbb8108b28bb3d7135a993d9053150d5f1e6e.

* save historical state on every save state

* fix hsitorical states

* set historical state in initialize state

* change to a bool

* fix error with empty retrieval of states

* Add missing import

* fix test

* lock in receive block

* remove state generator

* Revert "lock in receive block"

This reverts commit 151b10829d70b2dad3055a8db36d0e1269a853f2.

* Fix Initial Sync Not Processing Canonical Block to Produce Canonical State (#2152)

* fix init sync

* fatal if highest observed root does not match

* proto fields

* Update beacon-chain/sync/initial-sync/service.go

* confirm canonical state root

* fix most tests

* failing test

* fix PR tests

* lint

* no simbackend changes

* logf revert

* add todo

* fix off by one

* fix test with deleted property

* merge #2157

* passing tests :)
2019-04-04 22:39:51 -05:00
Raul Jordan
eb900a8193
Refactor Initial Sync, Enable Catching Up to Chain (#2111)
* refactor initial sync to prevent reorg infinite loops

* lint

* fixed build

* passing tests

* tests passing

* terence suggestion

* new attempt

* clean up and refactor sync service

* complete the new initial sync logic

* revert head

* init sync working

* config for blockchain receive block

* all works

* builds

* fix a few more tests

* init sync tests pass

* revert scripts

* revert accounts changes

* lint

* lint2

* travis lint

* fix build

* fix single use argument

* any peer

* imports spacing

* imports

* ready for a rolling restart

* add todo

* fork choice in blocks when exiting sync

* readd finalized state root to requests

* successful build

* revert blockchain config

* old config reversion

* initial sync tests pass

* initial sync full test works

* lint

* use the new block processing api

* new proto defs

* init sync functions again

* remove sync polling

* tests fixed

* fixed catching up with chain

* tests pass

* spacing

* lint

* goimports

* add changes

* add lock and conditional to prevent multiple goroutines

* make reg sync synchronous

* add

* fixed the parent block issue

* fix errors in chain service

* tests pass

* check nil block

* typo

* fix nil state

* merge & conflicts

* revert synchronus reg sync

* add more spans to state db

* fix lint

* lint
2019-04-03 10:13:19 -05:00
Preston Van Loon
76e30200fe Pass ctx to saveState (#2141)
* pass ctx to saveState

* add span
2019-04-02 16:49:45 +08:00
Raul Jordan
7489e52b99
Refactor Chain Service Block Processing API (#2124)
* new block processing api

* fix tests

* lint

* spacing

* Update beacon-chain/blockchain/block_processing.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* fix tests, remove enable powchain
2019-03-31 20:44:16 -05:00
Raul Jordan
5b89da1ce2
Revert "Prevent Reorg Infinite Loops in Initial Sync (#2109)" (#2110)
This reverts commit d0790246a2.
2019-03-29 16:51:58 -05:00
Raul Jordan
d0790246a2
Prevent Reorg Infinite Loops in Initial Sync (#2109)
* refactor initial sync to prevent reorg infinite loops

* lint

* fixed build

* passing tests

* tests passing

* terence suggestion
2019-03-29 13:11:13 -05:00
Nishant Das
7029ed109d
Prune Outdated Deposits after Chainstart (#2090)
* disable pending deposits

* Revert "disable pending deposits"

This reverts commit 1525e01602a70737f8688cdbd63a5ce2ab721d92.

* adding new methods

* gazelle and other fixes

* spacing

* spacing

* Update beacon-chain/sync/initial-sync/service.go

Co-Authored-By: nisdas <nish1993@hotmail.com>

* remove comment
2019-03-28 00:15:29 +08:00
Nishant Das
0809b96c32 Fixes Initial Sync (#2041)
* plug forkchoice to blockchain service's block processing

* fixed tests

* more fixes...

* clean ups

* fixed test

* Update beacon-chain/blockchain/block_processing.go

* merged with 2006 and started fixing tests

* remove prints

* fixed tests

* lint

* include ops service

* if there's a skip slot, slot--

* fixed typo

* started working on test

* no fork choice in propose

* bleh, need to fix state generator first

* state gen takes input slot

* feedback

* fixed tests

* preston's feedback

* fmt

* removed extra logging

* add more logs

* fixed validator attest

* builds

* fixed save block

* children fix

* removed verbose logs

* fix fork choice

* right logs

* Add Prometheus Counter for Reorg (#2051)

* fetch every slot (#2052)

* test Fixes

* lint

* only regenerate state if there was a reorg

* better logging

* fixed seed

* better logging

* process skip slots in assignment requests

* fix lint

* disable state root computation

* filter attestations in regular sync

* log important items

* better info logs

* added spans to stategen

* span in stategen

* set validator deadline

* randao stuff

* disable sig verify

* lint

* lint

* save only using historical states

* use new goroutine for handling sync messages

* change default buffer sizes

* better p2p

* rem some useless logs

* lint

* sync tests complete

* complete tests

* tests fixed

* lint

* fix flakey att service

* bug fixes

* Update beacon-chain/sync/initial-sync/service.go

* Update beacon-chain/sync/initial-sync/service.go

* reenable shuffle

* enable shuffle
2019-03-25 13:23:57 -05:00
Raul Jordan
053038446c
Allow 8 Validator Multinode Cluster to Run Indefinitely (#2050)
* plug forkchoice to blockchain service's block processing

* fixed tests

* more fixes...

* clean ups

* fixed test

* Update beacon-chain/blockchain/block_processing.go

* merged with 2006 and started fixing tests

* remove prints

* fixed tests

* lint

* include ops service

* if there's a skip slot, slot--

* fixed typo

* started working on test

* no fork choice in propose

* bleh, need to fix state generator first

* state gen takes input slot

* feedback

* fixed tests

* preston's feedback

* fmt

* removed extra logging

* add more logs

* fixed validator attest

* builds

* fixed save block

* children fix

* removed verbose logs

* fix fork choice

* right logs

* Add Prometheus Counter for Reorg (#2051)

* fetch every slot (#2052)

* test Fixes

* lint

* only regenerate state if there was a reorg

* better logging

* fixed seed

* better logging

* process skip slots in assignment requests

* fix lint

* disable state root computation

* filter attestations in regular sync

* log important items

* better info logs

* added spans to stategen

* span in stategen

* set validator deadline

* randao stuff

* disable sig verify

* lint

* lint

* save only using historical states

* use new goroutine for handling sync messages

* change default buffer sizes

* better p2p

* rem some useless logs

* lint

* sync tests complete

* complete tests

* tests fixed

* lint

* fix flakey att service

* PR feedback

* undo k8s changes

* Update beacon-chain/blockchain/block_processing.go

* Update beacon-chain/sync/regular_sync.go

* Add feature flag to enable compute state root

* add comment

* gazelle lint fix
2019-03-25 10:21:21 -05:00
Preston Van Loon
7f11cbd5ed
print stack trace on panic recovery (#2042) 2019-03-20 10:17:44 -04:00
terence tsao
4a2ef17be1 fixed typo (#2038) 2019-03-20 01:13:14 -04:00
Preston Van Loon
612bb38077 Cross p2p spans, more spans, synchronous attestations, minor fixes (#2009)
* Fix assignments bug where validators don't retry for assignments on failure

* synch only please

* trying to fix state issues

* trying random stuff

* do not explode

* use ctx

* working build, failing tests

* broadcast local addrs as well as relay addrs

* fixed p2p tests, more tests to fix still

* another test fixed, log warning instead of throw error

* Fix last tests

* godoc

* add test for broadcast in apply fork choiec

* remove unneeded code

* remove tracer adapter, not needed

* remove extra stuff

* remove any

* revert addr_factory

* revert addr_factory

* Revert "revert addr_factory"

This reverts commit e93fb706494a1070158b8db31e67146d6b0648ad.

* Revert "revert addr_factory"

This reverts commit dedaa405559cc818698870c4e4570953367f1e3c.

* revert removal of this code

* unused param
2019-03-17 10:56:05 +08:00
Raul Jordan
5371218b9b
Refactor ChainService for Receiving Blocks Synchronously (#1984)
* refactor chain service

* restructure service and lint

* all calls to receive block in chain service are now blocking

* begin fixing tests

* refactored blockchain tests

* builds correctly

* blockchain tests pass again

* lint

* sync and rpc tests pass again

* done

* add in open tracing

* span in fork choice

* Update beacon-chain/blockchain/block_processing.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>
2019-03-13 17:17:32 -04:00
Nishant Das
000a42deac Make State Requests only Request for Finalized State (#1955)
* remove hash field for state request

* change to finalized state root

* refactor code for bug

* add in saving of finalized state

* add method to db

* adding finalized state root

* add new changes

* add in new changes

* add in return vars

* goimports

* review comments

* error message

* fix broken build

* fix cache state tests

* lint
2019-03-12 16:44:21 -04:00
Nishant Das
7179b6930b Remove Incorrect Error Logs in Sync (#1943) 2019-03-11 15:07:16 -07:00
Nishant Das
028506e7ed Handle Panics in Remaining Services (#1926)
* add recovery for initial sync

* add recovery for rpc

* adding to other services

* remaining services and tests

* fix test

* remove changes to rpc

* handle powchain

* handle in powchain

* abstract to shared package

* gazelle

* lint

* remove for operations

* add in more for operations

* travis

* fix lint
2019-03-10 17:53:28 -05:00
Nishant Das
7c5a7fb044 Handle Skipped Slots in Sync (#1939)
* handle edge cases

* tests passing

* Update beacon-chain/sync/initial-sync/service.go

* Update beacon-chain/sync/initial-sync/service.go

* Update beacon-chain/sync/initial-sync/service_test.go
2019-03-08 10:09:24 -06:00
Raul Jordan
205594c5d6 Fix Travis Failures on Master (#1938)
* resolve travis conflicts

* cp proto

* revert

* remove unused context
2019-03-07 23:54:41 -05:00
Preston Van Loon
2e7b08f97e
p2p fixes / improvements (#1931)
* don't broadcast to yourself

* todo

* Also listen for incoming streams

* send to a specific peer

* add godoc comment

* handle EOF as a normal thing

* gazelle

* add test for subscribing via direct comm
2019-03-07 21:37:15 -05:00
Nishant Das
2f9de6f93c Clean Up Batched Request (#1932)
* goimports

* do not log empty batches

* invalid request
2019-03-07 16:43:48 -06:00
Nishant Das
07a25ca710 Adding Prometheus Metrics and Tracing to Sync (#1858)
* adding tracing and metrics

* clean up main routine

* added tracing to initial sync

* gazelle

* fix tests

* fix naming

* comments

* consistent style in promauto counters

* rename spans

* final comment
2019-03-05 11:31:19 -06:00
Nishant Das
b264cdc557 Allow Node to Request State if it is Behind (#1875)
* fix naming

* comment

* tests
2019-03-05 10:59:49 -06:00
Nishant Das
d162b5603e clean up context (#1850) 2019-03-04 12:57:59 -08:00
Ivan Martinez
5aa7320507 Clear Up Code Inconsistencies (#1836) 2019-03-03 09:31:29 -08:00
Nishant Das
fe2eb5a812 Fixing Sync (#1830)
* fix querier

* new fixes for sync

* reducing batch size and adding equality condition

* remove commented code

* fix test

* fix few more tests
2019-03-03 11:03:47 -06:00
Preston Van Loon
a19cb1864e Tracing additions (#1747) 2019-02-27 19:55:47 -08:00
Nishant Das
a27c52e3a3
Update Initial Sync (#1694)
* adding check for chainHead

* add batch block limit

* adding state root

* removing genesisHash

* removing setBlockForInitialSync

* fixing tests

* remove obsolete check

* changing to check for state initialization

* changing var name

* fix test

* add in tests

* fix all tests

* lint

* reduce test time

* merge fixes

* gazelle
2019-02-26 13:37:28 +08:00
Raul Jordan
20803b4c9d
Revert Usage of ssz.TreeHash Across Repo (#1711)
* revert using ssz.TreeHash

* gazelle
2019-02-25 21:42:31 -06:00
Raul Jordan
e19920aec1
Miscellaneous Runtime Fixes & Improvements - Raul (#1674)
* fatal if impossible to receive chainstart

* fix tests

* fix

* custom delay

* completed custom delay

* errors

* better logs, nothing at genesis

* use demo in val

* add gazelle

* log

* starting to log stuff

* pass in ops

* avoid printing the large #s for debug, still working on tests..

* all around better logging

* fixed build error in epoch process

* fixed state transiton tests

* fixed block tests

* lint

* verify sigs in randao

* ready for inclusion falg

* only print waiting when slot is not valid

* fix build

* mod config

* fixed last justified slot issue

* fix inclusion

* fixed attestation issue

* using zero hash from params instead

* fix tests

* update balance

* removed swp

* more `- genesis_slot` for logs

* rem unused log

* fix broken tests

* account for skip slots in state root computation

* fixes done

* validator guide bug fixes - 671

* epoch boundary at the last slot of the epoch

* fix epoch issue

* more balance cal logs for debugging

* greater balance

* attestaton fixes

* fixes

* addressed testrun

* fixed ejection balance

* fix tests with far future epoch

* revert sync change

* revert initial sync change

* fix changes

* off by one att fix

* revert the att fix

* address comments

* format

* fix build

* rem file
2019-02-24 20:09:45 -06:00
Nishant Das
f5c88e1bcb
Test Syncing with Another Fully Synced Service (#1594)
* clean up test

* adding test setup

* add chainstart check to sync querier

* goimports

* goimports

* backend cleanup

* adding getters

* lint

* moving to db

* gazelle

* adding more services setup

* adding more changes

* adding different test setups

* other merge issues

* imports

* fixing keys

* get intial sync able to be set up

* new changes and gazelle

* use simulated p2p

* everything finally works

* unexport fields

* revert changes from merge

* remove mock server

* add documentation

* gazelle

* add another node

* review comments

* fix tests
2019-02-23 11:36:20 +05:30
terence tsao
8c04ced1a6 Name Changes to Align with V0.3 (#1621)
* fixed epoch_processing

* penalize->slash

* exit -> voluntary_exit

*  SEED_LOOKAHEAD -> MIN_SEED_LOOKAHED

* ENTRY_EXIT_DELAY -> ACTIVATION_EXIT_DELAY

* `INCLUDER_REWARD_QUOTIENT` -> `ATTESTATION_INCLUSION_REWARD_QUOTIEN`

* LatestIndexRoots -> LatestActiveIndexRoots

* `MIN_VALIDATOR_WITHDRAWAL_EPOCHS` -> `MIN_VALIDATOR_WITHDRAWAL_DELAY`

* MAX_WITHDRAWALS_PER_EPOCH -> MAX_EXIT_DEQUEUES_PER_EPOCH

* ETH1_DATA_VOTING_PERIOD -> EPOCHS_PER_ETH1_VOTING_PERIOD

* SLOT_DURATION -> SECONDS_PER_SLOT

* EPOCH_LENGTH -> SLOTS_PER_EPOCH

* SLOT_DURATION -> SECONDS_PER_SLOT take 2

* rest of the misc fixes for config name changes

* remove tools/bootnode/.!74296!bootnode.go

* `current_epoch_start_shard` -> `current_shuffling_start_shard`, `current_shuffling_epoch`, `current_shuffling_see`

* go fmt

* fixed comment

* updated pseudocode comments

* merged master
2019-02-18 10:52:16 -06:00
Raul Jordan
ae13bb5c83
Use Tree Hashing Algorithm Everywhere for Block Roots and State Root (#1577)
* begin using tree hash instead of hash for state and blocks

* replace with ssz

* rem references to vote cache

* replace all instances of state/block hash with root

* goimports

* almost every test in the world fails

* goimports

* all tests pass, tree hash across repo
2019-02-14 14:04:47 -06:00
terence tsao
4cadd534db
Update Genesis Slot and Epoch (#1537) 2019-02-10 23:09:35 +01:00
Nishant Das
9ecef934d8 Update Beacon State to Match the Spec (#1489)
* first line of changes

* fix all tests in core package

* Finished fixing up all tests

* docs

* more docs
2019-02-06 10:44:27 -06:00
Raul Jordan
cec6c8de83 Standardize Naming of Shared Packages, Remove Deprecated Packages (#1435)
* rem txpool

* standardize repo shared utils, remove deprecated items

* complete

* package lvl godoc
2019-01-31 10:53:58 +08:00
Ed Mazurek
10b237e72d db methods: remove Get prefix (#1351)
* db methods: remove Get prefix for getter functions

* db methods: consistent test names by removing Get
2019-01-21 17:34:11 +08:00
Nishant Das
7ff544fddf
Fix travis and Other Linter Errors (#1356)
* fix travis and other linter errors

* remove megacheck
2019-01-21 13:35:34 +08:00
Preston Van Loon
592c5c3d92 Refactor hashing of beaconblock to shared (#1315)
* Add beacon block hash function

* Refactor/remove old hash method

* gazelle
2019-01-15 00:41:20 +08:00
terence tsao
e313e97666
Implement Validator Rotation Logic (#1279) 2019-01-13 12:52:31 -08:00
Andrei Ivasko
b29188128d ToBytes32 (#1306) 2019-01-13 09:04:14 -05:00
Raul Jordan
4bb6196f4f
Wrap Up Data Structure Renaming (#1270)
* spec renames

* fix goimports

* fix items
2019-01-09 03:49:17 -06:00
Raul Jordan
7e1af4fceb
Remove Use of Proto .GetX for Value Getters (#1235) 2019-01-04 21:58:19 -06:00
Raul Jordan
952dbe983e
Integrate Block Operations Into State Transition (#1135) 2018-12-26 21:15:12 -06:00
Nishant Das
5804dcfb67
Add Batch Syncing for Initial Sync (#1119)
* adding proto objects

* simulator changes

* adding stuff for initial sycn

* more changes

* finally fixed test

* fixing tests and sync

* test fix

* adding batched block response

* remove types references

* fix tests

* addressing review

* removing genesis block

* updating protos

* fix merge conflicts

* file permissions
2018-12-26 22:16:46 +08:00
Nishant Das
9db1002d87
Updating Genesis Block to Spec (#1172)
* Updating to spec

* addressing review comments
2018-12-25 14:47:07 +08:00
Raul Jordan
5d26efcfaa
Deprecate types.BeaconState with pb.BeaconState Generated Proto (#1164) 2018-12-23 16:51:04 -06:00
Preston Van Loon
10f45744d6
Use gogo protobuf as the protoc compiler (#1165)
* first commit, remote att types

* no more agg attestation proto

* regen mock

* only attestations

* proto

* att process

* fix att references

* more tests passing

* use att protos

* complete

* change visibility

* use gogoprotobu
2018-12-23 15:34:59 -05:00
Raul Jordan
e49a190754
Replace Types of Block Primitives With Proto Generated Types (#1137)
* refactor repo to use protos

* removed block successfully in types

* db package updated

* db, core passing

* chain service done

* no more block instances

* all pass

* deprecate all block protos

* tests pass
2018-12-20 17:00:38 -05:00
Nishant Das
724b036fff
Combine Sync Services (#1060)
* adding initial service

* got it working

* deleting redundant querier file

* golint

* formatting span messages

* lint

* lint

* addressing raul's comments
2018-12-10 13:26:44 +08:00
terence tsao
121654ff12
Removed Deprecated BeaconBlock Fields (#1035) 2018-12-03 08:10:47 -08:00
Raul Jordan
a99de08562
Merge Active/Cystallized State (#1019) 2018-12-01 16:09:12 -06:00
Nishant Das
16b04699d0 Allow Initial Sync to Work with Simulator (#669)
* polling interval

* adding proto message

* changing proto messages

* changing naming

* adding slot functionality

* initial sync working

* new changes

* more sync fixes

* its working now

* finally working

* add tests

* fix tests

* tests

* adding tests

* lint

* log checks

* making changes to simulator

* update logs

* fix tests

* get sync to work with crystallized state

* fixing race

* making requested changes

* unexport

* documentation

* gazelle and fix merge conflicts

* adding repeated requests

* fix lint

* adding new clock , db methods, and util func

* revert change to test

* gazelle

* add in test

* gazelle

* finally working

* save slot

* fix lint and constant
2018-11-21 10:00:36 -08:00
Nishant Das
f46ee335bd
Allow Sync Service to Query State Of Network (#787)
* adding proto messages

* adding fields

* adding new service

* new changes

* checking in file from syncFix

* fixing test

* add test

* adding to node

* goimports

* adding query service to sync services

* sync working

* finally works

* fix test

* lint

* fix build

* fix test
2018-11-19 09:59:11 +08:00
terence tsao
abfe430f03 Beacon Node Logging Enhancement (#826)
* fixed logging

* typo
2018-11-18 18:21:02 -05:00
Yutaro Mori
1f00cfe23e
Replaced HasStoredState with IsSyncedWithNetwork (#662) 2018-10-16 07:22:15 +09:00
Yutaro Mori
d1eefa5cc7 Use consistent hash formatting (#650) 2018-10-12 08:35:49 -07:00
terence tsao
efeb6976d1
Align Block and State Field Names with Spec (#628) 2018-10-09 19:34:50 -07:00
Raul Jordan
cd2073e84f
Cleanup and Refactor Shared Package (#634) 2018-10-09 00:58:54 -05:00
Yutaro Mori
ce99bd7605 Refactor to Create DB Package (#582) 2018-10-05 13:14:50 -04:00
Preston Van Loon
e5569c88f8
More todos (#558)
* Add TODO link for issue #555

* Remove database related TODO

* update todo for issue #426
2018-09-22 12:58:24 -04:00
Raul Jordan
419cff2512
--Dev Mode Flag for Local Demo/Development (#537) 2018-09-21 16:02:17 -05:00
Preston Van Loon
3c264be1c1
Clean up some p2p related TODOs (#521)
* Remove error checking for type assertions on p2p messages. The code should fail earlier in the p2p layer. Also fixed format on some TODOs

* format TODO again

* Add a bit more tests for p2p stuff

* goimports on feed_mock

* remove unused import
2018-09-16 14:02:53 -04:00
Nishant Das
cddae7a306
Implement RPC Messaging Between Validators (#475)
* Adding Proto files

* Move to shared package

* adding attestation sub

* gazelle

* attestation check

* proposal finished

* Add in attesters responsibilities

* fixing dependency issues

* adding topics

* refactoring tests

* Adding more tests

* adding more changes

* gazelle

* removing attester p2p

* remove mock

* changing to new proposer model

* changing tests

* making changes

* gazelle

* adding gomock

* adding rpc methods, reverting changes to other proto files

* gazelle and test changes

* adding tests

* adding mocks and tests

* gazelle

* fixing merge issues

* lint

* lint
2018-09-16 09:12:36 +08:00
Preston Van Loon
db1138ae1f
Fix p2p issues in #465 (#489) 2018-09-09 18:15:24 -04:00
Yutaro Mori
7cfda8a149 Extract Initial Sync Into its Own Package (#404)
* Initial commit to extract initial sync code

* Add stop/resume of sync/intialsync

* bazel run

* gometalinter

* add tests

* cleanup

* build fixes

* use infof rather than errorf if sync is not in correct state

* Add TODO comment detailing eventual completion of transition from initial sync to sync

* fix build

* pr comments

* fix tests

* bump
2018-08-22 09:52:03 +08:00