Commit Graph

53 Commits

Author SHA1 Message Date
terence tsao
9bf55e53e7 Add context and tracing to attestation pool (#3737)
* Test case for overlapping aggregation bits

* Add ctx and tracing for attestation pool and beyond

* No nil

* Use real ctx

* Use real ctx

* Fix test

* Fix test

* Fix test

* Fix test

* Fixed imports
2019-10-09 06:42:17 +08:00
Preston Van Loon
23764c4640
Abstract verifySlot to helper package (#3731)
* Abstract verifySlot to helper package

* blocks -> slots

* fix test
2019-10-07 16:15:50 +09:00
terence tsao
750bc83369
Clean up feature flag namings (#3715) 2019-10-07 14:11:49 +09:00
Preston Van Loon
58dbdfb6f5 Remove attestation that fails to verify from queue (#3704) 2019-10-03 13:57:18 -07:00
Jim McDonald
323ee8dfac SetupInitialDeposits() now returns deposit data roots (#3683) 2019-10-02 10:50:34 +08:00
Raul Jordan
d9c0e65cef
Improve Beacon Node Logging UX (#3600)
* info logs beacon node improvements

* prom test fixes

* info logging changes

* wrapped up node info logging

* changed to debug level

* warn logs taken care of

* Terence suggestion

* warn spacing

* better logging in initial sync

* debug level standardized

* complete debug standardization

* participation at epoch end

* fix archive tests

* even more test fixes

* prom test

* ops test

* powtest

* rpc sync test

* rem part

* log formatting
2019-10-01 15:05:17 -05:00
terence tsao
944d3b16fd Batch save attestations in state transition (#3672)
* Batch save attestations

* Update test

* Revert config
2019-09-30 16:11:59 -07:00
Raul Jordan
8d215feb25
Large Prysm Performance Improvements (#3622)
* skip bls verification with a feature flag at runtime

* gazelle

* more bls mocks

* block roots efficiency

* db block roots now does not show up on the flame graphs

* save validator latest votes batch

* batch save att

* misc improvements to pprof

* import

* include validator index cache

* error if no filter criteria

* resolved comments

* build fix

* lint

* remove delay global

* attestation and block test fixes

* preston suggestions

* fix db tests

* fix missing broken tests

* tests pass
2019-09-30 15:45:53 -05:00
Nishant Das
8ece8fb44b Expose DB Metrics (#3663)
* add in bolt metrics

* unregister in db teardown

* unregister in Close()

* fix clear db case

* fix test error

* gaz

* remove unregister

* remove gaz
2019-09-30 10:24:47 -05:00
terence tsao
4c8269aca3
Part 4 of caching improvement - Use Cache (#3625) 2019-09-27 15:56:08 -07:00
Nishant Das
73938876b1 better log validator votes (#3590) 2019-09-26 06:32:06 -07:00
Preston Van Loon
0a2dfedf0f
Log warning with error instead of returning an error for fork choice votes update (#3587) 2019-09-25 20:39:19 -07:00
terence tsao
3ef681e649
Initial sync no verify block (#3586) 2019-09-25 20:32:00 -07:00
Nishant Das
8ce96428b1 Fixes our Attestation Aggregation Issues (#3579) 2019-09-25 07:39:16 -07:00
terence tsao
9577e2c123 Removed attestation check for votes (#3562) 2019-09-23 15:44:13 -05:00
terence tsao
ab2d4e8ad6 Implement attestation pool in memory (#3542)
* Run time bug

* Still failing

* Run time working

* Run time working

* Gazelle

* Fixed all the tests

* Revert config

* Revert back test configs

* Revert config

* Tested run time again, everything is good
2019-09-23 11:11:44 -05:00
terence tsao
4e6ed2744d Fixed 2 cosmetic errors (#3543)
* Fixed

* Treehash

* Updated error msg
2019-09-23 08:30:39 -07:00
terence tsao
315d4f0549 Save aggregated attestations in DB after processing (#3541)
* Added AggregatedAttestation helper

* Implemented `saveNewAttestation` for processing attestation

* Implemented `savesNewBlockAttestations` for processing block

* Tests

* Fix name

* Raul's feedback
2019-09-21 13:57:32 -05:00
terence tsao
2b2ef4f37c
Added AggregatedAttestation helper (#3539) 2019-09-21 09:43:18 -07:00
terence tsao
c29a7be0ec Fix justified check point mutation (#3534)
* first version of the watchtower api

* first version

* delete watchtower

* move to message loop

* roughtime

* one time

* fix test

* Fixed

* Fixed

* Revert unused lock
2019-09-20 10:44:28 -07:00
terence tsao
ac176a5078 Update validators votes from incoming block (#3530)
* first version of the watchtower api

* first version

* delete watchtower

* move to message loop

* roughtime

* one time

* fix test

* Update block attestation votes

* Clean up

* Found a bug

* Confirmed to be working in run time

* Confirmed to be working run time

* Raul's feedback

* Tests
2019-09-19 20:34:57 -05:00
terence tsao
b863004b2a
Aggregate attestations before verify and update votes (#3493) 2019-09-16 17:48:03 -07:00
Raul Jordan
1e086b63e8 Add Fork Choice Package Docs (#3463) 2019-09-13 13:03:14 -04:00
terence tsao
798bbbdc82 Cold start for interop (#3437)
* coldstart flags for validator

* WIP beacon node flags

* wip beacon chain, flag fix in validator, arg fix in validator

* checkpoint

* Added interop service

* working on mock chainstart

* save the state lol

* fix tests

* Save genesis validators

* gaz

* fix validator help flags

* WaitForChainStart actually waits for genesis time

* cold start fixes

* cache

* change back

* allow for genesis state too

* remove logs

* increase mmap size

* dont process if head doesn't exist

* add 10ms tolerance

* enable libp2p debug at debug, fix pubsub

* works with checkpt

* initialize justified and finalized in db

* Removed preloadStatePath from blockchain service

* Clean up

* Write to disk for now post state

* revert b466dd536f8eadbdae2264a545a755370223d917

* Builds

* Only RPC test fails now

* use minimal config, no demo config

* clean up branch

* Lint

* resolve lint

* more lint fixes

* lint

* fix viz

* Fixing RPC test

* skip before epoch 2

* RPC time out

* Fixed ordering

* rename

* remove some dbg statements

* ensure index is correct

* fix some panics

* getting closer

* fix tests

* Fix private key

* Fixed RPC test

* Fixed beacon chain build for docker

* Add interop.go to validator go_image

* Fixed docker build

* handle errors

* skip test, skip disconnecting peers

* Fixed docker build

* tolerance for attestation processing

* revert copy

* clearer err message parse

* fix launching from dep contract
2019-09-11 13:38:35 -05:00
Ivan Martinez
5e939378d0 Update to spec v0.8.3 (#3355)
* Ignore latest messages in fork choice prior to latest justified

* Make sure Compact Committee Roots isn't changed by process_final_updates

* WIP add attestation bitfields length to match committee length

* Begin work on updating spec tests to 0.8.2

* WIP set up for new spec test structure

* Fix slashings

* Get mainnet tests mostly passing for attestations and attester slashings

* Fix process attestation test

* Undo change

* Complete spec tests for all operations
Still need sanity block tests

* Fix BLS sigs

* Reduce amount of reused code in core/blocks/spectests/

* Fix tests

* Update block sanity tests to 0.8.2

* Update epoch spec tests to 0.8.2

* Clean up all tests and fix shuffling/epoch tests

* WIP update bls tests to 0.8.2

* WIP update bls tests to 0.8.3

* Finish BLS spectest update to 0.8.3

* Fix shuffling spec tests

* Fix more tests

* Update proto ssz spec tests to 0.8.3

* Attempt to fix PrevEpochFFGDataMismatches test

* Goimports

* Fix documentation

* fix test

* Use custom general spec tests

* Reduce code footprint

* Remove unneeded minimal skip

* Fix for comments

* Fix for comments

* Fix test

* Small fixes

* Cleanup block spec tests a bit

* Undo change

* fix validator

* Fix validator tests

* Run gazelle

* Fix error output for epoch spec tests
2019-09-08 12:41:52 -07:00
Raul Jordan
adc27a0bc2 Update WORKSPACE With Latest SSZ (#3416) 2019-09-08 12:58:22 -04:00
terence tsao
75bce9b7e1
Align metrics to interop (#3406) 2019-09-05 08:32:35 -07:00
Preston Van Loon
0a61c379a5
Rename / move logic about updating validator indices (#3402)
* rename, move

* pr feedback
2019-09-03 13:57:08 -07:00
terence tsao
60c048a0ec
Remove lock from store struct (#3400) 2019-09-03 13:14:23 -07:00
Preston Van Loon
5ec629af71 remove unnecessary lock (#3399) 2019-09-03 12:53:18 -07:00
terence tsao
25dbc5ea85
Add back state metrics (#3369) 2019-09-01 08:37:38 -07:00
terence tsao
146b611dc8
Use rough time for checking attestation is not from future epoch (#3370) 2019-08-31 13:18:31 -07:00
terence tsao
ca2a55874c
Save new validator index in DB (#3367) 2019-08-30 21:43:18 -07:00
Preston Van Loon
8e2dcb81ae
use roughtime (#3366) 2019-08-30 15:51:15 -07:00
terence tsao
f131585041 Initialize chain info w/o clear db (#3365)
* Initialize chain info upon restart

* Test
2019-08-30 15:24:37 -07:00
terence tsao
9a6410ec15
Lock when process attestation (#3364) 2019-08-30 14:07:42 -07:00
terence tsao
c425bf2c31 Refactor fork choice start up (#3360)
* Done

* Test

* Fixed test

* emit state initialized

* Fixed existing tests

* Lint

* Lint
2019-08-30 08:58:02 -05:00
terence tsao
5d7c33a8dc Check if slot is greater before process slots (#3349) 2019-08-28 14:24:33 -05:00
terence tsao
d84ae95309
Moved delay att inclusion to fork choice service (#3345) 2019-08-28 10:26:07 -07:00
terence tsao
323bbe10ed
Add checkpoint to state caching (#3333) 2019-08-27 15:01:27 -07:00
Raul Jordan
9ad00ffafb
Use New Attestation Receiver Method in RPC (#3287)
* next compatible, tests pass

* terence feedback

* skip comment

* fixes

* misc fix

* on block

* resolved err
2019-08-25 15:45:55 -05:00
terence tsao
122166b317
Fix transition logging (#3303) 2019-08-24 11:51:00 -06:00
Nishant Das
9d15196bed Runtime Fixes (#3300) 2019-08-24 07:26:25 -06:00
terence tsao
4484558d87 Part 11 of update fork choice - tracing and spans (#3285)
* Add tracing in forkchoice service

* Gazelle
2019-08-23 15:04:06 -05:00
terence tsao
02ca2290e1
Added metrics for monitoring processed objects and competing chain (#3283) 2019-08-23 12:18:39 -06:00
terence tsao
0d64f7b80e Part 6 of update fork choice - implement new ReceiveAttestation (#3246)
* Implemented new fork choice service and helpers

* Added rest of the tests

* Lint

* Add back helpers test

* Reformatted to doc, helpers and metrics.go

* include new getter for block

* create block filters from indices

* give every block index a unique bucket

* construct block indices by bucket mmap

* almost done save for the block filters

* include block filters, need a few more small touches for fetching the proper indices by bucket

* full functionality to filter by parent root

* tests pass when using the same logic as attestations

* todo

* proper todo formatting

* first minimum slot range filter

* slot range filters pass

* more filter criteria passing

* tests passing

* add todos

* all block tests pass and work

* rem fmt

* range retrieval test

* fixed test conditions

* instantiate the other buckets

* simplify bucket lookups

* deprecate non map code

* revamp to remove old index prefixes

* create indices from data

* create indices from data

* fetch block roots by slot range

* better abstractions

* simpler abstractions

* roots rename

* comment

* preston feedback

* Fixed existing tests

* allow blocks without parent root

* Cleaned up a few things

* Removed todo

* Lint

* Cleaned up a few things

* A few functions don't need to be exported

* Gaz

* Fixed visibility

* Review feedback

* Review feedback part1

* Raul's feedback, refactored OnBlock and OnAttestation to its own file

* Fixed grammar

* Lint

* Implemented ReceiveAttestation

* Use time.Time

* Implemented ReceiveAttestation

* All tests pass

* Lint

* Oooops

* Typo
2019-08-22 20:00:55 -05:00
Raul Jordan
bb542d2032
Basic Block/LatestVote Caching in DB Refactor (#3249)
* add block caching layer

* runlock

* lockinggg

* latest votes map

* validator latest vote deletion method added to interface

* seguin cache working

* cache size

* impl interface

* initialize caches at struct layer
2019-08-22 10:04:13 -05:00
terence tsao
bfbff885fe Part 7 of update fork choice - chain info access (#3263)
* Implemented new fork choice service and helpers

* Added rest of the tests

* Lint

* Add back helpers test

* Add benchmark tests

* Add yaml driven framework tests

* Reformatted to doc, helpers and metrics.go

* include new getter for block

* create block filters from indices

* give every block index a unique bucket

* construct block indices by bucket mmap

* almost done save for the block filters

* include block filters, need a few more small touches for fetching the proper indices by bucket

* full functionality to filter by parent root

* tests pass when using the same logic as attestations

* todo

* proper todo formatting

* first minimum slot range filter

* slot range filters pass

* more filter criteria passing

* tests passing

* add todos

* all block tests pass and work

* rem fmt

* range retrieval test

* fixed test conditions

* Implemented new receive block methods

* Comments

* Remove mark evil block

* instantiate the other buckets

* simplify bucket lookups

* deprecate non map code

* revamp to remove old index prefixes

* create indices from data

* create indices from data

* fetch block roots by slot range

* better abstractions

* simpler abstractions

* roots rename

* comment

* preston feedback

* Fixed existing tests

* allow blocks without parent root

* Cleaned up a few things

* Removed todo

* Lint

* Cleaned up a few things

* A few functions don't need to be exported

* Gaz

* Fixed visibility

* Review feedback

* Review feedback part1

* Raul's feedback, refactored OnBlock and OnAttestation to its own file

* Fixed grammar

* Lint

* Renamed to receive_block.go

* Use time.Time

* Preston's feedback, removed OnTick and Store.time

* Dont have to cast it to kv

* add block caching layer

* runlock

* lockinggg

* Fixed

* Avoid 2 fetches of the same data

* latest votes map

* Gaz

* Test passes

* Lint

* Fixed db set up

* Fixed all the tests

* Gazelle

* Added tests

* Remove todo

* remove kv

* Last clean up

* Last clean up

Last clean up

* Lint

* Preston's feedback

* Starting

* Gazelle
2019-08-21 14:50:27 -05:00
terence tsao
8fc3c55199
Forkchoice get head tiebreaker (#3253) 2019-08-20 21:26:04 -06:00
terence tsao
6195a0bfa1
Fix GetHeadFromYaml is flakey (#3251)
* Forgot clear cache needs to be within the loop

* space
2019-08-20 19:57:35 -06:00