Commit Graph

62 Commits

Author SHA1 Message Date
terence tsao
81c53c26fb Update run time to v0.9 (#3935)
* Update seed domains (#3872)

* Remove Transfers (#3870)

* Remove active index roots and compact committee roots (#3869)

* Update inclusion reward (#3886)

* Alter proposer selection logic (#3884)

* Fix early committee bias (#3888)

* Remove shards and committees (#3896)

* Epoch spec tests v0.9 (#3907)

* Block spec test v0.9 (#3905)

* rm'ed in protobuf

* build proto

* build proto

* build proto

* fix core package

* Gazelle

* Fixed all the tests

* Fixed static test

* Comment out spec test for now

* One more skip

* fix-roundRobinSync (#3862)

* Starting but need new seed function

* Revert initial sync

* Updated Proposer Slashing

* Fixed all tests

* Lint

* Update inclusion reward

* Fill randao mixes with eth1 data hash

* Test

* Fixing test part1

* All tests passing

* One last test

* Updated config

* Build proto

* Proper skip message

* Conflict and fmt

* Removed crosslinks and shards. Built

* Format and gazelle

* Fixed all the block package tests

* Fixed all the helper tests

* All epoch package tests pass

* All core package tests pass

* Fixed operation tests

* Started fixing rpc test

* RPC tests passed!

* Fixed all init sync tests

* All tests pass

* Fixed blockchain tests

* Lint

* Lint

* Preston's feedback

* Starting

* Remove container

* Fixed block spec tests

* All passing except for block_processing test

* Failing block processing test

* Starting

* Add AggregateAndProof

* All mainnet test passes

* Update deposit contract (#3906)

* Proto spec tests v0.9 (#3908)

* Starting

* Add AggregateAndProof

* Unskip block util tests (#3910)

* rm'ed in protobuf

* build proto

* build proto

* build proto

* fix core package

* Gazelle

* Fixed all the tests

* Fixed static test

* Comment out spec test for now

* One more skip

* fix-roundRobinSync (#3862)

* Starting but need new seed function

* Revert initial sync

* Updated Proposer Slashing

* Fixed all tests

* Lint

* Update inclusion reward

* Fill randao mixes with eth1 data hash

* Test

* Fixing test part1

* All tests passing

* One last test

* Updated config

* Build proto

* Proper skip message

* Conflict and fmt

* Removed crosslinks and shards. Built

* Format and gazelle

* Fixed all the block package tests

* Fixed all the helper tests

* All epoch package tests pass

* All core package tests pass

* Fixed operation tests

* Started fixing rpc test

* RPC tests passed!

* Fixed all init sync tests

* All tests pass

* Fixed blockchain tests

* Lint

* Lint

* Preston's feedback

* Starting

* Remove container

* Fixed block spec tests

* All passing except for block_processing test

* Failing block processing test

* Starting

* Add AggregateAndProof

* All mainnet test passes

* Unskip block util tests

* Slot processing spec test V0.9 (#3912)

* Starting

* Add AggregateAndProof

* Unskip slot processing mainnet test

* Unskip minimal spec test for finalization (#3920)

* Remove outdated interop tests (#3922)

* Rm outdated interop tests

* Rm test runner

* Gazelle

* Update validator to use proposer slot (#3919)

* Fix committee assignment (#3931)

* Replace shard with committee index (#3930)

* Conflict

* Clean up (#3933)

* Remove shard filter in db (#3936)

* Remove lightouse compatibility test (#3939)

* Update Committee Cache for v0.9 (#3948)

* Updated committee cache

* Removed shuffled indices cache

* Started testing run time

* Lint

* Fixed test

* Safeguard against nil head state

* address edge case

* add test

* Fixed TestRoundRobinSync by doubling the epochs

* Unskip TestProtoCompatability (#3958)

* Unskip TestProtoCompatability

* Update WORKSPACE

* Fix minimal config (#3959)

* fix minimal configs

* fix hardcoded value in test

* Simplify verify att time (#3961)

* update readme for deposit contract, regen bindings for vyper 0.1.0b12 (#3963)

* update readme for deposit contract, regen bindings

* medium

* Check nil base state (#3964)

* Copy Block When Receiving it From Sync (#3966)

* copy block

* clone for other service methods too

* Change logging of Bitfield  (#3956)

* change logging of bits

* preston's review

* Unskip Beacon Server Test (#3962)

* run test till the end

* fix up proto message types

* fmt

* resolve broken tests

* better error handling

* fixing new logic to use archived proposer info

* fix up logic

* clip using the max effective balance

* broken build fix with num arg mismatch

* amend archive

* archival logic changed

* rename test

* archive both proposer and attester seeds

* page size 100

* further experiments

* further experimentation, archivedProposerIndex seems wrong

* test passes

* rem log

* fix broken test

* fix test

* gaz

* fix imports

* ethapis
2019-11-11 17:03:43 -05:00
terence tsao
582c382771 Add tracing and annotation for descendent error (#3859)
* Add annotation

* Revert
2019-10-26 20:26:51 -07:00
terence tsao
c5b0b3c326 Prune states since last finalized (#3856)
* Prune states since last finalized

* Fixed test

* Renmaed to prevFinalizedCheckpt
2019-10-25 18:16:10 -07:00
terence tsao
57036d16f9
Fix remove older than last finalized state (#3847) 2019-10-25 09:51:08 -07:00
Preston Van Loon
acf11262de Add feature flag for state pruning (#3845)
* Add feature flag for state pruning

* gaz
2019-10-23 23:45:52 -07:00
terence tsao
ae1e435231 Guard against deleting genesis and finalized state in DB (#3842)
* Don't delete boundary state

* Lint

* Test

* Feedback

* Batch and better comment

* Fix test

* zzzzzz

* rmStatesOlderThanLastFinalized
2019-10-23 22:39:41 -07:00
terence tsao
9a0fb5dca1
Delete DB states older than finalized checkpoint (#3824) 2019-10-23 08:30:21 -07:00
terence tsao
e42af4f11d
Check attestations bitfield overlaps (#3804) 2019-10-18 17:41:27 -07:00
Preston Van Loon
6bf14dedcd
Better aggregated attestations pool (#3761)
* WIP of aggregated signatures in DB

* new lines at end

* taking a nap on the plane now

* fix tests

* remove duplication of attestations. so much for that airplane nap lol

* benchmark before flight lands

* gaz

* manual gaz

* fully contained checks

* quick improvement before landing

* new bitlist with fixes

* doesn't need real signatures

* it works, mostly

* print shard too

* some refactoring

* Revert "some refactoring"

This reverts commit 377ce7fbfb02321c20522da2eb27208a20a67669.

* Revert "Revert "some refactoring""

This reverts commit b46a458898acd214da44fc20dfae920d2725b6e8.

These changes are ok, just need to update the expected values

* fix tests

* lint

* lint

* upstream changes

* fix tests

* what

* resolve TODOs

* gofmt

* revert unrelated pb

* remove debug statement
2019-10-16 23:46:07 -07:00
terence tsao
9bf55e53e7 Add context and tracing to attestation pool (#3737)
* Test case for overlapping aggregation bits

* Add ctx and tracing for attestation pool and beyond

* No nil

* Use real ctx

* Use real ctx

* Fix test

* Fix test

* Fix test

* Fix test

* Fixed imports
2019-10-09 06:42:17 +08:00
Preston Van Loon
23764c4640
Abstract verifySlot to helper package (#3731)
* Abstract verifySlot to helper package

* blocks -> slots

* fix test
2019-10-07 16:15:50 +09:00
terence tsao
750bc83369
Clean up feature flag namings (#3715) 2019-10-07 14:11:49 +09:00
Preston Van Loon
58dbdfb6f5 Remove attestation that fails to verify from queue (#3704) 2019-10-03 13:57:18 -07:00
Jim McDonald
323ee8dfac SetupInitialDeposits() now returns deposit data roots (#3683) 2019-10-02 10:50:34 +08:00
Raul Jordan
d9c0e65cef
Improve Beacon Node Logging UX (#3600)
* info logs beacon node improvements

* prom test fixes

* info logging changes

* wrapped up node info logging

* changed to debug level

* warn logs taken care of

* Terence suggestion

* warn spacing

* better logging in initial sync

* debug level standardized

* complete debug standardization

* participation at epoch end

* fix archive tests

* even more test fixes

* prom test

* ops test

* powtest

* rpc sync test

* rem part

* log formatting
2019-10-01 15:05:17 -05:00
terence tsao
944d3b16fd Batch save attestations in state transition (#3672)
* Batch save attestations

* Update test

* Revert config
2019-09-30 16:11:59 -07:00
Raul Jordan
8d215feb25
Large Prysm Performance Improvements (#3622)
* skip bls verification with a feature flag at runtime

* gazelle

* more bls mocks

* block roots efficiency

* db block roots now does not show up on the flame graphs

* save validator latest votes batch

* batch save att

* misc improvements to pprof

* import

* include validator index cache

* error if no filter criteria

* resolved comments

* build fix

* lint

* remove delay global

* attestation and block test fixes

* preston suggestions

* fix db tests

* fix missing broken tests

* tests pass
2019-09-30 15:45:53 -05:00
Nishant Das
8ece8fb44b Expose DB Metrics (#3663)
* add in bolt metrics

* unregister in db teardown

* unregister in Close()

* fix clear db case

* fix test error

* gaz

* remove unregister

* remove gaz
2019-09-30 10:24:47 -05:00
terence tsao
4c8269aca3
Part 4 of caching improvement - Use Cache (#3625) 2019-09-27 15:56:08 -07:00
Nishant Das
73938876b1 better log validator votes (#3590) 2019-09-26 06:32:06 -07:00
Preston Van Loon
0a2dfedf0f
Log warning with error instead of returning an error for fork choice votes update (#3587) 2019-09-25 20:39:19 -07:00
terence tsao
3ef681e649
Initial sync no verify block (#3586) 2019-09-25 20:32:00 -07:00
Nishant Das
8ce96428b1 Fixes our Attestation Aggregation Issues (#3579) 2019-09-25 07:39:16 -07:00
terence tsao
9577e2c123 Removed attestation check for votes (#3562) 2019-09-23 15:44:13 -05:00
terence tsao
ab2d4e8ad6 Implement attestation pool in memory (#3542)
* Run time bug

* Still failing

* Run time working

* Run time working

* Gazelle

* Fixed all the tests

* Revert config

* Revert back test configs

* Revert config

* Tested run time again, everything is good
2019-09-23 11:11:44 -05:00
terence tsao
4e6ed2744d Fixed 2 cosmetic errors (#3543)
* Fixed

* Treehash

* Updated error msg
2019-09-23 08:30:39 -07:00
terence tsao
315d4f0549 Save aggregated attestations in DB after processing (#3541)
* Added AggregatedAttestation helper

* Implemented `saveNewAttestation` for processing attestation

* Implemented `savesNewBlockAttestations` for processing block

* Tests

* Fix name

* Raul's feedback
2019-09-21 13:57:32 -05:00
terence tsao
2b2ef4f37c
Added AggregatedAttestation helper (#3539) 2019-09-21 09:43:18 -07:00
terence tsao
c29a7be0ec Fix justified check point mutation (#3534)
* first version of the watchtower api

* first version

* delete watchtower

* move to message loop

* roughtime

* one time

* fix test

* Fixed

* Fixed

* Revert unused lock
2019-09-20 10:44:28 -07:00
terence tsao
ac176a5078 Update validators votes from incoming block (#3530)
* first version of the watchtower api

* first version

* delete watchtower

* move to message loop

* roughtime

* one time

* fix test

* Update block attestation votes

* Clean up

* Found a bug

* Confirmed to be working in run time

* Confirmed to be working run time

* Raul's feedback

* Tests
2019-09-19 20:34:57 -05:00
terence tsao
b863004b2a
Aggregate attestations before verify and update votes (#3493) 2019-09-16 17:48:03 -07:00
Raul Jordan
1e086b63e8 Add Fork Choice Package Docs (#3463) 2019-09-13 13:03:14 -04:00
terence tsao
798bbbdc82 Cold start for interop (#3437)
* coldstart flags for validator

* WIP beacon node flags

* wip beacon chain, flag fix in validator, arg fix in validator

* checkpoint

* Added interop service

* working on mock chainstart

* save the state lol

* fix tests

* Save genesis validators

* gaz

* fix validator help flags

* WaitForChainStart actually waits for genesis time

* cold start fixes

* cache

* change back

* allow for genesis state too

* remove logs

* increase mmap size

* dont process if head doesn't exist

* add 10ms tolerance

* enable libp2p debug at debug, fix pubsub

* works with checkpt

* initialize justified and finalized in db

* Removed preloadStatePath from blockchain service

* Clean up

* Write to disk for now post state

* revert b466dd536f8eadbdae2264a545a755370223d917

* Builds

* Only RPC test fails now

* use minimal config, no demo config

* clean up branch

* Lint

* resolve lint

* more lint fixes

* lint

* fix viz

* Fixing RPC test

* skip before epoch 2

* RPC time out

* Fixed ordering

* rename

* remove some dbg statements

* ensure index is correct

* fix some panics

* getting closer

* fix tests

* Fix private key

* Fixed RPC test

* Fixed beacon chain build for docker

* Add interop.go to validator go_image

* Fixed docker build

* handle errors

* skip test, skip disconnecting peers

* Fixed docker build

* tolerance for attestation processing

* revert copy

* clearer err message parse

* fix launching from dep contract
2019-09-11 13:38:35 -05:00
Ivan Martinez
5e939378d0 Update to spec v0.8.3 (#3355)
* Ignore latest messages in fork choice prior to latest justified

* Make sure Compact Committee Roots isn't changed by process_final_updates

* WIP add attestation bitfields length to match committee length

* Begin work on updating spec tests to 0.8.2

* WIP set up for new spec test structure

* Fix slashings

* Get mainnet tests mostly passing for attestations and attester slashings

* Fix process attestation test

* Undo change

* Complete spec tests for all operations
Still need sanity block tests

* Fix BLS sigs

* Reduce amount of reused code in core/blocks/spectests/

* Fix tests

* Update block sanity tests to 0.8.2

* Update epoch spec tests to 0.8.2

* Clean up all tests and fix shuffling/epoch tests

* WIP update bls tests to 0.8.2

* WIP update bls tests to 0.8.3

* Finish BLS spectest update to 0.8.3

* Fix shuffling spec tests

* Fix more tests

* Update proto ssz spec tests to 0.8.3

* Attempt to fix PrevEpochFFGDataMismatches test

* Goimports

* Fix documentation

* fix test

* Use custom general spec tests

* Reduce code footprint

* Remove unneeded minimal skip

* Fix for comments

* Fix for comments

* Fix test

* Small fixes

* Cleanup block spec tests a bit

* Undo change

* fix validator

* Fix validator tests

* Run gazelle

* Fix error output for epoch spec tests
2019-09-08 12:41:52 -07:00
Raul Jordan
adc27a0bc2 Update WORKSPACE With Latest SSZ (#3416) 2019-09-08 12:58:22 -04:00
terence tsao
75bce9b7e1
Align metrics to interop (#3406) 2019-09-05 08:32:35 -07:00
Preston Van Loon
0a61c379a5
Rename / move logic about updating validator indices (#3402)
* rename, move

* pr feedback
2019-09-03 13:57:08 -07:00
terence tsao
60c048a0ec
Remove lock from store struct (#3400) 2019-09-03 13:14:23 -07:00
Preston Van Loon
5ec629af71 remove unnecessary lock (#3399) 2019-09-03 12:53:18 -07:00
terence tsao
25dbc5ea85
Add back state metrics (#3369) 2019-09-01 08:37:38 -07:00
terence tsao
146b611dc8
Use rough time for checking attestation is not from future epoch (#3370) 2019-08-31 13:18:31 -07:00
terence tsao
ca2a55874c
Save new validator index in DB (#3367) 2019-08-30 21:43:18 -07:00
Preston Van Loon
8e2dcb81ae
use roughtime (#3366) 2019-08-30 15:51:15 -07:00
terence tsao
f131585041 Initialize chain info w/o clear db (#3365)
* Initialize chain info upon restart

* Test
2019-08-30 15:24:37 -07:00
terence tsao
9a6410ec15
Lock when process attestation (#3364) 2019-08-30 14:07:42 -07:00
terence tsao
c425bf2c31 Refactor fork choice start up (#3360)
* Done

* Test

* Fixed test

* emit state initialized

* Fixed existing tests

* Lint

* Lint
2019-08-30 08:58:02 -05:00
terence tsao
5d7c33a8dc Check if slot is greater before process slots (#3349) 2019-08-28 14:24:33 -05:00
terence tsao
d84ae95309
Moved delay att inclusion to fork choice service (#3345) 2019-08-28 10:26:07 -07:00
terence tsao
323bbe10ed
Add checkpoint to state caching (#3333) 2019-08-27 15:01:27 -07:00
Raul Jordan
9ad00ffafb
Use New Attestation Receiver Method in RPC (#3287)
* next compatible, tests pass

* terence feedback

* skip comment

* fixes

* misc fix

* on block

* resolved err
2019-08-25 15:45:55 -05:00