Commit Graph

18 Commits

Author SHA1 Message Date
terence tsao
def7b602e3
Hot states use no DB (#6488)
* Add cache to service struct
* Update hot getters/setters to use cache
* Update migration
* Update other services to adapt
* Fix initial sync get state
* Update getter related tests
* Update hot related tests
* Update migrate related tests
* New awesome tests for migration
* Clean up rest of the tests
* Merge refs/heads/master into hot-state-no-db
* Fix block chain head tests
* Fix block chain processor tests
* Fixed RPC tests
* Update cold getter and test
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
* Fix sync tests
* Short cut if state is already in DB
* Remove uneeded saves
* Update beacon-chain/state/stategen/hot_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/getter_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/getter_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/setter_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Preston's feedback
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
* Return a copy of cache states
* Remove hot state caches check in StateByRoot
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
* Merge refs/heads/master into hot-state-no-db
* Raul's feedback
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
2020-07-06 17:22:12 +00:00
Preston Van Loon
f9038674d1
Refactor setupDB to return the state summary cache used by the database (#6369) 2020-06-23 13:40:55 -07:00
terence tsao
678347e64b
Handle missing archived index (#5899)
* Make `archivedRoot` handle missing root case
* Regression tests
* Fixed existing tests
* Removed debug log
* Merge refs/heads/master into handle-missing-archived-index
* Merge refs/heads/master into handle-missing-archived-index
* More comments on the look back
* Merge branch 'handle-missing-archived-index' of github.com:prysmaticlabs/prysm into handle-missing-archived-index
* Merge refs/heads/master into handle-missing-archived-index
2020-05-18 19:46:06 +00:00
terence tsao
ef4dead3fe
Fix cold state generation (#5770) 2020-05-06 13:25:37 -07:00
Preston Van Loon
9be8a456eb
Use stateutil.BlockRoot everywhere (#5739)
* Sweeping replace of ssz.HashTreeRoot(block) to stateutil.BlockRoot, minor tweaks to critical path of proposeBlock
* imports
* Merge branch 'master' into better-block-htr
* Add --enable-custom-block-htr to e2e flags
* Merge branch 'better-block-htr' of github.com:prysmaticlabs/prysm into better-block-htr
2020-05-05 04:30:24 +00:00
Victor Farazdagi
b5c4dc2a75
Refactor db teardown to testing.TB.Cleanup (#5725)
* init-sync updates
* slasher/db/kv tests
* beacon-chain/rpc/beacon tests
* update kv_test
* beacon-chain/rpc-validator tests updated
* slasher/db/kv - remove teardown method
* beacon-chain/sync tests updated
* beacon-chain/db/kv tests updated
* beacon-chain/blockchain tests updated
* beacon-chain/state/stategen tests updated
* beacon-chain/powchain updates
* updates rest of slasher tests
* validator/db tests
* rest of the tests
* minor comments update
* gazelle
* Merge refs/heads/master into teardowndb-to-cleanup
2020-05-04 01:14:34 +00:00
terence tsao
480ddb7c3d
Continue if archival state is not in DB (#5565)
* Continue if archival state is not availble to generate
* Fixed test
* Merge branch 'master' into save-blocks-splitslot
* Merge refs/heads/master into save-blocks-splitslot
2020-04-21 18:18:55 +00:00
Victor Farazdagi
748d513c62
proper error checking and type assertions (#5424)
* proper error checking and type assertions
2020-04-14 16:41:09 +00:00
terence tsao
3b9a03d517
State service clean up - better cold state getter (#5393)
* Update load cold states by root and slot

* Update cold tests

* Add a way to recover state summary

* Short circuit slot 0 to return genesis state

* Short circuit if there's no block in DB

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-13 11:32:02 -05:00
terence tsao
e22365c4a8
Uncomment out cold state tests (#5252)
* Fixed most of the tests

* All tests passing

* All tests passing

* Fix merge conflict

* Fixed error test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-31 11:23:39 -05:00
terence tsao
7f7866ff2a
Micro optimizations on new-state-mgmt service for initial syncing (#5241)
* Starting a quick PoC

* Rate limit to one epoch worth of blocks in memory

* Proof of concept working

* Quick comment out

* Save previous finalized checkpoint

* Test

* Minor fixes

* More run time fixes

* Remove panic

* Feature flag

* Removed unused methods

* Fixed tests

* E2e test

* comment

* Compatible with current initial sync

* Starting

* New cache

* Cache getters and setters

* It should be part of state gen

* Need to use cache for DB

* Don't have to use finalized state

* Rm unused file

* some changes to memory mgmt when using mempool

* More run time fixes

* Can sync to head

* Feedback

* Revert "some changes to memory mgmt when using mempool"

This reverts commit f5b3e7ff4714fef9f0397007f519a45fa259ad24.

* Fixed sync tests

* Fixed existing tests

* Test for state summary getter

* Gaz

* Fix kafka passthrough

* Fixed inputs

* Gaz

* Fixed build

* Fixed visibility

* Trying without the ignore

* Didn't work..

* Fix kafka

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-03-30 17:10:45 -05:00
terence tsao
f2a3fadda7
Productionization new state service part 1 (#5230)
* Fixed last play methods

* Fixed a regression. Genesis case for state gen

* Comment

* Starting

* Update proto

* Remove boundary root usages

* Update migrate

* Clean up

* Remove unused db methods

* Kafta

* Kafta

* Update tests

* Comments

* Fix state summary tests

* Missed one pass through for kafta
2020-03-27 13:28:38 -07:00
terence tsao
eeffa4fb30
New state getter (#5101)
* getter.go
* getter_test.go
* fixed a cold bug
* fmt gaz
* All tests pass
* Merge branch 'master' into new-state-getter
* Merge refs/heads/master into new-state-getter
2020-03-14 18:39:23 +00:00
terence tsao
b024191887
Get cold intermediate state with slot (#5097)
* loadColdIntermediateStateWithSlot

* Starting test

* Tests
2020-03-14 10:34:37 -05:00
terence tsao
0974c02a00
Load cold state by root (#5086) 2020-03-12 15:27:55 -07:00
terence tsao
ed7ad4525e
Method to retrieve block slot via block root (#5084)
* blockRootSlot

* Tests

* Gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-12 16:04:24 -05:00
terence tsao
43722e45f4
Save cold state (#5077) 2020-03-12 05:58:06 -07:00
terence tsao
a90ffaba49
Archived point retrieval and recovery (#5075) 2020-03-11 17:38:30 -07:00