erigon-pulse/cmd/integration
ledgerwatch 5ea590c18e
State cache switching writes to reads during commit (#1368)
* State cache init

* More code

* Fix lint

* More tests

* More tests

* More tests

* Fix test

* Transformations

* remove writeQueue, before fixing the tests

* Fix tests

* Add more tests, incarnation to the code items

* Fix lint

* Fix lint

* Remove shards prototype, add incarnation to the state reader code

* Clean up and replace cache in call_traces stage

* fix flaky test

* Save changes

* Readers to use addrHash, writes - addresses

* Fix lint

* Fix lint

* More accurate tracking of size

* Optimise for smaller write batches

* Attempt to integrate state cache into Execution stage

* cacheSize to default flags

* Print correct cache sizes and batch sizes

* cacheSize in the integration

* Fix tests

* Fix lint

* Remove print

* Fix exec stage

* Fix test

* Refresh sequence on write

* No double increment

* heap.Remove

* Try to fix alignment

* Refactoring, adding hashItems

* More changes

* Fix compile errors

* Fix lint

* Wrapping cached reader

* Wrap writer into cached writer

* Turn state cache off by default

* Fix plain state writer

* Fix for code/storage mixup

* Fix tests

* Fix clique test

* Better fix for the tests

* Add test and fix some more

* Fix compile error|

* More functions

* Fixes

* Fix for the tests

* sepatate DeletedFlag and AbsentFlag

* Minor fixes

* Test refactoring

* More changes

* Fix some tests

* More test fixes

* More test fixes

* Fix lint

* Move blockchain_test to be able to use stagedsync

* More fixes

* Fixes and cleanup

* Fix tests in turbo/stages

* Fix lint

* Fix lint

* Intemediate

* Fix tests

* Intemediate

* More fixes

* Compilation fixes

* More fixes

* Fix compile errors

* More test fixes

* More fixes

* More test fixes

* Fix compile error

* Fixes

* Fix

* Fix

* More fixes

* Fixes

* More fixes and cleanup

* Further fix

* Check gas used and bloom with header

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2020-12-08 09:44:29 +00:00
..
commands State cache switching writes to reads during commit (#1368) 2020-12-08 09:44:29 +00:00
main.go Rpcdaemon as lib 2 (#943) 2020-08-20 10:52:27 +07:00
Readme.md integration_write_receipts (#1098) 2020-09-11 17:33:46 +07:00

Integration - tool to run TurboGeth stages in custom way: run single stage, or run all stages but reorg every X blocks, etc...

By examples. All commands require parameter --chaindata=/path/to/chaindata - I will skip it for readability.

integration --help
integration print_stages --block=1_000_000

# run all stages (which do not need internet). 
integration state_stages --unwind=1 --unwind_every=10  # process 10 blocks, then reorg 1 block, then process 10 blocks, ... 
integration state_stages --unwind=10 --unwind_every=1  # process 1 block, then reorg 10 blocks, then process 1 blocks, ...

# Run single stage 
integration stage_senders 
integration stage_exec  
integration stage_exec --block=1_000_000 # stop at 1M block
integration stage_hash_state 
integration stage_ih 
integration stage_history
integration stage_tx_lookup

# Drop data of single stage 
integration stage_exec --reset     
integration stage_history --reset
... 

# reset all data after stage_senders
integration reset_state

# hack which allows to force clear unwind stack of all stages
clear_unwind_stack

The way I usually run it:

go run -trimpath ./cmd/integration state_stages --chaindata=/path/to/chaindata --unwind=10 --unwind_every=20 --pprof 

Pre-requirements of state_stages command:

  • Headers/Bodies must be downloaded
  • TxSenders stage must be executed