erigon-pulse/cmd/integration/Readme.md
Alex Sharov c3e1cfdac8
Pruning for: exec, log_index, tx_lookup, history stages (#2399)
* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* add tvm flag

* save

* db migration for storage mode
add flag --prune=
remove flag --storage-mode=
add flag --experiments=tevm,...
rename integration set_storage_mode to set_prune

* fix

* forward move of stages must skip everything before PruneTo

* keep in db progress of prune method

* keep in db progress of prune method

* simplify logs

* simplify logs

* simplify logs

* fix test

* simplify logs

* simplify logs

* simplify logs

* simplify logs

* remove callTraceSet as dupsort
use etl transform for txlookup prune
remove some logs

* cleanup tests a bit

* print_stages and eth_sync to show prune progress

* fix print_stages

* add readme about --prune.to flag

* more docs

* add --prune.history.older and other flags support

* fix migration on empty db

* better toString

* better toString
2021-07-20 21:03:19 +01:00

72 lines
2.1 KiB
Markdown

Integration - tool to run Erigon stages in custom way: run/reset single stage, run all stages but reorg every X blocks,
etc...
## Examples
All commands require parameter `--datadir=<datadir>` - I will skip it for readability.
```
integration --help
integration print_stages
# Run single stage
integration stage_senders
integration stage_exec
integration stage_exec --block=1_000_000 # stop at 1M block
integration stage_hash_state
integration stage_trie
integration stage_history
integration stage_tx_lookup
# Unwind single stage 10 blocks backward
integration stage_exec --unwind=10
# Drop data of single stage
integration stage_exec --reset
integration stage_history --reset
# Unwind single stage N blocks backward
integration stage_exec --unwind=N
integration stage_history --unwind=N
# Run stage prune to block N
integration stage_exec --prune.to=N
integration stage_history --prune.to=N
...
# hack which allows to force clear unwind stack of all stages
clear_unwind_stack
```
## For testing run all stages in "N blocks forward M blocks re-org" loop
Pre-requirements of `state_stages` command:
- Headers/Bodies must be downloaded
- TxSenders stage must be executed
```
make all
./build/bin/integration state_stages --datadir=<datadir> --unwind=10 --unwind.every=20 --pprof
integration reset_state # drops all stages after Senders stage (including it's db tables DB tables)
```
For example:
```
--unwind=1 --unwind.every=10 # 10 blocks forward, 1 block back, 10 blocks forward, ...
--unwind=10 --unwind.every=1 # 1 block forward, 10 blocks back, 1 blocks forward, ...
--unwind=10 # 10 blocks back, then stop
--integrity.fast=false --integrity.slow=false # it performs DB integrity checks each step. You can disable slow or fast checks.
--block # stop at exact blocks
--chaindata.reference # When finish all cycles, does comparison to this db file.
```
## "Wrong trie root" problem - temporary solution
```
make all
./build/bin/integration stage_hash_state --datadir=<datadir> --reset
./build/bin/integration stage_trie --datadir=<datadir> --reset
# Then run TurobGeth as usually. It will take 2-3 hours to re-calculate dropped db tables
```