* Pruning for: exec, log_index, tx_lookup, history stages
* Pruning for: exec, log_index, tx_lookup, history stages
* Pruning for: exec, log_index, tx_lookup, history stages
* Pruning for: exec, log_index, tx_lookup, history stages
* add tvm flag
* save
* db migration for storage mode
add flag --prune=
remove flag --storage-mode=
add flag --experiments=tevm,...
rename integration set_storage_mode to set_prune
* fix
* forward move of stages must skip everything before PruneTo
* keep in db progress of prune method
* keep in db progress of prune method
* simplify logs
* simplify logs
* simplify logs
* fix test
* simplify logs
* simplify logs
* simplify logs
* simplify logs
* remove callTraceSet as dupsort
use etl transform for txlookup prune
remove some logs
* cleanup tests a bit
* print_stages and eth_sync to show prune progress
* fix print_stages
* add readme about --prune.to flag
* more docs
* add --prune.history.older and other flags support
* fix migration on empty db
* better toString
* better toString
* Initial work on RPC streaming
* Create the stream and pass it on
* trace_filter expressed as streamable (not finished)
* Reenable call trace index
* Add new line
* Remove storage mode override
* Tool for overriding storage mode
* Diagnostics
* Make trace_filter more streamy
* Streaming debug_traceTransaction and debug_traceCall
* Fix test
* Log error on stream flush
* Enable streaming for http
* Flush the stream too
* Make trace_filter flush too
* Stop streaming if client is not interested
* Try to quiet annoying test|
* Revert "Try to quiet annoying test|"
This reverts commit 42849257bfa52e90140aa535af34b957cd97a222.
* Debug log for test
* Proceed with handshake regardless of whether peer notification worked
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* SE
* State cache init
* More code
* Fix lint
* More tests
* More tests
* More tests
* Fix test
* Transformations
* remove writeQueue, before fixing the tests
* Fix tests
* Add more tests, incarnation to the code items
* Fix lint
* Fix lint
* Remove shards prototype, add incarnation to the state reader code
* Clean up and replace cache in call_traces stage
* fix flaky test
* Save changes
* Readers to use addrHash, writes - addresses
* Fix lint
* Fix lint
* More accurate tracking of size
* Optimise for smaller write batches
* Attempt to integrate state cache into Execution stage
* cacheSize to default flags
* Print correct cache sizes and batch sizes
* cacheSize in the integration
* Fix tests
* Fix lint
* Remove print
* Fix exec stage
* Fix test
* Refresh sequence on write
* No double increment
* heap.Remove
* Try to fix alignment
* Refactoring, adding hashItems
* More changes
* Fix compile errors
* Fix lint
* Wrapping cached reader
* Wrap writer into cached writer
* Turn state cache off by default
* Fix plain state writer
* Fix for code/storage mixup
* Fix tests
* Fix clique test
* Better fix for the tests
* Add test and fix some more
* Fix compile error|
* More functions
* Fixes
* Fix for the tests
* sepatate DeletedFlag and AbsentFlag
* Minor fixes
* Test refactoring
* More changes
* Fix some tests
* More test fixes
* More test fixes
* Fix lint
* Move blockchain_test to be able to use stagedsync
* More fixes
* Fixes and cleanup
* Fix tests in turbo/stages
* Fix lint
* Fix lint
* Intemediate
* Fix tests
* Intemediate
* More fixes
* Compilation fixes
* More fixes
* Fix compile errors
* More test fixes
* More fixes
* More test fixes
* Fix compile error
* Fixes
* Fix
* Fix
* More fixes
* Fixes
* More fixes and cleanup
* Further fix
* Check gas used and bloom with header
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Expose C Handle of lmdbTx
* LoadExecutionFunctionPointer
* silkworm ExecuteBlocks
* linter
* CLI flag for SilkwormExecutionFunc
* linter
* Call SilkwormExecutionFunc in SpawnExecuteBlocksStage
* Commit transaction after Silkworm execution
* Fix batch initialization
* Make batch_size uint64_t instead of size_t
* max_block in silkworm_execute_blocks
* More accurate logProgress
* Use CHandle exposed by lmdb-go
* Enable Silkworm on Linux only
* Update lmdb-go
* Error when attempting to use Silkworm not on Linux
* Move SilkwormFlag from cmd/utils/flags.go to turbo/cli/flags.go
* Integrate Silkworm into cmd/integration
* Check against ReaderBuilder/WriterBuilder/ChangeSetHook in Silkworm execution
* Refactor SpawnExecuteBlocksStage
* linters
* Small clean-ups
* Move ChangeSetHook inside executeBlockWithGo
* No need to tamper with batch size in executeBlocksWithSilkworm
* fix
* Fix
* Fixup
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Initial work on state shards
* Fix compile errors
* Intermediate
* Implement dispatcher
* Implemented sharded reader
* No cache
* Print comms
* No prepopulation
* Print dispatcher activity
* Print more
* Print errors
* Fix dispatcher
* Remove logging
* Dispatcher to enforce initial barrier
* Don't print that much
* Turn off cache pre-seeting for state sharding
* Artificial latency, fixed number of blocks to execute
* Fix
* Add print_migration command to integration
* Not run migrations for print commands
* Reverse migration
* Fix lint
* Add datadir to run_migrations
* Add lmdb.mapSize to integration
* Make lmdb.mapSize effective
* Fix
* receipt sizes
* fix print
* Switch to another version of lmdb
* Collector from existing files
* Not load
* Commit after clearing
* Add methods for clear/drop buckets by multiple transactions
* Fix to lmdb
* Not remove temp files
* Not load
* Re-enable loading
* Prints
* Try to fix refill
* Migration in 3 steps
* tidy mod
* Fix lint
* Fix tests
* Migrations for CBOR receipts in 3 txs
* Fix lint
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
* Add lmdb.mapSize to integration
* Make lmdb.mapSize effective
* Use mapSize for first opening
* Fix for cobra
* Extend lmdb.mapSize for all commands
* Fix lint
* save state
* torrent experiments
* torrent experiment passed
* fixes after merge
* snapshot headers processing passed
* save state
* save state
* download headers works after snapshot processing
* save state
* save state
* save state
* save state
* add lazy load tx to snapshots, increase number of trackers
* save state
* speedup getting info
* change logging
* move to turbo package
* save state
* save state
* save state
* cleanup
* save state
* add test test
* save state
* lmdb debugging
* fix readonly mode
* save state
* fix build
* sync works
* save state
* save state
* save state
* allow cmd stages stageSenders use snapshots
* debugging failed hashing
* remove experimental tests
* remove torrent experimental tests
* fix lint
* extract snapshot wrapper
* metainfo checker
* add remote seeder
* add logs
* update gomod
* remove useless code
* fix lint&remove useless code
* extract verify snapshot to separated command
* skip debug test
* fix test
* change type of seedSnapshot flag
* add eth logger to torrent lib
* skip debug test
* add Close method
* review fixes
* fix lint
* tidy mods
* Fix compile
* Fix lint
* Fix rpcdaemon running in the docker
Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
* save state
* torrent experiments
* torrent experiment passed
* fixes after merge
* snapshot headers processing passed
* save state
* save state
* download headers works after snapshot processing
* save state
* save state
* save state
* save state
* add lazy load tx to snapshots, increase number of trackers
* save state
* speedup getting info
* change logging
* move to turbo package
* save state
* save state
* save state
* cleanup
* save state
* add test test
* save state
* lmdb debugging
* fix readonly mode
* save state
* fix build
* sync works
* save state
* save state
* save state
* allow cmd stages stageSenders use snapshots
* debugging failed hashing
* remove experimental tests
* remove torrent experimental tests
* fix lint
* extract snapshot wrapper
* metainfo checker
* add remote seeder
* add logs
* update gomod
* remove useless code
* fix lint&remove useless code
* extract verify snapshot to separated command
* skip debug test
* fix test
* change type of seedSnapshot flag
* add eth logger to torrent lib
* skip debug test
* add Close method
* review fixes
* fix lint
Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
* save state
* torrent experiments
* torrent experiment passed
* fixes after merge
* snapshot headers processing passed
* save state
* save state
* download headers works after snapshot processing
* save state
* save state
* save state
* save state
* add lazy load tx to snapshots, increase number of trackers
* save state
* speedup getting info
* change logging
* move to turbo package
* save state
* save state
* save state
* cleanup
* save state
* add test test
* save state
* lmdb debugging
* fix readonly mode
* save state
* fix build
* sync works
* save state
* save state
* save state
* allow cmd stages stageSenders use snapshots
* debugging failed hashing
* remove experimental tests
* remove torrent experimental tests
* fix lint
* extract snapshot wrapper
* metainfo checker
* add remote seeder
* add logs
* update gomod
* remove useless code
* fix lint&remove useless code
* extract verify snapshot to separated command
* skip debug test
* fix test
* change type of seedSnapshot flag
* add eth logger to torrent lib
* skip debug test
* add Close method
* review fixes
* fix lint
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
* timer-based logs
* timer-based logs
* delegate progress calculation to user
* delegate progress calculation to user
* delegate progress calculation to user
* clear
* add logs to senders recovery
* use default dir in integration
* more logs
* more logs
