* State cache init
* More code
* Fix lint
* More tests
* More tests
* More tests
* Fix test
* Transformations
* remove writeQueue, before fixing the tests
* Fix tests
* Add more tests, incarnation to the code items
* Fix lint
* Fix lint
* Remove shards prototype, add incarnation to the state reader code
* Clean up and replace cache in call_traces stage
* fix flaky test
* Save changes
* Readers to use addrHash, writes - addresses
* Fix lint
* Fix lint
* More accurate tracking of size
* Optimise for smaller write batches
* Attempt to integrate state cache into Execution stage
* cacheSize to default flags
* Print correct cache sizes and batch sizes
* cacheSize in the integration
* Fix tests
* Fix lint
* Remove print
* Fix exec stage
* Fix test
* Refresh sequence on write
* No double increment
* heap.Remove
* Try to fix alignment
* Refactoring, adding hashItems
* More changes
* Fix compile errors
* Fix lint
* Wrapping cached reader
* Wrap writer into cached writer
* Turn state cache off by default
* Fix plain state writer
* Fix for code/storage mixup
* Fix tests
* Fix clique test
* Better fix for the tests
* Add test and fix some more
* Fix compile error|
* More functions
* Fixes
* Fix for the tests
* sepatate DeletedFlag and AbsentFlag
* Minor fixes
* Test refactoring
* More changes
* Fix some tests
* More test fixes
* More test fixes
* Fix lint
* Move blockchain_test to be able to use stagedsync
* More fixes
* Fixes and cleanup
* Fix tests in turbo/stages
* Fix lint
* Fix lint
* Intemediate
* Fix tests
* Intemediate
* More fixes
* Compilation fixes
* More fixes
* Fix compile errors
* More test fixes
* More fixes
* More test fixes
* Fix compile error
* Fixes
* Fix
* Fix
* More fixes
* Fixes
* More fixes and cleanup
* Further fix
* Check gas used and bloom with header
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Add a convenience method to replace stages
* allow more parameters to the execution function
* add a plug-in point for stuff like bloom filters, etc
* add comments
* fixups
This change adds a test framework for the "eth" protocol and some basic
tests. The tests can be run using the './devp2p rlpx eth-test' command.
# Conflicts:
# core/forkid/forkid.go
# eth/discovery.go
# eth/handler.go
# eth/helper_test.go
# eth/protocol_test.go
* Fix tx propagation and announcement tests
* Fix checkpoint challenge test
* Save current state of code
* Fix compile errors
* Fixes
* Fix formatting
* Printouts
* Printouts
* Printouts
* Too much printing, not close db when stopping
* Better handing of interruptions
* No need to reset after incremental interruption
* Fix stages
* Fixes
* Giving up
* Fix formatting
* Cleanup
* close goruotines
* fix using defer
* close pm managers a bit later
* fmt
* finalizer for in-mem db
* after cr
* revert finalizer
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
* Euphemerally -> Ephemerally
* Move StorageMode to ethdb and pass it to PrepareStagedSync
* linter
* Remove StorageModeThinHistory and move SetStorageModeIfNotExist into storage_mode.go
* Optionally write receipts in the execute stage
* memory profiler
* linter
* proper linter fix
* linter
* typo
* Merge stateDb with changeDb so that all-or-nothing is commited in stage_execute
* db based version of PrefixByCumulativeWitnessSize
* db based version of PrefixByCumulativeWitnessSize
* retain all in Trie by default
* fix WitnessLen logic in calcTrie roots
* Rename IntermediateTrieWitnessLenBucket to IntermediateWitnessLenBucket
* handle corner cases in WL
* Use correct incarnation for IH bucket
* use name WitnessSize
* save progress towards db-only witness estimation
* results from trie and from db are still different
* less recursion
* correct incarnation in CumulativeSearch
* reuse results from previous Tick, separate concepts of parent and startKey
* experiment: if not including trie structure to WitnessSize will reduce cumulative error
* tool to generate all IH and tool to calculate assessment of cumulative error
* tool to generate all IH
* Calculate totalWitnessSize based on DB data - then schedule will not overrun state during MGR cycle
* better stats
* Calculate totalWitnessSize based on DB data - then schedule will not overrun state during MGR cycle
* Calculate totalWitnessSize based on DB data - then schedule will not overrun state during MGR cycle
* calculate ticks size distribution
* estimate cumulative error
* fix linter
* resetIH from scratch if needed
* cleanup
* fix test
* fix test
* Not hash, keep the files
* Calculate savings
* Fix
* Fix
* Fix
* Fix
* RestAPI to support local boltdb
* Not error on read-only db
* Changes so far
* Continue
* More
* Roll back a bit
* Restore newline
* something compiles
* Fix restapi
* Fix block number
* Fix reads
* Use plain writer
* Maps for storage reads and writes
* Clean up coersions
* Fix accounts/abi/bind
* Fix tests
* More fixes
* more fixes
* More fixes
* Fixes
* Fixed core/state
* Fixed eth tests
* Move code, fix linter
* Fix test
* Fix linter
* Fix linter
* Fix linter, badger_db to support AbstractKV
* Increase IdealBatchSize for badger
* Fix linter
* Fix linter
* Rewrite
* Fix linter, make counter
* Skip hash stage
* Fix linter
* Add common
* Fix wrong bucket
* Use temp files to generate index
* Fix hack, defer
* Add logging when creating files
* Properly pass datadir
* Bigger buffers, less logging
* Log current key, enforce batching
* Limit to 4m, print more
* Use ReadFull
* Optimised version of storage generation
* Don't print entire key
* Commit at the end
* Fix linter
* Remove a bit of copy-paste
* More copy-paste reduction
* Fixes
* Fix key length
* Not delete files too early
* Fix linter
* Fix logging
* Fix starting block
* Skip test
* Remove limit on the stage 4
* Fix test while disabling the hash check
* Fix tests
* Fix unreachable
* save state
* add current index feature
* fix test
* remove logs
* Only execute 1000 blocks
* Reset history index
* Correct action
* Increase batch size
* Increase chunk size, print memory stats
* Fix linter
* Remove unused from
* Split into 2 staged
* Use storage history gen
* remove log
* Not to run tx_cacher in staged mode
* Not to recover during stage 2
* Not to recover during stage 2
* Remove counter
Co-authored-by: b00ris <b00ris@mail.ru>
* introduce fork
* Fix unwind test
* Unwind now really works in the test
* Fix linter
* Fix linter
* Fix linter
* Not to handle block and block hash announced in staged sync
* add_incarnation_to_acc_root_in_ih
* merge cached resolver into stateful resolver
* - move account root set to "post iteration" of resolver
- rename "cache" to IntermediateHash
* remove blockNR and bucket params from walker
* fix out of range panic
* calc acc.Root on the fly
* remove fieldSet field from resolver, make logic of root - lazy
* remove 2 parameters
* working version of forward-only walk over Acc and Storage
* improve test
* rebase master
* save progress - more tests for PrepareResolveParams, add dedicated ResolveSet for storage.
Problem: See duplicates in ResolveSet hexes. Next test failing: oracle_test.go
* skip old incarnations
* don't rebuild when 0 requests
* fix tests
* start from account key when need resolve storage
* Error: stateless prototype faced hashNode when extracting witness
* Statless works: copy touches
* Remove getAccRoot function
* Remove "isAccount" parameter from resolver signature
* Fix: use correct storageResolveSet in finaliseStorageRoot
* Fix: when startKey changed - reset storage buffers also
* Fix: if account incarnation=0 - set EmptyRoot
* Fix: remove account roots by default from IntermediateHash bucket
* Fix: skip abandoned storage - which appeared just after startKey
* Fix: did reset acc key incorrectly
* Fix: clean previous key if receive IH
* Fix: IH observer - subscribe only to branch nodes (was subscribed to value nodes also)
* Add DISABLE_IH and STORE_ACCOUNT_ROOT env variables for tests
* Remove accNode from IH cycle
* Fix flags
* Fix: reset succStorage also
* Fix: skip IH if it has wrong incarnation
* Fix: skip IH if it has wrong incarnation
* Fix: skip IH if it has wrong incarnation
* Fix: use rssStorage to HashOnly check
* Fix: remove termination symbol from resolveRequest
* cleanup
* Fix: skip abandoned storage after IH
* Debug
This reverts commit 9c5eb69465f25607d546b03359b2cbcb1bd46689.
* Fix linters
* add_incarnation_to_acc_root_in_ih
* merge cached resolver into stateful resolver
* - move account root set to "post iteration" of resolver
- rename "cache" to IntermediateHash
* remove blockNR and bucket params from walker
* fix out of range panic
* calc acc.Root on the fly
* remove fieldSet field from resolver, make logic of root - lazy
* remove 2 parameters
* working version of forward-only walk over Acc and Storage
* improve test
* rebase master
* save progress - more tests for PrepareResolveParams, add dedicated ResolveSet for storage.
Problem: See duplicates in ResolveSet hexes. Next test failing: oracle_test.go
* skip old incarnations
* don't rebuild when 0 requests
* fix tests
* start from account key when need resolve storage
* Error: stateless prototype faced hashNode when extracting witness
* Statless works: copy touches
* Remove getAccRoot function
* Remove "isAccount" parameter from resolver signature
* Fix: use correct storageResolveSet in finaliseStorageRoot
* Fix: when startKey changed - reset storage buffers also
* Fix: if account incarnation=0 - set EmptyRoot
* Fix: remove account roots by default from IntermediateHash bucket
* Fix: skip abandoned storage - which appeared just after startKey
* Fix: did reset acc key incorrectly
* Fix: clean previous key if receive IH
* Fix: IH observer - subscribe only to branch nodes (was subscribed to value nodes also)
* Add DISABLE_IH and STORE_ACCOUNT_ROOT env variables for tests
* Remove accNode from IH cycle
* Fix flags
* Fix: reset succStorage also
* Fix: skip IH if it has wrong incarnation
* Fix: skip IH if it has wrong incarnation
* Fix: skip IH if it has wrong incarnation
* Fix: use rssStorage to HashOnly check
* Fix: remove termination symbol from resolveRequest
* cleanup
* Fix: skip abandoned storage after IH
* remove inc
* remove inc from rss
* tr.succStorage.Reset()
* remove inc from rss
* Remove hard-coding
* succ.Reset
* Enable CalcTrieRoots
* Proper dumping of the trie
* Debug
* Fix for CalcTrieRoot
* Fix another inteference bug
* Temp
* Fix test
* Cleanup
* remove STORE_ACCOUNT_ROOT=true flag
* Fix linter
* Fix linter
* Disable getnodedata by default
* Fix test
* Fix test
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
