This PR makes use of go 1.13 error handling, wrapping errors and using
errors.Is to check a wrapped root-cause. It also removes the travis
builders for go 1.11 and go 1.12.
# Conflicts:
# eth/downloader/downloader.go
# eth/downloader/downloader_test.go
* Fix body fetch
* Reduce spurious reorgs
* Exit the sync cycle after unwinds
* Fix out of range
* No stalling check for staged sync
* Disable failing tests
* Remove duplicate log message
* Fix UnwindTest and add assertions
* Fix formatting
* Cleanup
* Fix off by one error with bodies
* Remove rollback
* save state
* txlookup full results
* save state
* save state
* remove experiments
* some fix&lint
* add end key to txLookup and index generation
* change log message
* change log
* fix lint
* lint
* fix test
* Euphemerally -> Ephemerally
* Move StorageMode to ethdb and pass it to PrepareStagedSync
* linter
* Remove StorageModeThinHistory and move SetStorageModeIfNotExist into storage_mode.go
* Optionally write receipts in the execute stage
* memory profiler
* linter
* proper linter fix
* linter
* typo
* Merge stateDb with changeDb so that all-or-nothing is commited in stage_execute
* uint256 in rlp
* uint256 rather than big.Int in Transation
* linters
* more linters
* still linters
* Reduce garbage in writeUint256
* Experiment with GC in writeByteArray
* db based version of PrefixByCumulativeWitnessSize
* db based version of PrefixByCumulativeWitnessSize
* retain all in Trie by default
* fix WitnessLen logic in calcTrie roots
* Rename IntermediateTrieWitnessLenBucket to IntermediateWitnessLenBucket
* handle corner cases in WL
* Use correct incarnation for IH bucket
* use name WitnessSize
* save progress towards db-only witness estimation
* results from trie and from db are still different
* less recursion
* correct incarnation in CumulativeSearch
* reuse results from previous Tick, separate concepts of parent and startKey
* experiment: if not including trie structure to WitnessSize will reduce cumulative error
* tool to generate all IH and tool to calculate assessment of cumulative error
* tool to generate all IH
* Calculate totalWitnessSize based on DB data - then schedule will not overrun state during MGR cycle
* better stats
* Calculate totalWitnessSize based on DB data - then schedule will not overrun state during MGR cycle
* Calculate totalWitnessSize based on DB data - then schedule will not overrun state during MGR cycle
* calculate ticks size distribution
* estimate cumulative error
* fix linter
* resetIH from scratch if needed
* cleanup
* fix test
* fix test
* Move promotion before the shortcut
* Correct the logs
* Trying without the encoder
* Revert "Trying without the encoder"
This reverts commit 24c526bc2791fb6806b1c0d4f52a43b87c3959b9.
* Commit to db, not to the mutation
* Fix log more
* No need to log every deleted file
* "Unwind" for the execution phase when plain state is selected
* test stub (fails)
* tests (one with incarnations fails)
* test fixups
* fix tests: cleanup contract code bucket
* Rewrite
* Fix linter, make counter
* Skip hash stage
* Fix linter
* Add common
* Fix wrong bucket
* Use temp files to generate index
* Fix hack, defer
* Add logging when creating files
* Properly pass datadir
* Bigger buffers, less logging
* Log current key, enforce batching
* Limit to 4m, print more
* Use ReadFull
* Optimised version of storage generation
* Don't print entire key
* Commit at the end
* Fix linter
* Remove a bit of copy-paste
* More copy-paste reduction
* Fixes
* Fix key length
* Not delete files too early
* Fix linter
* Fix logging
* Fix starting block
* Skip test
* Remove limit on the stage 4
* Fix test while disabling the hash check
* Fix tests
* Fix unreachable
* save state
* add current index feature
* fix test
* remove logs
* Only execute 1000 blocks
* Reset history index
* Correct action
* Increase batch size
* Increase chunk size, print memory stats
* Fix linter
* Remove unused from
* Split into 2 staged
* Use storage history gen
* remove log
* Not to run tx_cacher in staged mode
* Not to recover during stage 2
* Not to recover during stage 2
* Remove counter
Co-authored-by: b00ris <b00ris@mail.ru>
* introduce PlainStateReader with fallbacks
* no 10.000 changes in tests
* even less iterations
* remove even more iterations
* add `go run ./cmd/geth --syncmode staged --plainstate` flag
* fix serialization calls
* make a more sensible file default
doesn’t affect anything, because this flag is always overriden when parsing CLI. but still.
* introduce fork
* Fix unwind test
* Unwind now really works in the test
* Fix linter
* Fix linter
* Fix linter
* Not to handle block and block hash announced in staged sync
* Initial rework
* Fix lint
* boilerplate for the test
* Fix linter
* More to the test
* fix linter
* Save progress
* Test works
* Fix linter
* More on test
* Fix test, linter
* Fix lint
* Fix test
* add_incarnation_to_acc_root_in_ih
* merge cached resolver into stateful resolver
* - move account root set to "post iteration" of resolver
- rename "cache" to IntermediateHash
* remove blockNR and bucket params from walker
* fix out of range panic
* calc acc.Root on the fly
* remove fieldSet field from resolver, make logic of root - lazy
* remove 2 parameters
* working version of forward-only walk over Acc and Storage
* improve test
* rebase master
* save progress - more tests for PrepareResolveParams, add dedicated ResolveSet for storage.
Problem: See duplicates in ResolveSet hexes. Next test failing: oracle_test.go
* skip old incarnations
* don't rebuild when 0 requests
* fix tests
* start from account key when need resolve storage
* Error: stateless prototype faced hashNode when extracting witness
* Statless works: copy touches
* Remove getAccRoot function
* Remove "isAccount" parameter from resolver signature
* Fix: use correct storageResolveSet in finaliseStorageRoot
* Fix: when startKey changed - reset storage buffers also
* Fix: if account incarnation=0 - set EmptyRoot
* Fix: remove account roots by default from IntermediateHash bucket
* Fix: skip abandoned storage - which appeared just after startKey
* Fix: did reset acc key incorrectly
* Fix: clean previous key if receive IH
* Fix: IH observer - subscribe only to branch nodes (was subscribed to value nodes also)
* Add DISABLE_IH and STORE_ACCOUNT_ROOT env variables for tests
* Remove accNode from IH cycle
* Fix flags
* Fix: reset succStorage also
* Fix: skip IH if it has wrong incarnation
* Fix: skip IH if it has wrong incarnation
* Fix: skip IH if it has wrong incarnation
* Fix: use rssStorage to HashOnly check
* Fix: remove termination symbol from resolveRequest
* cleanup
* Fix: skip abandoned storage after IH
* Debug
This reverts commit 9c5eb69465f25607d546b03359b2cbcb1bd46689.
* Fix linters
* add_incarnation_to_acc_root_in_ih
* merge cached resolver into stateful resolver
* - move account root set to "post iteration" of resolver
- rename "cache" to IntermediateHash
* remove blockNR and bucket params from walker
* fix out of range panic
* calc acc.Root on the fly
* remove fieldSet field from resolver, make logic of root - lazy
* remove 2 parameters
* working version of forward-only walk over Acc and Storage
* improve test
* rebase master
* save progress - more tests for PrepareResolveParams, add dedicated ResolveSet for storage.
Problem: See duplicates in ResolveSet hexes. Next test failing: oracle_test.go
* skip old incarnations
* don't rebuild when 0 requests
* fix tests
* start from account key when need resolve storage
* Error: stateless prototype faced hashNode when extracting witness
* Statless works: copy touches
* Remove getAccRoot function
* Remove "isAccount" parameter from resolver signature
* Fix: use correct storageResolveSet in finaliseStorageRoot
* Fix: when startKey changed - reset storage buffers also
* Fix: if account incarnation=0 - set EmptyRoot
* Fix: remove account roots by default from IntermediateHash bucket
* Fix: skip abandoned storage - which appeared just after startKey
* Fix: did reset acc key incorrectly
* Fix: clean previous key if receive IH
* Fix: IH observer - subscribe only to branch nodes (was subscribed to value nodes also)
* Add DISABLE_IH and STORE_ACCOUNT_ROOT env variables for tests
* Remove accNode from IH cycle
* Fix flags
* Fix: reset succStorage also
* Fix: skip IH if it has wrong incarnation
* Fix: skip IH if it has wrong incarnation
* Fix: skip IH if it has wrong incarnation
* Fix: use rssStorage to HashOnly check
* Fix: remove termination symbol from resolveRequest
* cleanup
* Fix: skip abandoned storage after IH
* remove inc
* remove inc from rss
* tr.succStorage.Reset()
* remove inc from rss
* Remove hard-coding
* succ.Reset
* Enable CalcTrieRoots
* Proper dumping of the trie
* Debug
* Fix for CalcTrieRoot
* Fix another inteference bug
* Temp
* Fix test
* Cleanup
* remove STORE_ACCOUNT_ROOT=true flag
* Fix linter
* Fix linter
* Disable getnodedata by default
* Fix test
* Fix test
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
