* First algorithm
* Update doc
* Second algorithm implemented
* Prepend (unfinished)
* More on prepend
* More fixes
* Change CalcDifficulty interface
* fix make test
* Fix formatting
* Another test
* More on Preprend
* Another test for Prepend
* More tests for Prepend
* More tests for Prepend
* Move files to turbo
* Add another test for Prepend
* Started adding Append algorithm
* More work on Append
* Start changing the terminology
* Split trees into segments
* More fixes
* Fix compilation
* Connect
* FindAnchors, FindTip
* Tip has anchor and not anchorParent field
* Remove Prepend and Append
* Add Connect
* NewAnchor algorithm
* More to recovery from files
* Add more to AddAnchor
* Request more headers
* Header fix-sizes serialisation
* Upgrade CI to 1.15
* Fix lint
* Fix lint
* replace RLP with fixed-sized serialisation
* Fix lint
* More algos
* Fix lint
* Cut down the stateGrowth generation
* Fix NPE
* Fix for lastTimestamp
* Adjust to the current state
* Potential fix
* Fix for shorter keys in the state
* Extract code
* Iterate over code
* First
* More on SA interpreter
* Fixup
* Add cfg action to hack binary that invokes the SaInterpreter. Added an operation handler for PUSH1
* refactor cfg tests into separate file
* Move cfg tests into separate file
* More refactoring into new file
* dataflow interpreter
* work on cfg0
* finish cfg0
* df works on base examples
* refactor into dataflow spec
* add bounded stack
* add harder example
* fix switch pass thru
* fix switch pass thru
* bug fix, and better printing
* manual merge
* restore call to test gencfg
* abstract interpretation based cfg analysis
* fix post signature
* use uint256 instead uint64, add post function
* preprocess stmts
* initial implementation of resolve
* fix resolve
* fix resolve
* print stmts for edges
* print stmts for edges
* print states
* print states
* bug fixes, debugging
* fix jumpi dest - first working impl
* reachability analysis to filter out dead edges
* add all transfer functions
* larger contract bytecodes from solc compiler
* simple solidity contract goes thru
* add deposit contract bytecode
* rename deposit contract test
* fix new contract arg
* Address non-determinism leading to imprecise results
* improve debugging output
* improve debugging output
* improve debugging output
* fix for bug causing incorrect analysis results
* fix for bug causing incorrect analysis results
* fix for bug causing incorrect analysis results
* add more test cases
* fix coverage bug
* debugging for non-termination
* fix bad fixpoint check
* fix data inference
* fix transfer function for halting stmts
* switch to deposit contract test, disable debugging
* add anly counter to viz, fix stmt.valid check
* show all preds, adjust anlycounter behavior
* dfs instead of bfs to fail earlier
* viz improvements
* add worklist size to viz
* add test case for private functions
* valueset analysis
* add more checks to fail earlier in the analysis to help debugging, improve debugging output, catch additional bad jumps
* delete old code
* delete old code
* delete old code
* fix up minor changes to jump table
* copy over comments from cgf-1 branch
* remove minor diffs
* add recompiled deposit contract
* graph viz
* cleanup/refactoring
* initial impl of viz
* script to run cfg anly and generate dot file
* div example
* accept bytecode from cmd line
* add minimal deposit contract example
* replace valueset analysis with stackset analysis
* get in sync with master
* sync with master
* fix linting
* fix linting
* fix linting
* reformatting
* fix linting
* fix linting
* fix linting
* fix linting
* fix linting
* fix linting
* fix linting
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
* v0
* more docs, less customization
* more docs, less customization
* more docs, less customization
* don't store db reference
* rely on fact that lmdb key/value valid until end of transaction
* clean
* clean
* share config object
* create default config and logger
* move db connection to common func
* move server start to cli package
* clear
* clear
* rename cli to rpc
* use unified SetupLogger func
* make all root flag persistent
* use common flags in different packages
* use common flags in different packages
* move TraceTx method to eth package
* use native slice flags
* create package "turbo"
* disable geth api
* disable geth api
* move more data types to turbo/adapter package
* add support for customApiList
* run more
* run more
* run more
* dog-food
* move DoCall
* move DoCall
* fix tests
* fix test
* Fix tx propagation and announcement tests
* Fix checkpoint challenge test
* Save current state of code
* Fix compile errors
* Fixes
* Fix formatting
* Printouts
* Printouts
* Printouts
* Too much printing, not close db when stopping
* Better handing of interruptions
* No need to reset after incremental interruption
* Fix stages
* Fixes
* Giving up
* Fix formatting
* Cleanup
* close goruotines
* fix using defer
* close pm managers a bit later
* fmt
* finalizer for in-mem db
* after cr
* revert finalizer
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
```
go run ./cmd/integration reset --chaindata=...
go run ./cmd/integration state_stages -h
go run ./cmd/integration state_stages --chaindata=... --verbosity=3 --block=2_000_000 --unwind=10 --unwind_every=1_000
```
Also, it inherits flags from geth:
```
--pprof.cpuprofile=./cpu.out // to file
--pprof --pprof.port=6060 // launch pprof server
--metrics // sends to prometheus
```
* fix case: if header stage is ehead of body stage
* fix case: if header stage is ehead of body stage
* fix case: if header stage is ehead of body stage
* allow stage3 to sync to given block
* allow stage3 to sync to given block
* TestWatchNoDir not to be parallel
* Print ks and inc
* Print addrHash
* Change the buffer
* Print loading
* Skip
* More logging
* Error out earlier
* Handle empty codes
* Handle empty codes
* Remove logging
* Compare states
* Not do stage5
* compareBucket
* Preimage
* Clearer errors
* No need to clean up contract code
* Restore stage6
* Printing
* Skip the skipping
* Print all
* Change buffer type
* Add limit to stage5
* Always fail
* Remove exessive logging
* Restore buffer type
* Revert
* Print when exception
* Reenable skipping
* Skip storage items for deleted contracts
* not shortcut
* Remove removeAccount
* Re-enable state hashing
* Default to plain state
* Disable hashing state
* Reenable reset5
* Print unfurl list
* Enable removingAccount
* No printing
* Reenable stage5 commit
* Swap order of stages
* Prevent backwards promotion, reset tx lookup
* reset finish
* Introduce storage item replacement
* See if unwind works
* Restore removingAccount
* Don't do removeAccount for unwinding
* Possible fix
* Proper(er) fix
* Don't exclude unwinding
* Remove unwinding flag
* Fix formatting
* Fix lint
* Not to ignore blocks if they cause reorg
* Fix test, separate stages again
* Fix TestUnwind
* Fix stage
* Swap unwinding
* Revert to unwinding flag
* Print unfurl list
* Print
* Print inside receive
* Print after
* No printing
* Cleanup
* Not use blockCache when doing GetBlock
* Handle bucket error
* Replace with 0
* SetMaxDBs
* Set MaxDb before opening'
* Merge stage5 and stage6
* Fix lint
* Make downloader tests not parallel
* remove unused slice from MultiPut
* mutation: reuse tuples slice and preallocate bucketPuts
* use bucketPool in kv_lmdb
* remove duplicated check of context status
* more benchmarks
* remove reusage of puts
* remove ctx from MustOpen
* remove ctx from Open. Stop goroutines on Close.
* remove ctx from Open. Stop goroutines on Close.
* remove ctx from remote open (we have DialTimeout field to manage connection timeouts)
* enable RawReads and add native implementation of Get/Has methods
* save state
* txlookup full results
* save state
* save state
* remove experiments
* some fix&lint
* add end key to txLookup and index generation
* change log message
* change log
* fix lint
* lint
* fix test
* Euphemerally -> Ephemerally
* Move StorageMode to ethdb and pass it to PrepareStagedSync
* linter
* Remove StorageModeThinHistory and move SetStorageModeIfNotExist into storage_mode.go
* Optionally write receipts in the execute stage
* memory profiler
* linter
* proper linter fix
* linter
* typo
* Merge stateDb with changeDb so that all-or-nothing is commited in stage_execute
use `s.UpdateWithStageData(db, <block number>, <key>)` to store the key
use `s.StageData` with `etl.NextKey` to restart ETL from where it was interrupted.
* Start from 9m7
* Regenerate IH + receipts
* Only stats for iH bucket
* Persist receipts
* Go all in
* Start from block 10m
* Convert DbState to use plain state
* Fix findHistory
* Hard-code export
* More fixes
* Fix test
* Fix formatting
* Introduce PlainDbState
* Actually return PlainDbState
* Fix formatting
* Fix name style lint
* Fix linters
* Fix history_test
* Fix blockchain_test
* Fix compile error
* Bucket stats from all buckets
* Lmdb internal objects reuse (built-in feature of lmdb):
- lmdb read transactions pool
- lmdb also support cursors reuse, but not implemented in this PR
And kv abstraction objects reuse:
- lmdbKV pool of all tx objects
- boltKV pool of all tx objects
- badgerKV pool of all tx objects
* switch makefile back to bolt
* switch makefile back to bolt
* lmdb test run
* switch makefile back to bolt
* cursors pool
* run lmdb tests
* make kv objects pool global
* switch makefile back to bolt
* remove badgers GOMAXPROC setup, because our app tunned for sequential read/writes, not for random throughput
* simplify code
* Query progress
* Run stage4 offline
* More thorough resetState
* Correct BlockNumber
* Fix formatting
* State loop
* do every 200k blocks
* Shift to 6.6m
* Close dbs in tests
* Stage2 with option of no reset
* every 100k blocks
* Reset state before stage5
* Introduce another stage
* Check compile errors
* Fix linter
* Fix linter
* Disable unreliable test
* Fix test
* Remove unreachable code
