* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* squash
* squash
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* history_early_stop
* history_early_stop
* vmConfig with ReadOnly false
* auto_increment
* auto_increment
* rebase master
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
* accounts, signer: implement gnosis safe support
* common/math: add type for marshalling big to dec
* accounts, signer: properly sign gnosis requests
* signer, clef: implement account_signGnosisTx
* signer: fix auditlog print, change rpc-name (signGnosisTx to signGnosisSafeTx)
* signer: pass validation-messages/warnings to the UI for gnonsis-safe txs
* signer/core: minor change to validationmessages of typed data
* save progress of db migration
* save progress of db migration
* delete
* delete
* delete
* delete
* delete
* print full key if it has 8 leading zeroes
* print full key if it has 8 leading zeroes
* print full key if it has 8 leading zeroes
* save progress
* save progress
* save progress
* save progress
* etl bimaps
* etl bimaps
* etl bimaps
* etl bimaps
* a
* a
* a
* a
* a
* a
* a
* save progress
* save progress
* save progress
* a
* fix_race_condition_on_zstd_build
* clean
* clean
* clean
* clean
* clean
* go mod tidy
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* clean
* fix corner case
* commit every
* call traces etl
* call traces etl
* call traces etl
* return missed error
* go mod tidy
* go mod tidy
* go mod tidy
* rebase to master
* remove trash
* print full key if it has 8 leading zeroes
* Add print_migration command to integration
* Not run migrations for print commands
* Reverse migration
* Fix lint
* Add datadir to run_migrations
* Add lmdb.mapSize to integration
* Make lmdb.mapSize effective
* Fix
* receipt sizes
* fix print
* Switch to another version of lmdb
* Collector from existing files
* Not load
* Commit after clearing
* Add methods for clear/drop buckets by multiple transactions
* Fix to lmdb
* Not remove temp files
* Not load
* Re-enable loading
* Prints
* Try to fix refill
* Migration in 3 steps
* tidy mod
* Fix lint
* Fix tests
* Migrations for CBOR receipts in 3 txs
* Fix lint
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
* save state
* torrent experiments
* torrent experiment passed
* fixes after merge
* snapshot headers processing passed
* save state
* save state
* download headers works after snapshot processing
* save state
* save state
* save state
* save state
* add lazy load tx to snapshots, increase number of trackers
* save state
* speedup getting info
* change logging
* move to turbo package
* save state
* save state
* save state
* cleanup
* save state
* add test test
* save state
* lmdb debugging
* fix readonly mode
* save state
* fix build
* sync works
* save state
* save state
* save state
* allow cmd stages stageSenders use snapshots
* debugging failed hashing
* remove experimental tests
* remove torrent experimental tests
* fix lint
* extract snapshot wrapper
* metainfo checker
* add remote seeder
* add logs
* update gomod
* remove useless code
* fix lint&remove useless code
* extract verify snapshot to separated command
* skip debug test
* fix test
* change type of seedSnapshot flag
* add eth logger to torrent lib
* skip debug test
* add Close method
* review fixes
* fix lint
* tidy mods
* Fix compile
* Fix lint
* Fix rpcdaemon running in the docker
Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
* Initial commit for CallTraces index
* Fix compilation
* fix lint, add comment
* Fix integration
* Add Close function to ethdb.Cursor, fix some compile errors
* Try to stop cursor leak in Get
* Fix compile errors in RPC daemon
* Fix compile errors
* fixing another way
* Some fixes
* More fixes
* More fixes
* More fixes
* Fixes to core/state
* Fix lint
* Fix lint
* Fixes
* Stage caching for call trace stage
* Add mem stats
* Try to stop the leak
* Turn off debug
* Chunks for 10k blocks
* Print
* Revert "Print"
This reverts commit 5ffada4828d61e00e5dad1ca12c98258dfbbad00.
* Revert "Chunks for 10k blocks"
This reverts commit cfb9d498e782e5583d41c30abf0e2137da27383e.
* Trying to fix the leak
* Don't compute receipts in re-tracing
* Not compose block
* Print speed, fix receipts, bigger caches
* Fix lint
* Utilise changeset info
* Counters
* Use NoReceipts and ReadOnly
* ReadOnly is incompatible with caching
* Skip test leaking transactions
* Fix block test
* Change disable message for call-traces stage
* Use block option for call traces integration
* Fix retracing due to incarnation
* Added back fdlimit to increase number of file descriptors
* Fixing a test file that fails on Mac with too few file descriptors available
* Trying to fix ci
* Trying to fix ci
* Trying to fix ci 3
* Fixing ci
* save state
* torrent experiments
* torrent experiment passed
* fixes after merge
* snapshot headers processing passed
* save state
* save state
* download headers works after snapshot processing
* save state
* save state
* save state
* save state
* add lazy load tx to snapshots, increase number of trackers
* save state
* speedup getting info
* change logging
* move to turbo package
* save state
* save state
* save state
* cleanup
* save state
* add test test
* save state
* lmdb debugging
* fix readonly mode
* save state
* fix build
* sync works
* save state
* save state
* save state
* allow cmd stages stageSenders use snapshots
* debugging failed hashing
* remove experimental tests
* remove torrent experimental tests
* fix lint
* extract snapshot wrapper
* metainfo checker
* add remote seeder
* add logs
* update gomod
* remove useless code
* fix lint&remove useless code
* extract verify snapshot to separated command
* skip debug test
* fix test
* change type of seedSnapshot flag
* add eth logger to torrent lib
* skip debug test
* add Close method
* review fixes
* fix lint
Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
* save state
* torrent experiments
* torrent experiment passed
* fixes after merge
* snapshot headers processing passed
* save state
* save state
* download headers works after snapshot processing
* save state
* save state
* save state
* save state
* add lazy load tx to snapshots, increase number of trackers
* save state
* speedup getting info
* change logging
* move to turbo package
* save state
* save state
* save state
* cleanup
* save state
* add test test
* save state
* lmdb debugging
* fix readonly mode
* save state
* fix build
* sync works
* save state
* save state
* save state
* allow cmd stages stageSenders use snapshots
* debugging failed hashing
* remove experimental tests
* remove torrent experimental tests
* fix lint
* extract snapshot wrapper
* metainfo checker
* add remote seeder
* add logs
* update gomod
* remove useless code
* fix lint&remove useless code
* extract verify snapshot to separated command
* skip debug test
* fix test
* change type of seedSnapshot flag
* add eth logger to torrent lib
* skip debug test
* add Close method
* review fixes
* fix lint
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
* don't use global variables
* up deps
* clean
* use joined flags
* use single non-global buckets config
* clean
* copy map
* fix tests
* fix tests
* clean
* clean
* v0
* v1
* v3
* v4
* clean
* temporary fix of txpool
* Add debug logs about tx start/commit
* Add debug logs about tx start/commit
* Add debug logs about tx start/commit
* add condition
* tx pool to not hold own db
* try to enable TxPool in integration
* exclude tx pool from tx
* exclude tx pool from integration
* reduce limit
* fix integration
* clean
* clean
* clean
* clean
* clean
* exclude tx pool unwind
* exclude tx pool unwind in integration
* fix integration tx pool
* fix commit
* fix current stage after unwind
* fix current stage after unwind
* fix linter
* move unwind of tx_pool after unwind of unwind of senders, then all stages from body to tx_pool will be inside tx.
* move body and headers unwind out of tx
* fix unwind order after reboot
* add support external tx to exec stage
* clean
* clean
* clean
* clean
* clean
* add logs
* better id check
* better id check
* timer-based logs
* timer-based logs
* delegate progress calculation to user
* delegate progress calculation to user
* delegate progress calculation to user
* clear
* add logs to senders recovery
* use default dir in integration
* more logs
* more logs
* add logging to loader
* use pure tx in etl loading, logs in mutation commit
* clean
* bletter logging and more cleanup
* bletter logging and more cleanup
* increase batch size to 500M
* better batch commit logging
* async fsync
* sync fsync
* sync fsync
* unify logging
* fix corner-case when etl can use empty bucket name
* fix tests
* better logging
* better logging
* rebase master
* remove lmdb.NoMetaSync flag for now
* consistent walk and multi-walk
* clean
* sub tx
* add consistent multi-put
* implement dupsort support in one new cursor method
* clear
Reverting because this change started handling account balances as
uint64 in the transaction pool, which is incorrect.
This reverts commit af5c97aebe1d37486635521ef553cb8bd4bada13.
# Conflicts:
# core/types/transaction.go
* core: use uint64 for total tx costs instead of big.Int
* core: added local tx pool test case
* core, crypto: various allocation savings regarding tx handling
* Update core/tx_list.go
* core: added tx.GasPriceIntCmp for comparison without allocation
adds a method to remove unneeded allocation in comparison to tx.gasPrice
* core: handle pools full of locals better
* core/tests: benchmark for tx_list
* core/txlist, txpool: save a reheap operation, avoid some bigint allocs
Co-authored-by: Martin Holst Swende <martin@swende.se>
# Conflicts:
# common/math/integer.go
# core/tx_list.go
# core/tx_pool.go
# core/types/transaction.go
* return error from rawdb
* squash
* v14
* improve performance of put
* clean
* clean
* hide feature behind ENV variable
* cleanup
* cleanup
* disable ipc and make Readme less confusing (people thought points are depend on each-other)
* fix test
* cleanup
* cleanup
* limit incremental step size of stages 5 and 6
* remove limit, make incremental
* remove_load_start_key_from_etl
* switch to "struct of arrays" from "array of structs"
* switch to "struct of arrays" from "array of structs"
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
* save state
* walkAsOf by cs test passed
* cs search for plain state
* save state
* fix accounts tests
* refactor walkAsOf account tests
* fix storage test
* refactor walkAsOf storage tests
* fix lint
* fix test
* save state
* save state
* add test with fixed bits
* fmt
* add stages check
* fix lint
* fix lint
* remove obsoleted methods
```
go run ./cmd/integration reset --chaindata=...
go run ./cmd/integration state_stages -h
go run ./cmd/integration state_stages --chaindata=... --verbosity=3 --block=2_000_000 --unwind=10 --unwind_every=1_000
```
Also, it inherits flags from geth:
```
--pprof.cpuprofile=./cpu.out // to file
--pprof --pprof.port=6060 // launch pprof server
--metrics // sends to prometheus
```
* make lmdb default db
* switch nodes db to lmdb
* log errors in nodedb
* fix miss typing
* fix default test target in makefile
* report error of .Get method
* add debug info
* run tests with debug
* run tests with debug
* remove default db from TEST_DB env
* cleanup
* wait for goroutines finish
* wait for goroutines finish before close db
* wait for goroutines finish before close db
* wait for goroutines finish before close db
* wait for goroutines finish before close db
* save state
* txlookup full results
* save state
* save state
* remove experiments
* some fix&lint
* add end key to txLookup and index generation
* change log message
* change log
* fix lint
* lint
* fix test
* Euphemerally -> Ephemerally
* Move StorageMode to ethdb and pass it to PrepareStagedSync
* linter
* Remove StorageModeThinHistory and move SetStorageModeIfNotExist into storage_mode.go
* Optionally write receipts in the execute stage
* memory profiler
* linter
* proper linter fix
* linter
* typo
* Merge stateDb with changeDb so that all-or-nothing is commited in stage_execute
* Start from 9m7
* Regenerate IH + receipts
* Only stats for iH bucket
* Persist receipts
* Go all in
* Start from block 10m
* Convert DbState to use plain state
* Fix findHistory
* Hard-code export
* More fixes
* Fix test
* Fix formatting
* Introduce PlainDbState
* Actually return PlainDbState
* Fix formatting
* Fix name style lint
* Fix linters
* Fix history_test
* Fix blockchain_test
* Fix compile error
* Bucket stats from all buckets
* Lmdb internal objects reuse (built-in feature of lmdb):
- lmdb read transactions pool
- lmdb also support cursors reuse, but not implemented in this PR
And kv abstraction objects reuse:
- lmdbKV pool of all tx objects
- boltKV pool of all tx objects
- badgerKV pool of all tx objects
* switch makefile back to bolt
* switch makefile back to bolt
* lmdb test run
* switch makefile back to bolt
* cursors pool
* run lmdb tests
* make kv objects pool global
* switch makefile back to bolt
* remove badgers GOMAXPROC setup, because our app tunned for sequential read/writes, not for random throughput
* simplify code
