* add logging to loader
* use pure tx in etl loading, logs in mutation commit
* clean
* bletter logging and more cleanup
* bletter logging and more cleanup
* increase batch size to 500M
* better batch commit logging
* async fsync
* sync fsync
* sync fsync
* unify logging
* fix corner-case when etl can use empty bucket name
* fix tests
* better logging
* better logging
* rebase master
* remove lmdb.NoMetaSync flag for now
* consistent walk and multi-walk
* clean
* sub tx
* add consistent multi-put
* implement dupsort support in one new cursor method
* clear
* return error from rawdb
* squash
* v14
* improve performance of put
* clean
* clean
* hide feature behind ENV variable
* cleanup
* cleanup
* disable ipc and make Readme less confusing (people thought points are depend on each-other)
* fix test
* cleanup
* cleanup
* limit incremental step size of stages 5 and 6
* remove limit, make incremental
* remove_load_start_key_from_etl
* switch to "struct of arrays" from "array of structs"
* switch to "struct of arrays" from "array of structs"
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
```
go run ./cmd/integration reset --chaindata=...
go run ./cmd/integration state_stages -h
go run ./cmd/integration state_stages --chaindata=... --verbosity=3 --block=2_000_000 --unwind=10 --unwind_every=1_000
```
Also, it inherits flags from geth:
```
--pprof.cpuprofile=./cpu.out // to file
--pprof --pprof.port=6060 // launch pprof server
--metrics // sends to prometheus
```
* make lmdb default db
* switch nodes db to lmdb
* log errors in nodedb
* fix miss typing
* fix default test target in makefile
* report error of .Get method
* add debug info
* run tests with debug
* run tests with debug
* remove default db from TEST_DB env
* cleanup
* wait for goroutines finish
* wait for goroutines finish before close db
* wait for goroutines finish before close db
* wait for goroutines finish before close db
* wait for goroutines finish before close db
* avoid 0 length key/values and seek arguments
* less abstractions in .get method, and copy memory from C space without unsafe conversion into go memory
* fix test
* don't call initCursor on happy path
* don't call initCursor on happy path
* don't run stale reads goroutine for inMem mode
* don't call initCursor on happy path
* remove buffers from cursor object - they are useful only in Badger implementation
* commit kv benchmark
* remove buffers from cursor object - they are useful only in Badger implementation
* remove buffers from cursor object - they are useful only in Badger implementation
* cancel server before return pipe to pool
* try to fix test
* set field db in managed tx
* remove unused slice from MultiPut
* mutation: reuse tuples slice and preallocate bucketPuts
* use bucketPool in kv_lmdb
* remove duplicated check of context status
* more benchmarks
* remove reusage of puts
* remove ctx from MustOpen
* remove ctx from Open. Stop goroutines on Close.
* remove ctx from Open. Stop goroutines on Close.
* remove ctx from remote open (we have DialTimeout field to manage connection timeouts)
* enable RawReads and add native implementation of Get/Has methods
* Lmdb internal objects reuse (built-in feature of lmdb):
- lmdb read transactions pool
- lmdb also support cursors reuse, but not implemented in this PR
And kv abstraction objects reuse:
- lmdbKV pool of all tx objects
- boltKV pool of all tx objects
- badgerKV pool of all tx objects
* switch makefile back to bolt
* switch makefile back to bolt
* lmdb test run
* switch makefile back to bolt
* cursors pool
* run lmdb tests
* make kv objects pool global
* switch makefile back to bolt
* remove badgers GOMAXPROC setup, because our app tunned for sequential read/writes, not for random throughput
* simplify code
