Alex Sharov
62fe81e4be
IH stage speedup and lmdb custom comparators support ( #1080 )
...
* etl.Loader - allow use of custom comparator
* log timing
* try now
* try now
* more performance
* etl.Loader - allow use of custom comparator
* working version
* simplify IH cursor
* clean
* squash
* squash
* squash
* squash
* squash
* squash
* squash
* clean
* add only unwind support
* squash
* squash
* clean
* fix test
* clean
* clean
* clean
2020-09-10 13:35:58 +01:00
Alex Sharov
0e253e7336
lmdb transactions of unlimited size ( #918 )
...
* add logging to loader
* use pure tx in etl loading, logs in mutation commit
* clean
* bletter logging and more cleanup
* bletter logging and more cleanup
* increase batch size to 500M
* better batch commit logging
* async fsync
* sync fsync
* sync fsync
* unify logging
* fix corner-case when etl can use empty bucket name
* fix tests
* better logging
* better logging
* rebase master
* remove lmdb.NoMetaSync flag for now
* consistent walk and multi-walk
* clean
* sub tx
* add consistent multi-put
* implement dupsort support in one new cursor method
* clear
2020-08-17 07:45:52 +01:00
Alex Sharov
6043c09ddf
Replace global buff pool by local, because of buffer leaks to larger pools ( #739 )
2020-07-14 08:56:47 +07:00
ledgerwatch
d713de4713
Fix index generation for storage when incarnation changes ( #716 )
...
* Fixing history generation
* Try fixes
* Reset from 0
* Cleanup
2020-07-06 07:34:24 +01:00
ledgerwatch
b498419dcb
Fix loader crash (kludge) ( #706 )
...
* Kludge to remove duplicates
* Fix
* Fix formatting
* Fix in Len function
2020-07-03 08:08:35 +01:00
ledgerwatch
137daa6c67
Fixes in generated changeSet and checkChangeSets ( #698 )
...
* Debugging changesets
* Kinda works
* Fix compile error
* Fix formatting
* Fix lint
* Duplicate entries kludge
* Fix compile error
* Cleanup
2020-07-01 15:56:56 +01:00
Alex Sharov
b26b00f72b
increase IO buffers size ( #686 )
...
* increase IO buffers size
* found SSD threshold
2020-06-25 08:34:40 +01:00
ledgerwatch
fd98914c28
Stage6 - intermediate hashes ( #677 )
...
* First cut of the stage6
* Fix formatter
* Introduce state6
* Fix target number for stage6
* Fix linter
* Fix linter
* Reset in regenerate
* Correct block number
* Fix linter
* Fix linter
* Fix
* Upper case
* Skeleton to debug'
* Fix formatting
* Added codehash correction
* Fix TestUnwind
* fix test
* Fix linter
* Introduce unwind6
* Report adjustment error
* Code hashes included into stage5 incremental promotion and unwind
* Fix formatting
* Cleanup
* fix TestUnwind
* Cleanup
* Fix compile error
* Fix formatting
* unwind4
* Disable verifyRoot in stage5, fix hacks
* Remove verifyRoot function
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2020-06-18 22:27:11 +01:00
Igor Mandrigin
96750b3711
Support ETL interruptions while promoting plain state ( #662 )
...
* lil etl changez
* fix test compile
* fixups to the unwinds
* add new buffer mode
* support unfinished transitions
* fixups
* fix tests
* linters
* linters
2020-06-13 16:03:38 +01:00
b00ris
b4ba764fb1
[WIP] TxLookup stage ( #646 )
...
* save state
* txlookup full results
* save state
* save state
* remove experiments
* some fix&lint
* add end key to txLookup and index generation
* change log message
* change log
* fix lint
* lint
* fix test
2020-06-10 23:07:14 +03:00
