* dirty trie with direct reading of account/storage data from state
run with fixes
implemented trie with direct reading from state
* cleaner version without updates
* Switch back to MDBX
* Fix test
* No restore
* Fix overwrite
* Fix change file opening
* Verify state vs change files
* Add WriteMap
* Print
* Not to go to the end of the file when reading
* Fix rebuild
* prefixLen
* Print
* Print
* Print
* Fix for rebuild
* Remove prints
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
* Split handover time
* Clean up
* Correctly compute handover time
* not use compression when aggregating
* Fix for etl collector
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Elias fano search and merge
* Add first cut of search
* Iterator and test
* Changes in aggregator
* Elias fano bitmap
* Fix uncompress decompress
* Print
* Print
* No print
* Print
* Print
* Print
* Change to AppendBytes
* Print
* Fix NextUncompressed
* Remove print
* Fix history search
* Fix in history search
* More tracing
* More tracing
* Fix
* Print
* Print key
* More print
* Print
* No deletion for history records
* Remove print
* Fix
* Fix
* Fix test
* Fix lint
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Intermediate work
* Allow uncompressed words
* Fix
* Fix tests
* Add NextUncompressed, remove g.word buffer
* Code simplifications, no goroutines when workers == 1
* Fix lint|
* Add test for MatchPrefix
* Work on patricia
* Beginning of new matcher
* Fuzz test for new longest match
* No skip
* Fixes
* Fixes
* More tracing
* Fixes
* Fixes
* Change back to old FindLongestMatches
* Switch to old match finder
* Print mismatches
* Fix
* After fix
* After fix
* After fix
* Print pointers
* Fixes and tests
* Print
* Print
* Print
* More tests
* Intermediate
* Fix
* Fix
* Prints
* Fix
* Fix
* Initialise matchStack
* Compute only once
* Compute only once
* Switch back
* Switch to old Find
* Introduce sais
* Switch patricia to sais
* Use sais in compressor
* Use sais in compressor
* Remove unused code
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
* Optimisations in etl collector and compressor
* Not copy k and v in the collector
* Fix lint
* Optimisations
* Change Load1 back to Load
* Reduce allocations for tests
* preallocate inv
* counting hits and misses
* Try to fix
* Try to fix
* Relaxation 1
* Relaxation 2
* Add arch tables
* Fix
* Update arch tables and use them
* Not to override larger value
* Increase arch table size
* Increase arch table size
* Fixes to arch
* Print
* Off by one
* Print
* Fix
* Remove print
* Perform update of arch in the background
* Build up huffman tree
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* More tracing for history
* More tracing for history
* More tracing for history
* only under tracing
* Trace
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Another fix for history files
* Half way through
* Another fix
* Correct closing sequence
* Remove first byte insert marker
* More on think commitments
* Fixes
* Fixes
* Print
* Skip touchMap
* Merge branchData from trees and from files
* Fill branch commitment
* Fill branch commitment
* Print
* Fix?
* Merge branchData when updating in the tree
* Better panic
* Prints
* Prints
* Prints
* Create complete branch data if it did not exist before
* Cleanup printing
* Fix merge
* Fix merge use
* Fix transform
* Better startBlock panic
* Preserve touchMap
* Merge commitments during aggregation
* Merge commitments during aggregation
* Merge commitments during aggregation
* Merge commitments during aggregation
* Merge commitments
* Merge commitments
* Merge commitments
* Merge commitments
* Merge commitments
* Merge commitments
* Merge commitments
* Include fieldbits during transform
* Fix history reads
* Print
* Print
* Prints
* Prints
* Prints
* Prints
* Prints
* Prints
* Prints
* Prints
* Prints
* Prints
* Prints
* Print largestMerge, lock files in branchFn
* Add storage lock
* Prints
* prefixLen fix
* prefixLen fix
* Fixes
* Remove print
* Remove print
* Set changesets and commitments flags upfront
* Logging instead of printing
* Fix history merge, recsplit panic
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
* Initial code for history reader
* Print
* Print
* Fix npe
* Open files of all types
* don't close file in the constructor
* Print foundTxNum
* Print foundTxNum
* Print adding to bitmap
* Print when reducing
* Print when reducing
* Print when reducing
* Enum false for index
* Remove prints
* Remove prints
* Print loaded files
* Tracing
* Tracing
* Tracing
* Tracing
* Tracing
* Optinal commitments
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* corretly shutdown history goroutine
* Different final merge for history files
* Skip value
* Bitmap production bug
* Bitmap production bug
* Bitmap production bug
* Bitmap production bug
* Bitmap production bug
* Bitmap production bug
* Bitmap production bug
* Bitmap production bug
* Handle collision
* Handle collision
* Debug
* Debug
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
* Aggregator refactoring
* Restore the test
* Simplify commitment
* Lint
* onBefore
* Wip smaller commitments
* Fixes
* Fix lint
* Check for zero len
* Print
* Remove restriction
* Correct restrictions
* Correct restrictions
* More merging
* delBitmap
* Switch off merging
* Reintroduce background merge
* Try to fix
* Slight optimisation
* Produce initial changeset files
* Print changeset keys
* Print changeset keys
* Print changeset keys
* Print changeset keys
* Print changeset keys
* Merge for history and bitmaps
* Merge for history and bitmaps
* Merge for history and bitmaps
* Merge for history and bitmaps
* Merge for history and bitmaps
* Merge for history and bitmaps
* Merge for history and bitmaps
* Merge for history and bitmaps
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
* Try to fix deadlock
* Another lock
* Try to optimise
* comments and cleanup
* Prevent deadlock on panic
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Separate state file btrees, fix Match in the decompressor
* fix match
* Fix to match
* Switch back from Match
* Try to use match, close indices
* Fixing Match
* Use Skip
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* Remove warning message
* Calculate sizes of change sets for transactions
* Calculate sizes of change sets for transactions
* Produce changeset files
* Not to produce changesets for commitments
* Remove rewindTx
* Close decompressor
* Not generate changesets by default
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
* Introduce commitment files and table
* Introduce commitment files and table
* Introduce words buffer
* Move changes out of aggregator into the writer
* Start calc commitment
* Add commitment package
* More on hex patricia
* More
* account decorator
* More
* Add test
* More
* More to the test
* Use hex in tests
* More
* More
* More
* More
* Simplified cells
* Add delBitmap
* More advanced MockState
* More compact unfolding
* Separation of hashed Keys
* Sepatate downHashedKey and upHashedKey
* Carry extension node through accounts
* optimised row allocations
* Fix encoding/decoding, add trace flag
* Added account and storage into the cells
* Make accountKeyLen global setting
* Remove BranchNodeUpdaqte intermediate
* Started on computeHash
* Initial hash calculations
* Fix lint
* Change account encoding
* Fix commitment issues
* Fix lint
* Fix lint
* Fix lint
* Trace
* Small fix and tracing
* Print branch hashes, fix empty
* Fix
* Fix
* Fix
* Fix
* Print accountFn
* Trigger accountFn
* Trigger accountFn
* Another fix for extension nodes
* Return root hash and set trace
* Print depth for computeCellHash
* Fix for storage leaves
* Fix for storage leaves
* Fix for storage leaves
* Fix for storage leaves
* Fix for storage leaves
* Fix for storage leaves
* Ineffectual deletes
* Trace needUnfolding
* Fix
* Fix
* Fix
* trace deletes
* trace deletes
* trace deletes
* trace deletes
* trace deletes
* not overwrite downHashedKey in updateAccount
* simplify needUnfolding
* simplify needUnfolding
* simplify needUnfolding
* simplify needUnfolding
* simplify needUnfolding
* propagate deletes when unfolding
* remove deletes when folding
* remove deletes when folding
* remove upHashedKey when delete cell
* remove upHashedKey when delete cell
* aggregate storage changes later
* remove del bit when updating account
* remove del bit when updating account
* remove del bit when updating account
* perform delete after code update
* perform delete after code update
* perform delete after code update
* perform delete after code update
* perform delete after code update
* perform delete after code update
* latest update has precedence when aggregating
* Revert "latest update has precedence when aggregating"
This reverts commit 0234ea6c3e1a0b2e7599df7c470d4b8b8d56736a.
* latest update has precedence when aggregating
* Revert "latest update has precedence when aggregating"
This reverts commit cfa7b75327e4d5d1236f905c3c04eb640b21838d.
* Introduce FinishTx
* Separate commitment separation from computation
* Prevent spurious delete
* Spurious deletes
* Trace param
* Preserve upHashedKey when unfolding
* Fix
* Fix
* Correctly compute firstInsert
* Try to fix delete + fold
* Remove commented out
* updateStorage to remove deleted flag
* rename upHashedKey to extension, fix clear-up
* Carry extension up if account plain key is present
* Carry extension up if account plain key is present
* Not to fail for deleteAccount
* Not to fail for deleteAccount
* Not to fail for deleteAccount
* Print
* Print
* Print
* Print
* Print
* Print
* Print
* Print
* Copy keys before putting them into commTree
* Preserve changes to AccountData
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
