Commit Graph

62 Commits

Author SHA1 Message Date
Alex Sharov
11ab5bdbb8
Parallel compressor - allow empty words (#245)
* save

* save

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-18 13:57:35 +00:00
Alex Sharov
0f80e9941f
Switch to parallel compressor (#244) 2022-01-18 12:55:20 +07:00
ledgerwatch
0eaa0333e1
[erigon2] Optional writing of before changes, commitment decoupled from blocks (#237)
* Separate commitment from blocks

* Optional beforeOn

* Optional beforeOn

* Optional beforeOn

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-16 20:27:44 +00:00
ledgerwatch
e5c07ec901
[erigon2] Resumable prototype (#236)
* Not warn about files that don't match at all

* Not warn about files that don't match at all

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Cleanup

* Cleanup

* Cleanup

* Cleanup

* Cleanup

* Cleanup

* Cleanup

* Cleanup

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-15 22:09:06 +00:00
ledgerwatch
c08ed12256
[erigon2] Measure change set sizes with perfect hash table (#230)
* Remove warning message

* Calculate sizes of change sets for transactions

* Calculate sizes of change sets for transactions

* Produce changeset files

* Not to produce changesets for commitments

* Remove rewindTx

* Close decompressor

* Not generate changesets by default

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-14 14:10:10 +00:00
ledgerwatch
ae6f7202d2
[erigon2] Introduce commitment files and table (#201)
* Introduce commitment files and table

* Introduce commitment files and table

* Introduce words buffer

* Move changes out of aggregator into the writer

* Start calc commitment

* Add commitment package

* More on hex patricia

* More

* account decorator

* More

* Add test

* More

* More to the test

* Use hex in tests

* More

* More

* More

* More

* Simplified cells

* Add delBitmap

* More advanced MockState

* More compact unfolding

* Separation of hashed Keys

* Sepatate downHashedKey and upHashedKey

* Carry extension node through accounts

* optimised row allocations

* Fix encoding/decoding, add trace flag

* Added account and storage into the cells

* Make accountKeyLen global setting

* Remove BranchNodeUpdaqte intermediate

* Started on computeHash

* Initial hash calculations

* Fix lint

* Change account encoding

* Fix commitment issues

* Fix lint

* Fix lint

* Fix lint

* Trace

* Small fix and tracing

* Print branch hashes, fix empty

* Fix

* Fix

* Fix

* Fix

* Print accountFn

* Trigger accountFn

* Trigger accountFn

* Another fix for extension nodes

* Return root hash and set trace

* Print depth for computeCellHash

* Fix for storage leaves

* Fix for storage leaves

* Fix for storage leaves

* Fix for storage leaves

* Fix for storage leaves

* Fix for storage leaves

* Ineffectual deletes

* Trace needUnfolding

* Fix

* Fix

* Fix

* trace deletes

* trace deletes

* trace deletes

* trace deletes

* trace deletes

* not overwrite downHashedKey in updateAccount

* simplify needUnfolding

* simplify needUnfolding

* simplify needUnfolding

* simplify needUnfolding

* simplify needUnfolding

* propagate deletes when unfolding

* remove deletes when folding

* remove deletes when folding

* remove upHashedKey when delete cell

* remove upHashedKey when delete cell

* aggregate storage changes later

* remove del bit when updating account

* remove del bit when updating account

* remove del bit when updating account

* perform delete after code update

* perform delete after code update

* perform delete after code update

* perform delete after code update

* perform delete after code update

* perform delete after code update

* latest update has precedence when aggregating

* Revert "latest update has precedence when aggregating"

This reverts commit 0234ea6c3e1a0b2e7599df7c470d4b8b8d56736a.

* latest update has precedence when aggregating

* Revert "latest update has precedence when aggregating"

This reverts commit cfa7b75327e4d5d1236f905c3c04eb640b21838d.

* Introduce FinishTx

* Separate commitment separation from computation

* Prevent spurious delete

* Spurious deletes

* Trace param

* Preserve upHashedKey when unfolding

* Fix

* Fix

* Correctly compute firstInsert

* Try to fix delete + fold

* Remove commented out

* updateStorage to remove deleted flag

* rename upHashedKey to extension, fix clear-up

* Carry extension up if account plain key is present

* Carry extension up if account plain key is present

* Not to fail for deleteAccount

* Not to fail for deleteAccount

* Not to fail for deleteAccount

* Print

* Print

* Print

* Print

* Print

* Print

* Print

* Print

* Copy keys before putting them into commTree

* Preserve changes to AccountData

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-12 11:21:52 +00:00
ledgerwatch
e83c9f276a
Tracing in the aggregator (#194)
* Tracing in the aggregator

* Trace merge of keys

* print value when merging

* More tracing

* Fix trace keys

* Fix tracing

* Fix skipping

* Fix tracing for deleteAccount

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-11-27 08:34:56 +00:00
ledgerwatch
d5f6afc92d
Delete code with account, add code test (#171)
* Delete code with account, add code test

* Fix test

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-11-17 15:49:48 +00:00
ledgerwatch
14b6423388
[state aggregator] contract recreation storage test (#169)
* Remove incarnation, add recreation

* Fix recreation test

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-11-16 21:59:26 +00:00
ledgerwatch
ea56078a4c
state aggregator - tests for contract storage (#165)
* First test for storage

* Debugging

* Fixes to delete account test

* Fix the lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-11-16 00:11:40 +00:00
Alex Sharov
5b7f67deae
Snapshot naming (#163)
* save

* save

* save

* save

* save

* save
2021-11-15 14:19:56 +00:00
ledgerwatch
fd19ad8148
State aggregator (#114)
* State aggregator

* Compile fix

* More

* Add

* More

* More on aggregator

* Writes (still incorrect)

* Move table names

* More

* Start of aggregation

* Change files instead of db

* More on change files

* More

* More

* Dealing with state and change files

* More

* More

* More boilerplate

* More

* More

* Iteration over storage

* More boilerplate

* More fixes

* Insert flag

* More

* Unit test

* Add more to the test

* Expand the test a bit

* More testing

* Keep fixing the test

* More fixes to the test

* Clean up DB tables upon aggregation

* More fixes

* Remove update/insert indicator from returned values

* Add assertions

* close files before deleting

* close files before deleting

* close files before deleting

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-11-13 12:12:29 +00:00