mirror/erigon-pulse
1
0
Fork 0
mirror of https://gitlab.com/pulsechaincom/erigon-pulse.git synced 2025-01-01 00:31:21 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
902 Commits 5 Branches 14 Tags 294 MiB
df49481ddc
Commit Graph

11 Commits

Author SHA1 Message Date
ledgerwatch
df49481ddc
[erigon 2.2] Make keys always uncompressed, values compressed only for code (#492) 
* Reduce allocations in domain and aggregator

* Make keys always uncompressed, values compressed only for code

* Functions to remake index

* Fix index recreation

* Test for reindex, fix

* Use uncompress vals in history

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-06-17 12:39:49 +01:00
ledgerwatch
c71ac02a0f
[erigon2] Optimisations in etl collector and compressor (#339) 
* Optimisations in etl collector and compressor

* Not copy k and v in the collector

* Fix lint

* Optimisations

* Change Load1 back to Load

* Reduce allocations for tests

* preallocate inv

* counting hits and misses

* Try to fix

* Try to fix

* Relaxation 1

* Relaxation 2

* Add arch tables

* Fix

* Update arch tables and use them

* Not to override larger value

* Increase arch table size

* Increase arch table size

* Fixes to arch

* Print

* Off by one

* Print

* Fix

* Remove print

* Perform update of arch in the background

* Build up huffman tree

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-02-20 22:14:06 +00:00
ledgerwatch
4e8840256e
[erigon2] Use shorter references instead of full plain keys in the commitment files (#289) 
* Rearrange aggregations

* More rearranging before introducing 3 threads

* Background aggregation|

* Concurrency fixes

* Remove files under lock

* Better logging

* Remove files without lock

* Fix lint

* Fix locking

* Try

* Fix background Merge

* Log merging

* Log merging

* Less logging

* Millisecond

* Add Stats function

* Log merge only after 1m

* Wrong counting

* plain key extract and replace functions

* Insert valTransform function

* Not parse first byte

* Not parse first byte

* Fix lint

* Switch to thin state references

* Fix lint

* Fix lint

* Debug print|

* Fix decoding

* Turn off valTransform

* Not to reuse transformer

* Print

* Print

* Print

* Derive hashed keys later

* Fix

* Fix log

* Fix

* Debug

* Another fix

* Fix

* Fix

* Print

* Print

* Data race

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-01-31 22:32:00 +00:00
primal_concrete_sledge
d8a33270e8
issue/issue-249-add_index_reader (#273) 
* issue/issue-249-add_index_reader

* Add licence
 2022-01-24 20:39:04 +00:00
ledgerwatch
e5c07ec901
[erigon2] Resumable prototype (#236) 
* Not warn about files that don't match at all

* Not warn about files that don't match at all

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Cleanup

* Cleanup

* Cleanup

* Cleanup

* Cleanup

* Cleanup

* Cleanup

* Cleanup

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-01-15 22:09:06 +00:00
Alex Sharov
bb3f510d16
RecSplit: store BaseDataID in .idx file (helps to navigate over non blockNum-based entries) (#180) 
* save

* save

* save

* save
 2021-11-21 14:52:23 +00:00
Alex Sharov
5b7f67deae
Snapshot naming (#163) 
* save

* save

* save

* save

* save

* save
 2021-11-15 14:19:56 +00:00
ledgerwatch
fd19ad8148
State aggregator (#114) 
* State aggregator

* Compile fix

* More

* Add

* More

* More on aggregator

* Writes (still incorrect)

* Move table names

* More

* Start of aggregation

* Change files instead of db

* More on change files

* More

* More

* Dealing with state and change files

* More

* More

* More boilerplate

* More

* More

* Iteration over storage

* More boilerplate

* More fixes

* Insert flag

* More

* Unit test

* Add more to the test

* Expand the test a bit

* More testing

* Keep fixing the test

* More fixes to the test

* Clean up DB tables upon aggregation

* More fixes

* Remove update/insert indicator from returned values

* Add assertions

* close files before deleting

* close files before deleting

* close files before deleting

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2021-11-13 12:12:29 +00:00
Alex Sharov
3c86aa6290
ETL: use logPrefix as suffix of tmp files (#146) 2021-11-05 17:04:17 +07:00
Alex Sharov
8213f020f0
recsplit add MustOpen method (#138) 2021-10-31 09:38:10 +07:00
ledgerwatch
967937151d
Fixes for compress, decompressor, and tests (#110) 
* Fixes for compress, and first test

* Add decompressor and memory mapping

* Add decompressor and memory mapping

* Fix for windows

* Fix lint

* Fix compile for windows

* More on decompressor

* Fix lint

* Decompress

* Fix lint

* Use decompressor in tests, fixes

* Introduce Index for RecSplit

* Fix compilation on Windows

* close index file on failure

* Fixes to the tests

* Add single Elias Fano, fix recsplit fuzz test

* Fix elias fano

* Add two layer index

* Add two level index to the tests

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2021-10-16 10:43:41 +01:00