Alex Sharov
b3e8a1a02a
hack: remove createIdx and txLookup methods ( #3000 )
2021-11-21 12:06:36 +07:00
Alex Sharov
e55256296b
snapshots: read block from snapshots, add sender to txs file ( #2996 )
2021-11-21 10:32:14 +07:00
Alex Sharov
d4850b6adc
Hack: dump bodies and headers ( #2994 )
2021-11-20 09:41:01 +07:00
Alex Sharov
2a08dbd3d9
Recsplit: move files read/write helpers to erigon-lib ( #2993 )
2021-11-19 12:12:25 +07:00
alex.sharov
c1cbb21f3d
save
2021-11-11 10:39:44 +07:00
Alex Sharov
f31a76baea
Compress: add maxPatternLen=64 ( #2943 )
2021-11-10 22:10:01 +07:00
Alex Sharov
e670f3930f
Lcp must be int32 ( #2934 )
2021-11-09 17:47:12 +07:00
Alex Sharov
3842351293
move DictAggregator to erigon-lib ( #2933 )
2021-11-09 10:12:25 +07:00
Alex Sharov
deed48c01a
Small step towards torrent downloader ( #2929 )
2021-11-08 20:40:56 +07:00
Alex Sharov
9ab2bbee1b
recsplit: single offset bucket ( #2928 )
2021-11-08 14:27:47 +07:00
Alex Sharov
194330fa4b
Recsplit: preserve immutability threshold ( #2926 )
2021-11-07 13:01:33 +07:00
Alex Sharov
49ff8ff166
Recsplit: collision typed error ( #2925 )
2021-11-07 09:58:20 +07:00
alex.sharov
e802e43fa8
add pprof http to hack
2021-11-07 09:16:25 +07:00
Alex Sharov
a5bbe82a59
ETL: use logPrefix as suffix of tmp files ( #2921 )
2021-11-05 17:19:44 +07:00
Alex Sharov
686721b6c6
EliasFano: fix jump calculation, fuzzing to trigger jump logic ( #2912 )
2021-11-04 13:25:34 +07:00
Alex Sharov
70461e4e68
Recsplit: add first byte of hash to txn payload ( #2902 )
2021-11-02 14:35:09 +07:00
Alex Sharov
6f055e7adc
Recsplit: call ef.Build and set ef.prevOffset ( #2897 )
2021-11-01 09:24:06 +07:00
Alex Sharov
3d63c93b3d
Recsplit: add hack recsplitLookup for test ( #2895 )
2021-10-31 10:20:31 +07:00
ledgerwatch
04f7fe468a
hack to print headers ( #2892 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-10-30 12:27:30 +01:00
Alex Sharov
83593c28c2
recsplit: bigger bufio buffer ( #2867 )
2021-10-26 11:19:56 +07:00
Alex Sharov
5d7904c133
erl.collector - move logPrefix to constructor ( #2866 )
2021-10-25 15:09:43 +07:00
ledgerwatch
2dec76123c
Use decompressor from erigon-lib and experiments with tx lookup ( #2824 )
...
* Use decompressor
* Use decompressor
* Close compressor
* Upgrade to latest erigon-lib
* Fix compile errors
* Fix lint
* Remove count from files
* Remove count from files
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-10-16 22:19:13 +01:00
ledgerwatch
21d024b06a
More mature experiments with the state and transaction compression ( #2811 )
...
* Change to use Patten inside patricia tree
* Huffman codes for compression
* Huffman coding of positions, generate tx files to test compression
* Split state into 3 files, relative positions, average decoding time
* Small optimisations of decompression
* Optimise intermediate compression
* Update erigon-lib ref
* Fix lint
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-10-11 21:05:45 +01:00
ledgerwatch
7e4996bb09
Reworkings of state compression experiments ( #2790 )
...
* Changes
* Progress
* Another way
* More
* More
* Produce encoding
* Add uncoded characters
* cleanup
* Add sortdict
* Fixes
* Use patricia from erigon-lib
* Cleanup
* Switch to dynamic programming, optimise allocations in FindMatches
* Optimise allocations
* Reduce allocations
* Switch to main branch of erigon-lib, reduce allocations further
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-10-06 19:58:12 +01:00
Andrea Lanfranchi
c913f35c2e
Inner errors ( #2774 )
2021-10-04 22:16:52 +07:00
ledgerwatch
0d34a5b9c5
[State sync experiment] Building dictionary for state file compression ( #2721 )
...
* Kasai experiments
* Use divsufsort instead of standard lib
* Refactor experiments, split dictionary building into chunks
* Fixes
* Estimate compression, sample
* More stats
* Parallelise
* Fix lint
* dictionary aggregator
* Actual replacement
* Fixes, separate dictionary processing
* Test file
* More correct dictionary, decompression
* Use dynamic programming to reduce dictionary
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-09-27 21:32:53 +01:00
Alex Sharov
3de50637cd
remove changeset.Walk func ( #2716 )
2021-09-22 07:54:29 +07:00
ledgerwatch
a9e4cbc83e
Recsplit experiments ( #2706 )
...
* Only do 2m
* Optimisation
* Flexible count
* Count properly
* count
* Bijection check
* Add keys
* Print bucket info
* Collision map
* Collision map
* Add tracing
* Tracing
* Fix split
* No trace, print lookup time
* Not override count if block=0
* fix count
* Go mod tidy
* Fix lint
* Measurements
* Cleanup
* Latest erigon-lib
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-09-21 07:53:22 +01:00
Alex Sharp
c519b04551
Revert "state mphf experiment"
...
This reverts commit b0182d7834
2021-09-19 11:41:51 +01:00
Alex Sharp
b0182d7834
state mphf experiment
2021-09-19 11:40:23 +01:00
Alex Sharov
8d9e058eb7
Pool: chain config, non-mainnet ( #2702 )
2021-09-18 20:58:23 +07:00
Alex Sharov
3193f95152
Apply mined block ( #2650 )
2021-09-09 14:26:58 +07:00
Alex Sharov
c7c3a5932a
rename ReadBody to ReadBodyWithTransactions to make it obvious that it's heavier then just ReadBody ( #2534 )
...
* rename ReadBody to ReadBodyWithTransactions
* rename ReadBody to ReadBodyWithTransactions
2021-08-15 17:08:28 +07:00
Evgeny Danilenko
4cd72c8328
Keep readonly value while changing interpreters back and forth ( #2508 )
...
* restore TEVM
* fuzzing and property based
* comment
* lint
* stack callback into defer ater checking an error
* sequential tests
2021-08-10 09:48:56 +07:00
ledgerwatch
350d7fe7a9
Adding support for vmTrace into trace_ routines ( #2497 )
...
* VmTrace
* Fixes to gasCall, beginning of vmTrace
* Fix opcode tracer
* Add mem and store, enable vmTrace for all methods
* Fix lint
* More nuances and debugging|
* More fixes
* Fix for trace_callMany
* Fix for trace_callMany
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-08-08 13:28:03 +01:00
Alex Sharov
6c2beb273e
Remove tsdb dependency ( #2503 )
...
* remove tsdb dep
* remove tsdb dep
* remove tsdb dep
* remove tsdb dep
2021-08-08 16:56:54 +07:00
Alex Sharov
6bd44eb26c
move kv to erigon-lib ( #2467 )
2021-07-29 18:53:13 +07:00
Alex Sharov
5069558752
Apache licensed logger ( #2460 )
2021-07-29 17:23:23 +07:00
Alex Sharov
21cb7befa4
finish remove bucket suffix ( #2458 )
2021-07-28 10:43:51 +07:00
Alex Sharov
838e5f9ef2
Move bucket constants into kv package, move kv interface to kv package ( #2455 )
2021-07-28 09:47:38 +07:00
Alex Sharov
63a6c45ef1
Remove some db interfaces ( #2452 )
2021-07-27 19:35:32 +07:00
Alex Sharov
6801208497
move some tests to kv interface ( #2438 )
2021-07-24 14:14:11 +07:00
Alex Sharov
f735997fa1
Sokol v0: remove unused parameters and methods ( #2435 )
2021-07-24 11:04:07 +07:00
Alex Sharov
d2552196ad
Sokol v0: support for first epoch-set transition ( #2411 )
2021-07-21 18:13:26 +07:00
Alex Sharov
c3e1cfdac8
Pruning for: exec, log_index, tx_lookup, history stages ( #2399 )
...
* Pruning for: exec, log_index, tx_lookup, history stages
* Pruning for: exec, log_index, tx_lookup, history stages
* Pruning for: exec, log_index, tx_lookup, history stages
* Pruning for: exec, log_index, tx_lookup, history stages
* add tvm flag
* save
* db migration for storage mode
add flag --prune=
remove flag --storage-mode=
add flag --experiments=tevm,...
rename integration set_storage_mode to set_prune
* fix
* forward move of stages must skip everything before PruneTo
* keep in db progress of prune method
* keep in db progress of prune method
* simplify logs
* simplify logs
* simplify logs
* fix test
* simplify logs
* simplify logs
* simplify logs
* simplify logs
* remove callTraceSet as dupsort
use etl transform for txlookup prune
remove some logs
* cleanup tests a bit
* print_stages and eth_sync to show prune progress
* fix print_stages
* add readme about --prune.to flag
* more docs
* add --prune.history.older and other flags support
* fix migration on empty db
* better toString
* better toString
2021-07-20 21:03:19 +01:00
racytech
5e7bf5173d
Change freelist guide illustrations from lmdb to mdbx ( #2391 )
...
* lmdb to mdbx
* nolint added
* linting issues fixed
* linting issues fixed 2
* linting issues fixed 3
Co-authored-by: Kairat Abylkasymov <racytech0x0gmail.com>
2021-07-18 18:44:47 +01:00
Alex Sharov
4c53f51767
Remove "persistent unwind" concept ( #2388 )
2021-07-17 21:14:51 +07:00
Alex Sharov
a79b3cd6eb
unify stage params ( #2384 )
2021-07-17 09:09:56 +07:00
Alex Sharov
9f6ef74adc
Linter: to check tx.Rollback() by ruleguard ( #2383 )
2021-07-16 20:23:54 +07:00
Alex Sharov
5f55b6b89c
Remove stage builder and use ID's in unwindOrder ( #2381 )
2021-07-16 20:17:40 +07:00
