Alex Sharov
567d9ddfed
ParallelCompressor: Remove intermediate ETL collectors ( #302 )
2022-02-04 16:48:02 +07:00
alex.sharov
d4b9053aed
save
2022-02-04 11:20:02 +07:00
ledgerwatch
55080d5c01
Proper reset of decompressor getter ( #299 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-03 17:58:56 +00:00
ledgerwatch
11615db276
Fix memory leak in txpool ( #298 )
...
* Fix memory leak in txpool
* Cannot delete during iteration
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-03 17:52:20 +00:00
alex.sharov
d814adb13e
save
2022-02-03 16:28:43 +07:00
alex.sharov
02c407a14e
save
2022-02-03 16:28:32 +07:00
ledgerwatch
cbfd733672
[erigon2] optimisations ( #297 )
...
* Reuse getter and indexReader
* Properly reset Getter
* Use separate getter and indexReader for merge thread
* Remove DB
* Fixes
* Print
* Fixes
* Cleanup signatures
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-03 08:49:47 +00:00
Alex Sharov
ec354d1615
Fuzz fixes ( #295 )
...
* fuzz tests fixes
* fuzz tests fixes
2022-02-02 09:18:04 +00:00
Alex Sharov
a53f232322
pool: fix race in async broadcast ( #294 )
...
* pool: better async broadcast
* save
* save
2022-02-02 14:49:26 +07:00
Alex Sharov
74d49a9de1
non blocking broadcast of new txs ( #292 )
2022-02-01 12:16:43 +00:00
ledgerwatch
1cc07746a4
[erigon2] Try to fix deadlock ( #291 )
...
* Try to fix deadlock
* Another lock
* Try to optimise
* comments and cleanup
* Prevent deadlock on panic
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-01 10:28:05 +00:00
Alex Sharov
0feb7fd591
Decompressor.WithReadAhead ( #290 )
2022-02-01 11:19:11 +07:00
ledgerwatch
4e8840256e
[erigon2] Use shorter references instead of full plain keys in the commitment files ( #289 )
...
* Rearrange aggregations
* More rearranging before introducing 3 threads
* Background aggregation|
* Concurrency fixes
* Remove files under lock
* Better logging
* Remove files without lock
* Fix lint
* Fix locking
* Try
* Fix background Merge
* Log merging
* Log merging
* Less logging
* Millisecond
* Add Stats function
* Log merge only after 1m
* Wrong counting
* plain key extract and replace functions
* Insert valTransform function
* Not parse first byte
* Not parse first byte
* Fix lint
* Switch to thin state references
* Fix lint
* Fix lint
* Debug print|
* Fix decoding
* Turn off valTransform
* Not to reuse transformer
* Print
* Print
* Print
* Derive hashed keys later
* Fix
* Fix log
* Fix
* Debug
* Another fix
* Fix
* Fix
* Print
* Print
* Data race
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-31 22:32:00 +00:00
ledgerwatch
586ab3e6b3
Separate state btree files ( #287 )
...
* Separate state file btrees, fix Match in the decompressor
* fix match
* Fix to match
* Switch back from Match
* Try to use match, close indices
* Fixing Match
* Use Skip
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-29 11:12:38 +00:00
Enrique Jose Avila Asapche
a5c2d83108
Added bor tables ( #286 )
...
Co-authored-by: Krishna Upadhyaya <krishnau1604@gmail.com>
2022-01-28 21:34:03 +00:00
primal_concrete_sledge
24098d94d8
WIP: issue/issue-250-use_compressed_keys ( #285 )
...
* issue/issue-250-use_compressed_keys
* Step into the twilight zone
* fix zero byte
* Add zerolen check
* Remove branhc data check
* Check lengths
2022-01-28 19:30:32 +00:00
Alex Sharov
c8397a0433
Mdbx: more 4tb fixes ( #282 )
2022-01-27 23:10:30 +07:00
Alex Sharov
dfdf7c8a66
[wip] parallel compress: less read of dat file ( #284 )
...
* save
* save
* save
2022-01-27 17:13:26 +07:00
Alex Sharov
ec11eb3d91
parallel compressor: don't save dict ( #283 )
...
* save
* save
2022-01-27 12:54:38 +07:00
ledgerwatch
1d4022960a
Fix RLP inconsistency in txpool ( #280 )
...
* Fix RLP inconsistency in txpool
* Add debug
* hasEnvelope argument when parsing transaction
* Handle incorrect packets without panic
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-26 16:09:02 +00:00
ledgerwatch
df95313a31
Add blockGasLimit to the state stream ( #261 )
...
* Remove interfaces for replacement
* Squashed 'interfaces/' content from commit eff9f8f
git-subtree-dir: interfaces
git-subtree-split: eff9f8f7273dda7c78575f2474e3103da61825db
* Add blockGasLimit to the state stream
* Pass blockGasLimit into txpool
* Add NotTooMuchGas bit, refactored promote
* Replace effectiveTip with minTip in the queues and slices
* Fix lint
* Refactor
* Improve terminology of ordering function
* Optimise baseFee change
* cleanup
* Safe operations
* Fix UnsafeAdd
* also print pendingBaseFee
* trace txpool_content
* trace txpool_content
* user sender ID inverted index to speed up txpool_content
* copy rlp from Db
* remove debug printing
* change from string to []byte
* safe storage of sender in the map
* Fix compile
* safe storage of sender in the map
* Print errors
* Prints
* Prints
* Prints
* Change worst function
* Change worst function
* Change worst function
* Change worst function
* Fix minTip
* Fix minTip
* Warn on db error, but not stop
* Fix interface
* More prints
* Cleanup
* Cleanup
* Differentiate baseFee pool
* Bump version of remove KV interface
* Not print every block
* Not print every block
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-25 10:20:11 +00:00
Alex Sharov
5932009110
MDBX: configurable corrupt error recommendations ( #275 )
...
* save
* save
2022-01-25 15:25:09 +07:00
ledgerwatch
a7ec201c0a
Update aggregator.go ( #274 )
2022-01-24 22:43:41 +00:00
ledgerwatch
7ec016b160
Fixes in compress ( #260 )
...
* Fixes in compress
* Reuse outputFile also as uncompressed file
* Close file before renaming
* Trace
* Untrace
* Use 8 threads
* Print aggregations
* Print merge and timing
* Print merge and timing
* readonly mode for patricia
* Fix to infinite loop
* Fix file names
* Cleanup
* Cleanup
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-24 22:13:48 +00:00
primal_concrete_sledge
d8a33270e8
issue/issue-249-add_index_reader ( #273 )
...
* issue/issue-249-add_index_reader
* Add licence
2022-01-24 20:39:04 +00:00
primal_concrete_sledge
e69a5da702
Issue 248 refinements for decompressor api ( #271 )
...
* issue/ISSUE-248-refinements_for_decompressor_api
* Fix match tedst expectations
* Remove unneeded comment
2022-01-24 09:18:08 +00:00
Alex Sharov
3a2d474940
MDBX: Add recommendations to corruption error message #3337
2022-01-24 10:35:38 +07:00
Alex Sharov
5ba56ae360
Mdbx 0.11.4 ( #266 )
2022-01-23 09:07:46 +07:00
Alex Sharov
1c92f7bc00
mdbx: consensus label #265
2022-01-22 12:49:25 +07:00
alex.sharov
ec2a482087
allow configure pagesize
2022-01-22 12:46:22 +07:00
Alex Sharov
8791c5eb7a
Etl less allocs ( #263 )
...
* etl less allocs
* etl less allocs
* allow configure pagesize
2022-01-22 12:21:04 +07:00
Alex Sharov
bcef5f091c
allow configure pagesize ( #264 )
2022-01-22 12:13:35 +07:00
Alex Sharov
79eb27d3f1
Helper to prohibit cli flag changes ( #262 )
2022-01-22 10:48:22 +07:00
canepat
e58d0fa537
Add B+ tree prototype implementation ( #258 )
...
* Add B+ tree prototype implementation
* Add Apache license header to all files
Fix linter errors
2022-01-21 07:06:09 +00:00
Alex Sharov
f16aa09cb1
mdbx: MDBX_FORCE_ASSERTIONS=0 ( #256 )
2022-01-21 11:45:32 +07:00
e-danko
43105470fd
added starknet gointerfaces ( #259 )
2022-01-20 14:11:08 +00:00
e-danko
3dea173d42
Added starknet proto to gointerfaces ( #254 )
2022-01-20 09:31:34 +00:00
Alex Sharov
ac8c29cacf
kv helpers ( #255 )
2022-01-20 15:31:01 +07:00
Giulio rebuffo
ac26beb190
added cumulative index tables ( #253 )
2022-01-20 08:25:03 +00:00
Alex Sharov
302946c5e5
Move cli root context to erigon-lib ( #251 )
...
* save
* save
2022-01-19 10:48:58 +07:00
ledgerwatch
340195df93
Less verbose parallel compressor ( #247 )
...
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-18 14:20:05 +00:00
Alex Sharov
11ab5bdbb8
Parallel compressor - allow empty words ( #245 )
...
* save
* save
* Fix lint
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-18 13:57:35 +00:00
Zhang Zhuo
10b163bbf1
doc: fix a broken link in etl/README.md ( #246 )
2022-01-18 13:43:30 +00:00
Alex Sharov
0f80e9941f
Switch to parallel compressor ( #244 )
2022-01-18 12:55:20 +07:00
Alexandr Borodulin
ac4119191d
Encoding StarknetTx salt parameter ( #243 )
...
Co-authored-by: Aleksandr Borodulin <a.borodulin@axioma.lv>
2022-01-17 13:47:16 +01:00
Alex Sharov
7c2104e2e1
fix to no prealloc (because max size unknown)
2022-01-17 17:05:37 +07:00
Alexandr Borodulin
fa6c709fcd
Encoding StarknetTx salt parameter ( #240 )
...
Co-authored-by: Aleksandr Borodulin <a.borodulin@axioma.lv>
2022-01-17 10:54:29 +01:00
Alex Sharov
5042e16218
enforce exactly once semantic in BigChunks helper
2022-01-17 16:49:02 +07:00
Alex Sharov
51220cfe43
ParallelCompressor class, DecompressedFile class ( #234 )
...
* save
* save
* save
* remove major jump check
* remove major jump check
* log
* log
* save
* format docs
* format docs
* issue-260
* issue-260
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
* save
2022-01-17 08:50:42 +00:00
Neo
26dcd28433
Update PriceBump, change tx replacement logic, update test ( #238 )
2022-01-17 07:45:57 +00:00
