mirror/erigon-pulse
1
0
Fork 0
mirror of https://gitlab.com/pulsechaincom/erigon-pulse.git synced 2025-01-19 00:54:12 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
726 Commits 5 Branches 14 Tags 295 MiB
cbfd733672
Commit Graph

726 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
ledgerwatch
cbfd733672
[erigon2] optimisations (#297) 
* Reuse getter and indexReader

* Properly reset Getter

* Use separate getter and indexReader for merge thread

* Remove DB

* Fixes

* Print

* Fixes

* Cleanup signatures

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-02-03 08:49:47 +00:00
Alex Sharov
ec354d1615
Fuzz fixes (#295) 
* fuzz tests fixes

* fuzz tests fixes
 2022-02-02 09:18:04 +00:00
Alex Sharov
a53f232322
pool: fix race in async broadcast (#294) 
* pool: better async broadcast

* save

* save
 2022-02-02 14:49:26 +07:00
Alex Sharov
74d49a9de1
non blocking broadcast of new txs (#292) 2022-02-01 12:16:43 +00:00
ledgerwatch
1cc07746a4
[erigon2] Try to fix deadlock (#291) 
* Try to fix deadlock

* Another lock

* Try to optimise

* comments and cleanup

* Prevent deadlock on panic

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-02-01 10:28:05 +00:00
Alex Sharov
0feb7fd591
Decompressor.WithReadAhead (#290) 2022-02-01 11:19:11 +07:00
ledgerwatch
4e8840256e
[erigon2] Use shorter references instead of full plain keys in the commitment files (#289) 
* Rearrange aggregations

* More rearranging before introducing 3 threads

* Background aggregation|

* Concurrency fixes

* Remove files under lock

* Better logging

* Remove files without lock

* Fix lint

* Fix locking

* Try

* Fix background Merge

* Log merging

* Log merging

* Less logging

* Millisecond

* Add Stats function

* Log merge only after 1m

* Wrong counting

* plain key extract and replace functions

* Insert valTransform function

* Not parse first byte

* Not parse first byte

* Fix lint

* Switch to thin state references

* Fix lint

* Fix lint

* Debug print|

* Fix decoding

* Turn off valTransform

* Not to reuse transformer

* Print

* Print

* Print

* Derive hashed keys later

* Fix

* Fix log

* Fix

* Debug

* Another fix

* Fix

* Fix

* Print

* Print

* Data race

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-01-31 22:32:00 +00:00
ledgerwatch
586ab3e6b3
Separate state btree files (#287) 
* Separate state file btrees, fix Match in the decompressor

* fix match

* Fix to match

* Switch back from Match

* Try to use match, close indices

* Fixing Match

* Use Skip

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-01-29 11:12:38 +00:00
Enrique Jose Avila Asapche
a5c2d83108
Added bor tables (#286) 
Co-authored-by: Krishna Upadhyaya <krishnau1604@gmail.com>
 2022-01-28 21:34:03 +00:00
primal_concrete_sledge
24098d94d8
WIP: issue/issue-250-use_compressed_keys (#285) 
* issue/issue-250-use_compressed_keys

* Step into the twilight zone

* fix zero byte

* Add zerolen check

* Remove branhc data check

* Check lengths
 2022-01-28 19:30:32 +00:00
Alex Sharov
c8397a0433
Mdbx: more 4tb fixes (#282) 2022-01-27 23:10:30 +07:00
Alex Sharov
dfdf7c8a66
[wip] parallel compress: less read of dat file (#284) 
* save

* save

* save
 2022-01-27 17:13:26 +07:00
Alex Sharov
ec11eb3d91
parallel compressor: don't save dict (#283) 
* save

* save
 2022-01-27 12:54:38 +07:00
ledgerwatch
1d4022960a
Fix RLP inconsistency in txpool (#280) 
* Fix RLP inconsistency in txpool

* Add debug

* hasEnvelope argument when parsing transaction

* Handle incorrect packets without panic

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-01-26 16:09:02 +00:00
ledgerwatch
df95313a31
Add blockGasLimit to the state stream (#261) 
* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit eff9f8f

git-subtree-dir: interfaces
git-subtree-split: eff9f8f7273dda7c78575f2474e3103da61825db

* Add blockGasLimit to the state stream

* Pass blockGasLimit into txpool

* Add NotTooMuchGas bit, refactored promote

* Replace effectiveTip with minTip in the queues and slices

* Fix lint

* Refactor

* Improve terminology of ordering function

* Optimise baseFee change

* cleanup

* Safe operations

* Fix UnsafeAdd

* also print pendingBaseFee

* trace txpool_content

* trace txpool_content

* user sender ID inverted index to speed up txpool_content

* copy rlp from Db

* remove debug printing

* change from string to []byte

* safe storage of sender in the map

* Fix compile

* safe storage of sender in the map

* Print errors

* Prints

* Prints

* Prints

* Change worst function

* Change worst function

* Change worst function

* Change worst function

* Fix minTip

* Fix minTip

* Warn on db error, but not stop

* Fix interface

* More prints

* Cleanup

* Cleanup

* Differentiate baseFee pool

* Bump version of remove KV interface

* Not print every block

* Not print every block

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-01-25 10:20:11 +00:00
Alex Sharov
5932009110
MDBX: configurable corrupt error recommendations (#275) 
* save

* save
 2022-01-25 15:25:09 +07:00
ledgerwatch
a7ec201c0a
Update aggregator.go (#274) 2022-01-24 22:43:41 +00:00
ledgerwatch
7ec016b160
Fixes in compress (#260) 
* Fixes in compress

* Reuse outputFile also as uncompressed file

* Close file before renaming

* Trace

* Untrace

* Use 8 threads

* Print aggregations

* Print merge and timing

* Print merge and timing

* readonly mode for patricia

* Fix to infinite loop

* Fix file names

* Cleanup

* Cleanup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-01-24 22:13:48 +00:00
primal_concrete_sledge
d8a33270e8
issue/issue-249-add_index_reader (#273) 
* issue/issue-249-add_index_reader

* Add licence
 2022-01-24 20:39:04 +00:00
primal_concrete_sledge
e69a5da702
Issue 248 refinements for decompressor api (#271) 
* issue/ISSUE-248-refinements_for_decompressor_api

* Fix match tedst expectations

* Remove unneeded comment
 2022-01-24 09:18:08 +00:00
Alex Sharov
3a2d474940
MDBX: Add recommendations to corruption error message #3337 2022-01-24 10:35:38 +07:00
Alex Sharov
5ba56ae360
Mdbx 0.11.4 (#266) 2022-01-23 09:07:46 +07:00
Alex Sharov
1c92f7bc00
mdbx: consensus label #265 2022-01-22 12:49:25 +07:00
alex.sharov
ec2a482087 allow configure pagesize 2022-01-22 12:46:22 +07:00
Alex Sharov
8791c5eb7a
Etl less allocs (#263) 
* etl less allocs

* etl less allocs

* allow configure pagesize
 2022-01-22 12:21:04 +07:00
Alex Sharov
bcef5f091c
allow configure pagesize (#264) 2022-01-22 12:13:35 +07:00
Alex Sharov
79eb27d3f1
Helper to prohibit cli flag changes (#262) 2022-01-22 10:48:22 +07:00
canepat
e58d0fa537
Add B+ tree prototype implementation (#258) 
* Add B+ tree prototype implementation

* Add Apache license header to all files
Fix linter errors
 2022-01-21 07:06:09 +00:00
Alex Sharov
f16aa09cb1
mdbx: MDBX_FORCE_ASSERTIONS=0 (#256) 2022-01-21 11:45:32 +07:00
e-danko
43105470fd
added starknet gointerfaces (#259) 2022-01-20 14:11:08 +00:00
e-danko
3dea173d42
Added starknet proto to gointerfaces (#254) 2022-01-20 09:31:34 +00:00
Alex Sharov
ac8c29cacf
kv helpers (#255) 2022-01-20 15:31:01 +07:00
Giulio rebuffo
ac26beb190
added cumulative index tables (#253) 2022-01-20 08:25:03 +00:00
Alex Sharov
302946c5e5
Move cli root context to erigon-lib (#251) 
* save

* save
 2022-01-19 10:48:58 +07:00
ledgerwatch
340195df93
Less verbose parallel compressor (#247) 
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-01-18 14:20:05 +00:00
Alex Sharov
11ab5bdbb8
Parallel compressor - allow empty words (#245) 
* save

* save

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
 2022-01-18 13:57:35 +00:00
Zhang Zhuo
10b163bbf1
doc: fix a broken link in etl/README.md (#246) 2022-01-18 13:43:30 +00:00
Alex Sharov
0f80e9941f
Switch to parallel compressor (#244) 2022-01-18 12:55:20 +07:00
Alexandr Borodulin
ac4119191d
Encoding StarknetTx salt parameter (#243) 
Co-authored-by: Aleksandr Borodulin <a.borodulin@axioma.lv>
 2022-01-17 13:47:16 +01:00
Alex Sharov
7c2104e2e1
fix to no prealloc (because max size unknown) 2022-01-17 17:05:37 +07:00
Alexandr Borodulin
fa6c709fcd
Encoding StarknetTx salt parameter (#240) 
Co-authored-by: Aleksandr Borodulin <a.borodulin@axioma.lv>
 2022-01-17 10:54:29 +01:00
Alex Sharov
5042e16218
enforce exactly once semantic in BigChunks helper 2022-01-17 16:49:02 +07:00
Alex Sharov
51220cfe43
ParallelCompressor class, DecompressedFile class (#234) 
* save

* save

* save

* remove major jump check

* remove major jump check

* log

* log

* save

* format docs

* format docs

* issue-260

* issue-260

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
 2022-01-17 08:50:42 +00:00
Neo
26dcd28433
Update PriceBump, change tx replacement logic, update test (#238) 2022-01-17 07:45:57 +00:00
ledgerwatch
0eaa0333e1
[erigon2] Optional writing of before changes, commitment decoupled from blocks (#237) 
* Separate commitment from blocks

* Optional beforeOn

* Optional beforeOn

* Optional beforeOn

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-01-16 20:27:44 +00:00
ledgerwatch
e5c07ec901
[erigon2] Resumable prototype (#236) 
* Not warn about files that don't match at all

* Not warn about files that don't match at all

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Add trace

* Cleanup

* Cleanup

* Cleanup

* Cleanup

* Cleanup

* Cleanup

* Cleanup

* Cleanup

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-01-15 22:09:06 +00:00
ledgerwatch
b95cff2ce3
No empty RLPs in TxPacket messages, consistent parsing of TxPacket and PooledTxPacket (#232) 
* Not include empty RLPs into TransactionPacket messages

* Consistent parsing of TransactionPacket and PooledTransactionPacket

* The other way around

* Cleaner form

* Add test for encoding and parsing TransctionsPacket

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-01-15 09:09:33 +00:00
Alex Sharov
01a6417505
snapshots: same workers amount #233 2022-01-15 11:23:19 +07:00
ledgerwatch
c08ed12256
[erigon2] Measure change set sizes with perfect hash table (#230) 
* Remove warning message

* Calculate sizes of change sets for transactions

* Calculate sizes of change sets for transactions

* Produce changeset files

* Not to produce changesets for commitments

* Remove rewindTx

* Close decompressor

* Not generate changesets by default

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
 2022-01-14 14:10:10 +00:00
alex.sharov
3e856aba62 format docs 2022-01-13 11:08:05 +07:00