Commit Graph

676 Commits

Author SHA1 Message Date
ledgerwatch
ae6f7202d2
[erigon2] Introduce commitment files and table (#201)
* Introduce commitment files and table

* Introduce commitment files and table

* Introduce words buffer

* Move changes out of aggregator into the writer

* Start calc commitment

* Add commitment package

* More on hex patricia

* More

* account decorator

* More

* Add test

* More

* More to the test

* Use hex in tests

* More

* More

* More

* More

* Simplified cells

* Add delBitmap

* More advanced MockState

* More compact unfolding

* Separation of hashed Keys

* Sepatate downHashedKey and upHashedKey

* Carry extension node through accounts

* optimised row allocations

* Fix encoding/decoding, add trace flag

* Added account and storage into the cells

* Make accountKeyLen global setting

* Remove BranchNodeUpdaqte intermediate

* Started on computeHash

* Initial hash calculations

* Fix lint

* Change account encoding

* Fix commitment issues

* Fix lint

* Fix lint

* Fix lint

* Trace

* Small fix and tracing

* Print branch hashes, fix empty

* Fix

* Fix

* Fix

* Fix

* Print accountFn

* Trigger accountFn

* Trigger accountFn

* Another fix for extension nodes

* Return root hash and set trace

* Print depth for computeCellHash

* Fix for storage leaves

* Fix for storage leaves

* Fix for storage leaves

* Fix for storage leaves

* Fix for storage leaves

* Fix for storage leaves

* Ineffectual deletes

* Trace needUnfolding

* Fix

* Fix

* Fix

* trace deletes

* trace deletes

* trace deletes

* trace deletes

* trace deletes

* not overwrite downHashedKey in updateAccount

* simplify needUnfolding

* simplify needUnfolding

* simplify needUnfolding

* simplify needUnfolding

* simplify needUnfolding

* propagate deletes when unfolding

* remove deletes when folding

* remove deletes when folding

* remove upHashedKey when delete cell

* remove upHashedKey when delete cell

* aggregate storage changes later

* remove del bit when updating account

* remove del bit when updating account

* remove del bit when updating account

* perform delete after code update

* perform delete after code update

* perform delete after code update

* perform delete after code update

* perform delete after code update

* perform delete after code update

* latest update has precedence when aggregating

* Revert "latest update has precedence when aggregating"

This reverts commit 0234ea6c3e1a0b2e7599df7c470d4b8b8d56736a.

* latest update has precedence when aggregating

* Revert "latest update has precedence when aggregating"

This reverts commit cfa7b75327e4d5d1236f905c3c04eb640b21838d.

* Introduce FinishTx

* Separate commitment separation from computation

* Prevent spurious delete

* Spurious deletes

* Trace param

* Preserve upHashedKey when unfolding

* Fix

* Fix

* Correctly compute firstInsert

* Try to fix delete + fold

* Remove commented out

* updateStorage to remove deleted flag

* rename upHashedKey to extension, fix clear-up

* Carry extension up if account plain key is present

* Carry extension up if account plain key is present

* Not to fail for deleteAccount

* Not to fail for deleteAccount

* Not to fail for deleteAccount

* Print

* Print

* Print

* Print

* Print

* Print

* Print

* Print

* Copy keys before putting them into commTree

* Preserve changes to AccountData

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-12 11:21:52 +00:00
Alex Sharov
1647faec37
Fix bigChunk helper (#229) 2022-01-12 10:46:26 +07:00
Alex Sharov
e98482670a
mdbx: fix gc "retry" issue (slowness of gc during commit) (#227)
* mdbx: fix gc "retry" issue (slowness of gc during commit)

* mdbx: fix gc "retry" issue (slowness of gc during commit)
2022-01-11 08:12:20 +00:00
Alex Sharov
f92c12855d
Decompressor: fast .Count method(#226) 2022-01-09 17:32:56 +07:00
alex.sharov
a8c2481967 create huffman_codes.txt in tmpdir 2022-01-09 14:52:52 +07:00
alex.sharov
8bc0f26a49 create .seg in tmpdir 2022-01-09 14:49:56 +07:00
Alex Sharov
0d5d8975d9
Snapshots: create .dat in tmpdir (#225) 2022-01-09 14:43:55 +07:00
Alex Sharov
71c8187a64
Snapshot: Txn lookup (#224) 2022-01-07 17:14:21 +07:00
alex.sharov
8a0d41693f create idx in tmpdir 2022-01-07 14:38:38 +07:00
alex.sharov
6aa0a5f08e create idx in tmpdir 2022-01-07 14:37:27 +07:00
alex.sharov
68b0fe6030 create dir automatically 2022-01-07 14:27:26 +07:00
Alex Sharov
f5733d438f
Parallel compression (#223) 2022-01-06 14:13:03 +07:00
Alex Sharov
d4faf5beee
Rlp base err2 (#221) 2022-01-05 17:59:26 +07:00
Alex Sharov
2fe3f06702
Decompressor internal file path getter (#220) 2021-12-31 18:42:43 +07:00
Giulio rebuffo
bd5e7706f0
ran make (#218) 2021-12-28 01:50:12 +01:00
Giulio rebuffo
910ae8eec6
updated to latest merge spec (#217)
* updated to latest merge spec

* now

* gointerfaces

* done

* better naming

* corrected extra data

* corrected extra data

* correct order

* again, correct order
2021-12-27 18:59:32 +01:00
alex.sharov
dd363aea75 Revert "save"
This reverts commit ef20b06b01.
2021-12-27 15:11:24 +07:00
alex.sharov
ef20b06b01 save 2021-12-27 15:11:00 +07:00
alex.sharov
b6a6b34deb add go.work to .gitignore 2021-12-26 12:29:31 +07:00
Alex Sharov
bf21599d23
RLP: base error - allows skip them if need (#215) 2021-12-22 14:34:34 +07:00
Alex Sharov
583c3f9b5b
count in .seg (#212) 2021-12-21 10:45:20 +07:00
ledgerwatch
8d06531e4e
txpool - removed mined transactions if they were missed by state stream (#213)
* Copy rlp for stream, remove txs

* Fix low nonce removal

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-12-17 09:35:46 +00:00
ledgerwatch
d79d43f908
Downgrade PropagatePooledTxsToPeersList to Debug (#210)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-12-16 21:33:33 +00:00
ledgerwatch
7f6eb71c4c
Split up txpool Broadcast and Announce (#209)
* Split up Broadcast and Announce

* Split up Broadcast and Announce

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-12-16 21:16:04 +00:00
ledgerwatch
7f82ddaa75
txpool broadcasting (#208)
* txpool broadcasting

* Fix lint

* Broadcast transaction to random peers

* Fix broadcast

* Fix panic

* Change terminology

* fix for broadcasting

* Rebroadcast transactions promoted to pending subpool

* Trace moving between subpools

* Deduplicate promoted hashes, fix basefee promotion

* Tx propagation to be more resilient

* Fix dedup

* Change collection of promoted hashes

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-12-16 20:58:40 +00:00
Alex Sharov
bb6dfef7c8
add downloader.proto (#206)
* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit be07fc9

git-subtree-dir: interfaces
git-subtree-split: be07fc99183a7cef1ca08961033bdb917d85d9b5

* save

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit be07fc9

git-subtree-dir: interfaces
git-subtree-split: be07fc99183a7cef1ca08961033bdb917d85d9b5

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit 23d68fd

git-subtree-dir: interfaces
git-subtree-split: 23d68fda86c3a2c322e7b430c817b06b6f6ca311

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit 500dde8

git-subtree-dir: interfaces
git-subtree-split: 500dde8293c0a12d19d94b5ff633722a03d7fc4e

* save

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit e5b1945

git-subtree-dir: interfaces
git-subtree-split: e5b1945d02da7a7f00e2289034ee90a6edd60184

* save

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit f36d878

git-subtree-dir: interfaces
git-subtree-split: f36d878080fe018a41fb903f81fc8f8ac9981cbd

* save

* save

* save
2021-12-14 13:33:32 +00:00
ledgerwatch
c7cd5d65d3
Introduce transaction tracing in tx pool (#205)
* Introduce transaction tracing in tx pool

* Add tracing prints

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-12-14 09:40:07 +00:00
Andrew Ashikhmin
b06f3cec6b
RLP: Values in [0, 128) must be encoded as single byte (#203)
* RLP: Values in [0, 128) must be encoded as single byte

* Fix error message
2021-12-06 14:00:18 +00:00
ledgerwatch
4f39e6b8bc
Update ci.yml 2021-12-01 13:30:16 +00:00
ledgerwatch
e2c8696eef
move CI to MacOS 11 (#199) 2021-11-30 14:43:36 +00:00
Dmitry Savelev
da3c239ed6
Add nodeInfo GRPC methods (#196)
* Add nodeInfo

* Remove interfaces for replacement

Signed-off-by: Dmitry Savelev <darksawa@gmail.com>

* Squashed 'interfaces/' content from commit 67abfdd

git-subtree-dir: interfaces
git-subtree-split: 67abfdd23b3bbef0d2f0c038275b6f47820d7f5e
2021-11-30 14:41:31 +00:00
Alex Sharov
53c160deb1
save (#197) 2021-11-30 07:06:57 +00:00
TBC Dev
a27d9202e3
Refactor and simplify txpool.sendersBatch (#195)
* De-dup and extract sendersBatch.getOrCreateId()

* Misc simplify
2021-11-28 10:46:21 +00:00
ledgerwatch
e83c9f276a
Tracing in the aggregator (#194)
* Tracing in the aggregator

* Trace merge of keys

* print value when merging

* More tracing

* Fix trace keys

* Fix tracing

* Fix skipping

* Fix tracing for deleteAccount

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-11-27 08:34:56 +00:00
ledgerwatch
0404744146
Introduce PriceBump, change tx replacement logic, add test (#192)
* Introduce PriceBump, change tx replacement logic, add test

* Fix TestNonceFromAddress

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-11-26 13:55:58 +00:00
brendan-kelly
057925ba08
Add ParliaSnapshot table to database for Binance Smart Chain support (#190)
* Add ParliaSnapshot table to database for Binance Smart Chain support

* Add schema description for ParliaSnapshot table
2021-11-25 06:48:55 +00:00
Giulio rebuffo
4258567eb7
added util to convert to H2048 interface (#188)
* added util to convert to H2048 interface

* plain bytes

* lint
2021-11-25 05:18:17 +00:00
Alex Sharov
34e6ca3c35
Pool: support multi sentry (#191)
* save

* save

* save
2021-11-24 17:13:17 +00:00
Alex Sharov
6d88f06d4d
save (#187) 2021-11-22 11:41:44 +00:00
TBC Dev
d8870b19c3
Change sentry peer_id from H512 pubkey to H256 keccak256(pubkey) (#183)
* Rename h1 to h0 for clarity

* Simplify toHashes() and toPeerIDs()

* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit ed9b048

git-subtree-dir: interfaces
git-subtree-split: ed9b04802fe8891a2bc3fea274c498c758d3fcac

* Change sentry peer_id from H512 pubkey to H256 keccak256(pubkey)
2021-11-21 19:35:59 +00:00
Alex Sharov
bb3f510d16
RecSplit: store BaseDataID in .idx file (helps to navigate over non blockNum-based entries) (#180)
* save

* save

* save

* save
2021-11-21 14:52:23 +00:00
Alex Sharov
06fb85a0ed
Pool: reject low nonce and low balance txs (#182)
* save

* save
2021-11-21 14:52:17 +00:00
Giulio rebuffo
8a7f4ca4e9
Update interfaces (#179)
* Added CurrentExecutionPayload Bucked

* lint

* updated

* added util to decode blooms
2021-11-21 14:52:08 +00:00
Giulio rebuffo
4c973f1ee4
Added CurrentExecutionPayload Bucket (#178)
* Added CurrentExecutionPayload Bucked

* lint
2021-11-20 09:40:26 +07:00
Alex Sharov
bb1d712834
Hack: dump bodies and headers (#177)
* save

* save

* save

* save
2021-11-19 22:00:55 +07:00
Alex Sharov
a4ee647902
Recsplit: move files read/write helpers to erigon-lib (#176) 2021-11-19 12:03:36 +07:00
Alex Sharov
3256008c58
Canonical tx ids (#174)
* save

* save

* save
2021-11-18 12:02:03 +00:00
TBC Dev
5b0c5e7fdb
TxPool efficiency improvements (#173)
* Avoid iteration for Spammer tests by tracking senderID tx counts

* Add BySenderAndNonce.ascendAll()

* Add BySenderAndNonce.descend()

* Avoid iteration by descending, and pre-allocate known size

* Resize() once rather than for each good txn

* Use range value in loops

* Add comments to clarify NotSet default values
2021-11-18 07:59:25 +00:00
ledgerwatch
d5f6afc92d
Delete code with account, add code test (#171)
* Delete code with account, add code test

* Fix test

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-11-17 15:49:48 +00:00
Alex Sharov
5108c95b85
txpool.Pending() method, which preserve ready-for-mining order (#172) 2021-11-17 17:23:51 +07:00