Commit Graph

102 Commits

Author SHA1 Message Date
Alex Sharov
3c52757526
Downloader tables (#422)
* save

* save

* save
2022-04-21 10:43:46 +07:00
Alex Sharov
1ac7357005
KV: ReadAhead helper #419 2022-04-18 09:37:01 +07:00
Andrew Ashikhmin
8c89288cbc
Add LastForkchoice table (#405)
* Deprecate TransitionBlock

* Add ForkchoiceHead/Safe/Finalized tables

* Merge Forkchoice* into LastForkchoice
2022-03-31 10:51:23 +02:00
Alex Sharov
54629aaf7c
Mdbx fix after v0.11.6 (#398)
* save

* save
2022-03-27 07:23:35 +01:00
ledgerwatch
4e8d577d1d
[erigon2] Switch back to MDBX (#388)
* Switch back to MDBX

* Fix test

* No restore

* Fix overwrite

* Fix change file opening

* Verify state vs change files

* Add WriteMap

* Print

* Not to go to the end of the file when reading

* Fix rebuild

* prefixLen

* Print

* Print

* Print

* Fix for rebuild

* Remove prints

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-03-23 14:35:13 +00:00
Alex Sharov
83951a1d62
Enable more linters (#381) 2022-03-19 11:38:37 +07:00
Alex Sharov
2193d570a6
Mdbx: WriteMap fallback on open error (#369)
* save

* save
2022-03-16 11:59:11 +00:00
battlmonstr
3123b6d895
kv: add NodeRecords nodedb table (#359) 2022-03-10 13:15:15 +01:00
battlmonstr
f54e4c7eae
kv: use SetSyncPeriod from mdbx-go v0.22.16 (#358) 2022-03-10 13:04:08 +01:00
Alex Sharov
6512e3c941
add emptyWordsCount field to .seg file header (breaking .seg format) (#355)
* up torrent

* save

* save

* save

* save

* save

* save

* save
2022-03-10 07:48:37 +00:00
battlmonstr
009358ab2f
kv: expose mdbx SafeNoSync and OptSyncPeriod options (#356) 2022-03-10 10:25:11 +07:00
Alex Sharov
a2cbecf373
[wip] KV: limit amount of parallel read txs (#352)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* a

* a

* a

* a

* save

* save

* save

* save

* save
2022-03-07 12:21:59 +00:00
Alex Sharov
56d528b1f0
save (#353) 2022-03-04 09:13:15 +07:00
Alex Sharov
5be29adf1a
Grpc err utils (#334)
* save

* save
2022-02-18 09:40:11 +07:00
alex.sharov
a2ca4cf55f save 2022-02-17 10:10:23 +07:00
Alex Sharov
7aec036bdd
remotedb server: ro db (#313) 2022-02-11 16:02:57 +07:00
Alex Sharov
8e4618c57d
state and newHead: 1 goroutine per subscriber #309 2022-02-10 08:26:08 +07:00
alex.sharov
d814adb13e save 2022-02-03 16:28:43 +07:00
alex.sharov
02c407a14e save 2022-02-03 16:28:32 +07:00
Enrique Jose Avila Asapche
a5c2d83108
Added bor tables (#286)
Co-authored-by: Krishna Upadhyaya <krishnau1604@gmail.com>
2022-01-28 21:34:03 +00:00
ledgerwatch
df95313a31
Add blockGasLimit to the state stream (#261)
* Remove interfaces for replacement

* Squashed 'interfaces/' content from commit eff9f8f

git-subtree-dir: interfaces
git-subtree-split: eff9f8f7273dda7c78575f2474e3103da61825db

* Add blockGasLimit to the state stream

* Pass blockGasLimit into txpool

* Add NotTooMuchGas bit, refactored promote

* Replace effectiveTip with minTip in the queues and slices

* Fix lint

* Refactor

* Improve terminology of ordering function

* Optimise baseFee change

* cleanup

* Safe operations

* Fix UnsafeAdd

* also print pendingBaseFee

* trace txpool_content

* trace txpool_content

* user sender ID inverted index to speed up txpool_content

* copy rlp from Db

* remove debug printing

* change from string to []byte

* safe storage of sender in the map

* Fix compile

* safe storage of sender in the map

* Print errors

* Prints

* Prints

* Prints

* Change worst function

* Change worst function

* Change worst function

* Change worst function

* Fix minTip

* Fix minTip

* Warn on db error, but not stop

* Fix interface

* More prints

* Cleanup

* Cleanup

* Differentiate baseFee pool

* Bump version of remove KV interface

* Not print every block

* Not print every block

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-25 10:20:11 +00:00
Alex Sharov
1c92f7bc00
mdbx: consensus label #265 2022-01-22 12:49:25 +07:00
alex.sharov
ec2a482087 allow configure pagesize 2022-01-22 12:46:22 +07:00
Alex Sharov
bcef5f091c
allow configure pagesize (#264) 2022-01-22 12:13:35 +07:00
Alex Sharov
79eb27d3f1
Helper to prohibit cli flag changes (#262) 2022-01-22 10:48:22 +07:00
Alex Sharov
ac8c29cacf
kv helpers (#255) 2022-01-20 15:31:01 +07:00
Giulio rebuffo
ac26beb190
added cumulative index tables (#253) 2022-01-20 08:25:03 +00:00
Alex Sharov
0f80e9941f
Switch to parallel compressor (#244) 2022-01-18 12:55:20 +07:00
Alex Sharov
5042e16218
enforce exactly once semantic in BigChunks helper 2022-01-17 16:49:02 +07:00
alex.sharov
3e856aba62 format docs 2022-01-13 11:08:05 +07:00
ledgerwatch
ae6f7202d2
[erigon2] Introduce commitment files and table (#201)
* Introduce commitment files and table

* Introduce commitment files and table

* Introduce words buffer

* Move changes out of aggregator into the writer

* Start calc commitment

* Add commitment package

* More on hex patricia

* More

* account decorator

* More

* Add test

* More

* More to the test

* Use hex in tests

* More

* More

* More

* More

* Simplified cells

* Add delBitmap

* More advanced MockState

* More compact unfolding

* Separation of hashed Keys

* Sepatate downHashedKey and upHashedKey

* Carry extension node through accounts

* optimised row allocations

* Fix encoding/decoding, add trace flag

* Added account and storage into the cells

* Make accountKeyLen global setting

* Remove BranchNodeUpdaqte intermediate

* Started on computeHash

* Initial hash calculations

* Fix lint

* Change account encoding

* Fix commitment issues

* Fix lint

* Fix lint

* Fix lint

* Trace

* Small fix and tracing

* Print branch hashes, fix empty

* Fix

* Fix

* Fix

* Fix

* Print accountFn

* Trigger accountFn

* Trigger accountFn

* Another fix for extension nodes

* Return root hash and set trace

* Print depth for computeCellHash

* Fix for storage leaves

* Fix for storage leaves

* Fix for storage leaves

* Fix for storage leaves

* Fix for storage leaves

* Fix for storage leaves

* Ineffectual deletes

* Trace needUnfolding

* Fix

* Fix

* Fix

* trace deletes

* trace deletes

* trace deletes

* trace deletes

* trace deletes

* not overwrite downHashedKey in updateAccount

* simplify needUnfolding

* simplify needUnfolding

* simplify needUnfolding

* simplify needUnfolding

* simplify needUnfolding

* propagate deletes when unfolding

* remove deletes when folding

* remove deletes when folding

* remove upHashedKey when delete cell

* remove upHashedKey when delete cell

* aggregate storage changes later

* remove del bit when updating account

* remove del bit when updating account

* remove del bit when updating account

* perform delete after code update

* perform delete after code update

* perform delete after code update

* perform delete after code update

* perform delete after code update

* perform delete after code update

* latest update has precedence when aggregating

* Revert "latest update has precedence when aggregating"

This reverts commit 0234ea6c3e1a0b2e7599df7c470d4b8b8d56736a.

* latest update has precedence when aggregating

* Revert "latest update has precedence when aggregating"

This reverts commit cfa7b75327e4d5d1236f905c3c04eb640b21838d.

* Introduce FinishTx

* Separate commitment separation from computation

* Prevent spurious delete

* Spurious deletes

* Trace param

* Preserve upHashedKey when unfolding

* Fix

* Fix

* Correctly compute firstInsert

* Try to fix delete + fold

* Remove commented out

* updateStorage to remove deleted flag

* rename upHashedKey to extension, fix clear-up

* Carry extension up if account plain key is present

* Carry extension up if account plain key is present

* Not to fail for deleteAccount

* Not to fail for deleteAccount

* Not to fail for deleteAccount

* Print

* Print

* Print

* Print

* Print

* Print

* Print

* Print

* Copy keys before putting them into commTree

* Preserve changes to AccountData

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-01-12 11:21:52 +00:00
Alex Sharov
1647faec37
Fix bigChunk helper (#229) 2022-01-12 10:46:26 +07:00
Alex Sharov
e98482670a
mdbx: fix gc "retry" issue (slowness of gc during commit) (#227)
* mdbx: fix gc "retry" issue (slowness of gc during commit)

* mdbx: fix gc "retry" issue (slowness of gc during commit)
2022-01-11 08:12:20 +00:00
Alex Sharov
2fe3f06702
Decompressor internal file path getter (#220) 2021-12-31 18:42:43 +07:00
alex.sharov
dd363aea75 Revert "save"
This reverts commit ef20b06b01.
2021-12-27 15:11:24 +07:00
alex.sharov
ef20b06b01 save 2021-12-27 15:11:00 +07:00
Alex Sharov
53c160deb1
save (#197) 2021-11-30 07:06:57 +00:00
brendan-kelly
057925ba08
Add ParliaSnapshot table to database for Binance Smart Chain support (#190)
* Add ParliaSnapshot table to database for Binance Smart Chain support

* Add schema description for ParliaSnapshot table
2021-11-25 06:48:55 +00:00
Giulio rebuffo
4c973f1ee4
Added CurrentExecutionPayload Bucket (#178)
* Added CurrentExecutionPayload Bucked

* lint
2021-11-20 09:40:26 +07:00
Alex Sharov
a4ee647902
Recsplit: move files read/write helpers to erigon-lib (#176) 2021-11-19 12:03:36 +07:00
Alex Sharov
3256008c58
Canonical tx ids (#174)
* save

* save

* save
2021-11-18 12:02:03 +00:00
Giulio rebuffo
71603ba29a
TransitionKey added for proof-of-stake transition (#170)
* transition data

* Update tables.go
2021-11-17 00:17:13 +01:00
Alex Sharov
921a20df2b
mdbx: set back big augment limit (#157) 2021-11-14 12:20:49 +07:00
ledgerwatch
fd19ad8148
State aggregator (#114)
* State aggregator

* Compile fix

* More

* Add

* More

* More on aggregator

* Writes (still incorrect)

* Move table names

* More

* Start of aggregation

* Change files instead of db

* More on change files

* More

* More

* Dealing with state and change files

* More

* More

* More boilerplate

* More

* More

* Iteration over storage

* More boilerplate

* More fixes

* Insert flag

* More

* Unit test

* Add more to the test

* Expand the test a bit

* More testing

* Keep fixing the test

* More fixes to the test

* Clean up DB tables upon aggregation

* More fixes

* Remove update/insert indicator from returned values

* Add assertions

* close files before deleting

* close files before deleting

* close files before deleting

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-11-13 12:12:29 +00:00
alex.sharov
9b7e0459cf save 2021-10-28 21:52:22 +07:00
Alex Sharov
5d84a53060
Pool2 integration test support (#136) 2021-10-28 18:45:18 +07:00
Alex Sharov
b11097448f
Mdbx reduce augment limit to 8*default (#134) 2021-10-27 13:17:11 +07:00
Alex Sharov
585354391b
force some rw tx when db open, to allow mdbx up version signature (#125) 2021-10-22 10:13:11 +07:00
Alex Sharov
ec0155d4ed
Mdbx v0.11.0 (#123) 2021-10-22 09:12:39 +07:00
alex.sharov
9eae3ce63a add codeEvict list metric 2021-10-12 15:59:35 +07:00