Commit Graph

222 Commits

Author SHA1 Message Date
Alex Sharov
a78cb54132
step towards kv interface (#2311) 2021-07-07 23:15:49 +07:00
Alex Sharov
a68b5ba361
Replace ChainConfig.WithEIPsFlags by go-ethereum's ChainConfig.Rules (#2304)
* use chainRules

* use chainRules

* use chainRules

* use chainRules

* use chainRules
2021-07-05 19:52:50 +01:00
ledgerwatch
9023f2a29c
Investigate slow receipt repair (#2306)
* Collect data about broken receipts

* Actually collect data

* Fix CBOR problem more efficiently

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-05 19:14:58 +01:00
ledgerwatch
63ccabc72f
Receipt repair migration (#2291)
* Fixes in rpctest and rpcdaemon for debug_traceTransaction

* Fix for opcode tracer

* Tool to fix receipts

* Better mechanism for detecting broken receipts

* Fixes

* Introduce receipt_repair migration

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-04 14:48:13 +01:00
ledgerwatch
f5e42b1e7a
Fixes in rpctest and rpcdaemon for debug_traceTransaction, and opcode tracer, tool for repairing broken receipts (#2284)
* Fixes in rpctest and rpcdaemon for debug_traceTransaction

* Fix for opcode tracer

* Tool to fix receipts

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-03 14:34:23 +01:00
Alex Sharov
6a5f86423c
Remove "numgc" from logs (#2275) 2021-07-03 14:44:23 +07:00
ledgerwatch
ba55611f29
rpctest for eth_getTransactionReceipt (#2271)
* rpctest bench for eth_getTransationReceipt

* Add blockHash to re-generated receipts

* Modify scanReceipts

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-02 14:34:20 +01:00
Alex Sharov
3d6d45a82f
remove subtrie loader and dcmp (#2258) 2021-06-30 10:31:36 +07:00
ledgerwatch
7baf3ec484
Move secp256k1 into a separate go module, hook up erigon-lib (#2239)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-26 15:03:05 +01:00
ledgerwatch
adf4766191
Scan transaction to select test cases (#2225)
* Scan txs

* Find examples with access lists

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-23 23:34:17 +01:00
ledgerwatch
5ad449ef22
Migration to fix receipts (broken due to change of CBOR codec) (#2221)
* Scan receipts

* Not check tx type before Berlin

* Reverse

* scan receipt code in the migration

* Turn migration on

* Apply migration

* Fix

* Print last

* Remove last

* Not remove last

* Not updata in migration

* no cursors in migration

* Handle empty blocks

* Handle empty blocks

* Use the same code in hack

* Print

* Print

* Save

* Fix

* Fix lint

* Fix lint

* Fix for empty DB

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-23 15:05:16 +01:00
Alex Sharov
59d05dc5fe
hide file exists err (#2218) 2021-06-22 11:09:45 +01:00
ledgerwatch
ab685e90d9
Improvement to the trimTxs (#2215)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-22 08:35:34 +01:00
ledgerwatch
a79ed2fe0b
Utilities to fix DB corruption (#2214)
* Detect broken ancestry

* Print last canonical headers

* Add backExec

* Cleanup

* Add unwind util

* Add trimTxs utility

* Actually delete tx records

* Avoid infinite loop

* Change strategy

* Close cursor after commmit

* Fix RwBegin issue

* 10m records per transaction

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-21 21:52:51 +01:00
ledgerwatch
04943eaa52
Fix for state corruption without performance regression (#2208)
* extractHeaders

* Print diff

* Fix for state corruption without performance regression

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-21 06:56:45 +01:00
Artem Vorotnikov
d8a009837b
Nuke preimage remnants (#2203) 2021-06-20 13:00:22 +07:00
Alex Sharov
a535e8f30c
switch to mdbx-go module, and db-tools to libmdbx submodule (#2199)
* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule
2021-06-19 21:30:12 +01:00
Alex Sharov
bc271f92ce
To break dependency to db implementation - move all db classes to "ethdb/kv" and leave in "ethdb" only interfaces (#2195) 2021-06-19 15:21:53 +07:00
Artem Vorotnikov
af836a6200
Nuke LMDB (#2167) 2021-06-16 13:57:58 +03:00
Artem Vorotnikov
0a4e0d1fa1
Various code simplificaitons (#2165)
* simplify randomaccount

* simplify execution unwind

* Remove standalone account accessors
2021-06-14 19:52:30 +01:00
ledgerwatch
2c1fe2a3a7
Restore TD records (#2164)
* Insert missing TD values

* Print missing TD records

* Fix typo

* Apply fix

* Fixes in execution stage

* Fix tx/sec output

* Advance exec

* Reset unwind

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-14 19:51:57 +01:00
ledgerwatch
0f64df626a
Util to dump state (#2159)
* Util to dump state

* Extract bodies info

* Fix for extractBodies

* Fix typo

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-14 07:34:36 +01:00
BitBaseBit
7ed337cdcc
Implemented panic handling, graceful shutdown and reporting for all goroutines that don't explicitly handle them. (#2153)
* implemented crash reporting for all goroutine panics that aren't handled explicitly

* implemented crash reporting for all goroutine panics that aren't handled explicitly

* changed node defaults back to originals after testing

* implemented panic handling for all goroutines that don't explicitly handle them, outputting the stack trace to a file in crashreports

* handling panics on all goroutines gracefully

* updated missing call

* error assignment

* implemented suggestions

* path.Join added

* implemented Evgeny's suggestions

* changed path.Join to filepath.Join for cross-platform

* added err check

* updated RecoverStackTrace to LogPanic

* updated closures

* removed call of common.Go to some goroutines

* updated scope capture

* removed testing files

* reverted back to original method, I feel like its less intrusive

* update filename for clarity
2021-06-13 17:41:39 +01:00
Alex Sharov
925d17a238
remove lmdb from hack, integration, state packages. remove lmdb dbtools (#2146) 2021-06-11 20:31:37 +07:00
ledgerwatch
73bb98f686
Move tests 6 (#2107)
* Move tests 6

* Compile fixes

* Fix lint

* Compile fixes

* Fix compile

* compile fixes

* Compile fix

* Fix consesus/clique

* Cleanup

* Add gas limit

* Print

* Prints

* More print

* Fix

* Reinstate TestGetBlockReceipts66

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-06 22:44:14 +01:00
Alex Sharov
1f13f73045
up version of uint256 lib (#2082)
* eip 1559_tx_constraints

* eip 1559_tx_constraints

* return error on uint256 overflow

* up uint256 version

* merge master

* fix master

Co-authored-by: ledgerwatch <akhounov@gmail.com>
2021-06-04 17:25:28 +01:00
ledgerwatch
67d0a2d3ab
Reproduce and fix trace_filter index problem (#2084)
* Intermediate

* Test for trace_filter

* Clean up

* Clean up

* clean up

* More to the test

* Hack

* Block parameter

* Block parameter

* use mdbx

* Print number of records

* Print keys

* Print index records

* Print block numbers

* Print values in tracecallset

* Print some more

* More printing

* Print froms and tos

* Print startBlock and endBlock

* Tests and fixes

* Cleanup

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-04 09:09:57 +01:00
Evgeny Danilenko
74847d77e6
Rename to Erigon (#2018)
* turbo-geth to erigon

* tg, turbo to erigon
2021-05-26 11:35:39 +01:00
Alex Sharov
545fe32f68
Mdbx - make it default db. Lazy buckets renaming. (#1982) 2021-05-22 10:03:02 +07:00
Alex Sharov
0be3044b7e
rename (#1978)
* rename

* rename "make grpc"

* rename "abi bindings templates"

* rename "abi bindings templates"
2021-05-20 19:25:53 +01:00
ledgerwatch
5f048bc82f
Wrong receipt problem - utility to recover DB (#1970)
* Wrong receipt hack

* Advance5

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-20 14:28:38 +01:00
b00ris
565a4250d6
Snapshot sync headers stage (#1836)
* save state

* snapshot update works

* save state

* snapshot migrator

* tx test

* save state

* migrations stages refactor

* refactor snapshot migrator

* compilation fixed

* integrate snapshot migrator

* goerli sync headers

* debug async snapshotter on goerly

* move verify headers, remove experiments, fix remove old snapshot

* save state

* refactor snapshotsync injection

* fix deadlock

* replace snapshot generation stage logic to migrate method

* change done for body snapshot

* clean

* clean&&change deleted value

* clean

* fix hash len

* fix hash len

* remove one of wrap methods, add remove snapshots on start

* add err check

* fix shadowing

* stages unwind order debug

* matryoshka experiments

* steam test

* fix build

* fix test

* fix lint

* fix test

* fix test datarace

* add get test

* return timeout

* fix mdbx overlap

* fix after merge

* change epoch size

* clean todo

* fix

* return testdata

* added return from sndownloader gorutine

* fix review comments

* Fix

* More info

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-08 09:45:40 +01:00
Alex Sharov
c7eedf0091
Add kv to stage settings (because it doesn't change in runtime, tx will be passed as Stage func argument) (#1872)
* add kv to config

* move forward tx use

* move forward tx use

* gc stat

* gc stat

* changeset writer - move to tx

* clean

* clean

* clean

* clean

* clean
2021-05-04 13:36:03 +01:00
Artem Vorotnikov
2148885637
Simplify ChangeSet implementation and tests (#1878)
* Move storage changeset code into one file

* Simplify storage changeset tests

* simplify more

* more

* more

* more

* move account changeset into one file

* simplify account changeset test

* NewStorageChangesetPlain -> NewStorageChangeSet

* EncodeStoragePlain -> EncodeStorage

* StorageChangeSetPlain -> StorageChangeSet

* AccountChangeSetPlain -> AccountChangeSet

* more

* more

* more

* Simplify EncodeStorage

* more

* Simplify FromDBFormat

* more rename

* rename changeset buckets (constant names only)
2021-05-04 13:34:08 +01:00
ledgerwatch
4303465e96
Address dump utility (#1856)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-01 21:04:02 +01:00
Alex Sharov
aacc457ea8
rawdb deprecated methods with Database interface (#1854) 2021-05-01 14:42:23 +07:00
Alex Sharov
3af0b9b591
add hashstate, trie, history, logIndex stages (#1799)
* add hashstate and trie stages

* add history stages

* add history stages

* add history stages

* add logindex stage

* add logindex stage

* add other stages
2021-04-25 09:29:39 +01:00
ledgerwatch
12cde41772
Aleut support (Eip1559) (#1704)
* Where I am at

* Refactoring of transaction types

* More refactoring

* Use Homested signer in rpc daemon

* Unified signer

* Continue unified signer

* A bit more

* Fixes and down the rabbit hole...

* More tx pool fixes

* More refactoring fixes

* More fixes'

* more fixes

* More fixes

* More compile fixes

* More RLP hand-writing

* Finish RLP encoding/decoding of transactions

* Fixes to header encoding, start on protocol packets

* Transaction decoding

* Use DecodeTransaction function

* Decoding BlockBodyPacket

* Encode and decode for pool txs

* Start fixing tests

* Introduce SigningHash

* Fixes to SignHash

* RLP encoding fixes

* Fixes for encoding/decoding

* More test fixes

* Fix more tests

* More test fixes

* More test fixes

* Fix core tests

* More fixes for signer

* Fix for tx

* Fixes to string encoding/size

* Fix eip2930 test

* Fix rest of ./tests

* More fixes

* Fix compilation

* More test fixes

* More test fixes

* Test fixes

* More fixes

* Reuse EncodingSize in EncodeRLP for accessList

* Rearrange things in dynamic fee tx

* Add MarshalBinary

* More fixes

* Make V,R,S non-pointers

* More NPE fixes

* More fixes

* Receipt fixes

* Fix core/types

* Fix ./eth

* More compile fixes for tests

* More test fixes

* More test fixes

* Try to see lint errors better

* Try to see lint errors better

* Fix lint

* Debugging eip1559 test

* Fix TestEIP1559Transition test

* Fix NewBlockPacket encoding/decoding

* Fix calculation of TxHash

* Fix perf problem with senders

* Update aleut config values

* Try adding static peers

* Add staticpeers to defaul flags

* Change aleut networkID

* Fix test

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-22 18:11:37 +01:00
Artem Vorotnikov
9a3e47a498
Remove StateCache where unused (#1770) 2021-04-21 16:14:23 +07:00
Evgeny Danilenko
17c07c50a5
Move clique buckets to separate DB (#1703)
* debug

* debug

* it works

* rename clique bucket

* remove genesis special case

* copy snapshot

* remove debug

* migration

* debug

* regenerate snapshots

* simplify

* regeneration

* after merge

* tests

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-04-19 22:58:05 +01:00
Alex Sharov
4ef5e4b2f6
Continue move to rwtx (#1722) 2021-04-15 12:23:10 +07:00
Artem Vorotnikov
b71c6323a0
KVGetter, replace old geth interfaces with it (#1676) 2021-04-05 20:04:58 +07:00
Alex Sharov
07e96fc055
clean (#1677) 2021-04-05 14:48:16 +07:00
Artem Vorotnikov
d66555268d
Harmonize DB APIs (#1669) 2021-04-03 09:26:00 +03:00
Alex Sharov
9a9c5b298a
bitmapdb to use rwtx (#1661) 2021-04-03 08:52:45 +07:00
Alex Sharov
fbfc43da7e
etl to use rwtx (#1660) 2021-04-02 18:22:25 +07:00
Alex Sharov
ef77eb3053
tx.RwCursor() to return err (first step of removing lazy cursors) (#1656) 2021-04-02 13:36:49 +07:00
Alex Sharov
b4a2d823c4
old loader remove v1 (#1641) 2021-03-31 14:43:12 +07:00
ledgerwatch
08174e8e2d
Fix compilation error in hack.go (#1633)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-03-30 13:13:51 +01:00
ledgerwatch
6af1288831
Add readAccount hack (#1629)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-03-30 12:29:57 +01:00