Commit Graph

240 Commits

Author SHA1 Message Date
Evgeny Danilenko
4cd72c8328
Keep readonly value while changing interpreters back and forth (#2508)
* restore TEVM

* fuzzing and property based

* comment

* lint

* stack callback into defer ater checking an error

* sequential tests
2021-08-10 09:48:56 +07:00
ledgerwatch
350d7fe7a9
Adding support for vmTrace into trace_ routines (#2497)
* VmTrace

* Fixes to gasCall, beginning of vmTrace

* Fix opcode tracer

* Add mem and store, enable vmTrace for all methods

* Fix lint

* More nuances and debugging|

* More fixes

* Fix for trace_callMany

* Fix for trace_callMany

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-08-08 13:28:03 +01:00
Alex Sharov
6c2beb273e
Remove tsdb dependency (#2503)
* remove tsdb dep

* remove tsdb dep

* remove tsdb dep

* remove tsdb dep
2021-08-08 16:56:54 +07:00
Alex Sharov
6bd44eb26c
move kv to erigon-lib (#2467) 2021-07-29 18:53:13 +07:00
Alex Sharov
5069558752
Apache licensed logger (#2460) 2021-07-29 17:23:23 +07:00
Alex Sharov
21cb7befa4
finish remove bucket suffix (#2458) 2021-07-28 10:43:51 +07:00
Alex Sharov
838e5f9ef2
Move bucket constants into kv package, move kv interface to kv package (#2455) 2021-07-28 09:47:38 +07:00
Alex Sharov
63a6c45ef1
Remove some db interfaces (#2452) 2021-07-27 19:35:32 +07:00
Alex Sharov
6801208497
move some tests to kv interface (#2438) 2021-07-24 14:14:11 +07:00
Alex Sharov
f735997fa1
Sokol v0: remove unused parameters and methods (#2435) 2021-07-24 11:04:07 +07:00
Alex Sharov
d2552196ad
Sokol v0: support for first epoch-set transition (#2411) 2021-07-21 18:13:26 +07:00
Alex Sharov
c3e1cfdac8
Pruning for: exec, log_index, tx_lookup, history stages (#2399)
* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* add tvm flag

* save

* db migration for storage mode
add flag --prune=
remove flag --storage-mode=
add flag --experiments=tevm,...
rename integration set_storage_mode to set_prune

* fix

* forward move of stages must skip everything before PruneTo

* keep in db progress of prune method

* keep in db progress of prune method

* simplify logs

* simplify logs

* simplify logs

* fix test

* simplify logs

* simplify logs

* simplify logs

* simplify logs

* remove callTraceSet as dupsort
use etl transform for txlookup prune
remove some logs

* cleanup tests a bit

* print_stages and eth_sync to show prune progress

* fix print_stages

* add readme about --prune.to flag

* more docs

* add --prune.history.older and other flags support

* fix migration on empty db

* better toString

* better toString
2021-07-20 21:03:19 +01:00
racytech
5e7bf5173d
Change freelist guide illustrations from lmdb to mdbx (#2391)
* lmdb to mdbx

* nolint added

* linting issues fixed

* linting issues fixed 2

* linting issues fixed 3

Co-authored-by: Kairat Abylkasymov <racytech0x0gmail.com>
2021-07-18 18:44:47 +01:00
Alex Sharov
4c53f51767
Remove "persistent unwind" concept (#2388) 2021-07-17 21:14:51 +07:00
Alex Sharov
a79b3cd6eb
unify stage params (#2384) 2021-07-17 09:09:56 +07:00
Alex Sharov
9f6ef74adc
Linter: to check tx.Rollback() by ruleguard (#2383) 2021-07-16 20:23:54 +07:00
Alex Sharov
5f55b6b89c
Remove stage builder and use ID's in unwindOrder (#2381) 2021-07-16 20:17:40 +07:00
Alex Sharov
557d082c51
Sokol v0: epoch support (#2348)
* remove ctx from state writer interface

* call me baby

* save

* save

* can process block 1

* can process block 1

* can process block 1

* can process block 1

* save

* save

* parse logs logic

* cleans

* cleans

* cleans
2021-07-12 16:27:25 +01:00
Alex Sharov
a78cb54132
step towards kv interface (#2311) 2021-07-07 23:15:49 +07:00
Alex Sharov
a68b5ba361
Replace ChainConfig.WithEIPsFlags by go-ethereum's ChainConfig.Rules (#2304)
* use chainRules

* use chainRules

* use chainRules

* use chainRules

* use chainRules
2021-07-05 19:52:50 +01:00
ledgerwatch
9023f2a29c
Investigate slow receipt repair (#2306)
* Collect data about broken receipts

* Actually collect data

* Fix CBOR problem more efficiently

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-05 19:14:58 +01:00
ledgerwatch
63ccabc72f
Receipt repair migration (#2291)
* Fixes in rpctest and rpcdaemon for debug_traceTransaction

* Fix for opcode tracer

* Tool to fix receipts

* Better mechanism for detecting broken receipts

* Fixes

* Introduce receipt_repair migration

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-04 14:48:13 +01:00
ledgerwatch
f5e42b1e7a
Fixes in rpctest and rpcdaemon for debug_traceTransaction, and opcode tracer, tool for repairing broken receipts (#2284)
* Fixes in rpctest and rpcdaemon for debug_traceTransaction

* Fix for opcode tracer

* Tool to fix receipts

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-03 14:34:23 +01:00
Alex Sharov
6a5f86423c
Remove "numgc" from logs (#2275) 2021-07-03 14:44:23 +07:00
ledgerwatch
ba55611f29
rpctest for eth_getTransactionReceipt (#2271)
* rpctest bench for eth_getTransationReceipt

* Add blockHash to re-generated receipts

* Modify scanReceipts

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-02 14:34:20 +01:00
Alex Sharov
3d6d45a82f
remove subtrie loader and dcmp (#2258) 2021-06-30 10:31:36 +07:00
ledgerwatch
7baf3ec484
Move secp256k1 into a separate go module, hook up erigon-lib (#2239)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-26 15:03:05 +01:00
ledgerwatch
adf4766191
Scan transaction to select test cases (#2225)
* Scan txs

* Find examples with access lists

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-23 23:34:17 +01:00
ledgerwatch
5ad449ef22
Migration to fix receipts (broken due to change of CBOR codec) (#2221)
* Scan receipts

* Not check tx type before Berlin

* Reverse

* scan receipt code in the migration

* Turn migration on

* Apply migration

* Fix

* Print last

* Remove last

* Not remove last

* Not updata in migration

* no cursors in migration

* Handle empty blocks

* Handle empty blocks

* Use the same code in hack

* Print

* Print

* Save

* Fix

* Fix lint

* Fix lint

* Fix for empty DB

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-23 15:05:16 +01:00
Alex Sharov
59d05dc5fe
hide file exists err (#2218) 2021-06-22 11:09:45 +01:00
ledgerwatch
ab685e90d9
Improvement to the trimTxs (#2215)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-22 08:35:34 +01:00
ledgerwatch
a79ed2fe0b
Utilities to fix DB corruption (#2214)
* Detect broken ancestry

* Print last canonical headers

* Add backExec

* Cleanup

* Add unwind util

* Add trimTxs utility

* Actually delete tx records

* Avoid infinite loop

* Change strategy

* Close cursor after commmit

* Fix RwBegin issue

* 10m records per transaction

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-21 21:52:51 +01:00
ledgerwatch
04943eaa52
Fix for state corruption without performance regression (#2208)
* extractHeaders

* Print diff

* Fix for state corruption without performance regression

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-21 06:56:45 +01:00
Artem Vorotnikov
d8a009837b
Nuke preimage remnants (#2203) 2021-06-20 13:00:22 +07:00
Alex Sharov
a535e8f30c
switch to mdbx-go module, and db-tools to libmdbx submodule (#2199)
* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule
2021-06-19 21:30:12 +01:00
Alex Sharov
bc271f92ce
To break dependency to db implementation - move all db classes to "ethdb/kv" and leave in "ethdb" only interfaces (#2195) 2021-06-19 15:21:53 +07:00
Artem Vorotnikov
af836a6200
Nuke LMDB (#2167) 2021-06-16 13:57:58 +03:00
Artem Vorotnikov
0a4e0d1fa1
Various code simplificaitons (#2165)
* simplify randomaccount

* simplify execution unwind

* Remove standalone account accessors
2021-06-14 19:52:30 +01:00
ledgerwatch
2c1fe2a3a7
Restore TD records (#2164)
* Insert missing TD values

* Print missing TD records

* Fix typo

* Apply fix

* Fixes in execution stage

* Fix tx/sec output

* Advance exec

* Reset unwind

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-14 19:51:57 +01:00
ledgerwatch
0f64df626a
Util to dump state (#2159)
* Util to dump state

* Extract bodies info

* Fix for extractBodies

* Fix typo

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-14 07:34:36 +01:00
BitBaseBit
7ed337cdcc
Implemented panic handling, graceful shutdown and reporting for all goroutines that don't explicitly handle them. (#2153)
* implemented crash reporting for all goroutine panics that aren't handled explicitly

* implemented crash reporting for all goroutine panics that aren't handled explicitly

* changed node defaults back to originals after testing

* implemented panic handling for all goroutines that don't explicitly handle them, outputting the stack trace to a file in crashreports

* handling panics on all goroutines gracefully

* updated missing call

* error assignment

* implemented suggestions

* path.Join added

* implemented Evgeny's suggestions

* changed path.Join to filepath.Join for cross-platform

* added err check

* updated RecoverStackTrace to LogPanic

* updated closures

* removed call of common.Go to some goroutines

* updated scope capture

* removed testing files

* reverted back to original method, I feel like its less intrusive

* update filename for clarity
2021-06-13 17:41:39 +01:00
Alex Sharov
925d17a238
remove lmdb from hack, integration, state packages. remove lmdb dbtools (#2146) 2021-06-11 20:31:37 +07:00
ledgerwatch
73bb98f686
Move tests 6 (#2107)
* Move tests 6

* Compile fixes

* Fix lint

* Compile fixes

* Fix compile

* compile fixes

* Compile fix

* Fix consesus/clique

* Cleanup

* Add gas limit

* Print

* Prints

* More print

* Fix

* Reinstate TestGetBlockReceipts66

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-06 22:44:14 +01:00
Alex Sharov
1f13f73045
up version of uint256 lib (#2082)
* eip 1559_tx_constraints

* eip 1559_tx_constraints

* return error on uint256 overflow

* up uint256 version

* merge master

* fix master

Co-authored-by: ledgerwatch <akhounov@gmail.com>
2021-06-04 17:25:28 +01:00
ledgerwatch
67d0a2d3ab
Reproduce and fix trace_filter index problem (#2084)
* Intermediate

* Test for trace_filter

* Clean up

* Clean up

* clean up

* More to the test

* Hack

* Block parameter

* Block parameter

* use mdbx

* Print number of records

* Print keys

* Print index records

* Print block numbers

* Print values in tracecallset

* Print some more

* More printing

* Print froms and tos

* Print startBlock and endBlock

* Tests and fixes

* Cleanup

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-04 09:09:57 +01:00
Evgeny Danilenko
74847d77e6
Rename to Erigon (#2018)
* turbo-geth to erigon

* tg, turbo to erigon
2021-05-26 11:35:39 +01:00
Alex Sharov
545fe32f68
Mdbx - make it default db. Lazy buckets renaming. (#1982) 2021-05-22 10:03:02 +07:00
Alex Sharov
0be3044b7e
rename (#1978)
* rename

* rename "make grpc"

* rename "abi bindings templates"

* rename "abi bindings templates"
2021-05-20 19:25:53 +01:00
ledgerwatch
5f048bc82f
Wrong receipt problem - utility to recover DB (#1970)
* Wrong receipt hack

* Advance5

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-20 14:28:38 +01:00
b00ris
565a4250d6
Snapshot sync headers stage (#1836)
* save state

* snapshot update works

* save state

* snapshot migrator

* tx test

* save state

* migrations stages refactor

* refactor snapshot migrator

* compilation fixed

* integrate snapshot migrator

* goerli sync headers

* debug async snapshotter on goerly

* move verify headers, remove experiments, fix remove old snapshot

* save state

* refactor snapshotsync injection

* fix deadlock

* replace snapshot generation stage logic to migrate method

* change done for body snapshot

* clean

* clean&&change deleted value

* clean

* fix hash len

* fix hash len

* remove one of wrap methods, add remove snapshots on start

* add err check

* fix shadowing

* stages unwind order debug

* matryoshka experiments

* steam test

* fix build

* fix test

* fix lint

* fix test

* fix test datarace

* add get test

* return timeout

* fix mdbx overlap

* fix after merge

* change epoch size

* clean todo

* fix

* return testdata

* added return from sndownloader gorutine

* fix review comments

* Fix

* More info

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-08 09:45:40 +01:00