Commit Graph

333 Commits

Author SHA1 Message Date
TBC Dev
bf06095811
hack: Allow blockTotal <= 0 as offset from head (#3818) 2022-04-05 08:07:07 +07:00
Alex Sharov
8b03582558
compressor: log lvl #408
* save

* save
2022-04-01 10:44:30 +07:00
battlmonstr
47df98a499
Use "err" key for logging errors. (#3632)
log.Warn/Error uses "err" key to log errors in most places.
This renames "error" to "err" in some places to adhere to this convention.
2022-03-01 15:40:51 +00:00
ledgerwatch
ef6013faff
Automation for gathering erigon2 history data (#3501)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-12 22:16:06 +00:00
ledgerwatch
2074b98568
Tools for gathering data on erigon2 history and bitmaps (#3488)
* Tools for gathering data on erigon2 history and bitmaps

* Always start with 0

* Always start with 0

* Clean up commitment files, trace block

* traceblock

* Add tracing to checkChangeSets

* Fixes to DeleteAccount

* Update to latest erigon-lib

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-12 15:48:38 +00:00
ledgerwatch
079a177614
[erigon2] Computing data dependencies between transactions (#3290)
* Use ibs per transaction instead of per block

* Remove copy functions, introduce balance increase map

* Fix test

* Fix lint

* Introduce balanceIncrease journal action

* Introduce balanceIncrease journal action

* Simplifications

* Fix

* Reset

* Simplify

* Fix

* Fixes

* Fix for reading nil accounts

* Fix edge cases with reverts

* Clean up test output

* Fix transfers

* Handle special case of ripemd

* Try with fixed compress, only commit every commfreq blocks

* Updates

* Updates

* Update to fixed erigon-lib

* Update to erigon-lib

* Remove stop

* Update to latest erigon-lib

* fixup

* Cleanup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-24 23:12:25 +00:00
Alex Sharov
9188705849
Switch to parallel (#3285)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
2022-01-18 15:01:36 +07:00
ledgerwatch
3655916eb9
Junk db util (#3284)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-18 12:03:23 +07:00
Alex Sharov
f9983edc76
Snapshots: ParallelCompressor class, DecompressedFile class (#3265) 2022-01-17 16:57:57 +07:00
Alex Sharov
359af9e49d
Snapshots: files list (#3264) 2022-01-15 11:23:37 +07:00
Dmitry Savonin
a49d409457
Full BSC support with validator mode (#3233)
* migrated consensus and chain config files for bsc support

* migrated more files from bsc

* fixed consensus crashing

* updated erigon lib for parlia snapshot prefix

* added staticpeers for bsc

* [+] added system contracts
[*] fixed bug with loading snapshot
[+] enabled gas bailout
[+] added fix to prevent syncing more than 1000 headers (for testing only)
[*] fixed bug with crashing sender recover sometimes

* migrated system contract calls

* [*] fixed bug with returning mutable balance object
[+] migrated lightclient contracts from bsc
[*] fixed parlia consensus config param

* [*] fixed tendermint deps

* [+] added some logs

* [+] enabled bsc forks
[*] fixed syscalls from coinbase
[*] more logging

* Fix call sys contract gas calculation

* [*] fixed executing system transactions

* [*] enabled receipt hash, gas and bloom filter checks

* [-] removed some logging scripts
[*] set header checkpoint to 10 million blocks (for testing forks)

* [*] fixed bug with commiting dirty inter block state state after system transaction execution
[-] removed some extra logs and comments

* [+] added chapel and rialto testnet support

* [*] fixed chapel allocs

* [-] removed 6 mil block limit for headers sync

* Fix hardforks on chapel and other testnets

* [*] fixed header sync issue after merge

* [*] tiny code cleanup

* [-] removed some comments

* [*] increased mdbx map size to 4 TB

* [*] increased max chaindata size to 6 tb

* [*] bring more compatibility with origin erigon and some code cleanup

* [+] added support of validator mode for BSC chain

* [*] enable private key load for bsc, rialto and chapel chains

* [*] fixed running BSC validator node

* Fix the branch list

* [*] tiny fixes for linter

* [*] formatted imports for core and parlia packages

* [*] fixed import rules in other files

* Revert "[*] formatted imports for core and parlia packages"

This reverts commit c764b58b34fedc2b14d69458583ba0dad114f227.

* [*] changed import rules in more packages

* [*] fixed type mismatch in hack command

* [*] fixed crash on new epoch, enabled bootstrap flags

* [*] fixed linter errors

* [*] fixed missing err check for syscalls

* [*] now BSC implementation is fully compatible with erigon original sources

* Revert "Add chain config and CLI changes for Binance Smart Chain support (#3131)"

This reverts commit 3d048b7f1a.

* Revert "Add Parlia consensus engine for Binance Smart Chain support (#3086)"

This reverts commit ee99f17fbe.

* [*] fixed several issues after merge

* [*] fixed integration compilation

* Revert "Fix the branch list"

This reverts commit 8150ca57e5f2707a84a9f6a1c5b809b7cc84547b.

* [-] removed receipt repair migration

* [*] fixed parlia fork numbers output

* [*] bring more devel compatibility, fixed bsc address list for access list calculation

* [*] fixed bug with commiting state transition for bad blocks in BSC

* [*] fixed bsc changes apply for integration command and updated config print for parlia

* [*] fixed bug with applying bsc forks for chapel and rialto testnet chains
[*] let's use finalize and assemble for mining to  let consensus know for what it's finalizing block

* Fix compilation errors in hack.go

* Fix lint

* reset changes in erigon-snapshots to devel

* Remove unrelated changes

* Fix embed

* Remove more unrelated changes

* Remove more unrelated changes

* Restore clique and aura miner config

* Refactor interfaces not to use slice pointers

* Refactor parlia functions to return tx and receipt instead of dealing with slices

* Fix for header panic

* Fix lint, restore system contract addresses

* Remove more unrelated changes, unify GatherForks

Co-authored-by: Dmitry Ivanov <convexman18@gmail.com>
Co-authored-by: j75689 <j75689@gmail.com>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-14 19:06:35 +00:00
ledgerwatch
ea6dbd49e8
[erigon2] Running with commitment (#3203)
* [erigon2] Running with commitment

* [erigon2] Running with commitment

* [erigon2] Running with commitment

* go mod tidy

* [erigon2] Running with commitment

* More

* Debug

* fix

* Fix

* state root command

* More fixes

* Fix

* Progress to 164735

* Only trace when failing

* fix for firstInsert

* Over block 1.36m

* Update

* fix to deleteAccount

* Fixes for plainKeys

* Undo printing

* No changeset generation by default

* Print block number on interrupt, fix lint

* Fix lint

* Open history DB as read only

* Print error

* Open non read only

* Readonly again

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-14 16:02:44 +00:00
Alex Sharov
2bcc40fd3f
Snapshots: create .dat in tmpdir (#3222) 2022-01-09 14:43:58 +07:00
Alex Sharov
4ef541f5dc
Snapshots: tx lookup in RPC from snapshots (#3214) 2022-01-07 20:52:38 +07:00
Alex Sharov
eed7475af0
Snapshot: move parallel compression to erigon-lib (#3211) 2022-01-06 14:13:09 +07:00
Alex Sharov
5cac9fbff5
Move snapshot creation from hack.go to "erigon snapshot create" cli sub-commmand (#3177) 2021-12-27 14:59:29 +07:00
Alex Sharov
82753a6cda
add words count in .seg (breaking change in snapshot format) (#3140) 2021-12-21 10:48:34 +07:00
ledgerwatch
9f861e1b3f
Check upgrade/downgrade of database schema version to prevent accidental upgrade, remove migrations (#3133)
* Check upgrade/downgrade

* Remove old migrations

* Fix compilation

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-12-15 21:03:04 +00:00
Alex Sharov
ecb10e8548
Snapshots download and seed (#3117)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* Squashed 'interfaces/' content from commit e5b1945d0

git-subtree-dir: interfaces
git-subtree-split: e5b1945d02da7a7f00e2289034ee90a6edd60184

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
2021-12-14 10:13:17 +00:00
brendan-kelly
ee99f17fbe
Add Parlia consensus engine for Binance Smart Chain support (#3086)
* Add Parlia consensus engine for Binance Smart Chain support

* Leave RamanujanBlock as nil in params/config.go

* Run `go fmt` on files needing it

* Add comment for PoSA

* Remove empty branches and ineffectual assignments in parlia.go

* Remove commented imports

* Fix compilation error

* Remove EIP155Signer in transaction_signing.go

* Fix compilation issue

* Fix go fmt issues

* Remove Ramanujan from print statement

* Remove references to EthAPIBackend approach

* Fix Finalize method across consensus engines

* Run go fmt

* More linting

* Remove more changes

* remove a comment

* Remove unneeded hashing function

* Remove bytes check and fix actual vs expected mistake
2021-12-11 00:07:10 +00:00
Alex Sharov
178fd1931d
Genesis sync from existing snapshots (#3087) 2021-12-05 09:03:08 +07:00
TBC Dev
57d641b6f9
Avoid redundant Block.Header() deep-copy (#3050)
* Add separate Block.Nonce() and Block.NonceU64()

* Add Block.Seal()

* Avoid redundant Block.Header() deep-copy

* Add warning comment for Block.Header()
2021-11-29 08:52:36 +00:00
Alex Sharov
8104a90993
Snapshot: exec and trie to support snapshot files (#3033) 2021-11-29 10:43:19 +07:00
Alex Sharov
b3e8a1a02a
hack: remove createIdx and txLookup methods (#3000) 2021-11-21 12:06:36 +07:00
Alex Sharov
e55256296b
snapshots: read block from snapshots, add sender to txs file (#2996) 2021-11-21 10:32:14 +07:00
Alex Sharov
d4850b6adc
Hack: dump bodies and headers (#2994) 2021-11-20 09:41:01 +07:00
Alex Sharov
2a08dbd3d9
Recsplit: move files read/write helpers to erigon-lib (#2993) 2021-11-19 12:12:25 +07:00
alex.sharov
c1cbb21f3d save 2021-11-11 10:39:44 +07:00
Alex Sharov
f31a76baea
Compress: add maxPatternLen=64 (#2943) 2021-11-10 22:10:01 +07:00
Alex Sharov
e670f3930f
Lcp must be int32 (#2934) 2021-11-09 17:47:12 +07:00
Alex Sharov
3842351293
move DictAggregator to erigon-lib (#2933) 2021-11-09 10:12:25 +07:00
Alex Sharov
deed48c01a
Small step towards torrent downloader (#2929) 2021-11-08 20:40:56 +07:00
Alex Sharov
9ab2bbee1b
recsplit: single offset bucket (#2928) 2021-11-08 14:27:47 +07:00
Alex Sharov
194330fa4b
Recsplit: preserve immutability threshold (#2926) 2021-11-07 13:01:33 +07:00
Alex Sharov
49ff8ff166
Recsplit: collision typed error (#2925) 2021-11-07 09:58:20 +07:00
alex.sharov
e802e43fa8 add pprof http to hack 2021-11-07 09:16:25 +07:00
Alex Sharov
a5bbe82a59
ETL: use logPrefix as suffix of tmp files (#2921) 2021-11-05 17:19:44 +07:00
Alex Sharov
686721b6c6
EliasFano: fix jump calculation, fuzzing to trigger jump logic (#2912) 2021-11-04 13:25:34 +07:00
Alex Sharov
70461e4e68
Recsplit: add first byte of hash to txn payload (#2902) 2021-11-02 14:35:09 +07:00
Alex Sharov
6f055e7adc
Recsplit: call ef.Build and set ef.prevOffset (#2897) 2021-11-01 09:24:06 +07:00
Alex Sharov
3d63c93b3d
Recsplit: add hack recsplitLookup for test (#2895) 2021-10-31 10:20:31 +07:00
ledgerwatch
04f7fe468a
hack to print headers (#2892)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-10-30 12:27:30 +01:00
Alex Sharov
83593c28c2
recsplit: bigger bufio buffer (#2867) 2021-10-26 11:19:56 +07:00
Alex Sharov
5d7904c133
erl.collector - move logPrefix to constructor (#2866) 2021-10-25 15:09:43 +07:00
ledgerwatch
2dec76123c
Use decompressor from erigon-lib and experiments with tx lookup (#2824)
* Use decompressor

* Use decompressor

* Close compressor

* Upgrade to latest erigon-lib

* Fix compile errors

* Fix lint

* Remove count from files

* Remove count from files

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-10-16 22:19:13 +01:00
ledgerwatch
21d024b06a
More mature experiments with the state and transaction compression (#2811)
* Change to use Patten inside patricia tree

* Huffman codes for compression

* Huffman coding of positions, generate tx files to test compression

* Split state into 3 files, relative positions, average decoding time

* Small optimisations of decompression

* Optimise intermediate compression

* Update erigon-lib ref

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-10-11 21:05:45 +01:00
ledgerwatch
7e4996bb09
Reworkings of state compression experiments (#2790)
* Changes

* Progress

* Another way

* More

* More

* Produce encoding

* Add uncoded characters

* cleanup

* Add sortdict

* Fixes

* Use patricia from erigon-lib

* Cleanup

* Switch to dynamic programming, optimise allocations in FindMatches

* Optimise allocations

* Reduce allocations

* Switch to main branch of erigon-lib, reduce allocations further

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-10-06 19:58:12 +01:00
Andrea Lanfranchi
c913f35c2e
Inner errors (#2774) 2021-10-04 22:16:52 +07:00
ledgerwatch
0d34a5b9c5
[State sync experiment] Building dictionary for state file compression (#2721)
* Kasai experiments

* Use divsufsort instead of standard lib

* Refactor experiments, split dictionary building into chunks

* Fixes

* Estimate compression, sample

* More stats

* Parallelise

* Fix lint

* dictionary aggregator

* Actual replacement

* Fixes, separate dictionary processing

* Test file

* More correct dictionary, decompression

* Use dynamic programming to reduce dictionary

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-09-27 21:32:53 +01:00
Alex Sharov
3de50637cd
remove changeset.Walk func (#2716) 2021-09-22 07:54:29 +07:00
ledgerwatch
a9e4cbc83e
Recsplit experiments (#2706)
* Only do 2m

* Optimisation

* Flexible count

* Count properly

* count

* Bijection check

* Add keys

* Print bucket info

* Collision map

* Collision map

* Add tracing

* Tracing

* Fix split

* No trace, print lookup time

* Not override count if block=0

* fix count

* Go mod tidy

* Fix lint

* Measurements

* Cleanup

* Latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-09-21 07:53:22 +01:00
Alex Sharp
c519b04551 Revert "state mphf experiment"
This reverts commit b0182d7834.
2021-09-19 11:41:51 +01:00
Alex Sharp
b0182d7834 state mphf experiment 2021-09-19 11:40:23 +01:00
Alex Sharov
8d9e058eb7
Pool: chain config, non-mainnet (#2702) 2021-09-18 20:58:23 +07:00
Alex Sharov
3193f95152
Apply mined block (#2650) 2021-09-09 14:26:58 +07:00
Alex Sharov
c7c3a5932a
rename ReadBody to ReadBodyWithTransactions to make it obvious that it's heavier then just ReadBody (#2534)
* rename ReadBody to ReadBodyWithTransactions

* rename ReadBody to ReadBodyWithTransactions
2021-08-15 17:08:28 +07:00
Evgeny Danilenko
4cd72c8328
Keep readonly value while changing interpreters back and forth (#2508)
* restore TEVM

* fuzzing and property based

* comment

* lint

* stack callback into defer ater checking an error

* sequential tests
2021-08-10 09:48:56 +07:00
ledgerwatch
350d7fe7a9
Adding support for vmTrace into trace_ routines (#2497)
* VmTrace

* Fixes to gasCall, beginning of vmTrace

* Fix opcode tracer

* Add mem and store, enable vmTrace for all methods

* Fix lint

* More nuances and debugging|

* More fixes

* Fix for trace_callMany

* Fix for trace_callMany

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-08-08 13:28:03 +01:00
Alex Sharov
6c2beb273e
Remove tsdb dependency (#2503)
* remove tsdb dep

* remove tsdb dep

* remove tsdb dep

* remove tsdb dep
2021-08-08 16:56:54 +07:00
Alex Sharov
6bd44eb26c
move kv to erigon-lib (#2467) 2021-07-29 18:53:13 +07:00
Alex Sharov
5069558752
Apache licensed logger (#2460) 2021-07-29 17:23:23 +07:00
Alex Sharov
21cb7befa4
finish remove bucket suffix (#2458) 2021-07-28 10:43:51 +07:00
Alex Sharov
838e5f9ef2
Move bucket constants into kv package, move kv interface to kv package (#2455) 2021-07-28 09:47:38 +07:00
Alex Sharov
63a6c45ef1
Remove some db interfaces (#2452) 2021-07-27 19:35:32 +07:00
Alex Sharov
6801208497
move some tests to kv interface (#2438) 2021-07-24 14:14:11 +07:00
Alex Sharov
f735997fa1
Sokol v0: remove unused parameters and methods (#2435) 2021-07-24 11:04:07 +07:00
Alex Sharov
d2552196ad
Sokol v0: support for first epoch-set transition (#2411) 2021-07-21 18:13:26 +07:00
Alex Sharov
c3e1cfdac8
Pruning for: exec, log_index, tx_lookup, history stages (#2399)
* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* add tvm flag

* save

* db migration for storage mode
add flag --prune=
remove flag --storage-mode=
add flag --experiments=tevm,...
rename integration set_storage_mode to set_prune

* fix

* forward move of stages must skip everything before PruneTo

* keep in db progress of prune method

* keep in db progress of prune method

* simplify logs

* simplify logs

* simplify logs

* fix test

* simplify logs

* simplify logs

* simplify logs

* simplify logs

* remove callTraceSet as dupsort
use etl transform for txlookup prune
remove some logs

* cleanup tests a bit

* print_stages and eth_sync to show prune progress

* fix print_stages

* add readme about --prune.to flag

* more docs

* add --prune.history.older and other flags support

* fix migration on empty db

* better toString

* better toString
2021-07-20 21:03:19 +01:00
Alex Sharov
4c53f51767
Remove "persistent unwind" concept (#2388) 2021-07-17 21:14:51 +07:00
Alex Sharov
5f55b6b89c
Remove stage builder and use ID's in unwindOrder (#2381) 2021-07-16 20:17:40 +07:00
Alex Sharov
557d082c51
Sokol v0: epoch support (#2348)
* remove ctx from state writer interface

* call me baby

* save

* save

* can process block 1

* can process block 1

* can process block 1

* can process block 1

* save

* save

* parse logs logic

* cleans

* cleans

* cleans
2021-07-12 16:27:25 +01:00
Alex Sharov
a78cb54132
step towards kv interface (#2311) 2021-07-07 23:15:49 +07:00
Alex Sharov
a68b5ba361
Replace ChainConfig.WithEIPsFlags by go-ethereum's ChainConfig.Rules (#2304)
* use chainRules

* use chainRules

* use chainRules

* use chainRules

* use chainRules
2021-07-05 19:52:50 +01:00
ledgerwatch
9023f2a29c
Investigate slow receipt repair (#2306)
* Collect data about broken receipts

* Actually collect data

* Fix CBOR problem more efficiently

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-05 19:14:58 +01:00
ledgerwatch
63ccabc72f
Receipt repair migration (#2291)
* Fixes in rpctest and rpcdaemon for debug_traceTransaction

* Fix for opcode tracer

* Tool to fix receipts

* Better mechanism for detecting broken receipts

* Fixes

* Introduce receipt_repair migration

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-04 14:48:13 +01:00
ledgerwatch
f5e42b1e7a
Fixes in rpctest and rpcdaemon for debug_traceTransaction, and opcode tracer, tool for repairing broken receipts (#2284)
* Fixes in rpctest and rpcdaemon for debug_traceTransaction

* Fix for opcode tracer

* Tool to fix receipts

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-03 14:34:23 +01:00
Alex Sharov
6a5f86423c
Remove "numgc" from logs (#2275) 2021-07-03 14:44:23 +07:00
ledgerwatch
ba55611f29
rpctest for eth_getTransactionReceipt (#2271)
* rpctest bench for eth_getTransationReceipt

* Add blockHash to re-generated receipts

* Modify scanReceipts

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-07-02 14:34:20 +01:00
Alex Sharov
3d6d45a82f
remove subtrie loader and dcmp (#2258) 2021-06-30 10:31:36 +07:00
ledgerwatch
7baf3ec484
Move secp256k1 into a separate go module, hook up erigon-lib (#2239)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-26 15:03:05 +01:00
ledgerwatch
adf4766191
Scan transaction to select test cases (#2225)
* Scan txs

* Find examples with access lists

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-23 23:34:17 +01:00
ledgerwatch
5ad449ef22
Migration to fix receipts (broken due to change of CBOR codec) (#2221)
* Scan receipts

* Not check tx type before Berlin

* Reverse

* scan receipt code in the migration

* Turn migration on

* Apply migration

* Fix

* Print last

* Remove last

* Not remove last

* Not updata in migration

* no cursors in migration

* Handle empty blocks

* Handle empty blocks

* Use the same code in hack

* Print

* Print

* Save

* Fix

* Fix lint

* Fix lint

* Fix for empty DB

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-23 15:05:16 +01:00
ledgerwatch
ab685e90d9
Improvement to the trimTxs (#2215)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-22 08:35:34 +01:00
ledgerwatch
a79ed2fe0b
Utilities to fix DB corruption (#2214)
* Detect broken ancestry

* Print last canonical headers

* Add backExec

* Cleanup

* Add unwind util

* Add trimTxs utility

* Actually delete tx records

* Avoid infinite loop

* Change strategy

* Close cursor after commmit

* Fix RwBegin issue

* 10m records per transaction

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-21 21:52:51 +01:00
ledgerwatch
04943eaa52
Fix for state corruption without performance regression (#2208)
* extractHeaders

* Print diff

* Fix for state corruption without performance regression

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-21 06:56:45 +01:00
Artem Vorotnikov
d8a009837b
Nuke preimage remnants (#2203) 2021-06-20 13:00:22 +07:00
Alex Sharov
a535e8f30c
switch to mdbx-go module, and db-tools to libmdbx submodule (#2199)
* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule

* move mdbx to submodule
2021-06-19 21:30:12 +01:00
Alex Sharov
bc271f92ce
To break dependency to db implementation - move all db classes to "ethdb/kv" and leave in "ethdb" only interfaces (#2195) 2021-06-19 15:21:53 +07:00
Artem Vorotnikov
0a4e0d1fa1
Various code simplificaitons (#2165)
* simplify randomaccount

* simplify execution unwind

* Remove standalone account accessors
2021-06-14 19:52:30 +01:00
ledgerwatch
2c1fe2a3a7
Restore TD records (#2164)
* Insert missing TD values

* Print missing TD records

* Fix typo

* Apply fix

* Fixes in execution stage

* Fix tx/sec output

* Advance exec

* Reset unwind

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-14 19:51:57 +01:00
ledgerwatch
0f64df626a
Util to dump state (#2159)
* Util to dump state

* Extract bodies info

* Fix for extractBodies

* Fix typo

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-06-14 07:34:36 +01:00
Alex Sharov
925d17a238
remove lmdb from hack, integration, state packages. remove lmdb dbtools (#2146) 2021-06-11 20:31:37 +07:00
ledgerwatch
73bb98f686
Move tests 6 (#2107)
* Move tests 6

* Compile fixes

* Fix lint

* Compile fixes

* Fix compile

* compile fixes

* Compile fix

* Fix consesus/clique

* Cleanup

* Add gas limit

* Print

* Prints

* More print

* Fix

* Reinstate TestGetBlockReceipts66

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-06 22:44:14 +01:00
Alex Sharov
1f13f73045
up version of uint256 lib (#2082)
* eip 1559_tx_constraints

* eip 1559_tx_constraints

* return error on uint256 overflow

* up uint256 version

* merge master

* fix master

Co-authored-by: ledgerwatch <akhounov@gmail.com>
2021-06-04 17:25:28 +01:00
ledgerwatch
67d0a2d3ab
Reproduce and fix trace_filter index problem (#2084)
* Intermediate

* Test for trace_filter

* Clean up

* Clean up

* clean up

* More to the test

* Hack

* Block parameter

* Block parameter

* use mdbx

* Print number of records

* Print keys

* Print index records

* Print block numbers

* Print values in tracecallset

* Print some more

* More printing

* Print froms and tos

* Print startBlock and endBlock

* Tests and fixes

* Cleanup

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-06-04 09:09:57 +01:00
Evgeny Danilenko
74847d77e6
Rename to Erigon (#2018)
* turbo-geth to erigon

* tg, turbo to erigon
2021-05-26 11:35:39 +01:00
Alex Sharov
545fe32f68
Mdbx - make it default db. Lazy buckets renaming. (#1982) 2021-05-22 10:03:02 +07:00
Alex Sharov
0be3044b7e
rename (#1978)
* rename

* rename "make grpc"

* rename "abi bindings templates"

* rename "abi bindings templates"
2021-05-20 19:25:53 +01:00
ledgerwatch
5f048bc82f
Wrong receipt problem - utility to recover DB (#1970)
* Wrong receipt hack

* Advance5

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-05-20 14:28:38 +01:00
b00ris
565a4250d6
Snapshot sync headers stage (#1836)
* save state

* snapshot update works

* save state

* snapshot migrator

* tx test

* save state

* migrations stages refactor

* refactor snapshot migrator

* compilation fixed

* integrate snapshot migrator

* goerli sync headers

* debug async snapshotter on goerly

* move verify headers, remove experiments, fix remove old snapshot

* save state

* refactor snapshotsync injection

* fix deadlock

* replace snapshot generation stage logic to migrate method

* change done for body snapshot

* clean

* clean&&change deleted value

* clean

* fix hash len

* fix hash len

* remove one of wrap methods, add remove snapshots on start

* add err check

* fix shadowing

* stages unwind order debug

* matryoshka experiments

* steam test

* fix build

* fix test

* fix lint

* fix test

* fix test datarace

* add get test

* return timeout

* fix mdbx overlap

* fix after merge

* change epoch size

* clean todo

* fix

* return testdata

* added return from sndownloader gorutine

* fix review comments

* Fix

* More info

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-05-08 09:45:40 +01:00