Commit Graph

294 Commits

Author SHA1 Message Date
battlmonstr
e6c056e666
Rename Datadir/dataDir to follow conventions. (#3576)
* rename from Datadir to DataDir (functions and public vars)
* rename dataDir to datadir (local variables)
2022-02-22 17:39:48 +00:00
Alex Sharov
f5da8833b6
erigon2: exec metric (#3567) 2022-02-22 08:55:49 +07:00
Alex Sharov
27b87f44d6
Erigon2: support block snapshots (#3557)
* save

* erigon

* save
2022-02-22 08:42:12 +07:00
ledgerwatch
e784f2b809
[erigon2] ETL optimisations (#3550) 2022-02-21 09:31:06 +07:00
ledgerwatch
80d7734ea8
[erigon2] fix history creation (txNum confusion) (#3547)
* More tracing

* More tracing

* More tracing

* Keep correct txNum

* Keep correct txNum

* Update to tracing

* Off by one

* Print memory info

* Quicker startup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-19 08:15:35 +00:00
ledgerwatch
c5b4d3e352
[erigon2] Thin commitment (2nd attempt) (#3515)
* Another fix for history files

* Fix NPE

* Another fix

* Proper closing sequence

* Remove first byte insert flag

* Fixes

* Update

* Add traceblock to erigon2 command

* Update

* Update

* Update

* Update

* print panic

* Fix history reads

* Update

* Update locks

* Update

* Fixes

* Set changesets and commitments upfront

* Logging instead of printing

* Fixes in aggregator and recsplit

* Update to erigon-lib

* Remove empty commitments

* Update

* Update

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-17 10:56:20 +07:00
Andrew Ashikhmin
3663f8c709
Explicit error for purged change sets (#3528) 2022-02-16 16:44:16 +01:00
Andrew Ashikhmin
7bad3a9c59
state.PlainState is now at the beginning of blockNbr, not the end (#3517)
* state.PlainState is now at the beginning of blockNbr, not the end (fix discprepancy with GetAsOf)

* Fix tests
2022-02-16 08:38:12 +00:00
ledgerwatch
c865290502
Add traceblock to erigon2 command (#3516)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-14 23:15:24 +00:00
ledgerwatch
dffcc57ecc
[erigon2] Fix history bug, optimise bitmap search (#3510)
* Fix history bug, optimise bitmap search

* Optimise SeekInBitmap64

* Update erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-13 20:02:18 +00:00
ledgerwatch
2074b98568
Tools for gathering data on erigon2 history and bitmaps (#3488)
* Tools for gathering data on erigon2 history and bitmaps

* Always start with 0

* Always start with 0

* Clean up commitment files, trace block

* traceblock

* Add tracing to checkChangeSets

* Fixes to DeleteAccount

* Update to latest erigon-lib

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-12 15:48:38 +00:00
Alex Sharov
c9cee7a884
path -> filepath (path package is for urls) (#3493) 2022-02-12 20:33:09 +07:00
ledgerwatch
eb6a168603
[erigon2] Chain history and bitmap indices (#3437)
* First update

* Update to latest erigon-lib

* Fix lint

* Update

* Update

* Update

* Add history2 command

* Updates

* Update erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-12 00:06:57 +00:00
ledgerwatch
093bfbad39
[erigon2] Optimisations (#3412)
* Update

* [erigon2] optimisations

* Update to latest erigon-lib

* Update to latest erigon-lib, fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-03 09:36:27 +00:00
ledgerwatch
4ef14215b4
[erigon2] three threads, better stats in the logs (#3378)
* [erigon2] Use compact keys in commitments

* Better stats in the logs

* Update

* Update

* Update

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-31 23:40:51 +00:00
ledgerwatch
079a177614
[erigon2] Computing data dependencies between transactions (#3290)
* Use ibs per transaction instead of per block

* Remove copy functions, introduce balance increase map

* Fix test

* Fix lint

* Introduce balanceIncrease journal action

* Introduce balanceIncrease journal action

* Simplifications

* Fix

* Reset

* Simplify

* Fix

* Fixes

* Fix for reading nil accounts

* Fix edge cases with reverts

* Clean up test output

* Fix transfers

* Handle special case of ripemd

* Try with fixed compress, only commit every commfreq blocks

* Updates

* Updates

* Update to fixed erigon-lib

* Update to erigon-lib

* Remove stop

* Update to latest erigon-lib

* fixup

* Cleanup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-24 23:12:25 +00:00
Alex Sharov
1e3228124a
Move cli root context to erigon-lib (#3294)
* save

* save

* save
2022-01-19 10:49:07 +07:00
ledgerwatch
1af72faf71
[erigon2] optional writing of "before" value for changes, decouple commitment from blocks (#3272)
* Sepatate commitment calculation from blocks

* Fix

* Fix lint

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-16 21:24:54 +00:00
ledgerwatch
17aff6090f
[erigon2] Resumable prototype (#3263)
* [erigon2] Resumable prototype

* Fix compile error

* Resumable prototype

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-16 08:40:16 +00:00
ledgerwatch
c774bff61e
Update erigon2.go (#3266) 2022-01-15 09:41:56 +00:00
Dmitry Savonin
a49d409457
Full BSC support with validator mode (#3233)
* migrated consensus and chain config files for bsc support

* migrated more files from bsc

* fixed consensus crashing

* updated erigon lib for parlia snapshot prefix

* added staticpeers for bsc

* [+] added system contracts
[*] fixed bug with loading snapshot
[+] enabled gas bailout
[+] added fix to prevent syncing more than 1000 headers (for testing only)
[*] fixed bug with crashing sender recover sometimes

* migrated system contract calls

* [*] fixed bug with returning mutable balance object
[+] migrated lightclient contracts from bsc
[*] fixed parlia consensus config param

* [*] fixed tendermint deps

* [+] added some logs

* [+] enabled bsc forks
[*] fixed syscalls from coinbase
[*] more logging

* Fix call sys contract gas calculation

* [*] fixed executing system transactions

* [*] enabled receipt hash, gas and bloom filter checks

* [-] removed some logging scripts
[*] set header checkpoint to 10 million blocks (for testing forks)

* [*] fixed bug with commiting dirty inter block state state after system transaction execution
[-] removed some extra logs and comments

* [+] added chapel and rialto testnet support

* [*] fixed chapel allocs

* [-] removed 6 mil block limit for headers sync

* Fix hardforks on chapel and other testnets

* [*] fixed header sync issue after merge

* [*] tiny code cleanup

* [-] removed some comments

* [*] increased mdbx map size to 4 TB

* [*] increased max chaindata size to 6 tb

* [*] bring more compatibility with origin erigon and some code cleanup

* [+] added support of validator mode for BSC chain

* [*] enable private key load for bsc, rialto and chapel chains

* [*] fixed running BSC validator node

* Fix the branch list

* [*] tiny fixes for linter

* [*] formatted imports for core and parlia packages

* [*] fixed import rules in other files

* Revert "[*] formatted imports for core and parlia packages"

This reverts commit c764b58b34fedc2b14d69458583ba0dad114f227.

* [*] changed import rules in more packages

* [*] fixed type mismatch in hack command

* [*] fixed crash on new epoch, enabled bootstrap flags

* [*] fixed linter errors

* [*] fixed missing err check for syscalls

* [*] now BSC implementation is fully compatible with erigon original sources

* Revert "Add chain config and CLI changes for Binance Smart Chain support (#3131)"

This reverts commit 3d048b7f1a.

* Revert "Add Parlia consensus engine for Binance Smart Chain support (#3086)"

This reverts commit ee99f17fbe.

* [*] fixed several issues after merge

* [*] fixed integration compilation

* Revert "Fix the branch list"

This reverts commit 8150ca57e5f2707a84a9f6a1c5b809b7cc84547b.

* [-] removed receipt repair migration

* [*] fixed parlia fork numbers output

* [*] bring more devel compatibility, fixed bsc address list for access list calculation

* [*] fixed bug with commiting state transition for bad blocks in BSC

* [*] fixed bsc changes apply for integration command and updated config print for parlia

* [*] fixed bug with applying bsc forks for chapel and rialto testnet chains
[*] let's use finalize and assemble for mining to  let consensus know for what it's finalizing block

* Fix compilation errors in hack.go

* Fix lint

* reset changes in erigon-snapshots to devel

* Remove unrelated changes

* Fix embed

* Remove more unrelated changes

* Remove more unrelated changes

* Restore clique and aura miner config

* Refactor interfaces not to use slice pointers

* Refactor parlia functions to return tx and receipt instead of dealing with slices

* Fix for header panic

* Fix lint, restore system contract addresses

* Remove more unrelated changes, unify GatherForks

Co-authored-by: Dmitry Ivanov <convexman18@gmail.com>
Co-authored-by: j75689 <j75689@gmail.com>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-14 19:06:35 +00:00
ledgerwatch
ea6dbd49e8
[erigon2] Running with commitment (#3203)
* [erigon2] Running with commitment

* [erigon2] Running with commitment

* [erigon2] Running with commitment

* go mod tidy

* [erigon2] Running with commitment

* More

* Debug

* fix

* Fix

* state root command

* More fixes

* Fix

* Progress to 164735

* Only trace when failing

* fix for firstInsert

* Over block 1.36m

* Update

* fix to deleteAccount

* Fixes for plainKeys

* Undo printing

* No changeset generation by default

* Print block number on interrupt, fix lint

* Fix lint

* Open history DB as read only

* Print error

* Open non read only

* Readonly again

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-14 16:02:44 +00:00
Alex Sharov
4ef541f5dc
Snapshots: tx lookup in RPC from snapshots (#3214) 2022-01-07 20:52:38 +07:00
Frojdi Dymylja
499c27d2e1
change: port js tracer changes (#3128)
* change: expose callCtx as ScopeContext

* change: update tracer API

* change: logger API
update: jst tracer

* add: port jst tracer 2

* add: port jst tracer 3

* add: access list tracer

* chore: lint
2021-12-15 13:19:58 +00:00
brendan-kelly
ee99f17fbe
Add Parlia consensus engine for Binance Smart Chain support (#3086)
* Add Parlia consensus engine for Binance Smart Chain support

* Leave RamanujanBlock as nil in params/config.go

* Run `go fmt` on files needing it

* Add comment for PoSA

* Remove empty branches and ineffectual assignments in parlia.go

* Remove commented imports

* Fix compilation error

* Remove EIP155Signer in transaction_signing.go

* Fix compilation issue

* Fix go fmt issues

* Remove Ramanujan from print statement

* Remove references to EthAPIBackend approach

* Fix Finalize method across consensus engines

* Run go fmt

* More linting

* Remove more changes

* remove a comment

* Remove unneeded hashing function

* Remove bytes check and fix actual vs expected mistake
2021-12-11 00:07:10 +00:00
Alexandr Borodulin
bbb3cc978f
Starknet getcode (#3038)
* deploy_cairo_smartcontract

* deploy_cairo_smartcontract / 2

Add new transaction type for cairo and vm factory

* starknet_getcode

* deploy_cairo_smartcontract / 3

* deploy_cairo_smartcontract / 4

* deploy_cairo_smartcontract / 5

Co-authored-by: Aleksandr Borodulin <a.borodulin@axioma.lv>
2021-12-06 14:58:53 +00:00
Alex Sharov
b7aa5be149
Remove snapshot migrator class (#3088) 2021-12-05 16:13:40 +07:00
Alex Sharov
178fd1931d
Genesis sync from existing snapshots (#3087) 2021-12-05 09:03:08 +07:00
ledgerwatch
b1b68e05ea
Erigon2 state runner - option to compare with historical state (#3061)
* Comment out comparisons

* Introduce check option

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-11-30 16:46:43 +00:00
TBC Dev
ae4819ddad
Prefer Block.NumberU64() over Block.Number.Uint64() (#3051) 2021-11-29 11:32:41 +00:00
TBC Dev
57d641b6f9
Avoid redundant Block.Header() deep-copy (#3050)
* Add separate Block.Nonce() and Block.NonceU64()

* Add Block.Seal()

* Avoid redundant Block.Header() deep-copy

* Add warning comment for Block.Header()
2021-11-29 08:52:36 +00:00
ledgerwatch
5a09c1c8c6
Erigon2 state runner (#3034)
* erigon 2 state experiment

* Some boilerplate

* More boilerplate

* More boilerplate

* Introduce comparison

* trace merges

* Fix

* Update erigon-lib dependency

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-11-27 09:43:25 +00:00
Alex Sharov
702631bf6f
prohibit change prune mode (#2908) 2021-11-03 20:05:52 +07:00
Alex Sharov
5ca558c667
Move lengths to erigon-lib other packages depend on it (#2799)
* move lengths to erigon-lib

* move lengths to erigon-lib
2021-10-08 10:20:45 +07:00
Andrea Lanfranchi
c913f35c2e
Inner errors (#2774) 2021-10-04 22:16:52 +07:00
Alex Sharov
3de50637cd
remove changeset.Walk func (#2716) 2021-09-22 07:54:29 +07:00
ledgerwatch
15b4095718
Move ETL to erigon-lib (#2667)
* Move ETL to erigon-lib

* Update link in the readme

* go mod tidy

* Use common/chan.go from erigon-lib

* Clean up

* Fix lint

* Fix test

* Fix compilation

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2021-09-12 08:50:17 +01:00
Andrea Lanfranchi
00d627bdb0
Remove config.name (const "erigon") from paths (#2531)
* Remove config.name (const "erigon") from paths

* Fix unit test

* Safety check

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-08-18 16:24:03 +07:00
Alex Sharov
c7c3a5932a
rename ReadBody to ReadBodyWithTransactions to make it obvious that it's heavier then just ReadBody (#2534)
* rename ReadBody to ReadBodyWithTransactions

* rename ReadBody to ReadBodyWithTransactions
2021-08-15 17:08:28 +07:00
Evgeny Danilenko
4cd72c8328
Keep readonly value while changing interpreters back and forth (#2508)
* restore TEVM

* fuzzing and property based

* comment

* lint

* stack callback into defer ater checking an error

* sequential tests
2021-08-10 09:48:56 +07:00
ledgerwatch
350d7fe7a9
Adding support for vmTrace into trace_ routines (#2497)
* VmTrace

* Fixes to gasCall, beginning of vmTrace

* Fix opcode tracer

* Add mem and store, enable vmTrace for all methods

* Fix lint

* More nuances and debugging|

* More fixes

* Fix for trace_callMany

* Fix for trace_callMany

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2021-08-08 13:28:03 +01:00
Alex Sharov
6bd44eb26c
move kv to erigon-lib (#2467) 2021-07-29 18:53:13 +07:00
Alex Sharov
5069558752
Apache licensed logger (#2460) 2021-07-29 17:23:23 +07:00
Alex Sharov
21cb7befa4
finish remove bucket suffix (#2458) 2021-07-28 10:43:51 +07:00
Alex Sharov
838e5f9ef2
Move bucket constants into kv package, move kv interface to kv package (#2455) 2021-07-28 09:47:38 +07:00
Alex Sharov
6801208497
move some tests to kv interface (#2438) 2021-07-24 14:14:11 +07:00
Alex Sharov
d2552196ad
Sokol v0: support for first epoch-set transition (#2411) 2021-07-21 18:13:26 +07:00
Alex Sharov
c3e1cfdac8
Pruning for: exec, log_index, tx_lookup, history stages (#2399)
* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* Pruning for: exec, log_index, tx_lookup, history stages

* add tvm flag

* save

* db migration for storage mode
add flag --prune=
remove flag --storage-mode=
add flag --experiments=tevm,...
rename integration set_storage_mode to set_prune

* fix

* forward move of stages must skip everything before PruneTo

* keep in db progress of prune method

* keep in db progress of prune method

* simplify logs

* simplify logs

* simplify logs

* fix test

* simplify logs

* simplify logs

* simplify logs

* simplify logs

* remove callTraceSet as dupsort
use etl transform for txlookup prune
remove some logs

* cleanup tests a bit

* print_stages and eth_sync to show prune progress

* fix print_stages

* add readme about --prune.to flag

* more docs

* add --prune.history.older and other flags support

* fix migration on empty db

* better toString

* better toString
2021-07-20 21:03:19 +01:00
Alex Sharov
9f6ef74adc
Linter: to check tx.Rollback() by ruleguard (#2383) 2021-07-16 20:23:54 +07:00
Alex Sharov
557d082c51
Sokol v0: epoch support (#2348)
* remove ctx from state writer interface

* call me baby

* save

* save

* can process block 1

* can process block 1

* can process block 1

* can process block 1

* save

* save

* parse logs logic

* cleans

* cleans

* cleans
2021-07-12 16:27:25 +01:00