Commit Graph

184 Commits

Author SHA1 Message Date
ledgerwatch
2b517b7630
Erigon22 prototype to support BSC (#4867)
* Reduce workers and queue size

* Add traces and event logs

* Add chain flag

* Print

* Open consensus db as read only

* Use only one engine instance

* chainDb

* Display palia error

* Not to panic

* Not to exit

* Add syscall

* Print

* Pointer to TxTask

* Not register senders

* Print

* Relax check

* Skip system transactions

* increase workerCount again

* Remove print

* Change recon

* Fix types

* Skip system txs

* Print

* No rules for genesis commit

* Print genesis

* Print|

* Print

* Print

* Update

* Fix

* Remove print

* No print

* Support contract upgrades

* Use the same dif

* Print create2

* Fix

* Fix

* Print

* Hack iterate

* Print empty vals

* Fix hack

* Code hashes

* Code hashes

* Print

* No panic

* No panic

* Fix?

* another incarnation fix

* Remove print

* Remove print

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-07-31 21:54:23 +01:00
ledgerwatch
167de529a8
Fixes for erigon22 prototypes (#4851)
* Fixes for erigon22 prototypes

* Add waiting/applying stats

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-07-28 13:26:48 +01:00
ledgerwatch
09c104803a
Split erigon2.2 and erigon2.3 prototypes (#4811)
* Split erigon2.2 and erigon2.3 prototypes

* Renaming

* Interruptible and resumable erigon22

* Always regenerate trie

* Introduce aggregator

* Fixes

* cleanup

* Fix lint

* Update to erigon-lib main

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-07-28 12:16:37 +01:00
Alex Sharov
aee4b53788
save list of snapshots in db (#4777)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
2022-07-28 16:57:38 +07:00
ledgerwatch
81d106bc9d
Experiment in parallel execution (#4652)
* Restructure tx execution

* fixes

* Fixes and traces

* Tracing

* More tracing

* Drain the result channel

* Intermediate

* more efficient parallel exec

* Sorted buffer

* Fix results size

* fix for the recon

* Fix compilation

* Sort keys in Write and Read sets, fix compilation in rpcdaemon22

* Update to latest erigon-lib

* Update to erigon-lib

* Remove go.mod replace

* Update erigon-lib

* Update to erigon-lib main

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-07-23 18:39:08 +01:00
Alex Sharov
66758c7960
RetireBlocks: less arguments (#4785)
* save

* save
2022-07-22 13:44:42 +07:00
Artem Tsebrovskiy
9637b25a42
[erigon2] print trie variant derived from trie instead switch log (#4672)
* [erigon2] print trie variant derived from trie instead switch log

* updated erigon-lib version

* bumped erigon-lib version
2022-07-11 15:58:08 +01:00
sudeep
e13a318e0b
evm t8n to use ExecuteBlockEphemerally api (#4642)
* evm t8n tool to use ExecuteBlockEphemerally api (#4512)

* fix to set V, R, S in legacy transaction

* fix to dump post-execution alloc for evm t8n

* close tx in evm t8n

* populate current difficulty and gas used in output result

- update the ExecutionResult to include corresponding info (like
  Difficulty/GasUsed)

* initial attempt at migrating 'evm t8n' to use ExecuteBlockEphemerally

* using ExecutionResult in ExecuteBlockEphemerally

* bypass validations and integrate with EphemeralExecResult

* fixing output of 'evm t8n'

- remaining bits are "stateRoot" in results.txt and "balance" field for one account in
  alloc.txt (for testdata=1)

* get ExecuteBlockEphemerally to accept getTracer lambda

* fix build failure

* test cases for evm t8n

* more test cases for evm t8n

* fix stateRoot computation in evm t8n

* remove reward argument, as EBE itself takes care of it

* final cleanups for migration to using ExecuteBlockEphemerally

* change EBEforBSC to match EBE

* fix linter issues

* manually revert an unwanted diff

* avoid calculating ReceiptHash twice

* linter check

* minor correction

* remove unnecessary logic in EBEforBsc

* fix integration tests

* fix build
2022-07-07 12:47:00 +01:00
ledgerwatch
73b0659bb7
[erigon2.2] Optimisation of state reconstitution (#4621)
* Init

* Optimise ReconState flushes

* Updates

* Update

* Updates

* Updates

* More on parallel execution

* More on parallel exec

* Fix lint

* Improvements to parallel exec

* comment

* Fix history access, include incarnation

* Close work channel

* TxTask

* more fixes to parallel exec

* Update to latest erigon-lib

* Fix compilation

* Cleanup and timing

* Fixes to recon1

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-07-06 07:49:00 +01:00
Giulio rebuffo
d83b7a4ae0
Proper PoS Error reporting (#4631)
* better reporting

* removed debug log

* proper error reporting
2022-07-05 01:31:16 +02:00
ledgerwatch
77dc35bb85
Revert "evm t8n tool to use ExecuteBlockEphemerally api (#4512)" (#4610)
* Revert "evm t8n tool to use ExecuteBlockEphemerally api (#4512)"

This reverts commit db93d2ea37.

* Fix compilation

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-07-02 23:24:20 +01:00
ledgerwatch
8599dceec7
[erigon2.2] State reconstitution prototype (#4508)
* reconstitution

* Add history access without state function

* More on state reconstitution

* More on state recon

* More on state recon

* More

* More

* support dao fork

* More on state reconstitution

* Update to erigon-lib

* More

* Added genesis block and filling with history

* update

* Genesis works

* Start on parallel

* Preparation for parallel reconstitution, stats for EfSearch

* continue with parallel work

* Fix history reader

* Remove time measurements

* Fixes

* Fixes and UX improvements

* Fixes

* More tracing

* More fixes

* More fixes

* Fix code size

* Update to latest erigon-lib

* Fix for dao fork

* Remove hacks

* Update to erigon-lib, fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-07-02 20:48:42 +01:00
sudeep
db93d2ea37
evm t8n tool to use ExecuteBlockEphemerally api (#4512)
* fix to set V, R, S in legacy transaction

* fix to dump post-execution alloc for evm t8n

* close tx in evm t8n

* populate current difficulty and gas used in output result

- update the ExecutionResult to include corresponding info (like
  Difficulty/GasUsed)

* initial attempt at migrating 'evm t8n' to use ExecuteBlockEphemerally

* using ExecutionResult in ExecuteBlockEphemerally

* bypass validations and integrate with EphemeralExecResult

* fixing output of 'evm t8n'

- remaining bits are "stateRoot" in results.txt and "balance" field for one account in
  alloc.txt (for testdata=1)

* get ExecuteBlockEphemerally to accept getTracer lambda

* fix build failure

* test cases for evm t8n

* more test cases for evm t8n

* fix stateRoot computation in evm t8n

* remove reward argument, as EBE itself takes care of it

* final cleanups for migration to using ExecuteBlockEphemerally

* change EBEforBSC to match EBE

* fix linter issues

* manually revert an unwanted diff

* avoid calculating ReceiptHash twice

* linter check

* minor correction

* remove unnecessary logic in EBEforBsc
2022-07-02 06:52:23 +01:00
Alex Sharov
1f4f850b89
pass context around hased state stage (#4537)
* save

* save

* save
2022-06-25 20:34:42 +06:00
ledgerwatch
79830adefa
[erigon2.2] collecting read indices (#4499)
* [erigon2.2] collecting read indices

* Fix compile issue

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-20 09:00:45 +01:00
ledgerwatch
e47dd15e68
[erigon2.2] Prototype of eth_getLogs (#4437)
* Initial work

* Fix compile errors

* Update

* Debug

* Cleanup

* Temp changes

* Temp

* Remove temp

* Remove print

* Upgate to erigon-lib main

* go mod tidy

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-13 21:25:17 +01:00
ledgerwatch
fd8adddce8
trace_filter implementation based on erigon 2 update 2 data (#4431)
* trace_filter implementation based on erigon 2 update 2 data

* Fix test compile, extract txNums

* Update to erigon-lib, add generic heap

* Fix getHeaderHash for RPC methods

* Missing files

* Skip tests

* Add reward traces

* Filter trace lint

* Reinstate filtering

* Print

* Print

* Fix

* Print

* Print txNums

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Remove prints

* Fix nil dereference

* Inclusive toBlock bound

* Print

* Print

* Hack

* remove some deps

* Fix lint

* Update erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-12 12:44:01 +01:00
ledgerwatch
8e3ac8a21c
Erigon2 upgrade 2 prototype (#4341)
* Erigon2 upgrade 2 prototype

* Latest erigon-lib

* Fixes

* Fix print

* Fix maxSpan

* Reduce maxSpan

* Remove duplicate joins

* TxNum

* Fix resuming

* first draft of history22

* Introduce historical reads

* Update to erigon-lib

* Update erigon-lib

* Update erigon-lib

* Fixes and tracing for checkChangeSets

* More trace

* Print account details

* fix getHeader

* Update to erigon-lib main

* Add tracer indices and event log indices

* Fix calltracer

* Fix calltracer

* Duplicate rpcdaemon into rpcdaemon22

* Fix tests

* Fix tests

* Fix tests

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-10 16:18:43 +01:00
Enrique Jose Avila Asapche
9f1cc40c89
changed syncmode flag to snapshots flag (#4353)
* changed syncmode flag to snapshots flag

* using boolT

* Remove syncmode concept

* Add mumbai

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-05 07:33:55 +01:00
ledgerwatch
ca2b0f19c1
Fix checkChangeSets and RPC daemon commands accessing history (#4365)
* Simplifications

* Print

* Fix

* Clean up

* Fixed RPC daemon command accessing history

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-04 23:06:30 +01:00
ledgerwatch
ee4ffd190d
Update skip analysis (#4364)
* Skip analysis advance, fix for change sets

* Optimisations for integration

* Print

* Print

* Fix

* Fix

* Fix

* Remove prints

* No panic

* More checks

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-04 16:54:22 +01:00
Enrique Jose Avila Asapche
bc21ab9d97
syncmode changes (#4334)
* log syncmode

* little comment

* ops

* calling log in correct place

* got rid of error instead we warn

* no return
2022-06-02 14:15:00 +01:00
Alex Sharov
0795bcb865
integration: validate syncmode flag (#4300) 2022-05-30 09:59:38 +07:00
Andrew Ashikhmin
0c76286509
Add rules to AsMessage to vet transaction types (#4276)
* Add rules to AsMessage to vet transaction types

* Fix TestState
2022-05-26 18:20:34 +02:00
Alex Sharov
66248c4bfb
Torrent: verify all files (#4270) 2022-05-26 12:27:44 +07:00
Alex Sharov
fc1a37e6dc
create turbo/services pkg (#4269) 2022-05-26 10:31:06 +07:00
Alex Sharov
c2e0956e3e
return err on invalid syncmode (#4219)
* save

* save

* save
2022-05-20 16:04:38 +07:00
Artem Tsebrovskiy
f4c8f44f61
removed obsolete trie variant (#4172) 2022-05-20 15:03:27 +07:00
Alex Sharov
726ef081ce
option to disable runtime.ReadMemStats (#4200) 2022-05-19 11:48:07 +07:00
ledgerwatch
0e457ba9d7
Restore skip analysis (#4146)
* Restore skip analysis optimisation

* Make checkChangeSets work with snapshots

* Add syncmode to checkChangeSets

* Add block number for BSC

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-14 20:17:23 +01:00
ledgerwatch
14557a2348
[erigon2] Support for binary tree commitments (#4077)
* binary tree

* Binary commitment tree

* [erigon2] Bin tree support

* Point to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-05 13:54:29 +01:00
ledgerwatch
285c782b1b
More efficient header verification of headers for Parlia when snapshots are used (#3998)
* Update stageloop.go

* Print

* Consider snapshot headers as parlia checkpoints

* Not fail after not loading snapshot

* Lazy snapshots

* Print number of validators

* More printing

* Use epoch instead of checkpoint interval

* Reduce logging

* Fix compilation

* Remove trace jump dest

* Fix lint

* Not store snapshots every epoch

* Separate snapshot for verification and finalisation

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-29 07:10:31 +01:00
Alex Sharov
831da9ba41
reduce downloader deps (#4011) 2022-04-28 12:42:14 +07:00
Alex Sharov
9b47daee44
Default syncmode=snap for Mainnet, Bsc, Goerli (#3861)
* change syncmode default to snap sync

* save
2022-04-10 07:20:20 +01:00
alex.sharov
10f195578b fix linter 2022-04-01 15:11:02 +07:00
Alex Sharov
d1218181b0
Snapshots: geth compatibility, use --syncmode=snap flag (#3808) 2022-04-01 15:00:50 +07:00
Alex Sharov
85e47d83d2
Snapshots: remove "experimental" prefix from cli flag (#3760) 2022-03-24 12:17:31 +07:00
ledgerwatch
f8668da905
[erigon2] Bring back MDBX as storage for recent data (#3756)
* fixes

* Update

* Update to erigon-lib main

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-23 16:00:06 +00:00
ledgerwatch
c47cf1fd49
[erigon2] Optimise Huffman decoding (#3727)
* Update to erigon-lib

* Fix for block number

* Update

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-03-18 09:55:38 +00:00
Alex Sharov
f314222180
Snapshots: start seed new large .seg files (#3724) 2022-03-18 11:12:18 +07:00
Artem Tsebrovskiy
f14df3faa3
added support of Clique consensus for block processing for erigon2 (#3705)
* added support of Clique consensus for block processing for erigon2

* erigon2: remove block reader which doesn't use system pre and post block transactions
2022-03-17 07:30:47 +00:00
Alex Sharov
469b75c3d3
Snapshots: retire blocks by default (#3707) 2022-03-16 09:57:48 +07:00
ledgerwatch
8b00636e21
[erigon2] history indices based on Elias Fano (#3679)
* [erigon2] history indices based on Elias Fano

* Tidy

* Update

* tx lookup to support blocTo in integration

* Update to fix

* Update

* Fix

* Update to erigon-lib main

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-13 23:11:49 +00:00
Artem Tsebrovskiy
d300c2821b
erigon2: implemented chain switching by --chain flag (#3668)
erigon2: fix linter
2022-03-10 14:01:37 +00:00
battlmonstr
e6c056e666
Rename Datadir/dataDir to follow conventions. (#3576)
* rename from Datadir to DataDir (functions and public vars)
* rename dataDir to datadir (local variables)
2022-02-22 17:39:48 +00:00
Alex Sharov
f5da8833b6
erigon2: exec metric (#3567) 2022-02-22 08:55:49 +07:00
Alex Sharov
27b87f44d6
Erigon2: support block snapshots (#3557)
* save

* erigon

* save
2022-02-22 08:42:12 +07:00
ledgerwatch
e784f2b809
[erigon2] ETL optimisations (#3550) 2022-02-21 09:31:06 +07:00
ledgerwatch
80d7734ea8
[erigon2] fix history creation (txNum confusion) (#3547)
* More tracing

* More tracing

* More tracing

* Keep correct txNum

* Keep correct txNum

* Update to tracing

* Off by one

* Print memory info

* Quicker startup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-19 08:15:35 +00:00
ledgerwatch
c5b4d3e352
[erigon2] Thin commitment (2nd attempt) (#3515)
* Another fix for history files

* Fix NPE

* Another fix

* Proper closing sequence

* Remove first byte insert flag

* Fixes

* Update

* Add traceblock to erigon2 command

* Update

* Update

* Update

* Update

* print panic

* Fix history reads

* Update

* Update locks

* Update

* Fixes

* Set changesets and commitments upfront

* Logging instead of printing

* Fixes in aggregator and recsplit

* Update to erigon-lib

* Remove empty commitments

* Update

* Update

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-17 10:56:20 +07:00