Commit Graph

176 Commits

Author SHA1 Message Date
ledgerwatch
73b0659bb7
[erigon2.2] Optimisation of state reconstitution (#4621)
* Init

* Optimise ReconState flushes

* Updates

* Update

* Updates

* Updates

* More on parallel execution

* More on parallel exec

* Fix lint

* Improvements to parallel exec

* comment

* Fix history access, include incarnation

* Close work channel

* TxTask

* more fixes to parallel exec

* Update to latest erigon-lib

* Fix compilation

* Cleanup and timing

* Fixes to recon1

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-07-06 07:49:00 +01:00
Giulio rebuffo
d83b7a4ae0
Proper PoS Error reporting (#4631)
* better reporting

* removed debug log

* proper error reporting
2022-07-05 01:31:16 +02:00
ledgerwatch
77dc35bb85
Revert "evm t8n tool to use ExecuteBlockEphemerally api (#4512)" (#4610)
* Revert "evm t8n tool to use ExecuteBlockEphemerally api (#4512)"

This reverts commit db93d2ea37.

* Fix compilation

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-07-02 23:24:20 +01:00
ledgerwatch
8599dceec7
[erigon2.2] State reconstitution prototype (#4508)
* reconstitution

* Add history access without state function

* More on state reconstitution

* More on state recon

* More on state recon

* More

* More

* support dao fork

* More on state reconstitution

* Update to erigon-lib

* More

* Added genesis block and filling with history

* update

* Genesis works

* Start on parallel

* Preparation for parallel reconstitution, stats for EfSearch

* continue with parallel work

* Fix history reader

* Remove time measurements

* Fixes

* Fixes and UX improvements

* Fixes

* More tracing

* More fixes

* More fixes

* Fix code size

* Update to latest erigon-lib

* Fix for dao fork

* Remove hacks

* Update to erigon-lib, fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-07-02 20:48:42 +01:00
sudeep
db93d2ea37
evm t8n tool to use ExecuteBlockEphemerally api (#4512)
* fix to set V, R, S in legacy transaction

* fix to dump post-execution alloc for evm t8n

* close tx in evm t8n

* populate current difficulty and gas used in output result

- update the ExecutionResult to include corresponding info (like
  Difficulty/GasUsed)

* initial attempt at migrating 'evm t8n' to use ExecuteBlockEphemerally

* using ExecutionResult in ExecuteBlockEphemerally

* bypass validations and integrate with EphemeralExecResult

* fixing output of 'evm t8n'

- remaining bits are "stateRoot" in results.txt and "balance" field for one account in
  alloc.txt (for testdata=1)

* get ExecuteBlockEphemerally to accept getTracer lambda

* fix build failure

* test cases for evm t8n

* more test cases for evm t8n

* fix stateRoot computation in evm t8n

* remove reward argument, as EBE itself takes care of it

* final cleanups for migration to using ExecuteBlockEphemerally

* change EBEforBSC to match EBE

* fix linter issues

* manually revert an unwanted diff

* avoid calculating ReceiptHash twice

* linter check

* minor correction

* remove unnecessary logic in EBEforBsc
2022-07-02 06:52:23 +01:00
Alex Sharov
1f4f850b89
pass context around hased state stage (#4537)
* save

* save

* save
2022-06-25 20:34:42 +06:00
ledgerwatch
79830adefa
[erigon2.2] collecting read indices (#4499)
* [erigon2.2] collecting read indices

* Fix compile issue

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-20 09:00:45 +01:00
ledgerwatch
e47dd15e68
[erigon2.2] Prototype of eth_getLogs (#4437)
* Initial work

* Fix compile errors

* Update

* Debug

* Cleanup

* Temp changes

* Temp

* Remove temp

* Remove print

* Upgate to erigon-lib main

* go mod tidy

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-13 21:25:17 +01:00
ledgerwatch
fd8adddce8
trace_filter implementation based on erigon 2 update 2 data (#4431)
* trace_filter implementation based on erigon 2 update 2 data

* Fix test compile, extract txNums

* Update to erigon-lib, add generic heap

* Fix getHeaderHash for RPC methods

* Missing files

* Skip tests

* Add reward traces

* Filter trace lint

* Reinstate filtering

* Print

* Print

* Fix

* Print

* Print txNums

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Fix

* Remove prints

* Fix nil dereference

* Inclusive toBlock bound

* Print

* Print

* Hack

* remove some deps

* Fix lint

* Update erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-12 12:44:01 +01:00
ledgerwatch
8e3ac8a21c
Erigon2 upgrade 2 prototype (#4341)
* Erigon2 upgrade 2 prototype

* Latest erigon-lib

* Fixes

* Fix print

* Fix maxSpan

* Reduce maxSpan

* Remove duplicate joins

* TxNum

* Fix resuming

* first draft of history22

* Introduce historical reads

* Update to erigon-lib

* Update erigon-lib

* Update erigon-lib

* Fixes and tracing for checkChangeSets

* More trace

* Print account details

* fix getHeader

* Update to erigon-lib main

* Add tracer indices and event log indices

* Fix calltracer

* Fix calltracer

* Duplicate rpcdaemon into rpcdaemon22

* Fix tests

* Fix tests

* Fix tests

* Update to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-10 16:18:43 +01:00
Enrique Jose Avila Asapche
9f1cc40c89
changed syncmode flag to snapshots flag (#4353)
* changed syncmode flag to snapshots flag

* using boolT

* Remove syncmode concept

* Add mumbai

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-05 07:33:55 +01:00
ledgerwatch
ca2b0f19c1
Fix checkChangeSets and RPC daemon commands accessing history (#4365)
* Simplifications

* Print

* Fix

* Clean up

* Fixed RPC daemon command accessing history

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-04 23:06:30 +01:00
ledgerwatch
ee4ffd190d
Update skip analysis (#4364)
* Skip analysis advance, fix for change sets

* Optimisations for integration

* Print

* Print

* Fix

* Fix

* Fix

* Remove prints

* No panic

* More checks

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-04 16:54:22 +01:00
Enrique Jose Avila Asapche
bc21ab9d97
syncmode changes (#4334)
* log syncmode

* little comment

* ops

* calling log in correct place

* got rid of error instead we warn

* no return
2022-06-02 14:15:00 +01:00
Alex Sharov
0795bcb865
integration: validate syncmode flag (#4300) 2022-05-30 09:59:38 +07:00
Andrew Ashikhmin
0c76286509
Add rules to AsMessage to vet transaction types (#4276)
* Add rules to AsMessage to vet transaction types

* Fix TestState
2022-05-26 18:20:34 +02:00
Alex Sharov
66248c4bfb
Torrent: verify all files (#4270) 2022-05-26 12:27:44 +07:00
Alex Sharov
fc1a37e6dc
create turbo/services pkg (#4269) 2022-05-26 10:31:06 +07:00
Alex Sharov
c2e0956e3e
return err on invalid syncmode (#4219)
* save

* save

* save
2022-05-20 16:04:38 +07:00
Artem Tsebrovskiy
f4c8f44f61
removed obsolete trie variant (#4172) 2022-05-20 15:03:27 +07:00
Alex Sharov
726ef081ce
option to disable runtime.ReadMemStats (#4200) 2022-05-19 11:48:07 +07:00
ledgerwatch
0e457ba9d7
Restore skip analysis (#4146)
* Restore skip analysis optimisation

* Make checkChangeSets work with snapshots

* Add syncmode to checkChangeSets

* Add block number for BSC

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-14 20:17:23 +01:00
ledgerwatch
14557a2348
[erigon2] Support for binary tree commitments (#4077)
* binary tree

* Binary commitment tree

* [erigon2] Bin tree support

* Point to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-05 13:54:29 +01:00
ledgerwatch
285c782b1b
More efficient header verification of headers for Parlia when snapshots are used (#3998)
* Update stageloop.go

* Print

* Consider snapshot headers as parlia checkpoints

* Not fail after not loading snapshot

* Lazy snapshots

* Print number of validators

* More printing

* Use epoch instead of checkpoint interval

* Reduce logging

* Fix compilation

* Remove trace jump dest

* Fix lint

* Not store snapshots every epoch

* Separate snapshot for verification and finalisation

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-29 07:10:31 +01:00
Alex Sharov
831da9ba41
reduce downloader deps (#4011) 2022-04-28 12:42:14 +07:00
Alex Sharov
9b47daee44
Default syncmode=snap for Mainnet, Bsc, Goerli (#3861)
* change syncmode default to snap sync

* save
2022-04-10 07:20:20 +01:00
alex.sharov
10f195578b fix linter 2022-04-01 15:11:02 +07:00
Alex Sharov
d1218181b0
Snapshots: geth compatibility, use --syncmode=snap flag (#3808) 2022-04-01 15:00:50 +07:00
Alex Sharov
85e47d83d2
Snapshots: remove "experimental" prefix from cli flag (#3760) 2022-03-24 12:17:31 +07:00
ledgerwatch
f8668da905
[erigon2] Bring back MDBX as storage for recent data (#3756)
* fixes

* Update

* Update to erigon-lib main

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-23 16:00:06 +00:00
ledgerwatch
c47cf1fd49
[erigon2] Optimise Huffman decoding (#3727)
* Update to erigon-lib

* Fix for block number

* Update

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-03-18 09:55:38 +00:00
Alex Sharov
f314222180
Snapshots: start seed new large .seg files (#3724) 2022-03-18 11:12:18 +07:00
Artem Tsebrovskiy
f14df3faa3
added support of Clique consensus for block processing for erigon2 (#3705)
* added support of Clique consensus for block processing for erigon2

* erigon2: remove block reader which doesn't use system pre and post block transactions
2022-03-17 07:30:47 +00:00
Alex Sharov
469b75c3d3
Snapshots: retire blocks by default (#3707) 2022-03-16 09:57:48 +07:00
ledgerwatch
8b00636e21
[erigon2] history indices based on Elias Fano (#3679)
* [erigon2] history indices based on Elias Fano

* Tidy

* Update

* tx lookup to support blocTo in integration

* Update to fix

* Update

* Fix

* Update to erigon-lib main

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-13 23:11:49 +00:00
Artem Tsebrovskiy
d300c2821b
erigon2: implemented chain switching by --chain flag (#3668)
erigon2: fix linter
2022-03-10 14:01:37 +00:00
battlmonstr
e6c056e666
Rename Datadir/dataDir to follow conventions. (#3576)
* rename from Datadir to DataDir (functions and public vars)
* rename dataDir to datadir (local variables)
2022-02-22 17:39:48 +00:00
Alex Sharov
f5da8833b6
erigon2: exec metric (#3567) 2022-02-22 08:55:49 +07:00
Alex Sharov
27b87f44d6
Erigon2: support block snapshots (#3557)
* save

* erigon

* save
2022-02-22 08:42:12 +07:00
ledgerwatch
e784f2b809
[erigon2] ETL optimisations (#3550) 2022-02-21 09:31:06 +07:00
ledgerwatch
80d7734ea8
[erigon2] fix history creation (txNum confusion) (#3547)
* More tracing

* More tracing

* More tracing

* Keep correct txNum

* Keep correct txNum

* Update to tracing

* Off by one

* Print memory info

* Quicker startup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-19 08:15:35 +00:00
ledgerwatch
c5b4d3e352
[erigon2] Thin commitment (2nd attempt) (#3515)
* Another fix for history files

* Fix NPE

* Another fix

* Proper closing sequence

* Remove first byte insert flag

* Fixes

* Update

* Add traceblock to erigon2 command

* Update

* Update

* Update

* Update

* print panic

* Fix history reads

* Update

* Update locks

* Update

* Fixes

* Set changesets and commitments upfront

* Logging instead of printing

* Fixes in aggregator and recsplit

* Update to erigon-lib

* Remove empty commitments

* Update

* Update

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-17 10:56:20 +07:00
Andrew Ashikhmin
7bad3a9c59
state.PlainState is now at the beginning of blockNbr, not the end (#3517)
* state.PlainState is now at the beginning of blockNbr, not the end (fix discprepancy with GetAsOf)

* Fix tests
2022-02-16 08:38:12 +00:00
ledgerwatch
c865290502
Add traceblock to erigon2 command (#3516)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-14 23:15:24 +00:00
ledgerwatch
dffcc57ecc
[erigon2] Fix history bug, optimise bitmap search (#3510)
* Fix history bug, optimise bitmap search

* Optimise SeekInBitmap64

* Update erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-13 20:02:18 +00:00
ledgerwatch
2074b98568
Tools for gathering data on erigon2 history and bitmaps (#3488)
* Tools for gathering data on erigon2 history and bitmaps

* Always start with 0

* Always start with 0

* Clean up commitment files, trace block

* traceblock

* Add tracing to checkChangeSets

* Fixes to DeleteAccount

* Update to latest erigon-lib

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-12 15:48:38 +00:00
Alex Sharov
c9cee7a884
path -> filepath (path package is for urls) (#3493) 2022-02-12 20:33:09 +07:00
ledgerwatch
eb6a168603
[erigon2] Chain history and bitmap indices (#3437)
* First update

* Update to latest erigon-lib

* Fix lint

* Update

* Update

* Update

* Add history2 command

* Updates

* Update erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-12 00:06:57 +00:00
ledgerwatch
093bfbad39
[erigon2] Optimisations (#3412)
* Update

* [erigon2] optimisations

* Update to latest erigon-lib

* Update to latest erigon-lib, fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-03 09:36:27 +00:00
ledgerwatch
4ef14215b4
[erigon2] three threads, better stats in the logs (#3378)
* [erigon2] Use compact keys in commitments

* Better stats in the logs

* Update

* Update

* Update

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-31 23:40:51 +00:00