Commit Graph

52 Commits

Author SHA1 Message Date
Alex Sharov
26fdf9169d
move all packages from "internal" folder - to simplify users live (#5857) 2022-10-25 09:58:25 +07:00
Alex Sharov
86551e82f4
e3: parallel build missed indices (#5817) 2022-10-21 13:45:20 +07:00
Max Revitt
07ffa36d44
File system logging (#5812)
- lives in internal/logging
- all log flags moved to internal/logging/flags
- allows continued use of root logger via log.Info etc.
- update logger to take change allowing string to lvl for 'trace'

Verbosity flag is overridden by log.console.verbosity. Logs will be
colocated if all run as one process, only split where progs are run as
separate processes, in a future update this will be addressed so for
example rpcdeamon will always log to it's own file
2022-10-20 19:25:06 +01:00
Krishna Upadhyaya
bac6b071ab
use HeimdallURL flag instead of hardcoded value (#5452) 2022-09-21 14:55:11 +07:00
ledgerwatch
693017c554
Cleanup Tevm experimental code (#5259)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-01 19:49:29 +01:00
Alex Sharov
de3a602a24
erigon22: bsc fixes 2022-08-13 16:09:52 +07:00
ledgerwatch
2b517b7630
Erigon22 prototype to support BSC (#4867)
* Reduce workers and queue size

* Add traces and event logs

* Add chain flag

* Print

* Open consensus db as read only

* Use only one engine instance

* chainDb

* Display palia error

* Not to panic

* Not to exit

* Add syscall

* Print

* Pointer to TxTask

* Not register senders

* Print

* Relax check

* Skip system transactions

* increase workerCount again

* Remove print

* Change recon

* Fix types

* Skip system txs

* Print

* No rules for genesis commit

* Print genesis

* Print|

* Print

* Print

* Update

* Fix

* Remove print

* No print

* Support contract upgrades

* Use the same dif

* Print create2

* Fix

* Fix

* Print

* Hack iterate

* Print empty vals

* Fix hack

* Code hashes

* Code hashes

* Print

* No panic

* No panic

* Fix?

* another incarnation fix

* Remove print

* Remove print

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-07-31 21:54:23 +01:00
Alex Sharov
aee4b53788
save list of snapshots in db (#4777)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
2022-07-28 16:57:38 +07:00
Artem Tsebrovskiy
9637b25a42
[erigon2] print trie variant derived from trie instead switch log (#4672)
* [erigon2] print trie variant derived from trie instead switch log

* updated erigon-lib version

* bumped erigon-lib version
2022-07-11 15:58:08 +01:00
sudeep
e13a318e0b
evm t8n to use ExecuteBlockEphemerally api (#4642)
* evm t8n tool to use ExecuteBlockEphemerally api (#4512)

* fix to set V, R, S in legacy transaction

* fix to dump post-execution alloc for evm t8n

* close tx in evm t8n

* populate current difficulty and gas used in output result

- update the ExecutionResult to include corresponding info (like
  Difficulty/GasUsed)

* initial attempt at migrating 'evm t8n' to use ExecuteBlockEphemerally

* using ExecutionResult in ExecuteBlockEphemerally

* bypass validations and integrate with EphemeralExecResult

* fixing output of 'evm t8n'

- remaining bits are "stateRoot" in results.txt and "balance" field for one account in
  alloc.txt (for testdata=1)

* get ExecuteBlockEphemerally to accept getTracer lambda

* fix build failure

* test cases for evm t8n

* more test cases for evm t8n

* fix stateRoot computation in evm t8n

* remove reward argument, as EBE itself takes care of it

* final cleanups for migration to using ExecuteBlockEphemerally

* change EBEforBSC to match EBE

* fix linter issues

* manually revert an unwanted diff

* avoid calculating ReceiptHash twice

* linter check

* minor correction

* remove unnecessary logic in EBEforBsc

* fix integration tests

* fix build
2022-07-07 12:47:00 +01:00
ledgerwatch
77dc35bb85
Revert "evm t8n tool to use ExecuteBlockEphemerally api (#4512)" (#4610)
* Revert "evm t8n tool to use ExecuteBlockEphemerally api (#4512)"

This reverts commit db93d2ea37.

* Fix compilation

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-07-02 23:24:20 +01:00
sudeep
db93d2ea37
evm t8n tool to use ExecuteBlockEphemerally api (#4512)
* fix to set V, R, S in legacy transaction

* fix to dump post-execution alloc for evm t8n

* close tx in evm t8n

* populate current difficulty and gas used in output result

- update the ExecutionResult to include corresponding info (like
  Difficulty/GasUsed)

* initial attempt at migrating 'evm t8n' to use ExecuteBlockEphemerally

* using ExecutionResult in ExecuteBlockEphemerally

* bypass validations and integrate with EphemeralExecResult

* fixing output of 'evm t8n'

- remaining bits are "stateRoot" in results.txt and "balance" field for one account in
  alloc.txt (for testdata=1)

* get ExecuteBlockEphemerally to accept getTracer lambda

* fix build failure

* test cases for evm t8n

* more test cases for evm t8n

* fix stateRoot computation in evm t8n

* remove reward argument, as EBE itself takes care of it

* final cleanups for migration to using ExecuteBlockEphemerally

* change EBEforBSC to match EBE

* fix linter issues

* manually revert an unwanted diff

* avoid calculating ReceiptHash twice

* linter check

* minor correction

* remove unnecessary logic in EBEforBsc
2022-07-02 06:52:23 +01:00
Enrique Jose Avila Asapche
9f1cc40c89
changed syncmode flag to snapshots flag (#4353)
* changed syncmode flag to snapshots flag

* using boolT

* Remove syncmode concept

* Add mumbai

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-05 07:33:55 +01:00
Enrique Jose Avila Asapche
bc21ab9d97
syncmode changes (#4334)
* log syncmode

* little comment

* ops

* calling log in correct place

* got rid of error instead we warn

* no return
2022-06-02 14:15:00 +01:00
Alex Sharov
0795bcb865
integration: validate syncmode flag (#4300) 2022-05-30 09:59:38 +07:00
Andrew Ashikhmin
0c76286509
Add rules to AsMessage to vet transaction types (#4276)
* Add rules to AsMessage to vet transaction types

* Fix TestState
2022-05-26 18:20:34 +02:00
Alex Sharov
66248c4bfb
Torrent: verify all files (#4270) 2022-05-26 12:27:44 +07:00
Alex Sharov
fc1a37e6dc
create turbo/services pkg (#4269) 2022-05-26 10:31:06 +07:00
Alex Sharov
c2e0956e3e
return err on invalid syncmode (#4219)
* save

* save

* save
2022-05-20 16:04:38 +07:00
Artem Tsebrovskiy
f4c8f44f61
removed obsolete trie variant (#4172) 2022-05-20 15:03:27 +07:00
Alex Sharov
726ef081ce
option to disable runtime.ReadMemStats (#4200) 2022-05-19 11:48:07 +07:00
ledgerwatch
14557a2348
[erigon2] Support for binary tree commitments (#4077)
* binary tree

* Binary commitment tree

* [erigon2] Bin tree support

* Point to latest erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-05-05 13:54:29 +01:00
ledgerwatch
285c782b1b
More efficient header verification of headers for Parlia when snapshots are used (#3998)
* Update stageloop.go

* Print

* Consider snapshot headers as parlia checkpoints

* Not fail after not loading snapshot

* Lazy snapshots

* Print number of validators

* More printing

* Use epoch instead of checkpoint interval

* Reduce logging

* Fix compilation

* Remove trace jump dest

* Fix lint

* Not store snapshots every epoch

* Separate snapshot for verification and finalisation

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-04-29 07:10:31 +01:00
Alex Sharov
831da9ba41
reduce downloader deps (#4011) 2022-04-28 12:42:14 +07:00
Alex Sharov
9b47daee44
Default syncmode=snap for Mainnet, Bsc, Goerli (#3861)
* change syncmode default to snap sync

* save
2022-04-10 07:20:20 +01:00
alex.sharov
10f195578b fix linter 2022-04-01 15:11:02 +07:00
Alex Sharov
d1218181b0
Snapshots: geth compatibility, use --syncmode=snap flag (#3808) 2022-04-01 15:00:50 +07:00
ledgerwatch
f8668da905
[erigon2] Bring back MDBX as storage for recent data (#3756)
* fixes

* Update

* Update to erigon-lib main

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-23 16:00:06 +00:00
ledgerwatch
c47cf1fd49
[erigon2] Optimise Huffman decoding (#3727)
* Update to erigon-lib

* Fix for block number

* Update

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-03-18 09:55:38 +00:00
Alex Sharov
f314222180
Snapshots: start seed new large .seg files (#3724) 2022-03-18 11:12:18 +07:00
Artem Tsebrovskiy
f14df3faa3
added support of Clique consensus for block processing for erigon2 (#3705)
* added support of Clique consensus for block processing for erigon2

* erigon2: remove block reader which doesn't use system pre and post block transactions
2022-03-17 07:30:47 +00:00
Alex Sharov
469b75c3d3
Snapshots: retire blocks by default (#3707) 2022-03-16 09:57:48 +07:00
ledgerwatch
8b00636e21
[erigon2] history indices based on Elias Fano (#3679)
* [erigon2] history indices based on Elias Fano

* Tidy

* Update

* tx lookup to support blocTo in integration

* Update to fix

* Update

* Fix

* Update to erigon-lib main

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-03-13 23:11:49 +00:00
Artem Tsebrovskiy
d300c2821b
erigon2: implemented chain switching by --chain flag (#3668)
erigon2: fix linter
2022-03-10 14:01:37 +00:00
battlmonstr
e6c056e666
Rename Datadir/dataDir to follow conventions. (#3576)
* rename from Datadir to DataDir (functions and public vars)
* rename dataDir to datadir (local variables)
2022-02-22 17:39:48 +00:00
Alex Sharov
f5da8833b6
erigon2: exec metric (#3567) 2022-02-22 08:55:49 +07:00
Alex Sharov
27b87f44d6
Erigon2: support block snapshots (#3557)
* save

* erigon

* save
2022-02-22 08:42:12 +07:00
ledgerwatch
e784f2b809
[erigon2] ETL optimisations (#3550) 2022-02-21 09:31:06 +07:00
ledgerwatch
80d7734ea8
[erigon2] fix history creation (txNum confusion) (#3547)
* More tracing

* More tracing

* More tracing

* Keep correct txNum

* Keep correct txNum

* Update to tracing

* Off by one

* Print memory info

* Quicker startup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-19 08:15:35 +00:00
ledgerwatch
c5b4d3e352
[erigon2] Thin commitment (2nd attempt) (#3515)
* Another fix for history files

* Fix NPE

* Another fix

* Proper closing sequence

* Remove first byte insert flag

* Fixes

* Update

* Add traceblock to erigon2 command

* Update

* Update

* Update

* Update

* print panic

* Fix history reads

* Update

* Update locks

* Update

* Fixes

* Set changesets and commitments upfront

* Logging instead of printing

* Fixes in aggregator and recsplit

* Update to erigon-lib

* Remove empty commitments

* Update

* Update

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-17 10:56:20 +07:00
ledgerwatch
c865290502
Add traceblock to erigon2 command (#3516)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-02-14 23:15:24 +00:00
Alex Sharov
c9cee7a884
path -> filepath (path package is for urls) (#3493) 2022-02-12 20:33:09 +07:00
ledgerwatch
eb6a168603
[erigon2] Chain history and bitmap indices (#3437)
* First update

* Update to latest erigon-lib

* Fix lint

* Update

* Update

* Update

* Add history2 command

* Updates

* Update erigon-lib

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-12 00:06:57 +00:00
ledgerwatch
093bfbad39
[erigon2] Optimisations (#3412)
* Update

* [erigon2] optimisations

* Update to latest erigon-lib

* Update to latest erigon-lib, fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-02-03 09:36:27 +00:00
ledgerwatch
4ef14215b4
[erigon2] three threads, better stats in the logs (#3378)
* [erigon2] Use compact keys in commitments

* Better stats in the logs

* Update

* Update

* Update

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-31 23:40:51 +00:00
ledgerwatch
079a177614
[erigon2] Computing data dependencies between transactions (#3290)
* Use ibs per transaction instead of per block

* Remove copy functions, introduce balance increase map

* Fix test

* Fix lint

* Introduce balanceIncrease journal action

* Introduce balanceIncrease journal action

* Simplifications

* Fix

* Reset

* Simplify

* Fix

* Fixes

* Fix for reading nil accounts

* Fix edge cases with reverts

* Clean up test output

* Fix transfers

* Handle special case of ripemd

* Try with fixed compress, only commit every commfreq blocks

* Updates

* Updates

* Update to fixed erigon-lib

* Update to erigon-lib

* Remove stop

* Update to latest erigon-lib

* fixup

* Cleanup

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-24 23:12:25 +00:00
ledgerwatch
1af72faf71
[erigon2] optional writing of "before" value for changes, decouple commitment from blocks (#3272)
* Sepatate commitment calculation from blocks

* Fix

* Fix lint

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-16 21:24:54 +00:00
ledgerwatch
17aff6090f
[erigon2] Resumable prototype (#3263)
* [erigon2] Resumable prototype

* Fix compile error

* Resumable prototype

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-16 08:40:16 +00:00
ledgerwatch
c774bff61e
Update erigon2.go (#3266) 2022-01-15 09:41:56 +00:00
ledgerwatch
ea6dbd49e8
[erigon2] Running with commitment (#3203)
* [erigon2] Running with commitment

* [erigon2] Running with commitment

* [erigon2] Running with commitment

* go mod tidy

* [erigon2] Running with commitment

* More

* Debug

* fix

* Fix

* state root command

* More fixes

* Fix

* Progress to 164735

* Only trace when failing

* fix for firstInsert

* Over block 1.36m

* Update

* fix to deleteAccount

* Fixes for plainKeys

* Undo printing

* No changeset generation by default

* Print block number on interrupt, fix lint

* Fix lint

* Open history DB as read only

* Print error

* Open non read only

* Readonly again

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-01-14 16:02:44 +00:00