Commit Graph

2862 Commits

Author SHA1 Message Date
Alex Sharov
3c7e6b114e
e3: locality index (#6513) 2023-01-07 12:31:00 +07:00
ledgerwatch
2941e754e9
Fix hive tests and reorganise the fix for body download problem (#6515)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-01-06 12:43:46 +00:00
Alex Sharov
4c3bb1cca5
kv_temporal: DomainGet (#6511) 2023-01-06 14:27:54 +07:00
Andrew Ashikhmin
0b8f4aea2f
Withdrawals in body downloader (#6496) 2023-01-04 10:09:44 +01:00
Alex Sharov
a4006ad2af
Integration: configure correct consensus engine (#6492) 2023-01-03 12:20:18 +07:00
hexoscott
1a09dcbdb3
use set rather than slice for tracking yielded when mining (#6488)
make use of sets when tracking yielded transactions during mining
2023-01-02 13:00:08 +00:00
Alex Sharov
11637a5c7d
e3: force merge snapshots before reconst (#6484) 2023-01-02 10:10:48 +07:00
Andrew Ashikhmin
58893937b0
core/vm: Make INVALID a defined opcode (#6477)
Cherry-pick https://github.com/ethereum/go-ethereum/pull/24017.

* core/vm: Define 0xfe opcode as INVALID

* core/vm: Remove opInvalid as opUndefined handles it

Co-authored-by: Alex Beregszaszi <alex@rtfs.hu>

Co-authored-by: Andrei Maiboroda <andrei@ethereum.org>
Co-authored-by: Alex Beregszaszi <alex@rtfs.hu>
2022-12-30 17:13:54 +01:00
Alex Sharov
4a9c871628
a bit rename 22 to v3 (#6476) 2022-12-30 21:53:42 +07:00
Alex Sharov
b0af04f2df
bsc: incrementally calc and store all old snapshots. slower initial stage_snapshots, but faster restart during initial sync (#6457) 2022-12-30 12:44:54 +07:00
Giulio rebuffo
4932aa5264
added indexes to beacon blocks (#6460) 2022-12-29 15:15:25 +01:00
Alex Sharov
e7104c9a68
e3: support withdrawals (#6467) 2022-12-29 19:57:58 +07:00
alex.sharov
82e7893837 e3: remove debug line 2022-12-29 18:52:33 +07:00
alex.sharov
ed94d60671 e3: disable some checks for genesis, to fix tests 2022-12-29 16:04:56 +07:00
Alex Sharov
c04668d0ff
e3: agg cancel background jobs (#6463) 2022-12-29 15:04:07 +07:00
Andrew Ashikhmin
0fdd60a0d1
Execution Spec Tests & HashCheck() (#6444)
Run tests from a new repo
https://github.com/ethereum/execution-spec-tests.

Also introduce `HashCheck()` function that checks correctness of block's
uncle, transaction, and withdrawals hashes.
2022-12-28 17:01:40 +01:00
ledgerwatch
387b4d7f35
Native tracers step 10 - remove duplicate SelfDestruct capture, clean up (#6440)
Moved some of the tracers to `eth/tracers/logger` to make it more
similar to go-ethereum.

Removed Erigon-specific `Capture-` functions

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2022-12-26 04:56:39 +00:00
Alex Sharov
8161e17694
e3: fix non-parallel nil (#6438) 2022-12-25 19:48:44 +07:00
ledgerwatch
24040f3044
Native tracers step 9 - add native tracers (#6434)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2022-12-25 07:28:41 +00:00
ledgerwatch
8198cc2740
Native tracers step 8 - added trace test suite, fixed some bugs (#6430)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2022-12-24 13:18:11 +00:00
Alex Sharov
4e26f53a75
e3: return error from inline-apply (#6427) 2022-12-24 17:30:26 +07:00
Alex Sharov
2fa3f69a1f
stage_snapshots: clean table before initial write (#6431) 2022-12-24 16:15:24 +07:00
Alex Sharov
ade933be6b
kv_remote: server to support thread-safe multi-streams per 1 tx (#6402) 2022-12-24 13:11:15 +07:00
ledgerwatch
f364eff389
Native tracers step 7 - restructure js tracer files (#6415)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2022-12-23 18:10:37 +00:00
Andrew Ashikhmin
4931f113c2
Add override.shanghaiTime flag (#6414)
and remove `--override.terminaltotaldifficulty` &
`--override.mergeNetsplitBlock`
2022-12-23 17:06:56 +01:00
ledgerwatch
a49e287618
Native tracers step 6: Rename Tracer to EVMLogger, reduce args CaptureExit (#6410)
Getting closer to Geth's tracer interfaces

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2022-12-23 05:43:08 +00:00
Alex Sharov
130ab85bea
e3: kv/temporal prototype 3 (#6395) 2022-12-22 09:37:32 +07:00
Max Revitt
4bdb35f94c
Withdrawals part 2 (#6180)
Continuation of PR #6009. Storage and retrieval of withdrawals.

- [x] storage of withdrawals on writeblock
- [x] composite key (block number | block hash)
- [x] value ([]withdrawal)
- [x] retrieval of withdrawals with block
- [x] tests around storing/retrieving withdrawals
- [x] tested in hive
- [x] commits tidied


## Hive Failures
- [x] Withdrawals Fork on Block 2
- [x] Withdrawals Fork on Block 3
- [ ] Sync after 2 blocks - Withdrawals on Block 1 - Single Withdrawal
Account - No Transactions ( Syncing client rejected valid chain)
- [ ] Sync after 2 blocks - Withdrawals on Block 1 - Single Withdrawal
Account (Timeout while waiting for secondary client to sync)
- [ ] Sync after 2 blocks - Withdrawals on Genesis - Single Withdrawal
Account (Timeout while waiting for secondary client to sync)
- [ ] Sync after 2 blocks - Withdrawals on Block 2 - Single Withdrawal
Account - No Transactions (Syncing client rejected valid chain)
- [ ] Sync after 2 blocks - Withdrawals on Block 2 - Single Withdrawal
Account (Timeout while waiting for secondary client to sync)
- [ ] Sync after 128 blocks - Withdrawals on Block 2 - Multiple
Withdrawal Accounts (Unexpected error on BalanceAt: Post
"http://172.17.0.4:8545/": context deadline exceeded, expected=<None>)
- [ ] Withdrawals Fork on Block 1 - 8 Block Re-Org, Sync (Expected no
error on EngineNewPayloadV2: error=Post "http://172.17.0.4:8551/":
context deadline exceeded)
- [ ] Withdrawals Fork on Block 8 - 10 Block Re-Org Sync (Expected no
error on EngineNewPayloadV2: error=Post "http://172.17.0.4:8551/":
context deadline exceeded)
- [ ] Withdrawals Fork on Canonical Block 8 / Side Block 7 - 10 Block
Re-Org Sync (Expected no error on EngineNewPayloadV2: error=Post
"http://172.17.0.4:8551/": context deadline exceeded)
- [ ] Withdrawals Fork on Canonical Block 8 / Side Block 9 - 10 Block
Re-Org Sync (Expected no error on EngineNewPayloadV2: error=Post
"http://172.17.0.4:8551/": context deadline exceeded)
2022-12-20 09:09:04 +00:00
Alex Sharov
cce0f251b4
e3: kv/temporal prototype 2 (#6377) 2022-12-20 09:28:10 +07:00
Alex Sharov
dfa6505f93
e3: kv/temporal prototype (#6367) 2022-12-19 15:38:54 +07:00
ledgerwatch
97c9a9108d
Native tracers - step 4 (#6363)
Remove `callType` argument from `CaptureStart`, and change `callType
vm.CallType` to `typ vm.OpCode` in `CaptureEnter` and move to the first
position (as it is in geth)

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2022-12-19 03:12:08 +00:00
ledgerwatch
0761671607
Native tracers step 3 (#6362)
Remove `env` argument from all Capture functions except for
`CaptureStart`

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2022-12-18 17:06:40 +00:00
ledgerwatch
b8c6a4d263
Native tracers - step 2 (#6360)
Splitting function `CaptureStart` into `CaptureStart` (when depth == 0)
and `CaptureEnter` (when depth > 0), while removing argument `depth`.
Same with splitting `CaptureEnd` into `CaptureEnd` and `CaptureExit`

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2022-12-18 16:11:31 +00:00
ledgerwatch
3999c46046
Native tracers - step 1 (#6356)
In this first step, the new functions `CaptureTxStart` and
`CaptureTxEnd` are introduced to all tracers

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2022-12-18 04:36:57 +00:00
Alex Sharov
4ef39ea43c
unify logs a bit (#6354) 2022-12-18 09:23:11 +07:00
hexoscott
d4623ec9d7
mining updates around txpool yielding (#6342)
Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
2022-12-18 08:45:03 +07:00
Giulio rebuffo
0a3bfef484
Added prototype for Beacon history reconstruction stage (#6351) 2022-12-17 16:05:56 +01:00
awskii
222dc96581
add missed break clause during protocols port picking (#6343)
Missing break clause leaded to unnecessary incrementing of `pi` when
port is already picked.
2022-12-16 16:59:22 +00:00
Andrew Ashikhmin
0fc2022a0e
Extend fork ID to timestamp-based forks (#6324)
Starting from
[Shanghai](https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/shanghai.md),
forks are based on timestamps rather than block heights (see PR #6238).
This PR extends [EIP-2124](https://eips.ethereum.org/EIPS/eip-2124) Fork
ID to include timestamp-based blocks. See also
https://github.com/ethereum/go-ethereum/pull/25878.

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2022-12-16 12:08:54 +01:00
Alex Sharov
d95f0ca51e
e3: apply inline-retry (#6336) 2022-12-16 14:14:00 +07:00
Alex Sharov
43f83cd7d8
e3: use of ordered-map generic (#6335) 2022-12-16 13:59:57 +07:00
Alex Sharov
895fe97f29
e3: fix devel overlap tx (#6334) 2022-12-16 13:13:18 +07:00
Alex Sharov
625bb39a82
E3: unlimited inline-apply (#6333) 2022-12-16 12:22:52 +07:00
Alex Sharov
a8c5318c56
e3: exec tx right after conflict (but only 1 such exec per applyLoopStep) (#6332) 2022-12-16 11:10:12 +07:00
ledgerwatch
e4d4a985b4
Demote TxLookup warning to debug (#6327) 2022-12-15 15:24:29 +00:00
Alex Sharov
b2bf12827c
e3: cancel background build (#6323) 2022-12-15 17:18:00 +07:00
Alex Sharov
863e1ed07b
e3: handle "kill -9" during merge, auto-hide history .ef file if not corresponding .v file (#6304) 2022-12-15 13:49:06 +07:00
Alex Sharov
7de7b37418
hash_state/ih: enable ReadAhead for "promote cleanly" case (#6317) 2022-12-15 09:25:31 +07:00
Igor Mandrigin
786610df3f
Fix debug_traceTransaction on Gnosis Chain (#6311)
Support 0-gas service transactions in tracing module.
2022-12-14 14:35:28 +01:00
Alex Sharov
1b4dbefa95
grafana/prometheus version up (#6301) 2022-12-14 10:57:47 +07:00
Alex Sharov
ce9203d80d
e3: more aggressive prune #780 (#6299) 2022-12-14 09:50:48 +07:00
Alex Sharov
f92f525d8f
e3: return err from parallel exec (#6283) 2022-12-14 09:02:57 +07:00
Andrew Ashikhmin
1e206e375b
Set default for externalcl based on whether embedded CL is supported (#6294)
Embedded CL is not supported for Gnosis Chain, so it makes sense to set
`externalcl` to true by default for it.

Also, this PR sets `terminalTotalDifficultyPassed` for Gnosis Chain &
Chiado (see https://docs.gnosischain.com/updates/20221210-merge).
2022-12-13 14:51:13 +01:00
awskii
a124bcf6ac
added --p2p.allowed-port flag (#6263)
Regarding https://github.com/ledgerwatch/erigon/issues/6260

added flag `--p2p.allowed-ports=<porta>,<portb>` to restrict which ports
to use for sentries for different protocol versions.

Default for this flag is `30303, 30304` (first port is inherited from
`--port` flag defaults.
If `--port` is changed and it's new value is not presented in allowed
port list, provided port will be allowed as well as list provided via
`--p2p.allowed-ports`

Port picking is straightforward, we create sentry gRPC server for
protocol over first allowed port that is not already taken.
If there are no allowed ports left, erigon exits with hint.
2022-12-12 13:25:47 +00:00
hexoscott
c0d2523b06
reset txpool yielded flags before starting mining (#6284) 2022-12-12 17:27:57 +07:00
Alex Sharov
9d20045ad3
e3: reconDB - to use 64kb pageSize (#6282) 2022-12-12 14:06:33 +07:00
Giulio rebuffo
0f0d0de783
Use of proper beacon state interface (#6277) 2022-12-11 15:12:38 +01:00
Alex Sharov
7a055a51d4
e3: graceful shutdown (#6272) 2022-12-11 13:15:55 +07:00
Andrew Ashikhmin
ecefc53dd0
Remove Ropsten & Fermion (#6262)
See https://blog.ethereum.org/2022/11/30/ropsten-shutdown-announcement
2022-12-10 22:55:31 +00:00
ledgerwatch
b5a7faa8db
[e3] Incremental reconstitution (#6270)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-12-10 09:34:17 +00:00
Alex Sharov
e21b1b6e25
e3: commit earlier (#6271) 2022-12-10 16:00:19 +07:00
Alex Sharov
b4fd07b671
e3: earlier flush (#6227) 2022-12-10 15:53:58 +07:00
hexoscott
47a6f86f76
use a shared simulated tx when mining (#6253)
This ensures that the filter bad transactions can use the same context
each time, so increases an account nonce in the first batch of
transactions is committed to the simulation just in case the same
account appears in the next batch of transactions.

Also makes use of the new txpool candidate functionality to ensure we're
getting fresh transactions each time.
2022-12-09 21:16:10 +07:00
Alex Sharov
5c05ff4c2a
Exec new blocks in TxnNoSync. Prune after sending notifications in TxnSync (#6222)
Change from: 
```
begin(TxNoSync)
exec new block, index nee data
do limited pruning 
commit()
send notifications about new data arrival to other apps, they may start reading new data at this time (by new read transactions) 
```
Change to: 
```
begin(TxNoSync)
exec new block, index nee data
commit() // no fsync here
send notifications about new data arrival to other apps, they may start reading new data at this time (by new read transactions) 
begin()
do pruning 
commit() // fsync here
```

it allows notify earlier. Fsync (of all changes) on modern drives is
fast, but on cloud-drives it’s about 1sec in worst cases.
2022-12-09 19:48:56 +07:00
hexoscott
c48040ca33
fix nil pointer in getting next transactions mining_exec (#6242) 2022-12-08 05:50:56 +07:00
Andrew Ashikhmin
7eab12d90d
Shanghai & Cancun forks should be based on timestamp (#6238) 2022-12-07 18:45:44 +01:00
hexoscott
d8d46b6312
fix v2 for mining stage exec (#6239) 2022-12-07 17:33:51 +00:00
Alex Sharov
bdc9da0162
e3: don't jump over StageSenders by reconstitution (#6218) 2022-12-06 13:00:23 +07:00
Krishna Upadhyaya
03d0dfe5ce
Update TxLookupRange error to warning (#6213) 2022-12-05 20:22:31 +07:00
Alex Sharov
f9cf18a4ac
e3: pool read list (#6210) 2022-12-05 18:48:08 +07:00
Alex Sharov
061c4ef744
E3: some clean (#6209) 2022-12-05 10:23:49 +07:00
Håvard Anda Estensen
1398703bc5
Use hex package to convert bytes to string (#6205) 2022-12-05 09:06:16 +07:00
Giulio rebuffo
6e76643c09
Added Handshake protocol to Erigon-CL Lightclient and Fullclient (#6206) 2022-12-05 01:25:12 +01:00
Alex Sharov
7e6a177eec
e3: recon speedup (#6201) 2022-12-04 16:11:24 +07:00
Nicolas Gotchac
961a0070cc
Fix trace error in Polygon | Pass Engin to the Base API (#6131)
So there is an issue with tracing certain blocks/transactions on
Polygon, for example:
```
> '{"method": "trace_transaction","params":["0xb198d93f640343a98f90d93aa2b74b4fc5c64f3a649f1608d2bfd1004f9dee0e"],"id":1,"jsonrpc":"2.0"}'
```
gives the error `first run for txIndex 1 error: insufficient funds for
gas * price + value: address 0x10AD27A96CDBffC90ab3b83bF695911426A69f5E
have 16927727762862809 want 17594166808296934`

The reason is that this transaction is from the author of the block,
which doesn't have enough ETH to pay for the gas fee + tx value if he's
not the block author receiving transactions fees.

The issue is that currently the APIs are using `ethash.NewFaker()`
Engine for running traces, etc. which doesn't know how to get the author
for a specific block (which is consensus dependant); as it was noting in
several TODO comments.

The fix is to pass the Engine to the BaseAPI, which can then be used to
create the right Block Context. I chose to split the current Engine
interface in 2, with Reader and Writer, so that the BaseAPI only
receives the Reader one, which might be safer (even though it's only
used for getting the block Author).
2022-12-04 12:17:39 +07:00
Alex Sharov
82c7779a85
evm: txContext.gasPrice to uint256 type (#6188) 2022-12-04 11:44:50 +07:00
Håvard Anda Estensen
31ec791d5a
Avoiding leaking the popped item (#6193)
Set the deleted item to its zero val so it can be garbage collected.
2022-12-04 11:17:02 +07:00
Alex Sharov
0efda1f19f
trace: change type of self destruct arg to uint256 (#6189) 2022-12-03 21:43:53 +07:00
alex.sharov
8108085012 fix integration merge conflict 2022-12-03 13:38:57 +07:00
Alex Sharov
9b65c533c9
e3: optimize incremental hashstate (#6179) 2022-12-03 12:23:01 +07:00
hexoscott
09a01bc6a0
txpool to pull blocks of transactions until no gas remains for the block (#6160)
Moving the txpool transaction pull into the execution phase and looping
until the gas is used up or the txpool runs dry.

Removed the concept of local vs remote transactions as comments/code
showed this split was no longer in use.

Created a `PreparedTxs` collection to satisfy the use case in the
integration tool when mining is active.

Untested locally as I have no way of mining/validating currently.

Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
2022-12-03 12:20:47 +07:00
Alex Sharov
5f5a9c23cd
e3: support of --snap.stop (#6175) 2022-12-02 09:03:29 +07:00
Andrew Ashikhmin
d82c778ab3
Withdrawals part 1 (#6009)
This PR partially implements
[EIP-4895](https://eips.ethereum.org/EIPS/eip-4895): Beacon chain push
withdrawals as operations. The new Engine API methods
(https://github.com/ethereum/execution-apis/pull/195) are implemented.

_Body downloader and saving withdrawals into DB are not implemented
yet!_
2022-12-01 09:15:01 +01:00
Alex Sharov
fd3c44f6e1
e3: fix dao integration test (#6171) 2022-12-01 14:42:39 +07:00
alex.sharov
5d9e74d39c save 2022-12-01 13:47:35 +07:00
Alex Sharov
81b438a6e2
e3: option to discard all history writes. useful to test execution performance. (#6157) 2022-11-30 16:06:53 +07:00
Alex Sharov
60cb4e2bbb
evm tracing interface to use uint256, to avoid value.ToBig() allocations (#5781) 2022-11-30 08:31:39 +07:00
Alex Sharov
16cd87748f
E3: fix unwind changes visibility (#6147) 2022-11-30 08:31:13 +07:00
J1ang
83dae8dad3
feat: enhance API erigon_getLatestLogs (#6078)
feat:
1. `erigon_getLatestLogs` doesn't have to match the exact position of
the topics. It will match logs that contain the topics regardless of the
topics' position with original bloom filter. And it accepts `blockCount`
& `crit.ToBlock` params for better pagination.
2022-11-29 15:14:41 +07:00
Manav Darji
80e29f29d3
sprint length and base fee denominator change for mumbai testnet (#6142)
This PR includes changes required for delhi hard fork schedule at block
`29638656` on mumbai testnet. It changes few major parameters.

1. Sprint length - the number of bor blocks post which a new validator
mines has been reduced from 64 to 16.
2. Block time - the block time which was increased earlier for some
experiments to 5 seconds has been reduced to 2 seconds (along with
backup multiplier and producer delay).
3. Base fee denominator - this fields has been increased from 8 to 16 to
smoothen the effect of EIP 1559.
2022-11-29 08:11:29 +07:00
Giulio rebuffo
8c38eeda5a
Added skeleton for beacon chain state transition (#6135) 2022-11-27 00:03:58 +01:00
Giulio rebuffo
8c67c55cc7
Created Erigon beacon downloader (#6130) 2022-11-25 16:38:22 +01:00
Andrew Ashikhmin
03057c4e61
Update consensus tests to v11.2 (#6111)
[v11.2](https://github.com/ethereum/tests/releases/tag/v11.2) includes,
among other things, tests for
[EIP-3860](https://eips.ethereum.org/EIPS/eip-3860): Limit and meter
initcode.
2022-11-22 16:07:31 +01:00
Alex Sharov
6cf3af8208
e3: some recon logs (#6107) 2022-11-22 10:49:33 +07:00
Alex Sharov
c582f750be
e3: 1-thread exec, send signal (#6101) 2022-11-21 17:22:51 +07:00
Alex Sharov
9bcd7fe6c3
e3: reduce state write lock (#6096) 2022-11-21 10:38:22 +07:00
Alex Sharov
7d485414c0
TxLookup: limit incremental prune range (#6086) 2022-11-20 13:03:21 +07:00
Alex Sharov
40264fd4d9
E3: state lock reduce (#6088) 2022-11-20 12:34:45 +07:00
Alex Sharov
92e6e56120
Fix test e3 (#6087) 2022-11-20 10:58:20 +07:00
Alex Sharov
8afeee56c8
Downloader extract, step2 (#6076) 2022-11-20 10:41:30 +07:00