Commit Graph

16980 Commits

Author SHA1 Message Date
awskii
e10e75fb3e
rename erigon23->erigon4 (#6128)
Rename directories db23/erigon23 to db4/erigon4
rename file ergion23.go to erigon4.go as well as `state` command (to
`erigon4`)
2022-11-25 15:11:01 +07:00
Max Revitt
8cbf316e49
dep(erigon-lib): update to version with hotfix for ipv6 detection (#6121)
#6117
2022-11-24 21:34:04 +07:00
Max Revitt
79fc64e7f2
fix(hive): prevent concurrent execution cancellation for same commit (#6122) 2022-11-24 21:33:43 +07:00
Alex Sharov
a0d2d585b8
sentinel: add logs prefix (#6124) 2022-11-24 21:29:51 +07:00
Alex Sharov
975c4960c7
mdbx: remove deprecated cmp_suffix32 feature (#6118) 2022-11-24 09:25:34 +07:00
Max Revitt
f362856836
ci(hive): also allow workflow_dispatch to trigger (#6115) 2022-11-24 09:11:00 +07:00
Max Revitt
1122e865be
ci(hive): use run context rather than artifacts for results (#6114)
When running from forks results would need to be uploaded as artifacts
and separately parsed. This is no longer true when running as a
scheduled job and we can therefore fix and get a speedup by reading the
results straight from the workspace.
2022-11-23 15:02:53 +00:00
Andrew Ashikhmin
03057c4e61
Update consensus tests to v11.2 (#6111)
[v11.2](https://github.com/ethereum/tests/releases/tag/v11.2) includes,
among other things, tests for
[EIP-3860](https://eips.ethereum.org/EIPS/eip-3860): Limit and meter
initcode.
2022-11-22 16:07:31 +01:00
Philippe Schommers
6f61f556a2
test: add fork id tests for Gnosis and Chiado (#6110)
Feels like there's an issue with the latest hardfork
(https://docs.gnosischain.com/specs/hard-forks/21735000), as the fork id
didn't change?

Related: https://github.com/NethermindEth/nethermind/pull/4925
2022-11-22 16:03:36 +01:00
Giulio rebuffo
a1de39d25c
added beacon state-transition first structure (#6105) 2022-11-22 15:36:08 +01:00
Leonard Chinonso
f76736e14a
Implemented mining of contract transactions on the devnet tool (#6102)
- Added a new method and type for contract transactions.
- Added functions to emit fallback events from contract transactions.
- Added GetLogs request generator
- Added tests for GetLogs request generator
2022-11-22 20:28:53 +07:00
Mike Neuder
9352ea3b57
Add get blocks by range request to recent block sync (#6104)
Part of https://github.com/ledgerwatch/erigon/issues/5965.
2022-11-22 18:28:15 +07:00
awskii
cd508a65c5
E4 commit key replace (#6109)
https://github.com/ledgerwatch/erigon-lib/pull/684#issuecomment-1321577450
2022-11-22 11:02:43 +00:00
Max Revitt
4669954fc5
ci(hive): move hive run to nightly job (#6100)
- remove hive run from CI yaml
- create nightly hive yaml which runs hive at 1am each day on a hosted
runner (can move to self hosted if needs be in future)
2022-11-22 17:12:17 +07:00
alexqrid
e81c074fe6
added miner address extracting for Bor in block api response (#6080)
This PR adds correct miner address to the response of
`eth_getBlockByNumber`/`eth_getBlockByHash` API methods.

Currently miner address is returned as zero address (`0x000...000`) for
Polygon -
https://wiki.polygon.technology/docs/edge/faq/validators/#why-do-the-json-rpc-commands-eth_getblockbynumber-and-eth_getblockbyhash-not-return-the-miners-address
.
Unfortunately this behaviour confuses a lot, I've succeeded in restoring
the miner address in block related API, want this to be the default
behaviour.

Co-authored-by: alexqrid <>
2022-11-22 17:11:44 +07:00
Manav Darji
d08c052193
consensus/bor: return txs and receipts in FinalizeAndAssemble (#6108)
Due to a change in a previous
[commit](https://github.com/ledgerwatch/erigon/pull/5212/files#diff-335eb65f5a52a99fd4cb182ce6385022828dfc04118b85336754d5a142f45be7R134),
the txs and receipts are overridden. This breaks compatibility with bor
consensus as the `bor.FinalizeAndAssemble` method returns nil txs and
receipts. This PR fixes that.
2022-11-22 13:06:18 +07:00
Alex Sharov
6cf3af8208
e3: some recon logs (#6107) 2022-11-22 10:49:33 +07:00
Mike Neuder
f33ec06572
Add Status fetching and made sentinel request logic generic (#6094)
Part of https://github.com/ledgerwatch/erigon/issues/5965.

An example logging output shows that we are getting accurate data:

the current finalized checkpoint block:
https://beaconcha.in/slot/5180288
the current head block: https://beaconcha.in/slot/5180361

```
INFO[11-20|16:52:36.631] Retrieved status.                        status="&{ForkDigest:[74 38 197 139] FinalizedRoot:[133 83 36 34 8 25 208 12 77 180 192 189 151 179 237 244 192 20 216 226 40 178 142 28 242 91 4 66 250 58 200 92] FinalizedEpoch:161884 HeadRoot:[164 219 216 224 200 71 201 76 70 253 61 144 64 238 11 14 224 173 49 5 107 109 238 32 215 126 85 23 10 170 107 223] HeadSlot:5180361}"
INFO[11-20|16:52:36.631] Current finalized root.                  root=855324220819d00c4db4c0bd97b3edf4c014d8e228b28e1cf25b0442fa3ac85c
INFO[11-20|16:52:36.631] Current finalized epoch.                 epoch=161884
INFO[11-20|16:52:36.631] Current head root.                       root=a4dbd8e0c847c94c46fd3d9040ee0b0ee0ad31056b6dee20d77e55170aaa6bdf
INFO[11-20|16:52:36.631] Current head slot.                       slot=5180361
```
2022-11-21 18:50:20 +01:00
Andrew Ashikhmin
0b28c7ad13
Support genesis constructors for Chiado (#6103)
This is a continuation of PR #6058 and fixes Chiado genesis state root
by supporting genesis constructors (deployment code).
2022-11-21 17:21:59 +01:00
Alex Sharov
c582f750be
e3: 1-thread exec, send signal (#6101) 2022-11-21 17:22:51 +07:00
Alex Sharov
e47c4f7a7a
e3: use usual headerfunc (#6099) 2022-11-21 12:39:51 +07:00
Alex Sharov
0bc56a4fd1
Revert "e3: enable integration testss" (#6098)
Reverts ledgerwatch/erigon#6097
2022-11-21 11:21:23 +07:00
Alex Sharov
17bbc7f262
e3: enable integration testss (#6097) 2022-11-21 10:50:01 +07:00
Alex Sharov
9bcd7fe6c3
e3: reduce state write lock (#6096) 2022-11-21 10:38:22 +07:00
Giulio rebuffo
d745b39df9
skip connection to private IPs (#6093)
Addresses #6079
2022-11-20 22:41:11 +01:00
Mike Neuder
45938124f5
WIP sync from checkpoint (#6042)
https://github.com/ledgerwatch/erigon/issues/5965. This PR starts the
implementation of syncing from a checkpoint to head.

Co-authored-by: Andrew Ashikhmin <34320705+yperbasis@users.noreply.github.com>
Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
Co-authored-by: Giulio rebuffo <giulio.rebuffo@gmail.com>
Co-authored-by: Nebojsa Urosevic <nebojsa94@users.noreply.github.com>
Co-authored-by: Mark Shields <4237425+beejiujitsu@users.noreply.github.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Enrique Jose  Avila Asapche <eavilaasapche@gmail.com>
Co-authored-by: Nathan (Blaise) Bruer <thegreatall@gmail.com>
Co-authored-by: lupin012 <58134934+lupin012@users.noreply.github.com>
Co-authored-by: Philippe Schommers <philippe@schommers.be>
2022-11-20 18:44:42 +01:00
Giulio rebuffo
487ea4e5f3
Refactored Lightclient prefix system (#6092) 2022-11-20 17:16:30 +01:00
Giulio rebuffo
5c5c19aa4c
added basic lightclient stream handlers (#6085) 2022-11-20 14:23:51 +01:00
Alex Sharov
7d485414c0
TxLookup: limit incremental prune range (#6086) 2022-11-20 13:03:21 +07:00
Alex Sharov
40264fd4d9
E3: state lock reduce (#6088) 2022-11-20 12:34:45 +07:00
Alex Sharov
68f3edf06a
devel fix (#6089) 2022-11-20 11:01:25 +07:00
Alex Sharov
92e6e56120
Fix test e3 (#6087) 2022-11-20 10:58:20 +07:00
Alex Sharov
8afeee56c8
Downloader extract, step2 (#6076) 2022-11-20 10:41:30 +07:00
Philippe Schommers
806117fbab
feat: add chiado config (#6058)
Hey guys, I'm trying to add the Chiado network ([Gnosis'
testnet](https://docs.gnosischain.com/about/networks/chiado)) now that
Gnosis can be synced fully with Erigon, so we can test it on the testnet
as well.

This is mostly inspired from
cd5ef32f37.

Probably missing:
- [ ] The right consensus config (currently only a copy of Gnosis)
- [ ] Fixes to the chainspec?
- [ ] Presumably something in `cl/clparams/config.go`

Current state:
```
$ ./build/bin/erigon --chain=chiado --log.console.verbosity=debug
WARN[11-16|11:52:28.188] no log dir set, console logging only 
WARN[11-16|11:52:28.193] no log dir set, console logging only 
INFO[11-16|11:52:28.193] Build info                               git_branch=feat/chiado git_tag=v2021.10.03-2291-g17fae73f8 git_commit=17fae73f8af5348ba7c04684f2a2978daf81b67e
INFO[11-16|11:52:28.193] Starting Erigon on                       devnet=chiado
INFO[11-16|11:52:28.194] Maximum peer count                       ETH=100 total=100
INFO[11-16|11:52:28.194] starting HTTP APIs                       APIs=eth,erigon,engine
INFO[11-16|11:52:28.194] torrent verbosity                        level=WRN
INFO[11-16|11:52:30.300] Set global gas cap                       cap=50000000
INFO[11-16|11:52:30.302] Opening Database                         label=chaindata path=/home/filoozom/.local/share/erigon/chiado/chaindata
INFO[11-16|11:52:30.310] Initialised chain configuration          config="{ChainID: 10200, Homestead: 0, DAO: <nil>, DAO Support: false, Tangerine Whistle: 0, Spurious Dragon: 0, Byzantium: 0, Constantinople: 0, Petersburg: 0, Istanbul: 0, Muir Glacier: <nil>, Berlin: 0, London: 0, Arrow Glacier: <nil>, Gray Glacier: <nil>, Terminal Total Difficulty: <nil>, Merge Netsplit: <nil>, Shanghai: <nil>, Cancun: <nil>, Engine: aura}" genesis=0xf463abeb7ee27fa62be3ac36a264e8174ee3458da451e6403df47618fd2cf415
WARN[11-16|11:52:30.311] Incorrect snapshot enablement            got=true change_to=false
INFO[11-16|11:52:30.311] Effective                                prune_flags= snapshot_flags= history.v3=false
INFO[11-16|11:52:30.312] Initialising Ethereum protocol           network=10200
INFO[11-16|11:52:30.329] Starting private RPC server              on=127.0.0.1:9090
INFO[11-16|11:52:30.329] new subscription to logs established 
INFO[11-16|11:52:30.329] rpc filters: subscribing to Erigon events 
DBUG[11-16|11:52:30.330] Establishing event subscription channel with the RPC daemon ... 
INFO[11-16|11:52:30.330] New txs subscriber joined 
INFO[11-16|11:52:30.330] new subscription to newHeaders established 
INFO[11-16|11:52:30.330] Reading JWT secret                       path=/home/filoozom/.local/share/erigon/chiado/jwt.hex
INFO[11-16|11:52:30.331] HTTP endpoint opened for Engine API      url=localhost:8551 ws=true ws.compression=true
INFO[11-16|11:52:30.332] HTTP endpoint opened                     url=localhost:8545 ws=false ws.compression=true grpc=false
DBUG[11-16|11:52:30.336] Couldn't add port mapping                proto=tcp extport=30304 intport=30304 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DBUG[11-16|11:52:30.336] Couldn't add port mapping                proto=udp extport=30304 intport=30304 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DBUG[11-16|11:52:30.336] QuerySeeds read nodes from the node DB   count=0
DBUG[11-16|11:52:30.341] [1/16 Snapshots] DONE                    in=134.7µs
INFO[11-16|11:52:30.341] [txpool] Started 
INFO[11-16|11:52:30.341] [2/16 Headers] Waiting for headers...    from=0
DBUG[11-16|11:52:30.341] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
DBUG[11-16|11:52:30.343] Couldn't add port mapping                proto=tcp extport=30303 intport=30303 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DBUG[11-16|11:52:30.343] Couldn't add port mapping                proto=udp extport=30303 intport=30303 interface="UPnP or NAT-PMP" err="no UPnP or NAT-PMP router discovered"
DBUG[11-16|11:52:30.344] QuerySeeds read nodes from the node DB   count=0
DBUG[11-16|11:52:30.346] QuerySeeds read nodes from the node DB   count=0
INFO[11-16|11:52:30.347] Started P2P networking                   version=67 self=enode://47d2e31d90fe140bfd967f147c1d4e8a4834b4c6b895a4bb7082100be60aa5e9a73e98e996da9b0257f02f9ad39b683755abe6ca3e9aefe0170a478a8559dcfb@127.0.0.1:30304 name=erigon/v2.30.0-dev-17fae73f/linux-amd64/go1.18.1
DBUG[11-16|11:52:30.351] QuerySeeds read nodes from the node DB   count=0
INFO[11-16|11:52:30.351] Started P2P networking                   version=66 self=enode://47d2e31d90fe140bfd967f147c1d4e8a4834b4c6b895a4bb7082100be60aa5e9a73e98e996da9b0257f02f9ad39b683755abe6ca3e9aefe0170a478a8559dcfb@127.0.0.1:30303 name=erigon/v2.30.0-dev-17fae73f/linux-amd64/go1.18.1
DBUG[11-16|11:52:31.342] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
[...]
DBUG[11-16|11:52:32.342] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
INFO[11-16|11:37:00.062] [p2p] GoodPeers                          eth66=0 eth67=0
INFO[11-16|11:37:00.077] [txpool] stat                            block=0 pending=0 baseFee=0 queued=0 alloc=42.7MB sys=79.5MB
INFO[11-16|11:37:00.089] [2/16 Headers] No block headers to write in this log period block number=0
INFO[11-16|11:37:00.089] Req/resp stats                           req=0 reqMin=0 reqMax=0 skel=0 skelMin=0 skelMax=0 resp=0 respMin=0 respMax=0 dups=0
DBUG[11-16|11:37:00.089] [Downloader] Queue sizes                 anchors=0 links=0 persisted=1
DBUG[11-16|11:37:00.089] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
[...]
DBUG[11-16|11:37:06.095] [Downloader] Request skeleton            anchors=0 top seen height=0 highestInDb=0
```

I guess it comes down to:
```
--- FAIL: TestDefaultBSCGenesisBlock (0.34s)
    genesis_test.go:27: 
                Error Trace:    /home/filoozom/projects/erigon/core/genesis_test.go:27
                                                        /home/filoozom/projects/erigon/core/genesis_test.go:30
                Error:          Not equal: 
                                expected: []byte{0xe8, 0x72, 0x46, 0x2c, 0x6b, 0xd5, 0xf4, 0x2, 0xec, 0x81, 0xde, 0x7c, 0x5b, 0xd2, 0x82, 0x3e, 0x13, 0x7c, 0x66, 0x6b, 0x78, 0xe8, 0x2b, 0x7e, 0xb0, 0xbe, 0x95, 0xaf, 0x5e, 0xce, 0xa1, 0x8d}
                                actual  : []byte{0xad, 0xa4, 0x4f, 0xd8, 0xd2, 0xec, 0xab, 0x8b, 0x8, 0xf2, 0x56, 0xaf, 0x7, 0xad, 0x3e, 0x77, 0x7f, 0x17, 0xfb, 0x43, 0x4f, 0x8f, 0x8e, 0x67, 0x8b, 0x31, 0x2f, 0x57, 0x62, 0x12, 0xba, 0x9a}
                            
                                Diff:
                                --- Expected
                                +++ Actual
                                @@ -1,4 +1,4 @@
                                 ([]uint8) (len=32) {
                                - 00000000  e8 72 46 2c 6b d5 f4 02  ec 81 de 7c 5b d2 82 3e  |.rF,k......|[..>|
                                - 00000010  13 7c 66 6b 78 e8 2b 7e  b0 be 95 af 5e ce a1 8d  |.|fkx.+~....^...|
                                + 00000000  ad a4 4f d8 d2 ec ab 8b  08 f2 56 af 07 ad 3e 77  |..O.......V...>w|
                                + 00000010  7f 17 fb 43 4f 8f 8e 67  8b 31 2f 57 62 12 ba 9a  |...CO..g.1/Wb...|
                                 }
                Test:           TestDefaultBSCGenesisBlock
                Messages:       chiado
FAIL
FAIL    github.com/ledgerwatch/erigon/core      0.823s
```

----------

Turns out that the `code` in Erigon's chainspec is not the same as
`constructor` in Nethermind for example.
Comparison for Sokol:
- Erigon:
https://github.com/ledgerwatch/erigon/blob/devel/core/allocs/sokol.json#L20-L34
- Nethermind:
https://github.com/NethermindEth/nethermind/blob/master/src/Nethermind/Chains/sokol.json#L248-L252
2022-11-18 12:54:18 +01:00
Alex Sharov
4ba8a6a381
Update README.md 2022-11-18 12:48:04 +03:00
alex.sharov
faa92f9bbc save 2022-11-18 04:31:11 +03:00
lupin012
a442293be1
ChainId conditional in Legacy transaction (#5856)
impacted API:
     eth_getTransactionByHash()
     eth_getTransactionByNumber()
     eth_getTransactionByBlockHashAndIndex()
     eth_getTransactionByBlockNumberAndIndex()
     eth_getBlockByHash()
     eth_getBlockByNumber()

1) In case of legacy transitions the chainId field should be inserted
only if V is not 27/28.
This seems also the Geth/v1.10.23 behaviour via infura

2) In case of dynamicFee/AccessList transaction the access_list should
be inserted in the response also if empty
This is done correctly in cmd/rpcdaemon/commands/eth_api.go and NOT in
internal/ethapi/api.go
This seems also the Geth/v1.10.23 behaviour via infura
2022-11-18 03:15:32 +03:00
lupin012
8ceb1638a0
add totalDifficulty to GetBlockByTimestamp (#5827)
Added totalDifficulty to the returned response to be similar to the data
returned by eth_getBlockByNumber() or eth_getBlockByHash()
2022-11-18 03:15:00 +03:00
lupin012
21d24a20a0
fix on number of samples for blocks (#5826)
eth_gasPrice(): A maximum of 3 samples should be taken per block (see
sample const)
The getBlockPrices() function takes 3 samples on the first block and one
on the others.
In the check s.Len() >= limit s.Len() after first block will contain 3
samples and so the loop will be executed only one time for each block
NOT three times
2022-11-18 03:14:49 +03:00
Alex Sharov
4509f496c6
bsc: new blocks snapshots (#6075) 2022-11-18 03:09:07 +03:00
Nathan (Blaise) Bruer
ca89620e56
SendMessageToRandomPeers() now is truely pseudo-random (#6050)
Fixes an issue where if the peer count didn't change much, we'd send the
message to the same peers nearly every time.

We now pick new random peers on every call to
SendMessageToRandomPeers().
2022-11-18 02:44:43 +03:00
Enrique Jose Avila Asapche
a4b9da9bd4
fixed nil ptr err (#6035)
The from field would sometimes be nil causing an error when trying to
access it, instead changed it to be an empty address #5882
2022-11-18 02:43:22 +03:00
Manav Darji
1da4d3abbf
eth, consensus/bor: fixes and improvements related to mining (#6051)
In context of https://github.com/ledgerwatch/erigon/issues/5694, this PR
adds some fixes and improvement in the mining flow. Also, a relevant
change in txpool (present in erigon-lib) is made here:
https://github.com/ledgerwatch/erigon-lib/pull/737

#### Changes in triggering mining in `startMining()`
The mining module didn't honour the block time as a simple 3 second
timer and a notifier from txpool was used to trigger mining. This would
cause inconsistencies, at least with the bor consensus. Hence, a geth
like approach is used instead for simplicity. A new head channel
subscription is added in the `startMining()` loop which would notify the
addition of new block. Hence, this would make sure that the block time
is being honoured. Moreover, the fixed 3 second timer is replaced by the
`miner.recommit` value set using flags.

####  Changes in the arrangement of calls made post mining
When all the mining stages are completed, erigon writes all the data in
a cache. It then processes the block through all the stages as it would
process a block received from P2P. In this case, some of the stages
aren't really required. Like the block header and body download stage is
not required as the block was mined locally. Even execution stage is not
required as it already went through it in the mining stages.

Now, we encountered an issue where the chain was halted and kept mining
the same block again and again (liveness issue). The root cause is
because of an error in a stage of it's parent block. This stage turns
out to be the 4th stage which is "Block body download" stage. This stage
tries to download the block body from peers using the headers. As, we
mined this block locally we don't really need to download anything (or
process anything again). Hence, it reaches out to the cache which we
store for the block body.

Interestingly that cache turned out to be empty for some blocks. This
was because post mining, before adding block header and body to a cache,
we call the broadcast method which starts the staged sync. So,
technically it’s a bit uncertain at any stage if the block header and
body has been written or not.(see
[this](https://github.com/ledgerwatch/erigon/blob/devel/eth/backend.go#L553-L572)).
To achieve complete certainty, we rearranged the calls with the write to
cache being called first and broadcast next. This pretty much solves the
issue as now we’re sure that we’d always have a block body in the cache
when we reach the body download stage.

#### Misc changes
This PR also adds some logs in bor consensus.
2022-11-18 02:39:16 +03:00
Mark Shields
0d854fea6e
feat(k8s): base with google-kubernetes-engine overlay example (#6041)
re:
https://discord.com/channels/687972960811745322/738982866670714901/1041381487754354689

Introduces a more modular base with a google-kubernetes-engine overlay
example.
2022-11-18 02:37:53 +03:00
Alex Sharov
874d96fb47
e3: Non parallel exec (with exec in RO goroutine) (#6073) 2022-11-17 20:52:12 +03:00
Alex Sharov
c50d4e6c0b
e3: panic logging (#6072) 2022-11-17 16:17:02 +03:00
Alex Sharov
d5c23c4df4
e3: enable test (#6071) 2022-11-17 16:06:57 +03:00
Alex Sharov
04293ef67a
Revert "e3: more mainnet snapshots" (#6069)
Reverts ledgerwatch/erigon#6068
2022-11-17 06:20:58 +03:00
Alex Sharov
17541b26da
e3: more mainnet snapshots (#6068) 2022-11-17 06:15:53 +03:00
Alex Sharov
66e1dc2dfd
better format p2p error (#6066) 2022-11-16 19:46:25 +03:00