Commit Graph

20325 Commits

Author SHA1 Message Date
Alex Sharov
f265bd8bfc
add "integration stage_headers --integrity.slow" to check gaps in headers or canonical markers (#8489) 2023-10-16 19:52:04 +07:00
Alex Sharov
c4b1930bad
mdbx: add lable to commit error (#8492) 2023-10-16 19:51:49 +07:00
Alex Sharov
21ebaab208
bor initFrozenSnapshot: parallel erecover (#8488)
on 16-core mumbai's initFrozenSnapshot took 10min (to 38M block)
2023-10-16 13:05:10 +01:00
Alex Sharov
ab04de6d9f
mdbx: don't mix modes (#8490)
db must not use "Durable" and "SyncPeriod" in same time

txpool and downloader must be: Durable
nodedb:  SafeNoSync+SyncPeriod

because txpool and downloader already can call sync when they need
(txpool after regular flush, downloader after piece download+validation
completion).
2023-10-16 16:21:44 +07:00
Giulio rebuffo
2e9436a7d0
Verbose --internalcl for whenever we fail at startup (#8487)
Just does not make it die silently
2023-10-16 11:09:43 +02:00
Giulio rebuffo
a7e63da066
CI Job for Consensus specs (#8483)
Consensus Specification Tests takes less than 8 minutes so I think they
can be in a PR's own CI for whenever it is ready. for reference it is
less than make test
2023-10-15 19:47:53 +02:00
Giulio rebuffo
bf2ef04c46
Optimized the checkpoint state layout (#8482) 2023-10-15 18:05:13 +02:00
Andrew Ashikhmin
42641bc6af
Fix Sepolia snapshot hashes (#8481)
Pick up https://github.com/ledgerwatch/erigon-snapshot/pull/41
2023-10-15 11:22:56 +02:00
Alex Sharov
3b6f38025f
up x/sync lib (#8479) 2023-10-15 10:10:36 +07:00
Alex Sharov
0a2cc303f3
db: don't create tables in accede mode (#8478) 2023-10-15 09:34:24 +07:00
Giulio rebuffo
1da6dc25f3
Caplin: Resetting cache head hash when new slot is avaiable (#8477) 2023-10-14 20:53:16 +02:00
Giulio rebuffo
54ce971084
Added separate Withdrawal data type to Caplin (#8474)
Reason: JSON formatting
2023-10-14 17:59:52 +02:00
Alex Sharov
9db82fee5b
downloader: don't loose debug logs (#8473) 2023-10-14 17:11:39 +07:00
Alex Sharov
71ee11af90
mumbai snapshots - 41m (#8461) 2023-10-14 08:40:59 +07:00
Andrew Ashikhmin
f1477e5b2c
Small extra check of ReceiptHash (#8469)
Small addition to PR #8467
2023-10-14 08:40:45 +07:00
battlmonstr
757a91c44d
sync: fix a memory leak when header verification fails (#8431)
If HeaderDownload.VerifyHeader always returns false, the memory usage
grows at a fast pace
due to Link objects (containing headers) not deallocated even after the
link queue pruning.
2023-10-14 08:39:43 +07:00
Léo Vincent
c1e99219d0
fix: newly mined blocks now include transactions and check new blocks with empty receipt hash (#8467)
## Description

This PR is a follow-up of #8442.

1. The previous PR resolved the issue where newly mined blocks had their
receipt hash set to an empty root hash. But I think it introduced an
issue where the new blocks include no transactions. So this PR addresses
that.

2. On top of that, this new PR introduces a validation step for the
receipt hash when peers receive new blocks at the p2p layer. If the
receipt hash is set to the empty root hash, but the block contains
transactions, it will now raise an error.

## Test

- When running a local erigon devnet with the new receipt hash check and
before #8442 fix (with a few more logs).

**=> error message: `newBlock66: block has empty receipt hash:
56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421 but it
includes 2 transactions`**

```sh
Received new block request: &{Block:0xc000bd0230 TD:+7891}
Block header: &{ParentHash:0x998f5be682d09b9d57dbf9c42032c23754bc9569d39670a1d7479f74946df0a9 UncleHash:0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347 Coinbase:0x0000000000000000000000000000000000000000 Root:0x422bd23663d6d52b94cd537491eaa2b250b03e5e252b9c1dfabee5dc12673628 TxHash:0xe4ec65ce3aba6347c9e7139571cd23ec5e42b3c1952706a3d965a322699b9b5e ReceiptHash:0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421 Bloom:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] Difficulty:+2 Number:+3945 GasLimit:30000000 GasUsed:70178 Time:1697192495 Extra:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 197 254 60 127 243 72 143 204 98 97 12 136 59 50 115 115 204 239 143 157 234 71 55 113 77 66 2 227 74 65 226 190 44 226 215 32 177 90 161 172 41 69 31 229 166 221 185 14 128 47 169 132 130 249 179 129 153 159 134 124 162 234 81 187 1] MixDigest:0x0000000000000000000000000000000000000000000000000000000000000000 Nonce:[0 0 0 0 0 0 0 0] AuRaStep:0 AuRaSeal:[] BaseFee:+8 WithdrawalsHash:<nil> BlobGasUsed:<nil> ExcessBlobGas:<nil> ParentBeaconBlockRoot:<nil> Verkle:false VerkleProof:[] VerkleKeyVals:[]}
Block body: &{Transactions:[0xc000cde100 0xc000cde200] Uncles:[] Withdrawals:[]}
DBUG[10-13|12:21:35.789] Handling incoming message                stream=RecvMessage err="newBlock66: block has empty receipt hash: 56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421 but it includes 2 transactions"
...
```

- When running a local erigon devnet with the new receipt hash check
after #8442 fix (with a few more logs).

**=> error message: `newBlock66: block has invalid transaction hash
...`**

```sh
Received new block request: &{Block:0xc002c38930 TD:+11963}
Block header: &{ParentHash:0x89a5134df82c85b38f2e02524143b575c44c5c6df444df9fc4d86c5972601ce0 UncleHash:0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347 Coinbase:0x0000000000000000000000000000000000000000 Root:0x1ab62696e1b585c8e084e12781e2f36f2591f91fe45e5b125df2468abd1f54e5 TxHash:0x6e7642d32215ab23f9096ed36463730ffa6c17c26dbba50c6100d64b5dee6349 ReceiptHash:0x9c8f5e4d71a8bb3ace2c0ce45e00357278fcac6da41b9e353872f106298cf6ec Bloom:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 16 0 0 0 0 0 32 0 0 0 0 0 0 0 16 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 128 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 8 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 16 0 16 0 0 0] Difficulty:+2 Number:+5981 GasLimit:30000000 GasUsed:70178 Time:1697193936 Extra:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 187 75 33 182 137 174 13 214 132 232 242 141 222 107 53 186 250 113 185 149 118 134 112 84 167 16 126 38 158 252 205 66 117 96 205 198 79 158 82 42 199 29 37 92 23 3 140 113 216 66 105 174 144 209 110 121 119 43 165 204 81 54 113 1] MixDigest:0x0000000000000000000000000000000000000000000000000000000000000000 Nonce:[0 0 0 0 0 0 0 0] AuRaStep:0 AuRaSeal:[] BaseFee:+8 WithdrawalsHash:<nil> BlobGasUsed:<nil> ExcessBlobGas:<nil> ParentBeaconBlockRoot:<nil> Verkle:false VerkleProof:[] VerkleKeyVals:[]}
Block body: &{Transactions:[] Uncles:[] Withdrawals:[]}
DBUG[10-13|12:45:36.908] Handling incoming message                stream=RecvMessage err="newBlock66: block has invalid transaction hash: have 56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421, exp: 6e7642d32215ab23f9096ed36463730ffa6c17c26dbba50c6100d64b5dee6349"
...
```

- When running a local erigon devnet with the new receipt hash check and
after this PR's fix (include transactions in block)

**=> no error message 🎉**

```sh
Received new block request: &{Block:0xc000d5a000 TD:+1227}
Block header: &{ParentHash:0xf5d6a19c8d56f230eeb8d2654e5562b934236136d5fcc50cbcde5c9ccc659810 UncleHash:0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347 Coinbase:0x0000000000000000000000000000000000000000 Root:0x9c17098ebfda6e59ceef147778367c628a44a6f1421ef21e844a9d8450f64a74 TxHash:0x507daf8bc5b329226614a0cf535273242cbf15d5ad582a4d5db0d5881732f791 ReceiptHash:0xaea5f86a33cba20385d4ce630b5f7ff613139b374718d89eafe224bd4b57e903 Bloom:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0] Difficulty:+2 Number:+613 GasLimit:20918370 GasUsed:54544 Time:1697195980 Extra:[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 118 30 19 173 212 211 186 172 0 200 117 103 101 19 76 147 18 161 141 70 70 237 243 162 239 236 162 35 154 248 10 28 90 20 214 32 140 72 23 151 36 212 210 99 122 154 244 229 171 249 29 105 109 23 80 112 142 65 235 189 59 238 8 27 1] MixDigest:0x0000000000000000000000000000000000000000000000000000000000000000 Nonce:[0 0 0 0 0 0 0 0] AuRaStep:0 AuRaSeal:[] BaseFee:+8 WithdrawalsHash:<nil> BlobGasUsed:<nil> ExcessBlobGas:<nil> ParentBeaconBlockRoot:<nil> Verkle:false VerkleProof:[] VerkleKeyVals:[]}
Block body: &{Transactions:[0xc0000dc400 0xc0000dc600] Uncles:[] Withdrawals:[]}
...
```
2023-10-13 18:05:38 +02:00
Giulio rebuffo
cfc2bcf707
Proper use of Dirs in Caplin (#8458) 2023-10-13 15:22:48 +02:00
Alex Sharov
7d357b4327
better webseed support (#8466) 2023-10-13 18:03:52 +07:00
Andrew Ashikhmin
b60642fa5a
Configure EIP-4844 parameters for Gnosis (#8464)
See https://github.com/gnosischain/specs/pull/20 &
https://github.com/gnosischain/specs/pull/24
2023-10-13 11:43:16 +02:00
GoodDaisy
ea328137a7
fix typos (#8452) 2023-10-13 11:36:23 +07:00
Mark Holt
7b3570c019
Add block producer progress check (#8447)
Add a check against the inbound headers before publishing a newly mined
block after the wait delay.

If the node received a block while it was processing transactions, or
waiting for its publish slot, do a final check that another node hasn't
already published a block.
2023-10-12 19:08:05 +01:00
Andrew Ashikhmin
52d3526b64
Maximize build space for goreleaser (#8455)
Mitigate [no space left on
device](https://github.com/ledgerwatch/erigon/actions/runs/6493703302/job/17639876934)
in GoReleaser. See
https://github.com/AdityaGarg8/remove-unwanted-software.

This is a simplified version of PR #8451 (there I had to add
`root-reserve-mb: 16384` to make it work).
2023-10-12 17:41:13 +02:00
Léo Vincent
62d9f3793e
chore(devnet): add rpc.host and rpc.port flags in devnet cmd (#8449)
## Description

Add the possibility to change the RPC address used in `devnet` without
requiring a rebuild.

## Test

- [x] `make devnet`
- [x] `./build/bin/devnet --datadir tmp`
- [x] `./build/bin/devnet --datadir tmp --rpc.host localhost --rpc.port
8546`
2023-10-12 15:18:51 +01:00
Manav Darji
9bf64eb9ef
cmd/sentry: remove empty headers check (#8450)
This PR removes the empty headers check when responding to
`GetBlockHeaders` p2p request. Currently, as we return from the
function, the requests on the caller's end gets timed out causing peer
drop. This will prevent that from happening by sending empty header list
which will be of no use but still helps in maintaining the peer
connection.

With this change, those running erigon 2.48 with
`--sentry.drop-useless-peers` will drop peers as they consider receiving
empty headers a harsh move which is fine.

We have also discovered that if we don't remove this check maintaining
connectivity to Bor is not maintained. This is significant as Erigon
becomes a minority validator, as bor chain stability (reducing forks) is
dependent upon good validator connectivity.
2023-10-12 14:28:24 +01:00
dependabot[bot]
7750ee5dbc
Bump golang.org/x/net from 0.15.0 to 0.17.0 (#8444)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.15.0 to
0.17.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="b225e7ca6d"><code>b225e7c</code></a>
http2: limit maximum handler goroutines to MaxConcurrentStreams</li>
<li><a
href="88194ad8ab"><code>88194ad</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="2b60a61f1e"><code>2b60a61</code></a>
quic: fix several bugs in flow control accounting</li>
<li><a
href="73d82efb96"><code>73d82ef</code></a>
quic: handle DATA_BLOCKED frames</li>
<li><a
href="5d5a036a50"><code>5d5a036</code></a>
quic: handle streams moving from the data queue to the meta queue</li>
<li><a
href="350aad2603"><code>350aad2</code></a>
quic: correctly extend peer's flow control window after MAX_DATA</li>
<li><a
href="21814e71db"><code>21814e7</code></a>
quic: validate connection id transport parameters</li>
<li><a
href="a600b3518e"><code>a600b35</code></a>
quic: avoid redundant MAX_DATA updates</li>
<li><a
href="ea633599b5"><code>ea63359</code></a>
http2: check stream body is present on read timeout</li>
<li><a
href="ddd8598e56"><code>ddd8598</code></a>
quic: version negotiation</li>
<li>Additional commits viewable in <a
href="https://github.com/golang/net/compare/v0.15.0...v0.17.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/net&package-manager=go_modules&previous-version=0.15.0&new-version=0.17.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/ledgerwatch/erigon/network/alerts).

</details>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: yperbasis <andrey.ashikhmin@gmail.com>
2023-10-12 10:54:30 +02:00
Mark Holt
6f7186e0f4
Fix invalid pre-fetched header broadcast (#8442)
Fixes and issue with Polygon validators where locally mined blocks are
broadcast with invalid header hashes because the NewBlock message
constructor was removing the ReceiptHash which contributed to the header
hash.

The results in the bor header validation code not being able to
correctly identify the signer of the header - so header validation
fails.

This also likely fixes part of the bogon-block issue which was
identified by the polygon team.
2023-10-12 08:27:02 +01:00
lupin012
02032ada42
rpcademon: changed erigon_getLogs() according changes done on erigon_getLatest() (#8417) 2023-10-12 14:18:21 +07:00
lupin012
2a760e9c02
rpctest: Update tx receipt test (#8415) 2023-10-12 14:17:38 +07:00
lupin012
67f85e55a1
rpc_test: Add test for get tx from hash (#8418) 2023-10-12 14:17:03 +07:00
Alex Sharov
c84193f02f
mainnet 18 snapshots (#8436) 2023-10-12 14:15:38 +07:00
Alex Sharov
6d9a4f4d94
rpcdaemon: must not create db - because doesn't know right parameters (#8445) 2023-10-12 14:11:46 +07:00
Alex Sharov
fe263ae02e
downloader: smaller ratelimit birst (#8435) 2023-10-11 16:18:21 +07:00
Alex Sharov
404719c292
Medbx: add label to error messages, UpdateForkChoice: add ctx to erorrs, MemDb: increase db-limit from 512Mb to 512Gb (#8434) 2023-10-11 12:53:34 +07:00
Alex Sharov
b8d8003618
move memdb to own package - to reduce cycle deps (#8428) 2023-10-11 08:48:36 +07:00
Andrew Ashikhmin
fe85648442
params: begin 2.53 release cycle (#8433) 2023-10-10 19:56:35 +02:00
Andrew Ashikhmin
761af94b22
params: remove dev from v2.52.0 (#8432) 2023-10-10 19:13:56 +02:00
Mark Holt
3fb89cc5c4
Prune all only on unwindBlock (#8430)
TruncateCanonicalHash should only have all pruned in the case of a
badBlock or a fork unwind
2023-10-10 17:13:19 +01:00
Giulio rebuffo
e36e5631a8
Better solution to bodies corrupted output in ByRange/ByHashes (#8427)
Marshal outside
2023-10-10 17:06:19 +02:00
Mark Holt
0d190ff9e9
Bor rpc config fix (#8413)
This is an additional fix for BorRo to add bor config in the constructor
- otherwise code which accesses chain config will panic.
2023-10-10 15:26:02 +01:00
Mark Holt
7deb69967f
Avoid marking blocks as bad at rewind (#8414)
Co-authored-by: Anshal Shukla <shukla.anshal85@gmail.com>
2023-10-10 19:47:51 +05:30
Giulio rebuffo
1d5d29709e
Revert "Fixed corrupted transactions in GetPayloadBodies" (#8426)
Reverts ledgerwatch/erigon#8416
2023-10-10 11:25:08 +02:00
Alex Sharov
1617f1802b
Sepolia more sn (#8422) 2023-10-10 11:19:10 +07:00
Alex Sharov
3ec23e083c
gnark lib up (#8420) 2023-10-10 10:42:06 +07:00
Giulio rebuffo
6b92dbcf13
Fixed corrupted transactions in GetPayloadBodies (#8416)
Should use marshalbinary
2023-10-09 23:47:56 +02:00
Anshal Shukla
076dc33232
move borfinality package out of eth (#8407)
- Move borfinality out of eth package
- Adds nil pointer check in bor_verifier
2023-10-09 19:13:31 +01:00
Somnath
4d97428c81
Fix expected value in test (#8412) 2023-10-09 18:12:54 +01:00
Giulio rebuffo
2a5c51dc57
ValidateChain not reliant on stageBodies (#8411) 2023-10-09 17:24:12 +02:00
Alex Sharov
62395c767e
move NewHashBatch to erigon-lib, remove oddb package (#8408) 2023-10-09 15:47:54 +07:00
Giulio rebuffo
afd88edea8
Fixed body downloader bug in Ethereum PoS of downloading blocks ahead of time (#8405) 2023-10-08 23:23:21 +02:00