Commit Graph

20627 Commits

Author SHA1 Message Date
Andrew Ashikhmin
9f9e3644ed
Update Polygon storage requirements (#9014)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-12-25 20:22:13 +07:00
Alex Sharov
0d31444eea
bor mainnet 51m (#9073) 2023-12-25 11:24:14 +07:00
Alex Sharov
bc50bd50a4
make 500K files also seedable (because they actually are) (#9077) 2023-12-25 10:07:31 +07:00
Alex Sharov
ab4a00ffb9
disable TestSentinelStatusRequest (failing on WIN CI) (#9076) 2023-12-25 09:19:17 +07:00
Giulio rebuffo
eaf0348bd0
[Grindmas] Added tests to Beacon API, also fixed stuff. (#9074)
* Testing Beacon API
* Fixed sentinel code (a little bit)
* Fixed sentinel tests
* Added historical state support
* Fixed state-related endpoints (i was drunk when writing them)
2023-12-25 02:34:13 +01:00
Alex Sharov
2944abbd19
fix typo in snap webseed server names (#9072) 2023-12-24 16:37:13 +07:00
Alex Sharov
e08003f1f7
block retire: merge all possible files (even bor) even if nothing to retire (#9068)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-12-24 07:32:52 +00:00
Alex Sharov
77d32ccdc6
"downloader manifest": cmd to produce manifest.txt (#9067)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-12-24 07:32:21 +00:00
Alex Sharov
4186213f22
disable TestSentinelGossipAverage (#9070) 2023-12-24 06:32:14 +00:00
lupin012
88a8aa6799
rpcademon: eth_createAccessList: fix in retrieve nonce (#9064) 2023-12-24 04:41:04 +00:00
Giulio rebuffo
be4036e132
Added tests to sentinel (FINALLY!) (#9066) 2023-12-24 03:54:50 +01:00
Giulio rebuffo
a4d7b6d33f
Switched Caplin snapshot format to ZSTD blinded blocks (#9058)
* Chunked format -> blinded
* LZ4 -> ZSTD
* Implemented parent block root support for history download
* Rationale: Allows to optimize GC collection easily on state
reconstruction and it allows to read fast attestations in historical
states reader
2023-12-23 15:56:35 +01:00
lupin012
6a83a91145
Add create access list test (#9062) 2023-12-23 21:38:47 +07:00
Bayram Guvanjov
a2f375c0b1
Add blocksByRange & blocksByRoot P2P rpc handlers (#8885)
This PR is ready to review. 

PR introduces `blocksByRange` and `blocksByRoot` P2P RPC methods
- `blocksByRange` - allows peers to request a range of blocks
- `blocksByRoot` - enables block requests using their root hashes(list
format)

Reference:
https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/p2p-interface.md#beaconblocksbyrange
2023-12-23 14:32:00 +01:00
Alex Sharov
2b87d65285
retire: handle case when bor snaps are behind block snaps (#9061) 2023-12-23 16:37:30 +07:00
ledgerwatch
ec970ac6bc
Fixed duplicate bucket names, remove presigned URLs (#9060)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-12-23 04:11:12 +00:00
Dmytro
a36071e7ff
dvovk/snapidx (#9049) 2023-12-22 11:25:55 +00:00
Mark Holt
a3a61701e2
Allow proxy paths in Heimdall URL (#8940)
Add paths to the hiemdall config URL when creating calls so that extra
paths needs by, for example proxy servers are not stripped from the flag
value passed into the process.
2023-12-22 10:48:25 +00:00
milen
64072ce6c7
polygon/sync: implement header downloader (#9030)
Co-authored-by: battlmonstr <battlmonstr@users.noreply.github.com>
2023-12-22 10:44:55 +00:00
Alex Sharov
eb4685f3e2
mode to produce block snapshots (#9053)
`STAGES_ONLY_BLOCKS=true` may help to produce BlockSnaps by Erigon2 on
weak machines - it disabling all stages after StageSenders.
2023-12-22 05:43:30 +00:00
Delweng
a45b4c79af
txpool: the same logging style of announced and broadcasted tx (#9054)
Found the different log style for announced and broadcasted tx:

```
[INFO] [12-22|05:18:01.363] Local tx broadcasted                     txHash=ec6b1c87aafd7f8ead5794477be50bda696f2ce17271ad4f6022a756722fa0be to peer=10
[INFO] [12-22|05:18:01.363] local tx announced                       tx_hash=ec6b1c87aafd7f8ead5794477be50bda696f2ce17271ad4f6022a756722fa0be to peer=40 baseFee=1
```

adjust them to the same style

Signed-off-by: jsvisa <delweng@gmail.com>
2023-12-22 12:39:19 +07:00
Alex Sharov
657aafd5b7
allow erigon download .torrent from webseed by default (#9052) 2023-12-22 11:42:35 +07:00
Alex Sharov
95434d100e
webseed: remove dependency on db state (#9051)
erigon didn't download form webseeds if StageSnapshots progress > 0. but
sometime we want "just delete files and re-download them"
2023-12-22 03:35:48 +00:00
Giulio rebuffo
0e18866ed9
Added fill-up with pre-existing API (#9042) 2023-12-21 17:49:40 +01:00
Michelangelo Riccobene
393dd18cad
QA Test - Clean exit (#8981)
First "smoke test", to check Erigon clean exiting on ctrl-c
2023-12-21 16:16:57 +01:00
ledgerwatch
3b10ba95d5
Fix creation of torrent files (#9048)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-12-21 12:42:07 +00:00
battlmonstr
55d37b938c
bor: spanID calculation refactoring (#9040) 2023-12-21 09:52:00 +01:00
Alex Sharov
9eb9151be4
atomic CRUD for .torrent files (#9043) 2023-12-21 05:15:32 +00:00
Alex Sharov
1499fbb582
reduce mutex-contention in downloader.ReCalcStats (#9044) 2023-12-21 04:00:03 +00:00
Alex Sharov
56c5f656d0
downloader: handle race between adding magnet link and files from webseed (#9045) 2023-12-21 03:59:26 +00:00
Alex Sharov
8c822256db
remove bor mainnet files (#9033) 2023-12-21 08:09:54 +07:00
Alex Sharov
4eecd8c86c
print_stages: bor snaps info (#9008) 2023-12-21 08:08:55 +07:00
Alex Sharov
379a5f8ea1
harness test: to use Mock for genesis write - for e35 compatibility (#9012)
`e35` doesn't write genesis state by special func anymore - and Mock
using `m.InsertBlocks` to process genesis block (as any other block).
2023-12-21 08:07:29 +07:00
Alex Sharov
7107cfed0f
snaps: stop merge to 500K and enjoy immutability (#9034) 2023-12-21 08:04:46 +07:00
battlmonstr
2760eeb961
polygon: astrid sync heimdall wrapper (#9017) 2023-12-20 16:48:37 +01:00
Alex Sharov
1ae3f41189
db migration: if stage_snapshots > 0, then create prohibit_new_downloads.lock file (#9036)
it will protect existing nodes from downloading `v2` files - even if
they do upgrade
2023-12-20 18:04:43 +07:00
kaliubuntu0206
0c4bebbf82
Update Ethereum Mainnet storage requirements (#9037)
Sync from scratch took about 4 days 15 hours, with total disk used about
2.8TB

```
ubuntu@localhost:/erigon$ cast block-number
18825796
ubuntu@localhost:/erigon$ ls
chaindata  downloader  jwt.hex  LOCK  nodekey  nodes  snapshots  temp  txpool
ubuntu@localhost:/erigon$ du -hs chaindata
2.2T    chaindata
ubuntu@localhost:/erigon$ du -hs downloader
17M     downloader
ubuntu@localhost:/erigon$ du -hs snapshots
473G    snapshots
ubuntu@localhost:/erigon$ du -hs temp
79G     temp
```
2023-12-20 15:44:49 +07:00
Giulio rebuffo
e75dc25181
Disabled caplin phase2 for Gnosis temporairly. (#9006)
Gnosis phase2 will be disabled until we get good snapshot automation.

* Just run it in "phase1" mode.
* Create jwt secret accordingly
2023-12-19 13:10:34 +01:00
Giulio rebuffo
a0b9d60265
Update Consensus Specification tests (#9024) 2023-12-19 09:42:11 +01:00
dependabot[bot]
33fc53f83d
build(deps): bump golang.org/x/crypto from 0.16.0 to 0.17.0 (#9020)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from
0.16.0 to 0.17.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9d2ee975ef"><code>9d2ee97</code></a>
ssh: implement strict KEX protocol changes</li>
<li><a
href="4e5a26183e"><code>4e5a261</code></a>
ssh: close net.Conn on all NewServerConn errors</li>
<li><a
href="152cdb1503"><code>152cdb1</code></a>
x509roots/fallback: update bundle</li>
<li><a
href="fdfe1f8531"><code>fdfe1f8</code></a>
ssh: defer channel window adjustment</li>
<li><a
href="b8ffc16e10"><code>b8ffc16</code></a>
blake2b: drop Go 1.6, Go 1.8 compatibility</li>
<li><a
href="7e6fbd82c8"><code>7e6fbd8</code></a>
ssh: wrap errors from client handshake</li>
<li><a
href="bda2f3f5cf"><code>bda2f3f</code></a>
argon2: avoid clobbering BP</li>
<li>See full diff in <a
href="https://github.com/golang/crypto/compare/v0.16.0...v0.17.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/crypto&package-manager=go_modules&previous-version=0.16.0&new-version=0.17.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/ledgerwatch/erigon/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-19 08:34:09 +07:00
milen
4f95342036
freezeblocks: fix blockreader last frozen bor span and event ids (#9018)
During testing we run into a "span 7813 not found (db)" due to a very
large unwind (1 million blocks).

This is because the block reader's `LastFrozenSpanID` and
`LastFrozenEventID` returned results that are not consistent with
`FrozenBorBlocks`. The latter is taking into account the existence of
`.idx` files while the former 2 functions were not.

Note such a large unwind is not likely to happen normally unless there
is a bug in our unwind logic or an operator is manually unwinding very
far back due to reasons like chain halts (ie mumbai bug problem from few
months ago), devel testing or anything else along these lines.
Regardless, it exposed the above discrepancy which is best to be fixed.
2023-12-18 19:13:21 +02:00
Andrew Ashikhmin
037754a177
rpc: add txHash to debug_traceBlock* results (#9016)
See https://github.com/ethereum/go-ethereum/pull/27183
2023-12-18 15:30:33 +01:00
Giulio rebuffo
a366629f8a
Cleanuped up request handling (#9007)
* Some requests may lead to undefined behaviour this will make sure to
call .Close once
2023-12-18 13:54:27 +01:00
Giulio rebuffo
ffc26b0f00
Added GET SyncCommittees (#9003) 2023-12-18 13:54:15 +01:00
Alex Sharov
1468317efd
erigon snapshots index: build bor indices (#9009) 2023-12-18 17:46:50 +07:00
Alex Sharov
47c5f16a8d
downloader: don't log complete files info (#9013) 2023-12-18 09:28:20 +01:00
Giulio rebuffo
6a1bb1dff1
Added Blinded blocks type and Beacon endpoints (#9002)
* Added blinded block for internal uses
* Added blinded block to the Beacon API
2023-12-17 00:22:47 +01:00
a
f934ecbd6a
Fix beacon api handler (#8999) 2023-12-16 19:12:40 +01:00
a
a53b1d8b29
flip this inequality (#8904)
this is funny
2023-12-15 21:52:55 -06:00
milen
d23f306fc1
borheimdall: add tests for state sync events persistence (#8989)
- adds a test for persisting state sync events at the beginning of every
sprint
2023-12-15 15:07:39 +02:00