Commit Graph

20589 Commits

Author SHA1 Message Date
Giulio rebuffo
a0b9d60265
Update Consensus Specification tests (#9024) 2023-12-19 09:42:11 +01:00
dependabot[bot]
33fc53f83d
build(deps): bump golang.org/x/crypto from 0.16.0 to 0.17.0 (#9020)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from
0.16.0 to 0.17.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9d2ee975ef"><code>9d2ee97</code></a>
ssh: implement strict KEX protocol changes</li>
<li><a
href="4e5a26183e"><code>4e5a261</code></a>
ssh: close net.Conn on all NewServerConn errors</li>
<li><a
href="152cdb1503"><code>152cdb1</code></a>
x509roots/fallback: update bundle</li>
<li><a
href="fdfe1f8531"><code>fdfe1f8</code></a>
ssh: defer channel window adjustment</li>
<li><a
href="b8ffc16e10"><code>b8ffc16</code></a>
blake2b: drop Go 1.6, Go 1.8 compatibility</li>
<li><a
href="7e6fbd82c8"><code>7e6fbd8</code></a>
ssh: wrap errors from client handshake</li>
<li><a
href="bda2f3f5cf"><code>bda2f3f</code></a>
argon2: avoid clobbering BP</li>
<li>See full diff in <a
href="https://github.com/golang/crypto/compare/v0.16.0...v0.17.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/crypto&package-manager=go_modules&previous-version=0.16.0&new-version=0.17.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/ledgerwatch/erigon/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-19 08:34:09 +07:00
milen
4f95342036
freezeblocks: fix blockreader last frozen bor span and event ids (#9018)
During testing we run into a "span 7813 not found (db)" due to a very
large unwind (1 million blocks).

This is because the block reader's `LastFrozenSpanID` and
`LastFrozenEventID` returned results that are not consistent with
`FrozenBorBlocks`. The latter is taking into account the existence of
`.idx` files while the former 2 functions were not.

Note such a large unwind is not likely to happen normally unless there
is a bug in our unwind logic or an operator is manually unwinding very
far back due to reasons like chain halts (ie mumbai bug problem from few
months ago), devel testing or anything else along these lines.
Regardless, it exposed the above discrepancy which is best to be fixed.
2023-12-18 19:13:21 +02:00
Andrew Ashikhmin
037754a177
rpc: add txHash to debug_traceBlock* results (#9016)
See https://github.com/ethereum/go-ethereum/pull/27183
2023-12-18 15:30:33 +01:00
Giulio rebuffo
a366629f8a
Cleanuped up request handling (#9007)
* Some requests may lead to undefined behaviour this will make sure to
call .Close once
2023-12-18 13:54:27 +01:00
Giulio rebuffo
ffc26b0f00
Added GET SyncCommittees (#9003) 2023-12-18 13:54:15 +01:00
Alex Sharov
1468317efd
erigon snapshots index: build bor indices (#9009) 2023-12-18 17:46:50 +07:00
Alex Sharov
47c5f16a8d
downloader: don't log complete files info (#9013) 2023-12-18 09:28:20 +01:00
Giulio rebuffo
6a1bb1dff1
Added Blinded blocks type and Beacon endpoints (#9002)
* Added blinded block for internal uses
* Added blinded block to the Beacon API
2023-12-17 00:22:47 +01:00
a
f934ecbd6a
Fix beacon api handler (#8999) 2023-12-16 19:12:40 +01:00
a
a53b1d8b29
flip this inequality (#8904)
this is funny
2023-12-15 21:52:55 -06:00
milen
d23f306fc1
borheimdall: add tests for state sync events persistence (#8989)
- adds a test for persisting state sync events at the beginning of every
sprint
2023-12-15 15:07:39 +02:00
Alex Sharov
9ccd8a1f91
snapshots: --metrics must not affect logs about peers rate (#8990) 2023-12-15 08:05:54 +01:00
Phillip LeBlanc
5d0627f148
Fix typo in log message (#8984) 2023-12-15 09:31:49 +07:00
Giulio rebuffo
eeb471d800
Added Finality_Checkpoints endpoint (#8979) 2023-12-15 02:27:27 +01:00
ddl
e018bb062a
cmd/evm:fix Env struct json tag (#8986)
use geth found this doc problem
https://github.com/ethereum/go-ethereum/pull/28635
2023-12-15 07:24:03 +07:00
Dmytro
e82147caf3
added collecting info about snapshot indexing, renamed downloading prop (#8987) 2023-12-15 07:23:26 +07:00
milen
1a6b83b82c
borheimdall: add test for span persistence (#8988)
1. Adds an eth/stagedsync/test package which provides a test Harness
object
2. Adds the first automated test to the bor-heimdall stage regarding
span persistence (more to come in subsequent PRs)
3. Fixes a bug in the bor-heimdall stage which was uncovered with the
test - we do not fetch span 0 when we sync straight from blockNum=0
without snapshots
4. Reorganises all mocks to be placed under ./mock sub-package within
their respective packages
2023-12-14 22:50:59 +02:00
battlmonstr
ce57b8f54f
silkworm: make install (#8985)
We've got a report from a user that erigon fails to run with this error:

```
/opt/erigon/releases/latest/bin/erigon: error while loading shared libraries: libsilkworm_capi.so: cannot open shared object file: No such file or directory
```

On this system erigon is executed using a service-account user which has
no permission to access the build user's $HOME where the
libsilkworm_capi.so resides (inside $GOPATH/pkg/mod).

This adds a support to silkworm-go to look for the library relative to
the executable path on Linux:

d4ec8a8bce

and a new `make DIST=<path> install` command which copies both the
library and erigon binary to any destination.
2023-12-14 21:45:18 +07:00
Andrew Ashikhmin
b26d0f201e
Engine API PR 498 (clarify payloadAttributes checks) (#8982)
Implements https://github.com/ethereum/execution-apis/pull/498
2023-12-14 13:59:46 +01:00
Dmytro
3b68d57c28
collect statistics even for download completed or not started files (#8956)
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2023-12-14 07:53:54 +01:00
ledgerwatch
58dabdc601
Fixup bittorrent/webseed download logs after refactoring (#8976)
After a refactoring, the detailed bittorrent vs webseeds log has
disappeared
2023-12-13 17:40:20 +01:00
Giulio rebuffo
4e83ef0ef5
Optimized beacon state retrieval (#8963)
* Reconstruct previous epoch without looking at DB: no hindrance to
performance -> removed 15GB
* Store inactivity scores and slashings in MDBX and do not store diffs
for them(they are tiny 700/400 bytes)
* Reduced dumps from every 2048 to 1024 -> Added 5 GB (maybe we should
down it to 768)
* Parallel processing of shuffled sets, 2x performance boost in reading
participation.
* Store balances diffs in a Btree diff matter, see:
https://github.com/ledgerwatch/erigon-documents/blob/master/caplin/design/data-model.md#uint64listuint64vector
2023-12-13 15:23:27 +01:00
Dmytro
ac1e42b68d
added grabbing info about downloaded metadata (#8972) 2023-12-13 21:04:14 +07:00
Alex Sharov
06e77d1705
don't log cancelation error at graceful shutdown (#8943) 2023-12-12 17:07:12 +07:00
battlmonstr
c1146bda49
p2p: skip TestUDPv4_smallNetConvergence on Linux (#8731) (#8962) 2023-12-12 17:06:48 +07:00
Andrew Ashikhmin
faaf5a0c86
Amend incarnation logic (#8954) 2023-12-12 11:03:41 +01:00
Alex Sharov
d41d523050
Downloader: add ProhibitNewDownloads() (#8939)
"whitelisting" mechanism (list of files - stored in DB) - which
protecting us from downloading new files after upgrade/downgrade was
broken. And seems it became over-complicated with time.
I replacing it by 1 persistent flag inside downloader:
"prohibit_new_downloads.lock"
Erigon will turn downloader into this mode after
downloading/verification of first snapshots.


```
//Corner cases:
	// - Erigon generated file X with hash H1. User upgraded Erigon. New version has preverified file X with hash H2. Must ignore H2 (don't send to Downloader)
	// - Erigon "download once": means restart/upgrade/downgrade must not download files (and will be fast)
	// - After "download once" - Erigon will produce and seed new files
```

------
`downloader --seedbox` is never "prohibit new downloads"
2023-12-12 16:05:56 +07:00
Jacek Glen
4217f78368
Fix case when there are more than one matching libraries (#8955)
Small fix to the script for the scenario where more than one matching
library can be returned.
For example, the command `/sbin/ldconfig -p | grep libstdc++ | awk '{
print $NF }'` can result in

```
/lib/x86_64-linux-gnu/libstdc++.so.6
/lib32/libstdc++.so.6
```

which then fails the check `if [[ ! -L "$link_path" ]]`
2023-12-12 08:50:37 +01:00
Giulio rebuffo
24987878e4
Resumable beacon state reconstruction (#8918)
* Most of the PR changed files are extra and slightly more complicated
unit tests.
* Fixed Eth1DataVotes not inheriting genesis
* Fixed Attestations simulation using wrong slot when reconstructing
partecipation
* Fixed Copy() operation on BeaconState on Eth1DataVotes
* Used correct ListSSZ type for Eth1DataVotes and HistoricalSummaries
* Fixed wrong []uint64 deltas on empty slots
2023-12-11 14:07:57 +01:00
Alex Sharov
7fb8f9db59
log blocks stat after downloading, before indexing (#8947) 2023-12-11 10:05:51 +00:00
Alex Sharov
34d93fbe18
makefile: unquote silkworm build tag - to make it concatenatable (#8948)
`--tags "a,b,nosilkworm",integration` doesn't work
2023-12-11 09:18:07 +01:00
ddl
3d9cb0494f
fix: cl/beacon genesisResponse name (#8923) 2023-12-11 10:08:00 +07:00
a
036446777c
max blocks in gossipsource (#8944) 2023-12-10 07:40:04 -06:00
Alex Sharov
be10b7de29
bor: block snaps 50m (#8941) 2023-12-10 14:52:10 +07:00
3commascapital
07331f900f
Adds access list sorting (#8933)
Because access lists use maps with the `StorageKey` as the key, they are
subject to inconsistent ordering in the results of the `.accessList()`
method.

To get around this, an `accessListSorted` method has been added, and
exposed with the same name. The `equal` method has also been exposed to
allow for equality checks at this level outside of this module.

Co-authored-by: 3commascapital <8562488-3commascapital@users.noreply.gitlab.com>
2023-12-09 17:25:20 +07:00
Alex Sharov
e006db6ed9
evm cli: no logs if --json (#8925) 2023-12-09 13:42:07 +07:00
Dmytro
4696769d25
dvovk/snapshotsstats (#8935)
Updated collecting snapshots, renamed keys
2023-12-08 21:07:59 +07:00
Dmytro
c91ca2db85
dvovk/sync file level (#8931) 2023-12-08 12:08:38 +01:00
battlmonstr
b86bdb7a7e
silkworm: disable in release binaries (#8927)
A short-term safety measure to keep release.yml GH workflow unaffected.
2023-12-08 11:07:19 +01:00
battlmonstr
dac73f4b57
silkworm: check glibcpp compatibility (#8932)
libsilkworm requires libstdc++.so.6.0.30, but Rocky Linux 9.3 has only
libstdc++.so.6.0.29,
and `make erigon` produces an error about the GLIBCXX Version needed
3.4.30 (available 3.4.29).

see:

https://stackoverflow.com/questions/10354636/how-do-you-find-what-version-of-libstdc-library-is-installed-on-your-linux-mac
https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html
2023-12-07 17:53:29 +01:00
Somnath
5987d4ef72
Enable superfluous ws.port flag to fix some Hive RPC tests (#8909)
### Context
**Websocket port flag**
Hive tests for RPC suite depend on the (geth) default 8546 port. So,
opening one more listener for this additional port if `ws.port` was
specified. This flag isn't used in Erigon, as it shares port with http
listener. Normally, one may not specify and it offers no other benefit.
2023-12-07 14:59:22 +04:00
Alex Sharov
f3822b17d4
sepolia: auto-gen snaps to 4.7M (#8898) 2023-12-07 08:24:50 +07:00
Alex Sharov
6960ec3d81
up golang/x deps (#8900) 2023-12-06 18:49:52 +00:00
battlmonstr
d78cbfeceb
silkworm: disable on incompatible Linux versions (#8893)
Silkworm built on Ubuntu 22 depends on glibc 2.34. In order to run on an
older OS, Silkworm needs to be built and linked with an older glibc, but
to build on an older OS we need a compatible compiler. Silkworm requires
gcc 11+ that is not available on Ubuntu 20 or Debian 11.

To simplify the deployment disable Silkworm support on versions before
Ubuntu 22, Debian 12, and glibc prior to 2.34. The check for Ubuntu and
Debian is explicit, because some Ubuntu 16 installations report glibc
2.35 with ldd, but `go build` still uses an older system one and fails.
2023-12-06 16:01:44 +01:00
Mark Holt
624e4d4201
Fix snap initialization from frozen blocks (#8908)
This fixes a bug on syncing from scratch if the start point is in a
frozen block.
2023-12-06 14:28:05 +00:00
Giulio rebuffo
0d2aecf829
Backfill only with flag (#8913)
Caplin snapshots only enabled with caplin.backfill
2023-12-06 14:22:13 +01:00
4rgon4ut
8bfff94941
Update gnosis bootnodes (#8907)
Title seems selfdescribing.

Bootnodes list source:

https://github.com/gnosischain/configs/blob/main/mainnet/bootnodes_execution.yaml
2023-12-06 14:20:25 +01:00
Alex Sharov
754276909b
bor snaps: "erigon snapshots retire" to build bor files (#8912) 2023-12-06 12:12:43 +00:00
Giulio rebuffo
c477281362
Caplin: Parallel historical states reconstruction (#8817)
What does this PR do:
* Optional Backfilling and Caplin Archive Node
* Create antiquary for historical states
* Fixed gaps of chain gap related to the Head of the chain and anchor of
the chain.
* Added basic reader object to Read the Historical state
2023-12-06 10:48:36 +01:00