Commit Graph

1151 Commits

Author SHA1 Message Date
Shane Bammel
43ef4304ee Add network config for Testnet-V4 2024-02-14 13:10:46 -06:00
Shane Bammel
4ea8213c42 Add v3 suffix to pulsechain-testnet 2024-02-14 13:10:46 -06:00
Shane Bammel
75db06fa1d Add support for PulseChain
Thanks @bretep for the original integration.
2024-02-14 13:10:45 -06:00
Alex Sharov
26ce6aa88b
return 1 lost mainnet file (#9242) 2024-01-17 14:21:51 +07:00
battlmonstr
cda48aeaf7
polygon/heimdall: drop GRPC support (#9230) 2024-01-15 16:49:46 +01:00
a
8d4d4d802c
refac some (#9185) 2024-01-14 23:22:34 -06:00
Andrew Ashikhmin
c1e3ec59af
deps: upgrade go-multiaddr to v0.12.1 (#9208) 2024-01-11 16:42:11 +01:00
dependabot[bot]
9a9808f715
build(deps): bump github.com/quic-go/quic-go from 0.38.1 to 0.38.2 (#9196)
Bumps [github.com/quic-go/quic-go](https://github.com/quic-go/quic-go)
from 0.38.1 to 0.38.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/quic-go/quic-go/releases">github.com/quic-go/quic-go's
releases</a>.</em></p>
<blockquote>
<h2>v0.38.2</h2>
<p>This release contains fixes for a resource exhaustion attack on
QUIC's path validation logic (CVE-2023-49295), see <a
href="https://seemann.io/posts/2023-12-18-exploiting-quics-path-validation">https://seemann.io/posts/2023-12-18-exploiting-quics-path-validation</a>
for details:</p>
<ul>
<li>limit the number of queued PATH_RESPONSE frames to 256 (<a
href="https://redirect.github.com/quic-go/quic-go/pull/4199">quic-go/quic-go#4199</a>)</li>
<li>don't retransmit PATH_CHALLENGE and PATH_RESPONSE frames (<a
href="https://redirect.github.com/quic-go/quic-go/pull/4200">quic-go/quic-go#4200</a>)</li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/quic-go/quic-go/compare/v0.38.1...v0.38.2">https://github.com/quic-go/quic-go/compare/v0.38.1...v0.38.2</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9aaefe19fc"><code>9aaefe1</code></a>
don't retransmit PATH_CHALLENGE and PATH_RESPONSE frames (<a
href="https://redirect.github.com/quic-go/quic-go/issues/4200">#4200</a>)</li>
<li><a
href="17fc98c2d8"><code>17fc98c</code></a>
limit the number of queued PATH_RESPONSE frames to 256 (<a
href="https://redirect.github.com/quic-go/quic-go/issues/4199">#4199</a>)</li>
<li>See full diff in <a
href="https://github.com/quic-go/quic-go/compare/v0.38.1...v0.38.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/quic-go/quic-go&package-manager=go_modules&previous-version=0.38.1&new-version=0.38.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/ledgerwatch/erigon/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-11 11:45:36 +01:00
Alex Sharov
7490e7dd63
up x and cli deps (#9154) 2024-01-08 16:01:36 +07:00
Giulio rebuffo
3d10cee49b
Make Caplin work with Otterscan (#9115)
* Fixed mispelling in json fields
* Added CORS
2024-01-02 23:29:00 +01:00
Giulio rebuffo
415b17ca85
Updated erigon-snapshots (#9111) 2024-01-02 09:23:16 +07:00
Giulio rebuffo
a959387aae
Make caplin sync snapshots (all of them) (#9106) 2024-01-01 08:18:56 +07:00
Giulio rebuffo
580c55517f
Make caplin snapshots downloadable (only sepolia) (#9105) 2023-12-31 13:22:41 +01:00
a
78bb3cdca8
[caplin] topic strings (#9000) 2023-12-30 16:55:01 +01:00
a
510d62ab8a
[caplin] sse handler placeholder (#9082) 2023-12-30 15:49:45 +01:00
ledgerwatch
ab27531fa7
Introduce new public buckets for the snapshots (#9094)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-12-28 12:39:25 +00:00
Alex Sharov
a48db431cb
return deleted caplin-v2 webseed (#9083) 2023-12-26 18:55:17 +07:00
Alex Sharov
0d31444eea
bor mainnet 51m (#9073) 2023-12-25 11:24:14 +07:00
Alex Sharov
2944abbd19
fix typo in snap webseed server names (#9072) 2023-12-24 16:37:13 +07:00
Giulio rebuffo
a4d7b6d33f
Switched Caplin snapshot format to ZSTD blinded blocks (#9058)
* Chunked format -> blinded
* LZ4 -> ZSTD
* Implemented parent block root support for history download
* Rationale: Allows to optimize GC collection easily on state
reconstruction and it allows to read fast attestations in historical
states reader
2023-12-23 15:56:35 +01:00
ledgerwatch
ec970ac6bc
Fixed duplicate bucket names, remove presigned URLs (#9060)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-12-23 04:11:12 +00:00
Alex Sharov
8c822256db
remove bor mainnet files (#9033) 2023-12-21 08:09:54 +07:00
dependabot[bot]
33fc53f83d
build(deps): bump golang.org/x/crypto from 0.16.0 to 0.17.0 (#9020)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from
0.16.0 to 0.17.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9d2ee975ef"><code>9d2ee97</code></a>
ssh: implement strict KEX protocol changes</li>
<li><a
href="4e5a26183e"><code>4e5a261</code></a>
ssh: close net.Conn on all NewServerConn errors</li>
<li><a
href="152cdb1503"><code>152cdb1</code></a>
x509roots/fallback: update bundle</li>
<li><a
href="fdfe1f8531"><code>fdfe1f8</code></a>
ssh: defer channel window adjustment</li>
<li><a
href="b8ffc16e10"><code>b8ffc16</code></a>
blake2b: drop Go 1.6, Go 1.8 compatibility</li>
<li><a
href="7e6fbd82c8"><code>7e6fbd8</code></a>
ssh: wrap errors from client handshake</li>
<li><a
href="bda2f3f5cf"><code>bda2f3f</code></a>
argon2: avoid clobbering BP</li>
<li>See full diff in <a
href="https://github.com/golang/crypto/compare/v0.16.0...v0.17.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/crypto&package-manager=go_modules&previous-version=0.16.0&new-version=0.17.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/ledgerwatch/erigon/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-12-19 08:34:09 +07:00
battlmonstr
ce57b8f54f
silkworm: make install (#8985)
We've got a report from a user that erigon fails to run with this error:

```
/opt/erigon/releases/latest/bin/erigon: error while loading shared libraries: libsilkworm_capi.so: cannot open shared object file: No such file or directory
```

On this system erigon is executed using a service-account user which has
no permission to access the build user's $HOME where the
libsilkworm_capi.so resides (inside $GOPATH/pkg/mod).

This adds a support to silkworm-go to look for the library relative to
the executable path on Linux:

d4ec8a8bce

and a new `make DIST=<path> install` command which copies both the
library and erigon binary to any destination.
2023-12-14 21:45:18 +07:00
Alex Sharov
be10b7de29
bor: block snaps 50m (#8941) 2023-12-10 14:52:10 +07:00
Alex Sharov
f3822b17d4
sepolia: auto-gen snaps to 4.7M (#8898) 2023-12-07 08:24:50 +07:00
Alex Sharov
6960ec3d81
up golang/x deps (#8900) 2023-12-06 18:49:52 +00:00
Giulio rebuffo
c477281362
Caplin: Parallel historical states reconstruction (#8817)
What does this PR do:
* Optional Backfilling and Caplin Archive Node
* Create antiquary for historical states
* Fixed gaps of chain gap related to the Head of the chain and anchor of
the chain.
* Added basic reader object to Read the Historical state
2023-12-06 10:48:36 +01:00
battlmonstr
96bb5ddb81
silkworm: macOS Intel library (#8891)
61782fa8d0

435dee4642
2023-12-06 11:02:54 +07:00
Alex Sharov
ad48ecdcbb
downloader: download rates per peer with webseeds (#8879) 2023-12-01 13:39:43 +00:00
Stéphane Loeuillet
b5640bb64b
Bump GraphQL version + implement blocks (block range) (#8872) 2023-12-01 10:21:16 +01:00
battlmonstr
ebdfbd8e54
silkworm: disable on arm64 Linux (#8870)
see
c5729e30cf
2023-11-30 22:49:00 +07:00
battlmonstr
bc0b727fc0
silkworm: use silkworm-go bindings (#8829) 2023-11-30 12:45:02 +01:00
Alex Sharov
0fbcd5b5d8
downloader: use manifest.txt for public bucket (#8863)
use manifest.txt instead of webseed.toml in public buckets
2023-11-30 16:58:23 +07:00
milen
28fff1b35e
metrics: remove VictoriaMetrics/metrics lib (#8766)
# Background

Erigon currently uses a combination of Victoria Metrics and Prometheus
client for providing metrics.

We want to rationalize this and use only the Prometheus client library,
but we want to maintain the simplified Victoria Metrics methods for
constructing metrics.

This task is currently partly complete and needs to be finished to a
stage where we can remove the Victoria Metrics module from the Erigon
code base.

# Summary of changes

- Remove `UsePrometheusClient` boolean flag
- Remove `VictoriaMetrics` client lib and related code (simplifies
registry and prometheus http handler initialisation since now we have
only 1 registry and can use default `promhttp.Handler`)
2023-11-23 20:31:38 +00:00
milen
34c0fe29ad
metrics: swap remaining VictoriaMetrics usages with erigon-lib/metrics (#8762)
# Background

Erigon currently uses a combination of Victoria Metrics and Prometheus
client for providing metrics.

We want to rationalize this and use only the Prometheus client library,
but we want to maintain the simplified Victoria Metrics methods for
constructing metrics.

This task is currently partly complete and needs to be finished to a
stage where we can remove the Victoria Metrics module from the Erigon
code base.

# Summary of changes

- Adds missing `NewCounter`, `NewSummary`, `NewHistogram`,
`GetOrCreateHistogram` functions to `erigon-lib/metrics` similar to the
interface VictoriaMetrics lib provides
- Minor tidy up for consistency inside `erigon-lib/metrics/set.go`
around return types (panic vs err consistency for funcs inside the
file), error messages, comments
- Replace all remaining usages of `github.com/VictoriaMetrics/metrics`
with `github.com/ledgerwatch/erigon-lib/metrics` - seamless (only import
changes) since interfaces match
2023-11-20 12:23:23 +00:00
Alex Sharov
33f42df10e
snaps: mumbai 42.5m (#8784) 2023-11-20 11:13:57 +07:00
Alex Sharov
950bb12098
metrics: race fix (#8701) 2023-11-11 18:09:04 +03:00
Alex Sharov
35bfffd621
sys deps up (#8695) 2023-11-11 15:04:18 +03:00
Alex Sharov
6722334671
fix mdbx version in devel branch (#8679) 2023-11-10 19:44:15 +03:00
Giulio rebuffo
e08b031f74
Updated go.sum and go.mod for snapshots (#8664) 2023-11-06 23:39:44 +01:00
Giulio rebuffo
4b580dcc2f
update caplin snapshots hashes (#8663)
This PR also adds snippets to download caplin snapshots
2023-11-06 21:05:07 +01:00
Alex Sharov
714fa0fdc7
mainnet 18.3M snaps (#8639) 2023-11-02 14:14:39 +07:00
Alex Sharov
329d18ef6f
snapshots: reduce merge limit of blocks to 100K (#8614)
Reason: 
- produce and seed snapshots earlier on chain tip. reduce depnedency on
"good peers with history" at p2p-network.
Some networks have no much archive peers, also ConsensusLayer clients
are not-good(not-incentivised) at serving history.
- avoiding having too much files:
more files(shards) - means "more metadata", "more lookups for
non-indexed queries", "more dictionaries", "more bittorrent
connections", ...
less files - means small files will be removed after merge (no peers for
this files).


ToDo:
[x] Recent 500K - merge up to 100K 
[x] Older than 500K - merge up to 500K 
[x] Start seeding 100k files
[x] Stop seeding 100k files after merge (right before delete)

In next PR: 
[] Old version of Erigon must be able download recent hashes. To achieve
it - at first start erigon will download preverified hashes .toml from
s3 - if it's newer that what we have (build-in) - use it.
2023-11-01 23:22:35 +07:00
Alex Sharov
b5ea87d8c5
mdbx_empty_kv_dup (#8635) 2023-11-01 15:50:27 +07:00
battlmonstr
eb65ffbaa7
config: avoid OOM in docker using cgroups v2 limit (#6646) (#8632)
Tested on Debian 11 by:

    $ git revert HEAD
    $ make docker
    $ docker run --memory=8.8G <sha>

before the fix outputs:

    15.6 GB

after the fix outputs:

    8.8 GB
2023-11-01 09:02:34 +07:00
Alex Sharov
c90bff7e22
mdbx: use release tag (#8628) 2023-10-31 10:11:40 +07:00
Giulio rebuffo
513fd50fa5
Compress snapshots for Caplin with lz4 level=1 (#8609) 2023-10-30 13:48:14 +01:00
Giulio rebuffo
0e5af0a69c
Added beacon snapshots download (#8601) 2023-10-28 17:41:50 +02:00
Alex Sharov
f36d090f39
make db-tools: suspend enum-int linter error (#8598)
```
 error: conflicting types for ‘mdbx_get_datacmp’ due to enum/integer mismatch; have ‘int (*(unsigned int))(const MDBX_val *, const MDBX_val *)’ {aka ‘int (*(unsigned int))(const struct iovec *, const struct iovec *)’} [-Werror=enum-int-mismatch]
26609 | __cold MDBX_cmp_func *mdbx_get_datacmp(unsigned flags) {
      |                       ^~~~~~~~~~~~~~~~
mdbx.h:4932:1: note: previous declaration of ‘mdbx_get_datacmp’ with type ‘int (*(MDBX_db_flags_t))(const MDBX_val *, const MDBX_val *)’ {aka ‘int (*(MDBX_db_flags_t))(const struct iovec *, const struct iovec *)’}
```
can't upgrade mdbx version for now
2023-10-27 11:01:11 +07:00