Commit Graph

20706 Commits

Author SHA1 Message Date
racytech
fa1e1bab27
Tests v13 fix (#9200) 2024-01-12 12:42:08 +01:00
Alex Sharov
e806db977f
mdbx: reduce 2 times hard dplimit (#9216)
can do it, because we don't do read-after-write in same rwtx
2024-01-12 11:27:18 +01:00
Alex Sharov
3bb1917e8a
recsplit: reduce ram pressure (#9218)
reasons: 
- indexing done in background (or in many workers)
- `recsplit` has 2 etl collectors
2024-01-12 11:26:20 +01:00
Mark Holt
66cd4e71fa
Mumbai uploader regression fixes (#9212)
This fixes a couple of regressions for running the uploader for mumbai

* Now flags have moved to a higher context they need to be set in the
context not the flag values
* Span 0 of mumbai has a header/span mismatch for span zero sprint 0. So
the check here needs to be suppressed
2024-01-11 21:15:26 +00:00
Giulio rebuffo
8315033a92
Added more checks on state reconstruction + made history reconstruction resumable. (#9211)
Co-authored-by: Bayram Guvanjov <bayramguwanjow@gmail.com>
2024-01-11 20:05:54 +01:00
Andrew Ashikhmin
c1e3ec59af
deps: upgrade go-multiaddr to v0.12.1 (#9208) 2024-01-11 16:42:11 +01:00
battlmonstr
04498180dc
p2p/discv4: revert gotreply handler change from #8661 (#9119) (#9195)
The handler had race conditions in the candidates processing goroutine.
2024-01-11 15:04:46 +00:00
dependabot[bot]
9a9808f715
build(deps): bump github.com/quic-go/quic-go from 0.38.1 to 0.38.2 (#9196)
Bumps [github.com/quic-go/quic-go](https://github.com/quic-go/quic-go)
from 0.38.1 to 0.38.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/quic-go/quic-go/releases">github.com/quic-go/quic-go's
releases</a>.</em></p>
<blockquote>
<h2>v0.38.2</h2>
<p>This release contains fixes for a resource exhaustion attack on
QUIC's path validation logic (CVE-2023-49295), see <a
href="https://seemann.io/posts/2023-12-18-exploiting-quics-path-validation">https://seemann.io/posts/2023-12-18-exploiting-quics-path-validation</a>
for details:</p>
<ul>
<li>limit the number of queued PATH_RESPONSE frames to 256 (<a
href="https://redirect.github.com/quic-go/quic-go/pull/4199">quic-go/quic-go#4199</a>)</li>
<li>don't retransmit PATH_CHALLENGE and PATH_RESPONSE frames (<a
href="https://redirect.github.com/quic-go/quic-go/pull/4200">quic-go/quic-go#4200</a>)</li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/quic-go/quic-go/compare/v0.38.1...v0.38.2">https://github.com/quic-go/quic-go/compare/v0.38.1...v0.38.2</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9aaefe19fc"><code>9aaefe1</code></a>
don't retransmit PATH_CHALLENGE and PATH_RESPONSE frames (<a
href="https://redirect.github.com/quic-go/quic-go/issues/4200">#4200</a>)</li>
<li><a
href="17fc98c2d8"><code>17fc98c</code></a>
limit the number of queued PATH_RESPONSE frames to 256 (<a
href="https://redirect.github.com/quic-go/quic-go/issues/4199">#4199</a>)</li>
<li>See full diff in <a
href="https://github.com/quic-go/quic-go/compare/v0.38.1...v0.38.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/quic-go/quic-go&package-manager=go_modules&previous-version=0.38.1&new-version=0.38.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/ledgerwatch/erigon/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-11 11:45:36 +01:00
Mark Holt
7308e87c0e
Fix txpool queue overflow (#9197)
When discarding spamming we need to remove the tx from the subpools as
well as the sender tx list. Otherwise the tx is ignored by other
operations and left in the subpool

As well as the fix here this PR also contains several changes to TX
TRACING and other logging to make it easier to see what is going on with
pool processing
2024-01-11 10:03:41 +00:00
Michelangelo Riccobene
470f05ee6f
clean-exit test: increase test time (#9193)
This PR
- increase the test time
- use the updated python scripts that reports as an issue a long exit
time
- change the test name to states that it is runned in the snapshot
downloading phase
(a later PR will add a similar test on the block downloading phase)
2024-01-11 08:47:23 +07:00
battlmonstr
b1c3006d7a
bor: remove duplicate validator_set file and debug logger (#9198) 2024-01-11 08:45:48 +07:00
Alex Sharov
1ee439e6ac
mdbx: dplimit - support cgroups/gomelimit, avoid using SetOptions after env.Open() (#9189) 2024-01-11 08:06:35 +07:00
Mark Holt
641f53e9d2
Remove duplicate flags (#9199)
This fixes the snapshot uploader start as flags have been moved to
default
2024-01-11 08:03:12 +07:00
milen
f690301c03
devnet: integration tests port clash fix (#9194)
Integration tests CI is failing due to a port clash in devnet tests. I
believe this is because there are 2 packages of devnet integration tests
and go can run tests from separate packages in parallel (by default it
does package level parallelism). A simple fix would be to just have all
devnet integration tests in 1 package and run all these tests
sequentially within the package (ie not use t.Parallel). This PR moves
all devnet integration tests in 1 package.

`"ContextStart devnet start failed: private api: could not create
listener: listen top 127.0.0.1:10090: bind: address already in use,
addr=localhost:10090"`

![Screenshot 2024-01-10 at 13 38
37](https://github.com/ledgerwatch/erigon/assets/94537774/06bda987-45e5-46ef-9e0b-3876b3f85c01)
2024-01-10 19:04:27 +00:00
Mark Holt
b05ffc909d
Fixes for Bor Block Production Synchronization (#9162)
This PR contains 3 fixes for interaction between the Bor mining loop and
the TX pool which where causing the regular creation of blocks with zero
transactions.

* Mining/Tx pool block synchronization
The synchronization of the tx pool between the sync loop and the mining
loop has been changed so that both are triggered by the same event and
synchronized via a sync.Cond rather than a polling loop with a hard
coded loop limit. This means that mining now waits for the pool to be
updated from the previous block before it starts the mining process.
* Txpool Startup consolidated into its MainLoop
Previously the tx pool start process was dynamically triggered at
various points in the code. This has all now been moved to the start of
the main loop. This is necessary to avoid a timing hole which can leave
the mining loop hanging waiting for a previously block broadcast which
it missed due to its delay start.
* Mining listens for block broadcast to avoid duplicate mining
operations
The mining loop for bor has a recommit timer in case blocks re not
produced on time. However in the case of sprint transitions where the
seal publication is delayed this can lead to duplicate block production.
This is suppressed by introducing a `waiting` state which is exited upon
the block being broadcast from the sealing operation.
2024-01-10 17:12:15 +00:00
Giulio rebuffo
284aa1dd0c
Added EthV1AggregateAndProofs API (#9192)
Added publishing of aggregates and proofs
2024-01-10 17:15:58 +01:00
Andrew Ashikhmin
09ec1f160c
Fix DenebForkEpoch for Chiado (#9188)
In PR #9184 I confused slot with epoch. Refer to
https://github.com/gnosischain/configs/pull/26.
2024-01-10 16:13:15 +01:00
milen
95109d9800
erigon-lib: cleanup unnnecessary import in tools.go added in prev PR (#9190) 2024-01-10 19:16:25 +07:00
Alex Sharov
ceeb090f48
mdbx: hard dirtyPages limit (#9178)
logic: 1/42 of ram, but not more than 2Gb for chandata and not more than
256mb for other databases.

---------

Co-authored-by: battlmonstr <battlmonstr@users.noreply.github.com>
2024-01-10 15:15:27 +07:00
milen
0cfe6617db
devnet: fix infinite recursion & enable TestStateSync & TestCallContract integration tests (#9183)
This PR fixes an infinite recursion (stack overflow) error in devnet
integration tests that surfaced in our integration CI a week or more ago
and additionally enables some integration tests that are now fixed -
`TestStateSync` & `TestCallContract`.

![Screenshot 2024-01-09 at 17 10
24](https://github.com/ledgerwatch/erigon/assets/94537774/a5a8c9c9-9f68-4084-9e08-1bf3c1601cab)
2024-01-09 20:02:14 +00:00
Andrew Ashikhmin
8db023bea6
Schedule Dencun for Chiado (#9184) 2024-01-09 20:57:02 +01:00
battlmonstr
9c47cce62c
bor: move to polygon directory (#9174) 2024-01-09 19:20:42 +01:00
milen
af913fdeeb
bor: fix for TestMiningBenchmark (#9182)
integration test TestMiningBenchmark is failing the CI on devel - this
PR aims to fix it
2024-01-09 15:41:00 +00:00
vuittont60
fec0c9eef5
docs: fix typos (#9173) 2024-01-09 15:34:47 +00:00
milen
e25b15b00e
remotedbserver: add support for bor snapshots (#9180) 2024-01-09 14:48:01 +00:00
milen
74ec3a9db7
stagedsync: fix bor heimdall mining flow (#9149)
Currently the mining loop is broken for the polygon chain. This PR fixes
this.

High level changes:

- Introduces new Bor<->Heimdall stage specifically for the needs of the
mining flow
- Extracts out common logic from Bor<->Heimdall sync and mining stages
into shared functions
- Removes `mine` flag for the Bor<->Heimdall sync stage
- Extends the current `StartMining` function to prefetch span zero if
needed before the mining loop is started
- Fixes Bor to read span zero (instead of span 1) from heimdall when the
span is not initially set in the local smart contract that the Spanner
uses

Test with devnet "state-sync" scenario:
![Screenshot 2024-01-05 at 17 41
23](https://github.com/ledgerwatch/erigon/assets/94537774/34ca903a-69b8-416a-900f-a32f2d4417fa)
2024-01-09 11:37:39 +00:00
battlmonstr
302f1f772d
ci: run macOS jobs on macos-13 instead of macos-13-xlarge (#9177)
macos-13-xlarge was needed to use M1 CPU,
because previously silkworm-go didn't have macOS Intel support
2024-01-09 11:49:04 +01:00
ledgerwatch
459ccf8de4
[E3] Some fixes for the in-memory database when working with Caplin (… (#9164)
…testing on Sepolia) (#9151)

---------

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2024-01-09 08:26:26 +07:00
Alex Sharov
22f761593f
disable TestStateAntiquaryCapella because of oom on CI (#9129)
https://github.com/ledgerwatch/erigon/issues/9128
2024-01-09 08:25:39 +07:00
Alex Sharov
b25d8cc873
disable TestGetValidatorsBalances: oom on CI (#9131)
https://github.com/ledgerwatch/erigon/issues/9130
2024-01-09 08:25:27 +07:00
Alex Sharov
36fefbbee1
downloader: verify fixempty (#9161) 2024-01-09 08:25:12 +07:00
milen
19b9504039
stagedsync: borheimdall segments min check minor improvement (#9168) 2024-01-08 18:37:22 +00:00
Dmytro
b4e6563e4a
fixed sync.loop flags (#9167) 2024-01-08 17:52:28 +00:00
Giulio rebuffo
b2fa618f74
Caplin: Fixed crash in OnAttestation (#9159)
* Added handling of AggregateAndProof
2024-01-08 17:13:25 +01:00
battlmonstr
2521f47e7b
polygon/sync: canonical chain builder unit tests (#9137) 2024-01-08 15:55:43 +01:00
Alex Sharov
de6c1593e4
"erigon snapshots retire" - doesn't see any files because version is 0 - fix (#9163) 2024-01-08 11:30:51 +00:00
Dmytro
ff92b701c3
dvovk/updsync (#9134)
refactored data structure for sync statistics
2024-01-08 10:43:04 +01:00
Michelangelo Riccobene
b4fd278533
clean-exit test: bump golang version to 1.21 (#9160) 2024-01-08 16:18:00 +07:00
Alex Sharov
7490e7dd63
up x and cli deps (#9154) 2024-01-08 16:01:36 +07:00
Alex Sharov
2cbe8b7fe7
downloader: --verify --verify.files --verify.failfast support (#9156) 2024-01-08 16:01:02 +07:00
Giulio rebuffo
fe9bc35e1e
Added GetEth1V1BuilderStatesExpectedWithdrawals (#9152) 2024-01-08 09:42:47 +01:00
Arpit Temani
134dc8573e
add amoy bootnodes (#9158)
this PR adds amoy bootnodes to the configuration
2024-01-08 12:26:08 +05:30
Mark Holt
15ff41876c
Change retire progress log level to debug (#9153)
This moved the log level of retire progress messaging to debug, to avoid
log nose on qa and test runs
2024-01-08 07:10:45 +07:00
a
235af8a321
glob filter (#9148) 2024-01-07 12:03:17 -06:00
Giulio rebuffo
e958d3584d
Added Heads method to Beacon API (#9135) 2024-01-06 21:49:23 +01:00
milen
98cc1ee808
stagedsync: implement bor span for chain reader and fix loggers (#9146)
While working on fixing the bor mining loop I stumbled across an error
in `ChainReader.BorSpan` - not implemented panic. Also hit a few other
panics due to missed logger in `ChainReaderImpl` struct initialisations.
This PR fixes both.
2024-01-05 14:20:21 +00:00
battlmonstr
ebe16d8360
bor: BorConfig setup fix (#9145)
A crash on startup happens on --chain=mumbai , because I've confused
chainConfig.Bor (from type chain.Config) and config.Bor (from type
ethconfig.Config) in the setup code.

ethconfig.Config.Bor property contained bogus values, and was used only
to check its type in CreateConsensusEngine(). Its value was never read
(before PR #9117).

This change removes the property to avoid confusion and fix the crash.

Devnet network.BorStateSyncDelay was implemented using
ethconfig.Config.Bor, but it wasn't taking any effect. It should be
fixed separately in a different way.
2024-01-05 20:03:19 +07:00
Andrew Ashikhmin
3eeb57218f
params: begin 2.57 release cycle (#9144) 2024-01-05 13:03:36 +01:00
a
9cd5f5837f
data (#9126) 2024-01-04 10:56:14 -06:00
battlmonstr
b57cbdcff7
polygon/sync: canonical chain builder (#9117) 2024-01-04 10:44:57 +01:00