Commit Graph

20717 Commits

Author SHA1 Message Date
battlmonstr
ac9f9e0a25
polygon/sync: fork choice logic (#9228) 2024-01-14 13:40:47 +00:00
Mark Holt
d8b91c4d02
Fix startup sync for txpool processing for bor block production (#9219)
When the sync loop first runs it suppresses block sync events both in
the initial loop and when the blocks being processed are greater than
1000.

This fix removed the first check, because otherwise the first block
received by the process ends up not getting sent to the tx pool. Which
means it won't produce new block for polygon.

As well as this fix - I have also moved the gas initialization to the
txpool start method rather than prompting it with a 'synthetic block
event'

As the txpool start has access to the core & tx DB's it can find the
current block and chain config internally so that it doesn't need to be
externally activated it can just do this itself on start up. This has
the advantage of making the txpool more self contained.
2024-01-13 10:33:34 +00:00
Pan chao
2b0fd6d447
chore: some proofreading in the docs (#9223)
Hi, caught a few mistakes while reading the docs. Hope I can be of any
help.
Have a nice day.
2024-01-13 12:04:55 +07:00
Giulio rebuffo
7b58dab998
Added LightClient Containers (#9222) 2024-01-13 01:12:02 +01:00
battlmonstr
52a948731c
polygon: refactor header validations for reuse in sync (#9224) 2024-01-12 21:01:28 +01:00
Mark Holt
320fe0ab8b
Don't run pos downloader or engine api for bor (#9220)
Pos downloader and the engine api code are not necessary for Bor. This
adds code to suppress their start up.
2024-01-12 16:07:59 +00:00
Arpit Temani
a592cbf163
Removed the check for milestoneID in the GetVoteOnHash() (#9021)
Removed the check for milestoneID in the GetVoteOnHash()

https://github.com/maticnetwork/bor/pull/1109
2024-01-12 21:20:53 +05:30
battlmonstr
c11e5047fb
polygon: refactor header.Time validation (#9213) 2024-01-12 16:11:01 +01:00
Bayram Guvanjov
a191296f05
Deneb integration to Caplin (#9093)
Pr is ready to review and merge. 

This PR includes implementing and integrating Ethereum Deneb's hard work
with the Caplin Ethereum client.

Changes:

- Full compatibility with Deneb Ethereum hard fork
- Added new EIPs introduced in Deneb. (`EIP-4788`, `EIP-4844`,
`EIP-7044`, `EIP-7045`, `EIP-7514`)
- Tests integration

---------

Co-authored-by: Giulio <giulio.rebuffo@gmail.com>
2024-01-12 14:20:26 +01:00
Alex Sharov
574ec8089d
caplin state_antiqate: reduce worst case ram limit (#9217) 2024-01-12 14:19:24 +01:00
ddl
db1dcbb9d7
refactor(erigon-lib/metrics): replace strings.IndexByte with strings.Cut (#9202)
above go1.18  Index calls are more clearly written using Cut.
https://github.com/golang/go/issues/46336
2024-01-12 13:07:49 +00:00
racytech
fa1e1bab27
Tests v13 fix (#9200) 2024-01-12 12:42:08 +01:00
Alex Sharov
e806db977f
mdbx: reduce 2 times hard dplimit (#9216)
can do it, because we don't do read-after-write in same rwtx
2024-01-12 11:27:18 +01:00
Alex Sharov
3bb1917e8a
recsplit: reduce ram pressure (#9218)
reasons: 
- indexing done in background (or in many workers)
- `recsplit` has 2 etl collectors
2024-01-12 11:26:20 +01:00
Mark Holt
66cd4e71fa
Mumbai uploader regression fixes (#9212)
This fixes a couple of regressions for running the uploader for mumbai

* Now flags have moved to a higher context they need to be set in the
context not the flag values
* Span 0 of mumbai has a header/span mismatch for span zero sprint 0. So
the check here needs to be suppressed
2024-01-11 21:15:26 +00:00
Giulio rebuffo
8315033a92
Added more checks on state reconstruction + made history reconstruction resumable. (#9211)
Co-authored-by: Bayram Guvanjov <bayramguwanjow@gmail.com>
2024-01-11 20:05:54 +01:00
Andrew Ashikhmin
c1e3ec59af
deps: upgrade go-multiaddr to v0.12.1 (#9208) 2024-01-11 16:42:11 +01:00
battlmonstr
04498180dc
p2p/discv4: revert gotreply handler change from #8661 (#9119) (#9195)
The handler had race conditions in the candidates processing goroutine.
2024-01-11 15:04:46 +00:00
dependabot[bot]
9a9808f715
build(deps): bump github.com/quic-go/quic-go from 0.38.1 to 0.38.2 (#9196)
Bumps [github.com/quic-go/quic-go](https://github.com/quic-go/quic-go)
from 0.38.1 to 0.38.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/quic-go/quic-go/releases">github.com/quic-go/quic-go's
releases</a>.</em></p>
<blockquote>
<h2>v0.38.2</h2>
<p>This release contains fixes for a resource exhaustion attack on
QUIC's path validation logic (CVE-2023-49295), see <a
href="https://seemann.io/posts/2023-12-18-exploiting-quics-path-validation">https://seemann.io/posts/2023-12-18-exploiting-quics-path-validation</a>
for details:</p>
<ul>
<li>limit the number of queued PATH_RESPONSE frames to 256 (<a
href="https://redirect.github.com/quic-go/quic-go/pull/4199">quic-go/quic-go#4199</a>)</li>
<li>don't retransmit PATH_CHALLENGE and PATH_RESPONSE frames (<a
href="https://redirect.github.com/quic-go/quic-go/pull/4200">quic-go/quic-go#4200</a>)</li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/quic-go/quic-go/compare/v0.38.1...v0.38.2">https://github.com/quic-go/quic-go/compare/v0.38.1...v0.38.2</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9aaefe19fc"><code>9aaefe1</code></a>
don't retransmit PATH_CHALLENGE and PATH_RESPONSE frames (<a
href="https://redirect.github.com/quic-go/quic-go/issues/4200">#4200</a>)</li>
<li><a
href="17fc98c2d8"><code>17fc98c</code></a>
limit the number of queued PATH_RESPONSE frames to 256 (<a
href="https://redirect.github.com/quic-go/quic-go/issues/4199">#4199</a>)</li>
<li>See full diff in <a
href="https://github.com/quic-go/quic-go/compare/v0.38.1...v0.38.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=github.com/quic-go/quic-go&package-manager=go_modules&previous-version=0.38.1&new-version=0.38.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/ledgerwatch/erigon/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-11 11:45:36 +01:00
Mark Holt
7308e87c0e
Fix txpool queue overflow (#9197)
When discarding spamming we need to remove the tx from the subpools as
well as the sender tx list. Otherwise the tx is ignored by other
operations and left in the subpool

As well as the fix here this PR also contains several changes to TX
TRACING and other logging to make it easier to see what is going on with
pool processing
2024-01-11 10:03:41 +00:00
Michelangelo Riccobene
470f05ee6f
clean-exit test: increase test time (#9193)
This PR
- increase the test time
- use the updated python scripts that reports as an issue a long exit
time
- change the test name to states that it is runned in the snapshot
downloading phase
(a later PR will add a similar test on the block downloading phase)
2024-01-11 08:47:23 +07:00
battlmonstr
b1c3006d7a
bor: remove duplicate validator_set file and debug logger (#9198) 2024-01-11 08:45:48 +07:00
Alex Sharov
1ee439e6ac
mdbx: dplimit - support cgroups/gomelimit, avoid using SetOptions after env.Open() (#9189) 2024-01-11 08:06:35 +07:00
Mark Holt
641f53e9d2
Remove duplicate flags (#9199)
This fixes the snapshot uploader start as flags have been moved to
default
2024-01-11 08:03:12 +07:00
milen
f690301c03
devnet: integration tests port clash fix (#9194)
Integration tests CI is failing due to a port clash in devnet tests. I
believe this is because there are 2 packages of devnet integration tests
and go can run tests from separate packages in parallel (by default it
does package level parallelism). A simple fix would be to just have all
devnet integration tests in 1 package and run all these tests
sequentially within the package (ie not use t.Parallel). This PR moves
all devnet integration tests in 1 package.

`"ContextStart devnet start failed: private api: could not create
listener: listen top 127.0.0.1:10090: bind: address already in use,
addr=localhost:10090"`

![Screenshot 2024-01-10 at 13 38
37](https://github.com/ledgerwatch/erigon/assets/94537774/06bda987-45e5-46ef-9e0b-3876b3f85c01)
2024-01-10 19:04:27 +00:00
Mark Holt
b05ffc909d
Fixes for Bor Block Production Synchronization (#9162)
This PR contains 3 fixes for interaction between the Bor mining loop and
the TX pool which where causing the regular creation of blocks with zero
transactions.

* Mining/Tx pool block synchronization
The synchronization of the tx pool between the sync loop and the mining
loop has been changed so that both are triggered by the same event and
synchronized via a sync.Cond rather than a polling loop with a hard
coded loop limit. This means that mining now waits for the pool to be
updated from the previous block before it starts the mining process.
* Txpool Startup consolidated into its MainLoop
Previously the tx pool start process was dynamically triggered at
various points in the code. This has all now been moved to the start of
the main loop. This is necessary to avoid a timing hole which can leave
the mining loop hanging waiting for a previously block broadcast which
it missed due to its delay start.
* Mining listens for block broadcast to avoid duplicate mining
operations
The mining loop for bor has a recommit timer in case blocks re not
produced on time. However in the case of sprint transitions where the
seal publication is delayed this can lead to duplicate block production.
This is suppressed by introducing a `waiting` state which is exited upon
the block being broadcast from the sealing operation.
2024-01-10 17:12:15 +00:00
Giulio rebuffo
284aa1dd0c
Added EthV1AggregateAndProofs API (#9192)
Added publishing of aggregates and proofs
2024-01-10 17:15:58 +01:00
Andrew Ashikhmin
09ec1f160c
Fix DenebForkEpoch for Chiado (#9188)
In PR #9184 I confused slot with epoch. Refer to
https://github.com/gnosischain/configs/pull/26.
2024-01-10 16:13:15 +01:00
milen
95109d9800
erigon-lib: cleanup unnnecessary import in tools.go added in prev PR (#9190) 2024-01-10 19:16:25 +07:00
Alex Sharov
ceeb090f48
mdbx: hard dirtyPages limit (#9178)
logic: 1/42 of ram, but not more than 2Gb for chandata and not more than
256mb for other databases.

---------

Co-authored-by: battlmonstr <battlmonstr@users.noreply.github.com>
2024-01-10 15:15:27 +07:00
milen
0cfe6617db
devnet: fix infinite recursion & enable TestStateSync & TestCallContract integration tests (#9183)
This PR fixes an infinite recursion (stack overflow) error in devnet
integration tests that surfaced in our integration CI a week or more ago
and additionally enables some integration tests that are now fixed -
`TestStateSync` & `TestCallContract`.

![Screenshot 2024-01-09 at 17 10
24](https://github.com/ledgerwatch/erigon/assets/94537774/a5a8c9c9-9f68-4084-9e08-1bf3c1601cab)
2024-01-09 20:02:14 +00:00
Andrew Ashikhmin
8db023bea6
Schedule Dencun for Chiado (#9184) 2024-01-09 20:57:02 +01:00
battlmonstr
9c47cce62c
bor: move to polygon directory (#9174) 2024-01-09 19:20:42 +01:00
milen
af913fdeeb
bor: fix for TestMiningBenchmark (#9182)
integration test TestMiningBenchmark is failing the CI on devel - this
PR aims to fix it
2024-01-09 15:41:00 +00:00
vuittont60
fec0c9eef5
docs: fix typos (#9173) 2024-01-09 15:34:47 +00:00
milen
e25b15b00e
remotedbserver: add support for bor snapshots (#9180) 2024-01-09 14:48:01 +00:00
milen
74ec3a9db7
stagedsync: fix bor heimdall mining flow (#9149)
Currently the mining loop is broken for the polygon chain. This PR fixes
this.

High level changes:

- Introduces new Bor<->Heimdall stage specifically for the needs of the
mining flow
- Extracts out common logic from Bor<->Heimdall sync and mining stages
into shared functions
- Removes `mine` flag for the Bor<->Heimdall sync stage
- Extends the current `StartMining` function to prefetch span zero if
needed before the mining loop is started
- Fixes Bor to read span zero (instead of span 1) from heimdall when the
span is not initially set in the local smart contract that the Spanner
uses

Test with devnet "state-sync" scenario:
![Screenshot 2024-01-05 at 17 41
23](https://github.com/ledgerwatch/erigon/assets/94537774/34ca903a-69b8-416a-900f-a32f2d4417fa)
2024-01-09 11:37:39 +00:00
battlmonstr
302f1f772d
ci: run macOS jobs on macos-13 instead of macos-13-xlarge (#9177)
macos-13-xlarge was needed to use M1 CPU,
because previously silkworm-go didn't have macOS Intel support
2024-01-09 11:49:04 +01:00
ledgerwatch
459ccf8de4
[E3] Some fixes for the in-memory database when working with Caplin (… (#9164)
…testing on Sepolia) (#9151)

---------

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2024-01-09 08:26:26 +07:00
Alex Sharov
22f761593f
disable TestStateAntiquaryCapella because of oom on CI (#9129)
https://github.com/ledgerwatch/erigon/issues/9128
2024-01-09 08:25:39 +07:00
Alex Sharov
b25d8cc873
disable TestGetValidatorsBalances: oom on CI (#9131)
https://github.com/ledgerwatch/erigon/issues/9130
2024-01-09 08:25:27 +07:00
Alex Sharov
36fefbbee1
downloader: verify fixempty (#9161) 2024-01-09 08:25:12 +07:00
milen
19b9504039
stagedsync: borheimdall segments min check minor improvement (#9168) 2024-01-08 18:37:22 +00:00
Dmytro
b4e6563e4a
fixed sync.loop flags (#9167) 2024-01-08 17:52:28 +00:00
Giulio rebuffo
b2fa618f74
Caplin: Fixed crash in OnAttestation (#9159)
* Added handling of AggregateAndProof
2024-01-08 17:13:25 +01:00
battlmonstr
2521f47e7b
polygon/sync: canonical chain builder unit tests (#9137) 2024-01-08 15:55:43 +01:00
Alex Sharov
de6c1593e4
"erigon snapshots retire" - doesn't see any files because version is 0 - fix (#9163) 2024-01-08 11:30:51 +00:00
Dmytro
ff92b701c3
dvovk/updsync (#9134)
refactored data structure for sync statistics
2024-01-08 10:43:04 +01:00
Michelangelo Riccobene
b4fd278533
clean-exit test: bump golang version to 1.21 (#9160) 2024-01-08 16:18:00 +07:00
Alex Sharov
7490e7dd63
up x and cli deps (#9154) 2024-01-08 16:01:36 +07:00