Commit Graph

20764 Commits

Author SHA1 Message Date
Andrew Ashikhmin
333e0e5666
flags: bump default value of db.size.limit to 8TB (#8869)
3TB is not enough for Polygon
2023-11-30 21:08:26 +07:00
battlmonstr
bc0b727fc0
silkworm: use silkworm-go bindings (#8829) 2023-11-30 12:45:02 +01:00
Alex Sharov
0fbcd5b5d8
downloader: use manifest.txt for public bucket (#8863)
use manifest.txt instead of webseed.toml in public buckets
2023-11-30 16:58:23 +07:00
Alex Sharov
5ff9ce802b
prnt_stages: to show value of --snapshots flag (#8862) 2023-11-30 16:16:53 +07:00
Guido Vranken
015f6eb4c5
Call Tracer.CaptureState before memory expansion (#8860)
This aligns the tracing behavior with Geth, Besu and Nimbus.
2023-11-30 11:33:11 +07:00
Anshal Shukla
8d1758ceea
Add support for amoy testnet (#8674)
Co-authored-by: Mark Holt <mark@distributed.vision>
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2023-11-30 08:19:52 +07:00
Alex Sharov
473e0414ef
estimate: at-lest 1 worker (#8859) 2023-11-29 16:29:15 +00:00
Alex Sharov
2e0f9b0411
"erigon snapshots diff": setup logger (#8834) 2023-11-29 16:27:58 +00:00
Alex Sharov
427f2637d2
mdbx: hard-limit of small db's dirty_space (#8850)
it didn't cause problems yet. but it seems a good idea in-general.
2023-11-29 15:09:55 +01:00
kaliubuntu0206
e91dc9f1bf
Update mingw cmake version for windows (#8856)
From comment
https://github.com/ledgerwatch/erigon/issues/8623#issuecomment-1831473288
2023-11-29 19:10:49 +07:00
a
71d7e2c957
fix (#8852)
- dont return nil for config :)
2023-11-29 12:18:38 +07:00
a
a2673c62c5
[caplin/sentinel] config reorganizations (#8844) 2023-11-28 22:45:58 -06:00
Alex Sharov
3e8a028cbb
evm: remove interpreter interface - step3 (#8842) 2023-11-29 09:29:16 +07:00
Dmytro
a63b89334b
added logging for slow RPC requests (#8818)
Changed distribution of httpcfg.HttpCfg to be pointer.
Added new flags:
rpc.slow.log - which is false by default, this flag need to enable
logging slow RPC requests
rpc.slow.log.threshold - which is 100 by default, this flag specify slow
threshold in milliseconds
Updated rpc handler to log slow requests:
- added map[request id] {method, timestamp}
- put every request details to map above
- delete request details from map above
- added time interval check for elements in map and if time difference
is more than given threshold print request id and the method
- app will print slow requests in next cases:
1. As soon as request take more than given threshold
2. Every 20 seconds if request still in process
3. After request finished and it took more than give threshold

---------

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2023-11-28 16:11:39 +07:00
Alex Sharov
8cfafa446a
add command "downloader torrent_cat" (#8824) 2023-11-27 08:49:19 +07:00
Alex Sharov
fd6f529c73
devnet: prevent tx-leak on panic (#8803) 2023-11-25 09:12:55 +07:00
Alex Sharov
9cfed88082
integration run_migrations: open in non-accede exclusive mode - to create new tables (#8821)
sometimes need tool to apply migrations - without starting erigon (if
erigon is buggy, or to prevent snapshots downloading, etc...)
2023-11-25 09:12:44 +07:00
milen
9b74cf0384
metrics: use prometheus histogram and summary interfaces (#8808) 2023-11-24 17:50:57 +00:00
milen
230b013096
metrics: separate usage of prometheus counter and gauge interfaces (#8793) 2023-11-24 16:15:12 +01:00
Alex Sharov
748359cf72
webseed: .torrent file validation must check - fileName and hash (#8820)
because files with different name can have same hash: BitTorrent is
content-addressable.
2023-11-24 18:46:17 +07:00
Alex Sharov
3db9467c94
increase peer tasks queue size (#8825)
Current value: 16 was added by me 1 year ago and didn't mean anything.
Never seen this field holding much data, probably can increase.

Currently I see logs like (and 10x like this): 
[DBUG] [11-24|06:59:38.353] slow peer or too many requests, dropping its old requests name=erigon/v2.54.0-aeec5...
2023-11-24 12:42:08 +01:00
Pratik Patil
59909a7efe
Added TxDependency Metadata to ExtraData in Block Header in Bor for Block-STM (#8037)
This PR adds support to store the transaction dependency (generated by
the block producer) in the block header for bor. This transaction
dependency will then be used by the parallel processor
([Block-STM](https://github.com/ledgerwatch/erigon/pull/7812/)).

I have created another
[PR](https://github.com/ledgerwatch/erigon-lib/pull/1064) in the
erigon-lib repo which adds the `IsParallelUniverse()` function.
2023-11-24 10:26:33 +00:00
milen
28fff1b35e
metrics: remove VictoriaMetrics/metrics lib (#8766)
# Background

Erigon currently uses a combination of Victoria Metrics and Prometheus
client for providing metrics.

We want to rationalize this and use only the Prometheus client library,
but we want to maintain the simplified Victoria Metrics methods for
constructing metrics.

This task is currently partly complete and needs to be finished to a
stage where we can remove the Victoria Metrics module from the Erigon
code base.

# Summary of changes

- Remove `UsePrometheusClient` boolean flag
- Remove `VictoriaMetrics` client lib and related code (simplifies
registry and prometheus http handler initialisation since now we have
only 1 registry and can use default `promhttp.Handler`)
2023-11-23 20:31:38 +00:00
Alex Sharov
fdc75df6b5
Bor: increase client timeout from 5 to 10sec (to cover remote server case) (#8801)
I using `https://heimdall-api-testnet.polygon.technology/` and seems
5sec timeout is not enough sometime - even that remote service working
well (node syncing well)

most of timeouts comes from same endpoint: 
```
 [bor.heimdall] request canceled          reason="context deadline exceeded" path=/milestone/lastNoAck attempt=2
```
2023-11-23 16:32:30 +00:00
Alex Sharov
34b9a70b02
bor: add more context to error - to understand where it happened (#8811) 2023-11-23 16:31:45 +00:00
ledgerwatch
19451ac610
Return difficulty check into bor header validation (#8815) 2023-11-23 16:30:58 +00:00
Alex Sharov
23f23bc971
disable disc tests on Mac (#8822)
TestUDPv4_smallNetConvergence tests are often timeout on mac - disabling
this tests on mac CI
2023-11-23 16:00:42 +07:00
Alex Sharov
55e05c440f
--txpool.gossip.disable (#8800)
Co-authored-by: cby3149 <cby3149@gmail.com>
2023-11-23 12:00:41 +07:00
Alex Sharov
e390b0e182
ots: stop at ctx.Done() (#8790) 2023-11-23 08:49:33 +07:00
Giulio rebuffo
57bcbaa21f
Adds flags to enable/disable backfilling and enable full historical beacon node (#8813)
* Correct naming of hash func in Eth2
* Customizable mode of operation for Caplin
2023-11-22 13:24:35 +01:00
Alex Sharov
30c05cc7a9
cl: disable spec-integration CI on mac - until flaky test fix (#8812)
See Issue #8789
2023-11-22 12:13:30 +01:00
Alex Sharov
43b8cbbdeb
bor: don't hide ctx.Err() (#8792)
log `ctx.Err()` - it can be canceled by many reasons: timeout, etc...
2023-11-22 09:27:50 +07:00
Giulio rebuffo
a2433455f9
Keep few beacon block headers in mdbx (#8809)
Now keep few beacon block headers in mdbx
2023-11-22 01:45:15 +01:00
battlmonstr
4abe1d59d8
silkworm: internal refactorings (#8675) 2023-11-21 15:22:56 +01:00
Dario Gagliano
0aa06af226
Add benchOtsGetBlockTransactions (#8672) 2023-11-21 15:53:10 +07:00
milen
aa55d9534a
temp fix for erigon-ext-test (#8797)
Fixes
https://github.com/ledgerwatch/erigon/actions/runs/6930037092/job/18850944152
2023-11-21 09:31:03 +07:00
battlmonstr
9623b7cefe
txpool: limit transactions outgoing messages (#8271) (#8742)
* limit remote transactions re-broadcast to 3-6 peers  
  Broadcasting to 100 peers generates too much outgoing traffic.
* limit transactions count/size in PooledTransactions replies

Before it was sending 5.5-6.5 MiB/sec:

![Screenshot 2023-11-17 at 15 50
15](https://github.com/ledgerwatch/erigon/assets/11477595/bd2f51c5-190b-4f3e-aabf-4ff42ab8972d)


With the fixes it stays at 3-3.5 MiB/sec:

![Screenshot 2023-11-17 at 15 39
10](https://github.com/ledgerwatch/erigon/assets/11477595/74b18037-6017-49f1-8c00-9d7f3d1818b3)

P.S. A baseline if everything is disabled (BroadcastPooledTxs,
AnnouncePooledTxs, responses to GetPooledTransactions) is 0.5-1 MiB/sec.
2023-11-20 21:19:20 +07:00
milen
34c0fe29ad
metrics: swap remaining VictoriaMetrics usages with erigon-lib/metrics (#8762)
# Background

Erigon currently uses a combination of Victoria Metrics and Prometheus
client for providing metrics.

We want to rationalize this and use only the Prometheus client library,
but we want to maintain the simplified Victoria Metrics methods for
constructing metrics.

This task is currently partly complete and needs to be finished to a
stage where we can remove the Victoria Metrics module from the Erigon
code base.

# Summary of changes

- Adds missing `NewCounter`, `NewSummary`, `NewHistogram`,
`GetOrCreateHistogram` functions to `erigon-lib/metrics` similar to the
interface VictoriaMetrics lib provides
- Minor tidy up for consistency inside `erigon-lib/metrics/set.go`
around return types (panic vs err consistency for funcs inside the
file), error messages, comments
- Replace all remaining usages of `github.com/VictoriaMetrics/metrics`
with `github.com/ledgerwatch/erigon-lib/metrics` - seamless (only import
changes) since interfaces match
2023-11-20 12:23:23 +00:00
Sixtysixter
0a374ab615
Review debug_trace* benches (#8773)
Review for following benches for the sake of clarity:

- debug_traceBlochByNumber
- debug_traceBlochByHsh
- debug_traceTransaction
- debug_traceCall 


Bench name `benchTraceBlockByHash` has been moved to
`benchDebugTraceBlockByHash`
Bench name `benchTraceTransaction` has been moved to
`benchDebugTraceTransaction` (to avoid confusion with future bench for
trace_transaction APIs)
2023-11-20 16:27:59 +07:00
Alex Sharov
daacd71116
bor: PersistValidatorSets nil header check (#8791) 2023-11-20 16:27:38 +07:00
Alex Sharov
f476fe690f
bor: check nil-blocks in other places (#8788) 2023-11-20 12:46:32 +07:00
Alex Sharov
d557679a0b
bor: check nil-header (#8779) 2023-11-20 12:27:42 +07:00
Alex Sharov
2f17848b76
bor: logs prefix, grep-friendly (#8787) 2023-11-20 12:16:06 +07:00
Alex Sharov
06c508c02c
downloader: don't create .torrent for too small files (#8785) 2023-11-20 11:14:05 +07:00
Alex Sharov
33f42df10e
snaps: mumbai 42.5m (#8784) 2023-11-20 11:13:57 +07:00
Giulio rebuffo
073dbe25ac
Added duties/proposer to beacon api (#8777) 2023-11-19 22:03:14 +01:00
Giulio rebuffo
7d28151a11
Added data model for public keys validator (#8774)
We store public keys contiguosly and batch update them on forkchoice
2023-11-18 19:03:56 +01:00
Giulio rebuffo
b72b5b95c5
working debug/state/ for early states (#8767)
Added jsonification to beacon state
2023-11-18 03:08:19 +01:00
Mark Holt
de60e03f03
Bor proof findpath (#8764)
Added trie findpath to support bor receipt format.

This requires testing against the polygon deployed contracts to confirm
parent path format
2023-11-17 16:39:59 +00:00
Andrew Ashikhmin
d4cd712da0
Revisit getPayloadBodiesByHash (#8758)
Revisit PR #8750
2023-11-17 13:04:02 +01:00