Commit Graph

1087 Commits

Author SHA1 Message Date
Alex Sharov
1617f1802b
Sepolia more sn (#8422) 2023-10-10 11:19:10 +07:00
Alex Sharov
3ec23e083c
gnark lib up (#8420) 2023-10-10 10:42:06 +07:00
Alex Sharov
2a24ea6f62
exp lib up (#8359) 2023-10-04 12:55:48 +07:00
Alex Sharov
0165d00ee6
downloader: don't drop torrents after download (performance problem there solved) (#8358) 2023-10-04 12:34:45 +07:00
Andrew Ashikhmin
3f590505bd
Bump gnark-crypto to v0.12.0 (#8355) 2023-10-03 17:40:03 +02:00
Mark Holt
0bdca6c457
Metrics label fixes (#8339)
Fixes for label discrepancies in collector for summaries etc which have
a template which includes a quantile.

Initial native Prometheus client implementation of metrics - which is
currently turned off except for local testing and interface exports.
2023-10-02 17:19:02 +01:00
Jason Yellick
5654ba07c9
Upgrade libp2p (enables go 1.21 support) (#8288)
Closes #8078 

This change is primarily intended to support go 1.21, but as a
side-effect requires updating libp2p, which in turn triggers an update
of golang.org/x/exp which creates quite a bit of (simple) churn in the
slice sorting.

This change introduces a new `cmp.Compare` function which can be used to
return an integer satisfying the compare interface for slice sorting.

In order to continue to support mplex for libp2p, the change references
github.com/libp2p/go-libp2p-mplex instead. Please see the PR at
https://github.com/libp2p/go-libp2p/pull/2498 for the official usptream
comment that indicates official support for mplex being moved to this
location.

Co-authored-by: Jason Yellick <jason@enya.ai>
2023-09-29 22:11:13 +02:00
Alex Sharov
d0366ae34e
snapshots: remove concept of separated hist .toml file (#8326) 2023-09-29 11:30:02 +07:00
Alex Sharov
e94f4d8199
mdbx: less warn (#8325) 2023-09-29 10:40:32 +07:00
Alex Sharov
b1c7824ecc
mdbx: less compile warnings (#8300) 2023-09-28 08:56:24 +07:00
Alex Sharov
71082b06ba
erigon-lib v1.0.0 release (#8277) 2023-09-23 10:28:08 +07:00
Alex Sharov
69941e0a9b
mdbx_go: use right version in devel (#8250) 2023-09-21 04:58:15 +07:00
battlmonstr
5fb5048511 refer to erigon-lib subdirectory module 2023-09-20 14:53:03 +02:00
Somnath Banerjee
f51d9b61a0
Txpool 4844 upgrades Part 2 (#8213)
Some peer-review changes from the last related PR. 
Addition of a flag for BlobSlots - for max allowed blobs per account in
txpool.
Use BlobFee from the block to validate txs in the pool.

See also https://github.com/ledgerwatch/erigon-lib/pull/1125
2023-09-20 17:29:30 +05:30
Alex Sharov
751de6a028
txpool: batch IdHashKnown calls - to reduce amount of mutex lock/unlock. less string(byte) casts (#8226) 2023-09-18 10:25:53 +07:00
Alex Sharov
eb747c042a
Downloader: add tests to protect from re-downloading (#8191) 2023-09-15 13:46:59 +07:00
Mark Holt
33d8c08c1c
Get vote on hash (#8172)
This is the initial merge for polygon milestones it implements an rpc
call used by heimdall but does not directly impact any chain processing
2023-09-13 11:49:49 +01:00
Alex Sharov
93fbda14a5
downloader: support --chain parameter (#8184) 2023-09-13 15:21:43 +07:00
Alex Sharov
2158b4916d
Downloader: correct logging when create .torrent files (#8182) 2023-09-13 11:49:45 +07:00
Alex Sharov
3cea1b9b9e
torrent: add --webseeds cli arg (#8176) 2023-09-12 12:18:47 +07:00
alex.sharov
babf8ef89b save 2023-09-12 09:09:14 +07:00
alex.sharov
787e3a0f05 merge stable to devel 2023-09-11 09:40:34 +07:00
Somnath Banerjee
a699f64761
Txpool upgrades for EIP-4844 Blob Transactions (#8004)
See https://github.com/ledgerwatch/erigon-lib/pull/1075
2023-09-11 09:38:58 +07:00
Alex Sharov
412b0e9ec5
torrent lib: fix requests orering bug (#8173) 2023-09-11 09:20:34 +07:00
alex.sharov
6c1cf0a9b0 save 2023-09-10 15:49:13 +07:00
Alex Sharov
2318138c6c
Downloader: don't fail when see unusual file, skip it (backward/forward compatibility) (#8170) 2023-09-10 15:46:30 +07:00
alex.sharov
b72dd9bcbd save 2023-09-09 17:25:42 +07:00
Alex Sharov
d3ab41034d
more sepolia snapshots (#8165) 2023-09-09 17:22:44 +07:00
Giulio rebuffo
b92f6d9814
Finished jsonification of solid package (#8163) 2023-09-08 23:02:34 +02:00
alex.sharov
97255efdb4 merge stable 2023-09-08 19:26:14 +07:00
Alex Sharov
5d3e294f3c
remove small snapshots from polygons .toml (#8158) 2023-09-08 19:09:31 +07:00
Alex Sharov
93f8a0602d
grpc v1.58 (#8157) 2023-09-08 12:32:54 +07:00
Giulio rebuffo
22bd173ab3
Updating Libs (#8147)
* BLS
* Sqlite
2023-09-06 23:20:26 +02:00
Somnath Banerjee
86cec41426
Update lib (#8146) 2023-09-06 22:29:19 +05:30
Park Changwan
ef84972e7c
Add addPeer RPC (#7804)
This PR mirrors https://github.com/testinprod-io/op-erigon/pull/54.

Actual implementation for `admin_addPeer` method.
RPC Spec: Refer to
https://geth.ethereum.org/docs/interacting-with-geth/rpc/ns-admin.

> The addPeer administrative method requests adding a new remote node to
the list of tracked static nodes. The node will try to maintain
connectivity to these nodes at all times, reconnecting every once in a
while if the remote connection goes down.

Requires https://github.com/ledgerwatch/erigon-lib/pull/1033/

After https://github.com/ledgerwatch/erigon-lib/pull/1033 is merged,
will update erigon-lib version, removing replace at go.mod.

Note that even if RPC response returns `true`, it does not guarantee
that RLPx protocol is established between peers. It just adds node
entrypoint to its static peer list, and periodically tries and tests
connections.

## Testing

This RPC needs integration testing, so I made some scenario.

Use below command for testing:

Spin up two dev nodes which p2p enabled:

Start Node 1: RPC running at port 8545:
```sh
./build/bin/erigon --datadir=dev --chain=dev --port=30303 --http.port=8545 --authrpc.port=8551 --torrent.port=42069  --no-downloader --nodiscover --private.api.addr=127.0.0.1:9090 --http --ws --http.api=admin --p2p.allowed-ports=30306,30307,30308  --authrpc.jwtsecret=/tmp/jwt1 --p2p.protocol=67,68  --log.console.verbosity=5
```

Start Node 2: RPC running at port 8546:
```sh
./build/bin/erigon --datadir=dev2 --chain=dev --port=30304 --http.port=8546 --authrpc.port=8552 --torrent.port=42068 --no-downloader --nodiscover --private.api.addr=127.0.0.1:9091 --http --ws --http.api=admin --p2p.allowed-ports=30309,30310,30311  --authrpc.jwtsecret=/tmp/jwt2  --p2p.protocol=67,68  --log.console.verbosity=5
```

Get nodeInfo of node 1 using `admin_nodeInfo` RPC:
```sh
curl --location 'localhost:8545/' \
--header 'Content-Type: application/json' \
--data '{
	"jsonrpc":"2.0",
	"method":"admin_nodeInfo",
	"params":[],
	"id":1
}' 
```
Example response:
```
{
    "jsonrpc": "2.0",
    "id": 1,
    "result": {
        "id": "b75e0c4d2113b6f144ea8fd356a8f90e612a2a5f48a13c78d7e0e176e5724eb2",
        "name": "erigon/v2.47.0-dev-5d86cdb5/darwin-arm64/go1.19.6",
        "enode": "enode://05ab575d947f2d73065ea0f795dc2d96ed0ad603f3e730ab90dc881122d552c9f59ffcb148fe50546bec8b319daeb3c22ec02e7d12a7c4f2ac4cd26456a04a7c@127.0.0.1:30303?discport=0",
   ...
```

Get nodeInfo of node 2 using `admin_nodeInfo` RPC:
```sh
curl --location 'localhost:8546/' \
--header 'Content-Type: application/json' \
--data '{
	"jsonrpc":"2.0",
	"method":"admin_nodeInfo",
	"params":[],
	"id":2
}' 
```
Example response:
```
{
    "jsonrpc": "2.0",
    "id": 2,
    "result": {
        "id": "32d721e4d75219b021d7f83235f1f1eb8b705d6f85e634bccde564b8f7f94d78",
        "name": "erigon/v2.47.0-dev-5d86cdb5/darwin-arm64/go1.19.6",
        "enode": "enode://1abb8579647779e13b7f68d18f9c776cbd29281841c7f950e9cf9afa996e31120a6f481cea8e90e0f42a0eb1aa00aeafee81c4bae6c31aa16810b795c6d6e069@127.0.0.1:30304?discport=0",
   ...
```

Call `admin_addPeer` RPC to node 2:
```sh
curl --location 'localhost:8546/' \
--header 'Content-Type: application/json' \
--data '{
	"jsonrpc":"2.0",
	"method":"admin_addPeer",
	"params":["enode://05ab575d947f2d73065ea0f795dc2d96ed0ad603f3e730ab90dc881122d552c9f59ffcb148fe50546bec8b319daeb3c22ec02e7d12a7c4f2ac4cd26456a04a7c@127.0.0.1:30303"],
	"id":2
}' 
```
Example response:
```
{
    "jsonrpc": "2.0",
    "id": 2,
    "result": true
}
```


Check peer info of node 1 using `admin_peers` RPC:
```sh
curl --location 'localhost:8545/' \
--header 'Content-Type: application/json' \
--data '{
	"jsonrpc":"2.0",
	"method":"admin_peers",
	"params":[],
	"id":1
}' 
```
Example response:
```
{
    "jsonrpc": "2.0",
    "id": 1,
    "result": [
        {
            "enode": "enode://1abb8579647779e13b7f68d18f9c776cbd29281841c7f950e9cf9afa996e31120a6f481cea8e90e0f42a0eb1aa00aeafee81c4bae6c31aa16810b795c6d6e069@127.0.0.1:55426",
            "id": "32d721e4d75219b021d7f83235f1f1eb8b705d6f85e634bccde564b8f7f94d78",
            "name": "erigon/v2.47.0-dev-5d86cdb5/darwin-arm64/go1.19.6",
            "caps": [
                "eth/66",
                "eth/67"
            ],
            "network": {
                "localAddress": "127.0.0.1:30303",
                "remoteAddress": "127.0.0.1:55426",
                "inbound": true,
                "trusted": false,
                "static": false
            },
            "protocols": null
        }
    ]
}
```

---------

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2023-09-06 15:31:02 +07:00
Alex Sharov
1c4819ffe3
compress.Next: return empty byte slice instead of nil when wordLen == 0 (#8143) 2023-09-06 15:03:16 +07:00
Alex Sharov
ff54df073e
etl: do sort and file flush in another goroutine (#7915) 2023-09-06 14:51:23 +07:00
Alex Sharov
4e1c86e5a2
deps up (after release) (#8142) 2023-09-06 14:43:48 +07:00
Alex Sharov
0b633cfa08
txpool: switch db to durable mode - with fsync outside of pool's global lock (#8141) 2023-09-06 14:30:36 +07:00
alex.sharov
70908def7e merge stable into devel 2023-09-06 10:26:13 +07:00
Alex Sharov
608341b636
shutdown deadlock (#8135) 2023-09-06 10:23:59 +07:00
Giulio rebuffo
841327e212
BeaconAPI: Implemented /Spec (#8131) 2023-09-05 22:37:18 +02:00
Alex Sharov
8983128d24
Restore genesis reads code - remote rpc case. Remove ctx parameter from baseApi func (tx already has internal ctx) (#8122) 2023-09-04 12:42:08 +07:00
Giulio rebuffo
78d05cb367
Fix devel lint and demote logs (#8114) 2023-09-01 21:49:57 +02:00
Mark Holt
e81852320c
Remove unused metrics and out of date versions (#8109)
ergon/metrics contains a lot of unused types - the pull is removing them
prior to rationalizing between Prometheus and Victoria Metrics types.

It also have upgrades of golang-lru + sets types to v2 when possible.
2023-09-01 07:13:13 +07:00
Alex Sharov
51b7b0ce1d
mdbx: m1 segfault (#8106) 2023-08-31 18:35:43 +07:00
Mark Holt
a4cfbe0d56
Heimdall metrics + Metrics HTTP server rationalization (#8094)
This is an update of:

https://github.com/ledgerwatch/erigon/pull/7846

which uses a local fork of victoria metrics to include the changes that
https://github.com/anshalshukla added to the original for we where
using.

It also includes code to address the duplicate metrics issue identified
here:

https://github.com/ledgerwatch/erigon/issues/8053

It has one more associated fix which is to correctly add a metadata
label to counters, these where previously labelled as gauges.

e.g. 

```
# TYPE p2p_peers counter
p2p_peers 0
```
rather than

```
# TYPE p2p_peers gauge
p2p_peers 0
```

---------

Co-authored-by: Anshal Shukla <53994948+anshalshukla@users.noreply.github.com>
Co-authored-by: Anshal Shukla <shukla.anshal85@gmail.com>
2023-08-31 09:04:27 +01:00
Mark Holt
f05a6ab43e
Bor mining benchmark (#8096)
Replacement for: https://github.com/ledgerwatch/erigon/pull/7998 with
windows fixes

---------

Co-authored-by: SHIVAM SHARMA <shivam691999@gmail.com>
2023-08-30 10:25:02 +01:00
Alex Sharov
bb16f29c04
Compress: graceful shutdown support (#8098) 2023-08-30 10:20:43 +07:00
Giulio rebuffo
f5cc67fe5b
Added ranges to the Execution.proto (#8081) 2023-08-26 02:25:48 +02:00