Commit Graph

80 Commits

Author SHA1 Message Date
Håvard Anda Estensen
4873502818
turbo: run tests in parallel (#8738)
Tests that don't affect each other should run in parallel
2023-11-16 16:29:31 +07:00
Anshal Shukla
076dc33232
move borfinality package out of eth (#8407)
- Move borfinality out of eth package
- Adds nil pointer check in bor_verifier
2023-10-09 19:13:31 +01:00
Anshal Shukla
c6f532d68e
Rewind fixes (#8380) 2023-10-05 10:12:47 +01:00
Anshal Shukla
3c8cbda809
fix finalized api for polygon chain (#8247)
Co-authored-by: Ubuntu <ubuntu@ip-10-0-0-234.eu-west-1.compute.internal>
2023-09-20 13:32:22 +01:00
Park Changwan
ef84972e7c
Add addPeer RPC (#7804)
This PR mirrors https://github.com/testinprod-io/op-erigon/pull/54.

Actual implementation for `admin_addPeer` method.
RPC Spec: Refer to
https://geth.ethereum.org/docs/interacting-with-geth/rpc/ns-admin.

> The addPeer administrative method requests adding a new remote node to
the list of tracked static nodes. The node will try to maintain
connectivity to these nodes at all times, reconnecting every once in a
while if the remote connection goes down.

Requires https://github.com/ledgerwatch/erigon-lib/pull/1033/

After https://github.com/ledgerwatch/erigon-lib/pull/1033 is merged,
will update erigon-lib version, removing replace at go.mod.

Note that even if RPC response returns `true`, it does not guarantee
that RLPx protocol is established between peers. It just adds node
entrypoint to its static peer list, and periodically tries and tests
connections.

## Testing

This RPC needs integration testing, so I made some scenario.

Use below command for testing:

Spin up two dev nodes which p2p enabled:

Start Node 1: RPC running at port 8545:
```sh
./build/bin/erigon --datadir=dev --chain=dev --port=30303 --http.port=8545 --authrpc.port=8551 --torrent.port=42069  --no-downloader --nodiscover --private.api.addr=127.0.0.1:9090 --http --ws --http.api=admin --p2p.allowed-ports=30306,30307,30308  --authrpc.jwtsecret=/tmp/jwt1 --p2p.protocol=67,68  --log.console.verbosity=5
```

Start Node 2: RPC running at port 8546:
```sh
./build/bin/erigon --datadir=dev2 --chain=dev --port=30304 --http.port=8546 --authrpc.port=8552 --torrent.port=42068 --no-downloader --nodiscover --private.api.addr=127.0.0.1:9091 --http --ws --http.api=admin --p2p.allowed-ports=30309,30310,30311  --authrpc.jwtsecret=/tmp/jwt2  --p2p.protocol=67,68  --log.console.verbosity=5
```

Get nodeInfo of node 1 using `admin_nodeInfo` RPC:
```sh
curl --location 'localhost:8545/' \
--header 'Content-Type: application/json' \
--data '{
	"jsonrpc":"2.0",
	"method":"admin_nodeInfo",
	"params":[],
	"id":1
}' 
```
Example response:
```
{
    "jsonrpc": "2.0",
    "id": 1,
    "result": {
        "id": "b75e0c4d2113b6f144ea8fd356a8f90e612a2a5f48a13c78d7e0e176e5724eb2",
        "name": "erigon/v2.47.0-dev-5d86cdb5/darwin-arm64/go1.19.6",
        "enode": "enode://05ab575d947f2d73065ea0f795dc2d96ed0ad603f3e730ab90dc881122d552c9f59ffcb148fe50546bec8b319daeb3c22ec02e7d12a7c4f2ac4cd26456a04a7c@127.0.0.1:30303?discport=0",
   ...
```

Get nodeInfo of node 2 using `admin_nodeInfo` RPC:
```sh
curl --location 'localhost:8546/' \
--header 'Content-Type: application/json' \
--data '{
	"jsonrpc":"2.0",
	"method":"admin_nodeInfo",
	"params":[],
	"id":2
}' 
```
Example response:
```
{
    "jsonrpc": "2.0",
    "id": 2,
    "result": {
        "id": "32d721e4d75219b021d7f83235f1f1eb8b705d6f85e634bccde564b8f7f94d78",
        "name": "erigon/v2.47.0-dev-5d86cdb5/darwin-arm64/go1.19.6",
        "enode": "enode://1abb8579647779e13b7f68d18f9c776cbd29281841c7f950e9cf9afa996e31120a6f481cea8e90e0f42a0eb1aa00aeafee81c4bae6c31aa16810b795c6d6e069@127.0.0.1:30304?discport=0",
   ...
```

Call `admin_addPeer` RPC to node 2:
```sh
curl --location 'localhost:8546/' \
--header 'Content-Type: application/json' \
--data '{
	"jsonrpc":"2.0",
	"method":"admin_addPeer",
	"params":["enode://05ab575d947f2d73065ea0f795dc2d96ed0ad603f3e730ab90dc881122d552c9f59ffcb148fe50546bec8b319daeb3c22ec02e7d12a7c4f2ac4cd26456a04a7c@127.0.0.1:30303"],
	"id":2
}' 
```
Example response:
```
{
    "jsonrpc": "2.0",
    "id": 2,
    "result": true
}
```


Check peer info of node 1 using `admin_peers` RPC:
```sh
curl --location 'localhost:8545/' \
--header 'Content-Type: application/json' \
--data '{
	"jsonrpc":"2.0",
	"method":"admin_peers",
	"params":[],
	"id":1
}' 
```
Example response:
```
{
    "jsonrpc": "2.0",
    "id": 1,
    "result": [
        {
            "enode": "enode://1abb8579647779e13b7f68d18f9c776cbd29281841c7f950e9cf9afa996e31120a6f481cea8e90e0f42a0eb1aa00aeafee81c4bae6c31aa16810b795c6d6e069@127.0.0.1:55426",
            "id": "32d721e4d75219b021d7f83235f1f1eb8b705d6f85e634bccde564b8f7f94d78",
            "name": "erigon/v2.47.0-dev-5d86cdb5/darwin-arm64/go1.19.6",
            "caps": [
                "eth/66",
                "eth/67"
            ],
            "network": {
                "localAddress": "127.0.0.1:30303",
                "remoteAddress": "127.0.0.1:55426",
                "inbound": true,
                "trusted": false,
                "static": false
            },
            "protocols": null
        }
    ]
}
```

---------

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2023-09-06 15:31:02 +07:00
Giulio rebuffo
bd63cb7c8c
Removed GRPC layer from Engine API (#7878) 2023-07-12 18:11:41 +02:00
Giulio rebuffo
6272559fb7
Separated PendingBlock behaviour to be chain agnostic (#7859)
Instead of getting the pending block with latest payload id, we just
store the latest block built and serve it in remote backend
2023-07-10 19:22:03 +02:00
Giulio rebuffo
84f31c873e
Separation of Engine from Ethbackend (#7821)
This PR separates ENGINE from Ethbackend. It makes it so:

1) EthBackend not a god class
2) We can abstract away engine API so that we can make it CL-like and
enable Consensus-Execution driven design
3) Objective is Json-RPC -> Engine Consensus Module -> Execution module.
2023-07-06 18:09:52 +02:00
Alex Sharov
f8cb4d6dbb
Fix subscription closing race (#7788)
`go test -p 1 -race -count=100 -run='TestFiltersDeadlock_Test'
./turbo/rpchelper/...`
2023-06-23 10:07:51 +07:00
Alex Sharov
1b14785d79
ws unfreeze (#7724)
attempt to address next issue: 

> when I'm having a lot of websocket connections the node is freezing
and then it needs like 10 mins to sync. Then if I keep pushing requests
it falls out of sync all the time
2023-06-14 16:25:05 +07:00
Alex Sharov
5ff8246e2a
e4: release some tests (#7729) 2023-06-14 10:01:00 +07:00
Alex Sharov
afa4f53ae2
e3: use deserializeV3 for history (#7691) 2023-06-09 10:53:04 +07:00
Andrew Ashikhmin
b2a90e0ea9
Merge EngineGetPayloadWithBlobs into EngineGetPayload (#7637)
Pick up https://github.com/ledgerwatch/interfaces/pull/171 &
https://github.com/ledgerwatch/interfaces/pull/173
2023-06-02 13:02:13 +02:00
ledgerwatch
05597cb195
[devnet tool] separare logging (#7531)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-17 17:36:15 +01:00
racytech
42e8db3958
eip-4844: modified DecodeTransaction version 1 (#7442)
Blob transactions are SSZ encoded, so it had to be added to decoding.
There are 2 encoding forms: `network` and `minimal` (usual). Network
encoded blob transactions include "wrapper data" which are `kzgs`,
`blobs` and `proofs`, and decoded by `DecodeWrappedTransaction`. For
previous types of transactions the network encoding is no different.
Execution-payloads / blocks use the minimal encoding of transactions. In
the transaction-pool and local transaction-journal the network encoding
is used.

Concerns: 
1. Possible performance reduction caused by these changes, not sure if
streams are better then slices. Go slices in this modifications are
read-only, so they should be referred to the same underlying array and
passed by a reference.
2. If `DecodeWrappedTransaction` and `DecodeTransaction` will create
confusion and should be merged into one function.
2023-05-09 18:44:53 +01:00
racytech
6588bca40b
eip-4844: NewMessage now expectes maxFeePerDataGas & GetPayloadV3 impl (#7365)
types.NewMessage now expects maxFeePerDataGas param, which will be used
in transaction verification (preCheck). GetPayloadV3 method added to
EngineAPI. Some cosmetic changes applied.
2023-04-23 18:27:05 +01:00
Alex Sharov
201572c6f5
enable more linters #954 (#7179) 2023-03-25 05:13:27 +00:00
a
966be04e6d
add eth_getFilterLogs, fix filter subscription ids (#6514)
this pr does the following:

1. adds new function to ApiImpl `GetFilterLogs` which should implement
`eth_getFilterLogs` (eth_getFilterChanges except with only logs filters)

2. changes the ID generator of rpchelper.Filters to use crypto/rand.

3. switched logs subscriptions to use the secure ID instead of number

4. changes subcription ids from an 8 byte hex string to a 16 byte
2023-02-06 10:18:10 +07:00
Alex Sharov
13d6415de8
e3: ots system txn handle 2023-01-27 16:05:09 +07:00
Alex Sharov
82c478a419
e3: getLogs on iterators (#6683) 2023-01-25 16:29:41 +07:00
hexoscott
4dcba50e99
engine payload bodies rpc endpoints (#6644)
Very basic implementation for get payload bodies rpc calls. Once we have
Hive tests for these calls I can pick this back up and work through any
issues.

Implementation of https://github.com/ethereum/execution-apis/pull/352.
2023-01-24 13:37:03 +01:00
Alex Sharov
539eaf0a51
e3: prepare trace_transaction env without execution (#6675) 2023-01-24 11:18:32 +07:00
Alex Sharov
4fbbdf9186
e3: move txnum to erigon-lib (#6663) 2023-01-22 19:39:33 +07:00
Alex Sharov
0ee8c175d9
e3: simplify history reader (#6659) 2023-01-22 16:42:24 +07:00
Andrew Ashikhmin
9927ca3171
Unify Engine API V1 & V2 methods (#6638)
Reduce code duplication.

Prerequisite: https://github.com/ledgerwatch/erigon-lib/pull/841.
2023-01-20 18:43:08 +01:00
hexoscott
7dcbfbc283
reference hash, address, and chain config from lib (#6536) 2023-01-13 18:12:18 +00:00
a
06de4aeb91
filterlogs deadlock fix (#6429)
Refactors filters.go such that map+locks are now in their own class 

Move logic for safely canceling & draining channel to its own class

changed subscriptions to ask for size of buffer and construct its own
channel

marked as draft b/c need to do live testing.
2023-01-02 11:42:40 +07:00
Alex Sharov
4a9c871628
a bit rename 22 to v3 (#6476) 2022-12-30 21:53:42 +07:00
alex.sharov
9efb8e838b save 2022-12-29 10:09:09 +07:00
Alex Sharov
ade933be6b
kv_remote: server to support thread-safe multi-streams per 1 tx (#6402) 2022-12-24 13:11:15 +07:00
Alex Sharov
130ab85bea
e3: kv/temporal prototype 3 (#6395) 2022-12-22 09:37:32 +07:00
Alex Sharov
65f3af30eb
use one state reader type (#6380) 2022-12-21 11:27:14 +07:00
ledgerwatch
0a31f5ac2a
Workaround for the code history of BSC system contracts (#6274)
Works around a flaw in the upgrade logic of the system contracts. Since
they are updated directly, without first being self-destructed and then
re-created, the usual incarnation logic does not get activated, and all
historical records of the code of these contracts are retrieved as the
most recent version. This problem will not exist in erigon3, but until
then, a workaround will be used to access code of such contracts through
a special structure, `SystemContractCodeLookup`

Fixes https://github.com/ledgerwatch/erigon/issues/5865

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-12-10 22:41:04 +00:00
Håvard Anda Estensen
1398703bc5
Use hex package to convert bytes to string (#6205) 2022-12-05 09:06:16 +07:00
Andrew Ashikhmin
d82c778ab3
Withdrawals part 1 (#6009)
This PR partially implements
[EIP-4895](https://eips.ethereum.org/EIPS/eip-4895): Beacon chain push
withdrawals as operations. The new Engine API methods
(https://github.com/ethereum/execution-apis/pull/195) are implemented.

_Body downloader and saving withdrawals into DB are not implemented
yet!_
2022-12-01 09:15:01 +01:00
Max Revitt
74308f2500
fix(test): eliminate race conditions (#6019) 2022-11-11 15:22:09 +00:00
hexoscott
12ed9d22cf
eth_estimateGas performance tweaks (#6001)
Will gain more perf improvements when the change for kvcache goes in
from erigon-lib.

Hopefully can be validated with hive? My main concern is the re-use of
the state reader, I couldn't find any manipulation of it so it looks
safe to re-use.
2022-11-08 21:48:11 +00:00
ledgerwatch
004c9005cb
Prevent double closing of the channel (#5962)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-11-05 08:19:48 +00:00
net.wyman
7548755b62
fix potential deadlock in eth_filters logs event (#5912)
Like the newPendingTransaction event, the logs event also has the
potential for deadlock when unsubscribed
2022-11-01 08:59:01 +07:00
Alex Sharov
8783521ba4
lint (#5920) 2022-11-01 08:57:05 +07:00
net.wyman
98420edd30
Update README.md (#5914)
the rpc_daemon README.md is out of date and miss the eth_subscribe logs
event
2022-11-01 08:56:22 +07:00
Alex Sharov
381bea5b92
e3: eth_traceTransaction implementation (#5909) 2022-10-31 12:31:38 +07:00
net.wyman
8981f22943
fix deadlock in filters.go (#5734)
When the filter is in the onNewTxs function, the subscription function
exits and no longer receives information from the msg channel. In this
case, the unsubscribe_xxx function is triggered, which will cause
filter.mu read and write locks to enter a deadlock state

Co-authored-by: dc <dctrlbox@gmail.com>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-10-13 22:14:09 +01:00
Alex Sharov
cb60382e6d
erigon3: rename "history.v2" to "history.v3" to avoid naming miss-match with "erigon3" (#5519) 2022-09-26 10:54:42 +07:00
Alex Sharov
9fb8a190bc
erigon22: folder snapshots/history (#5351) 2022-09-18 17:41:01 +07:00
hexoscott
cd8cad6a89
allow multiple log subscriptions at the same time (#5358) 2022-09-16 14:36:25 +01:00
Enrique Jose Avila Asapche
0c64c3f2c7
added a way to get latest executed block post POS (#5343)
* added a way to get latest executed block post POS

* added erigon_ExecutedBlockNumber into readme

* optional rpc.BlockNumber

* better message

* updated readme
2022-09-14 13:56:31 +03:00
Alex Sharov
25e615a878
erigon22: historyReader22 and more tests (#5281)
* save

* save
2022-09-05 21:31:00 +07:00
dmitriyselivanov
4499e04a05
rpcdaemon: wire the most recent of EthBackend.builders.block as a pending block into eth_getBlockByNumber (#5061) 2022-08-15 15:19:45 +07:00
hexoscott
a573f8356f
rpc: concurrent map issue with log subscription fix (#4971)
* rpc: concurrent map issue with log subscription fix

* rpc: subscription changes around locking
2022-08-10 16:01:53 +07:00