Commit Graph

926 Commits

Author SHA1 Message Date
Dmytro
a63b89334b
added logging for slow RPC requests ()
Changed distribution of httpcfg.HttpCfg to be pointer.
Added new flags:
rpc.slow.log - which is false by default, this flag need to enable
logging slow RPC requests
rpc.slow.log.threshold - which is 100 by default, this flag specify slow
threshold in milliseconds
Updated rpc handler to log slow requests:
- added map[request id] {method, timestamp}
- put every request details to map above
- delete request details from map above
- added time interval check for elements in map and if time difference
is more than given threshold print request id and the method
- app will print slow requests in next cases:
1. As soon as request take more than given threshold
2. Every 20 seconds if request still in process
3. After request finished and it took more than give threshold

---------

Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
2023-11-28 16:11:39 +07:00
milen
9b74cf0384
metrics: use prometheus histogram and summary interfaces () 2023-11-24 17:50:57 +00:00
Mark Holt
f3ce5f8a36
Bor proofgen tests ()
Added initial proof generation tests for polygon reverse flow for devnet

Blocks tested, receipts need trie proof clarification
2023-11-17 10:41:45 +00:00
Dmytro
a6b5297b3e
dvovk/tunnelwws ()
- changed communication tunnel to web socket in order to connect to
remote nodes
- changed diagnostics.url flag to diagnostics.addr as now user need to
enter only address and support command will connect to it through
websocket
- changed flag debug.urls to debug.addrs in order to have ability to
change connection type between erigon and support to websocket and don't
change user API
- added auto trying to connect to connect to ws if connection with was
failed
2023-11-16 16:37:29 +00:00
Alex Sharov
35bfffd621
sys deps up () 2023-11-11 15:04:18 +03:00
battlmonstr
3698e7f476
devnet: configuration fixes ()
* fix "genesis hash does not match" when dev nodes connect  
The "dev" nodes need to have the same --miner.etherbase in order to
generate the same genesis ExtraData by DeveloperGenesisBlock(). Override
DevnetEtherbase global var that's used if --miner.etherbase is not
passed. (for NonBlockProducer case)

* fix missing private key for the hardcoded DevnetEtherbase  
Fixes panic if SigKey is not found. Bor non-producers will use a default
`DevnetEtherbase` while Dev nodes modify it. Save hardcoded
DevnetEtherbase/DevnetSignPrivateKey into accounts so that SigKey can
recover it.

* refactor devnet.node to contain Node config  
This avoids interface{} type casts and fixes an error with
Heimdall.validatorSet == nil

* add connection retries to rpcCall and Subscribe of requestGenerator  
Fixes "connection refused" errors due to node not ready to handle early
RPC requests.

* fix deadlock in Heimdall.NodeStarted

* fix GetBlockByNumber
Fixes "cannot unmarshal string into Go struct field body.transactions of
type jsonrpc.RPCTransaction"

* demote "no of blocks on childchain is less than confirmations
required" to Info ()

* demote "mismatched pending subpool size" to Debug ()

* revert wiggle testing code
2023-11-01 11:08:47 +01:00
a
436493350e
Sentinel refactor ()
1. changes sentinel to use an http-like interface

2. moves hexutil, crypto/blake2b, metrics packages to erigon-lib
2023-10-22 01:17:18 +02:00
Mark Holt
0bdca6c457
Metrics label fixes ()
Fixes for label discrepancies in collector for summaries etc which have
a template which includes a quantile.

Initial native Prometheus client implementation of metrics - which is
currently turned off except for local testing and interface exports.
2023-10-02 17:19:02 +01:00
Mark Holt
f794438335
Diag session routing ()
Code to support react based UI for diagnostics:

* pprof, prometheus and diagnistics rationalized to use a single router
(i.e. they can all run in the same port)
* support_cmd updated to support node routing (was only first node)
* Multi content support in router tunnel (application/octet-stream &
appliaction/json)
* Routing requests changed from using http forms to rest + query params
* REST query requests can now be made against erigon base port and
diagnostics with the same url format/params

---------

Co-authored-by: dvovk <vovk.dimon@gmail.com>
Co-authored-by: Mark Holt <mark@disributed.vision>
2023-09-25 16:24:17 +01:00
Srdjan
dedf04caaa
Pre-allocate method metric labels ()
Closes 
2023-09-21 09:38:47 +02:00
Mark Holt
8ea0096d56
moved metrics sub packages types to metrics ()
This is a non functional change which consolidates the various packages
under metrics into the top level package now that the dead code is
removed.

It is a precursor to the removal of Victoria metrics after which all
erigon metrics code will be contained in this single package.
2023-09-03 08:09:27 +07:00
Mark Holt
e81852320c
Remove unused metrics and out of date versions ()
ergon/metrics contains a lot of unused types - the pull is removing them
prior to rationalizing between Prometheus and Victoria Metrics types.

It also have upgrades of golang-lru + sets types to v2 when possible.
2023-09-01 07:13:13 +07:00
Mark Holt
a4cfbe0d56
Heimdall metrics + Metrics HTTP server rationalization ()
This is an update of:

https://github.com/ledgerwatch/erigon/pull/7846

which uses a local fork of victoria metrics to include the changes that
https://github.com/anshalshukla added to the original for we where
using.

It also includes code to address the duplicate metrics issue identified
here:

https://github.com/ledgerwatch/erigon/issues/8053

It has one more associated fix which is to correctly add a metadata
label to counters, these where previously labelled as gauges.

e.g. 

```
# TYPE p2p_peers counter
p2p_peers 0
```
rather than

```
# TYPE p2p_peers gauge
p2p_peers 0
```

---------

Co-authored-by: Anshal Shukla <53994948+anshalshukla@users.noreply.github.com>
Co-authored-by: Anshal Shukla <shukla.anshal85@gmail.com>
2023-08-31 09:04:27 +01:00
Alex Sharov
ddf608f866
rpc: a bit less metrics allocations () 2023-08-26 11:20:08 +07:00
Mark Holt
c51573f333
Bor eth event flow ()
Implemented polygon->eth flow
2023-08-25 12:19:39 +01:00
a
db5b348673
Caplin block persistence ()
Co-authored-by: Giulio <giulio.rebuffo@gmail.com>
2023-08-09 01:21:19 +02:00
Andrew Ashikhmin
d69b20bc4e
Support engine_forkchoiceUpdatedV3 with ParentBeaconBlockRoot (EIP-4788) ()
Prerequisites: https://github.com/ledgerwatch/interfaces/pull/187 &
https://github.com/ledgerwatch/erigon-lib/pull/1069. Also implement
https://github.com/ethereum/execution-apis/pull/426.
2023-08-06 11:54:14 +02:00
Mark Holt
1e575ea172
rpc service and stageloop logger updates ()
This is another update to logging to replace the root logger with a
contextual logger
2023-06-10 07:39:39 +01:00
ledgerwatch
067f695fff
[devnet tool] Separate logging ()
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-20 14:48:16 +01:00
ledgerwatch
05597cb195
[devnet tool] separare logging ()
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-17 17:36:15 +01:00
Alex Sharov
201572c6f5
enable more linters () 2023-03-25 05:13:27 +00:00
hexoscott
165a6a950d
check for nil stream when running the null check in handler () 2023-03-15 04:03:16 +00:00
hexoscott
84ec0a0b4b
check for nil before returning invalid json in rpc streaming calls ()
should handle nil having already been written in any rpc call before
writing it again causing invalid json to be returned.
2023-03-14 11:07:53 +00:00
alex.sharov
7190d89af3 save 2023-03-10 16:01:55 +07:00
Andrew Ashikhmin
2fa448c45f
Validate params of GetPayloadBodiesByHash and ByRange ()
See https://github.com/ethereum/execution-apis/pull/366 and
https://github.com/ethereum/execution-apis/pull/370.

Also fix a couple of issues so that the Hive tests pass.
2023-02-06 18:38:45 +01:00
a
966be04e6d
add eth_getFilterLogs, fix filter subscription ids ()
this pr does the following:

1. adds new function to ApiImpl `GetFilterLogs` which should implement
`eth_getFilterLogs` (eth_getFilterChanges except with only logs filters)

2. changes the ID generator of rpchelper.Filters to use crypto/rand.

3. switched logs subscriptions to use the secure ID instead of number

4. changes subcription ids from an 8 byte hex string to a 16 byte
2023-02-06 10:18:10 +07:00
hexoscott
7dcbfbc283
reference hash, address, and chain config from lib () 2023-01-13 18:12:18 +00:00
ledgerwatch
389a2e8392
Eth call limits ()
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-01-07 21:16:43 +00:00
a
ad7be30679
jsonrpc HTTP Get handler & logs topic filter with map ()
this pr has two things in it

1. changed filter logs to use a map for the topics. this will speed up
queries with many topics in them. I still don't have a use case for this
though. i put is as a method of Logs, since that made sense to me, happy
to move it back out though.

2. allows json-rpc over http get request. since firefox is a great json
viewer (can search through, collapse large results) i often use it to
debug. it is also useful for sharing data with those who are less
familiar with command line / programming.
example get request:

http://rpcdaemon/?method=eth_getLogs&params=[{"fromBlock":"0xf2316b","toBlock":"0xf2316b"}]

it is based on the old jsonrpc http specification
https://www.jsonrpc.org/historical/json-rpc-over-http.html#encoded-parameters
except we also accept not base64 encoded params. since every eth rpc
request has a [], it will immediately fail validation for base64 and
attempt to use the parameters as a they are. otherwise it will attempt
to parse the rest of the payload as base64 and use that.

Co-authored-by: a <a@a.a>
Co-authored-by: gfx <86091021+gfxlabs@users.noreply.github.com>
2022-11-03 09:09:04 +07:00
dtan3847
42aa80e059
rpc: fix implemented filter methods being forbidden ()
Addresses issue 

Removes implemented methods from the forbidden method list. The
forbidden list seems to have been created as a bugfix before the filters
methods were implemented. Now that the filter methods are implemented,
these entries are causing unexpected behaviour.
2022-10-26 09:08:09 +07:00
Temirlan
44266a9237
rcpdaemon, erigon: add --rpc.evmtimeout flag () 2022-09-17 13:25:27 +07:00
ledgerwatch
566cd65664
Revert "cancel context before waiting for waitgroup to avoid deadlock in RPC on error ()" ()
This reverts commit 9ff5b21a1e.
2022-09-14 08:45:59 +01:00
ledgerwatch
9ff5b21a1e
cancel context before waiting for waitgroup to avoid deadlock in RPC on error ()
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-13 08:52:58 +07:00
Enrique Jose Avila Asapche
19a178bf19
missing return; () 2022-09-11 09:35:18 +07:00
Håvard Anda Estensen
ea6b92b132
Enable gosec linter ()
* Ignore pseudo random generators

* Enable gosec linter
2022-08-17 12:18:19 +07:00
Alex Sharov
52fd0d0e8b
Aggregator22.Unwind() ()
* save

* save
2022-08-13 18:51:25 +07:00
Alex Sharov
351cd49c21
go1.19 gofmt () 2022-08-10 19:04:13 +07:00
Enrique Jose Avila Asapche
c12d298f1c
new rpc block number ()
* new rpc block number

* log

* ops
2022-08-08 19:07:24 +07:00
Dan Kelly
cce64f6a33
Fix rpc json ()
* Fixing RPC JSON errors

* Use local rpc mod

* Revert "Use local rpc mod"

This reverts commit 61d06a72d218f5ecb84e9e4ddddf8762a04f714c.

* Make handle error accessible

* Port fixes to rpcdaemon22

* Remove unused...

* ... and use correct function name

* Handle errors outside iterator loop. Fix missing rpcdaemon22 error handling

* Bubble up higher level errors to the top-level object

* Fix linter error - semi-colon..
2022-08-08 09:15:58 +07:00
Alex Sharov
2681ee392a
rpc: fix error stream () 2022-07-29 10:01:13 +07:00
Andrew Ashikhmin
f927ac8037
Increase JWT issued-at window to 60s () 2022-07-27 09:16:50 +02:00
Alex Sharov
66758c7960
RetireBlocks: less arguments ()
* save

* save
2022-07-22 13:44:42 +07:00
Alex Sharov
d9cb87a149
RPC: Enable back json streaming for non-batch and non-websocket cases ()
* enable rpc streaming

* enable rpc streaming
2022-07-06 11:44:06 +01:00
Andrew Ashikhmin
8de7c5e41c
JSON parsing of safe & finalized () 2022-06-23 19:37:39 +02:00
ledgerwatch
64067a2b77
Debugging Engine API ()
* defer tx rollback

* Add http.trace flag

* Fixed http.trace flag

* Fix lint

* Fix lint

* Delete unused tests

* Fix lint

* Fix lint

* Fix lint

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-19 13:40:28 +01:00
Enrique Jose Avila Asapche
a5cb53d690
safe and finalized blocks from eth_getBlockByNumber ()
* added getFinalzed and getSafe block num

* added rpc finalized and safe block num

* getting nums

* returning nil

* returning nil

* added to helper.go

* removed repeated code

* added functions into rpchelper

* returning err

* simplified

* using previous latest getter

* getting pending block with filter/

* Fix plain state block number

* Fix test

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-14 11:07:46 +03:00
Alex Sharov
955c669d21
Group few sync configs ()
* save

* save
2022-05-30 17:08:49 +07:00
primal_concrete_sledge
5c055b9697
Fix/block new filters ()
* Add filters functions

* Fix: forbid new filters for requests

* Merge devel fix
2022-05-26 16:16:11 +07:00
Igor Mandrigin
7f02ff81f3
Fix rpc.BlockNumberOrHash unmarshaling ()
* add test

* fix unmarshaling bug

Co-authored-by: Igor Mandrigin <i@mandrigin.ru>
2022-05-20 16:04:24 +07:00
Håvard Anda Estensen
6844e74ad2
Replace ioutil with io and os () 2022-04-23 15:43:00 +01:00