Commit Graph

923 Commits

Author SHA1 Message Date
Dmytro
a6b5297b3e
dvovk/tunnelwws (#8745)
- changed communication tunnel to web socket in order to connect to
remote nodes
- changed diagnostics.url flag to diagnostics.addr as now user need to
enter only address and support command will connect to it through
websocket
- changed flag debug.urls to debug.addrs in order to have ability to
change connection type between erigon and support to websocket and don't
change user API
- added auto trying to connect to connect to ws if connection with was
failed
2023-11-16 16:37:29 +00:00
Alex Sharov
35bfffd621
sys deps up (#8695) 2023-11-11 15:04:18 +03:00
battlmonstr
3698e7f476
devnet: configuration fixes (#8592)
* fix "genesis hash does not match" when dev nodes connect  
The "dev" nodes need to have the same --miner.etherbase in order to
generate the same genesis ExtraData by DeveloperGenesisBlock(). Override
DevnetEtherbase global var that's used if --miner.etherbase is not
passed. (for NonBlockProducer case)

* fix missing private key for the hardcoded DevnetEtherbase  
Fixes panic if SigKey is not found. Bor non-producers will use a default
`DevnetEtherbase` while Dev nodes modify it. Save hardcoded
DevnetEtherbase/DevnetSignPrivateKey into accounts so that SigKey can
recover it.

* refactor devnet.node to contain Node config  
This avoids interface{} type casts and fixes an error with
Heimdall.validatorSet == nil

* add connection retries to rpcCall and Subscribe of requestGenerator  
Fixes "connection refused" errors due to node not ready to handle early
RPC requests.

* fix deadlock in Heimdall.NodeStarted

* fix GetBlockByNumber
Fixes "cannot unmarshal string into Go struct field body.transactions of
type jsonrpc.RPCTransaction"

* demote "no of blocks on childchain is less than confirmations
required" to Info (#8626)

* demote "mismatched pending subpool size" to Debug (#8615)

* revert wiggle testing code
2023-11-01 11:08:47 +01:00
a
436493350e
Sentinel refactor (#8296)
1. changes sentinel to use an http-like interface

2. moves hexutil, crypto/blake2b, metrics packages to erigon-lib
2023-10-22 01:17:18 +02:00
Mark Holt
0bdca6c457
Metrics label fixes (#8339)
Fixes for label discrepancies in collector for summaries etc which have
a template which includes a quantile.

Initial native Prometheus client implementation of metrics - which is
currently turned off except for local testing and interface exports.
2023-10-02 17:19:02 +01:00
Mark Holt
f794438335
Diag session routing (#8232)
Code to support react based UI for diagnostics:

* pprof, prometheus and diagnistics rationalized to use a single router
(i.e. they can all run in the same port)
* support_cmd updated to support node routing (was only first node)
* Multi content support in router tunnel (application/octet-stream &
appliaction/json)
* Routing requests changed from using http forms to rest + query params
* REST query requests can now be made against erigon base port and
diagnostics with the same url format/params

---------

Co-authored-by: dvovk <vovk.dimon@gmail.com>
Co-authored-by: Mark Holt <mark@disributed.vision>
2023-09-25 16:24:17 +01:00
Srdjan
dedf04caaa
Pre-allocate method metric labels (#8243)
Closes #8085
2023-09-21 09:38:47 +02:00
Mark Holt
8ea0096d56
moved metrics sub packages types to metrics (#8119)
This is a non functional change which consolidates the various packages
under metrics into the top level package now that the dead code is
removed.

It is a precursor to the removal of Victoria metrics after which all
erigon metrics code will be contained in this single package.
2023-09-03 08:09:27 +07:00
Mark Holt
e81852320c
Remove unused metrics and out of date versions (#8109)
ergon/metrics contains a lot of unused types - the pull is removing them
prior to rationalizing between Prometheus and Victoria Metrics types.

It also have upgrades of golang-lru + sets types to v2 when possible.
2023-09-01 07:13:13 +07:00
Mark Holt
a4cfbe0d56
Heimdall metrics + Metrics HTTP server rationalization (#8094)
This is an update of:

https://github.com/ledgerwatch/erigon/pull/7846

which uses a local fork of victoria metrics to include the changes that
https://github.com/anshalshukla added to the original for we where
using.

It also includes code to address the duplicate metrics issue identified
here:

https://github.com/ledgerwatch/erigon/issues/8053

It has one more associated fix which is to correctly add a metadata
label to counters, these where previously labelled as gauges.

e.g. 

```
# TYPE p2p_peers counter
p2p_peers 0
```
rather than

```
# TYPE p2p_peers gauge
p2p_peers 0
```

---------

Co-authored-by: Anshal Shukla <53994948+anshalshukla@users.noreply.github.com>
Co-authored-by: Anshal Shukla <shukla.anshal85@gmail.com>
2023-08-31 09:04:27 +01:00
Alex Sharov
ddf608f866
rpc: a bit less metrics allocations (#8084) 2023-08-26 11:20:08 +07:00
Mark Holt
c51573f333
Bor eth event flow (#8068)
Implemented polygon->eth flow
2023-08-25 12:19:39 +01:00
a
db5b348673
Caplin block persistence (#7941)
Co-authored-by: Giulio <giulio.rebuffo@gmail.com>
2023-08-09 01:21:19 +02:00
Andrew Ashikhmin
d69b20bc4e
Support engine_forkchoiceUpdatedV3 with ParentBeaconBlockRoot (EIP-4788) (#7969)
Prerequisites: https://github.com/ledgerwatch/interfaces/pull/187 &
https://github.com/ledgerwatch/erigon-lib/pull/1069. Also implement
https://github.com/ethereum/execution-apis/pull/426.
2023-08-06 11:54:14 +02:00
Mark Holt
1e575ea172
rpc service and stageloop logger updates (#7696)
This is another update to logging to replace the root logger with a
contextual logger
2023-06-10 07:39:39 +01:00
ledgerwatch
067f695fff
[devnet tool] Separate logging (#7553)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-20 14:48:16 +01:00
ledgerwatch
05597cb195
[devnet tool] separare logging (#7531)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-17 17:36:15 +01:00
Alex Sharov
201572c6f5
enable more linters #954 (#7179) 2023-03-25 05:13:27 +00:00
hexoscott
165a6a950d
check for nil stream when running the null check in handler (#7105) 2023-03-15 04:03:16 +00:00
hexoscott
84ec0a0b4b
check for nil before returning invalid json in rpc streaming calls (#7104)
should handle nil having already been written in any rpc call before
writing it again causing invalid json to be returned.
2023-03-14 11:07:53 +00:00
alex.sharov
7190d89af3 save 2023-03-10 16:01:55 +07:00
Andrew Ashikhmin
2fa448c45f
Validate params of GetPayloadBodiesByHash and ByRange (#6785)
See https://github.com/ethereum/execution-apis/pull/366 and
https://github.com/ethereum/execution-apis/pull/370.

Also fix a couple of issues so that the Hive tests pass.
2023-02-06 18:38:45 +01:00
a
966be04e6d
add eth_getFilterLogs, fix filter subscription ids (#6514)
this pr does the following:

1. adds new function to ApiImpl `GetFilterLogs` which should implement
`eth_getFilterLogs` (eth_getFilterChanges except with only logs filters)

2. changes the ID generator of rpchelper.Filters to use crypto/rand.

3. switched logs subscriptions to use the secure ID instead of number

4. changes subcription ids from an 8 byte hex string to a 16 byte
2023-02-06 10:18:10 +07:00
hexoscott
7dcbfbc283
reference hash, address, and chain config from lib (#6536) 2023-01-13 18:12:18 +00:00
ledgerwatch
389a2e8392
Eth call limits (#6523)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2023-01-07 21:16:43 +00:00
a
ad7be30679
jsonrpc HTTP Get handler & logs topic filter with map (#5922)
this pr has two things in it

1. changed filter logs to use a map for the topics. this will speed up
queries with many topics in them. I still don't have a use case for this
though. i put is as a method of Logs, since that made sense to me, happy
to move it back out though.

2. allows json-rpc over http get request. since firefox is a great json
viewer (can search through, collapse large results) i often use it to
debug. it is also useful for sharing data with those who are less
familiar with command line / programming.
example get request:

http://rpcdaemon/?method=eth_getLogs&params=[{"fromBlock":"0xf2316b","toBlock":"0xf2316b"}]

it is based on the old jsonrpc http specification
https://www.jsonrpc.org/historical/json-rpc-over-http.html#encoded-parameters
except we also accept not base64 encoded params. since every eth rpc
request has a [], it will immediately fail validation for base64 and
attempt to use the parameters as a they are. otherwise it will attempt
to parse the rest of the payload as base64 and use that.

Co-authored-by: a <a@a.a>
Co-authored-by: gfx <86091021+gfxlabs@users.noreply.github.com>
2022-11-03 09:09:04 +07:00
dtan3847
42aa80e059
rpc: fix implemented filter methods being forbidden (#5801)
Addresses issue #4833

Removes implemented methods from the forbidden method list. The
forbidden list seems to have been created as a bugfix before the filters
methods were implemented. Now that the filter methods are implemented,
these entries are causing unexpected behaviour.
2022-10-26 09:08:09 +07:00
Temirlan
44266a9237
rcpdaemon, erigon: add --rpc.evmtimeout flag (#5395) 2022-09-17 13:25:27 +07:00
ledgerwatch
566cd65664
Revert "cancel context before waiting for waitgroup to avoid deadlock in RPC on error (#5345)" (#5360)
This reverts commit 9ff5b21a1e.
2022-09-14 08:45:59 +01:00
ledgerwatch
9ff5b21a1e
cancel context before waiting for waitgroup to avoid deadlock in RPC on error (#5345)
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-09-13 08:52:58 +07:00
Enrique Jose Avila Asapche
19a178bf19
missing return; (#5328) 2022-09-11 09:35:18 +07:00
Håvard Anda Estensen
ea6b92b132
Enable gosec linter (#5083)
* Ignore pseudo random generators

* Enable gosec linter
2022-08-17 12:18:19 +07:00
Alex Sharov
52fd0d0e8b
Aggregator22.Unwind() (#5039)
* save

* save
2022-08-13 18:51:25 +07:00
Alex Sharov
351cd49c21
go1.19 gofmt (#4988) 2022-08-10 19:04:13 +07:00
Enrique Jose Avila Asapche
c12d298f1c
new rpc block number (#4953)
* new rpc block number

* log

* ops
2022-08-08 19:07:24 +07:00
Dan Kelly
cce64f6a33
Fix rpc json #4838 (#4894)
* Fixing RPC JSON errors

* Use local rpc mod

* Revert "Use local rpc mod"

This reverts commit 61d06a72d218f5ecb84e9e4ddddf8762a04f714c.

* Make handle error accessible

* Port fixes to rpcdaemon22

* Remove unused...

* ... and use correct function name

* Handle errors outside iterator loop. Fix missing rpcdaemon22 error handling

* Bubble up higher level errors to the top-level object

* Fix linter error - semi-colon..
2022-08-08 09:15:58 +07:00
Alex Sharov
2681ee392a
rpc: fix error stream (#4860) 2022-07-29 10:01:13 +07:00
Andrew Ashikhmin
f927ac8037
Increase JWT issued-at window to 60s (#4835) 2022-07-27 09:16:50 +02:00
Alex Sharov
66758c7960
RetireBlocks: less arguments (#4785)
* save

* save
2022-07-22 13:44:42 +07:00
Alex Sharov
d9cb87a149
RPC: Enable back json streaming for non-batch and non-websocket cases (#4647)
* enable rpc streaming

* enable rpc streaming
2022-07-06 11:44:06 +01:00
Andrew Ashikhmin
8de7c5e41c
JSON parsing of safe & finalized (#4524) 2022-06-23 19:37:39 +02:00
ledgerwatch
64067a2b77
Debugging Engine API (#4488)
* defer tx rollback

* Add http.trace flag

* Fixed http.trace flag

* Fix lint

* Fix lint

* Delete unused tests

* Fix lint

* Fix lint

* Fix lint

* Fix lint

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
2022-06-19 13:40:28 +01:00
Enrique Jose Avila Asapche
a5cb53d690
safe and finalized blocks from eth_getBlockByNumber (#4436)
* added getFinalzed and getSafe block num

* added rpc finalized and safe block num

* getting nums

* returning nil

* returning nil

* added to helper.go

* removed repeated code

* added functions into rpchelper

* returning err

* simplified

* using previous latest getter

* getting pending block with filter/

* Fix plain state block number

* Fix test

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
2022-06-14 11:07:46 +03:00
Alex Sharov
955c669d21
Group few sync configs (#4303)
* save

* save
2022-05-30 17:08:49 +07:00
primal_concrete_sledge
5c055b9697
Fix/block new filters (#4271)
* Add filters functions

* Fix: forbid new filters for requests

* Merge devel fix
2022-05-26 16:16:11 +07:00
Igor Mandrigin
7f02ff81f3
Fix rpc.BlockNumberOrHash unmarshaling (#4218)
* add test

* fix unmarshaling bug

Co-authored-by: Igor Mandrigin <i@mandrigin.ru>
2022-05-20 16:04:24 +07:00
Håvard Anda Estensen
6844e74ad2
Replace ioutil with io and os (#3946) 2022-04-23 15:43:00 +01:00
Giulio rebuffo
904674e1a1
added Websocket support and authentication to Engine API (#3752)
* added ws support and auth

* fixed lint
2022-03-23 17:12:19 +01:00
battlmonstr
47df98a499
Use "err" key for logging errors. (#3632)
log.Warn/Error uses "err" key to log errors in most places.
This renames "error" to "err" in some places to adhere to this convention.
2022-03-01 15:40:51 +00:00
Andrew Ashikhmin
37ccbc53d6
Refactor payload building (#3592)
* Bump MaxPendingPayloads

* UnknownPayload error

* Improve ExchangeTransitionConfigurationV1 comment

* Extract MakeEmptyHeader

* Move MakeEmptyHeader to chain_makers

* Start with empty block as pending payload

* Only build payloads with the right parent hash

* Double check parent hash in SpawnMiningCreateBlockStage

* Simpler error formatting

* ReadHeadBlockHash instead of ReadHeadHeaderHash

* Avoid rebuilding already built blocks

* getPayload should stop the build process

* Tweaking stageLoopIsBusy

* Don't update pending payload after engine_getPayload
2022-02-23 21:27:38 +01:00