Commit Graph

31 Commits

Author SHA1 Message Date
Mark Holt
19bc328a07
Added db loggers to all db callers and fixed flag settings (#9099)
Mdbx now takes a logger - but this has not been pushed to all callers -
meaning it had an invalid logger

This fixes the log propagation.

It also fixed a start-up issue for http.enabled and txpool.disable
created by a previous merge
2023-12-31 17:10:08 +07:00
Mark Holt
509a7af26a
Discovery zero refresh timer (#8661)
This fixes an issue where the mumbai testnet node struggle to find
peers. Before this fix in general test peer numbers are typically around
20 in total between eth66, eth67 and eth68. For new peers some can
struggle to find even a single peer after days of operation.

These are the numbers after 12 hours or running on a node which
previously could not find any peers: eth66=13, eth67=76, eth68=91.

The root cause of this issue is the following:

- A significant number of mumbai peers around the boot node return
network ids which are different from those currently available in the
DHT
- The available nodes are all consequently busy and return 'too many
peers' for long periods

These issues case a significant number of discovery timeouts, some of
the queries will never receive a response.

This causes the discovery read loop to enter a channel deadlock - which
means that no responses are processed, nor timeouts fired. This causes
the discovery process in the node to stop. From then on it just
re-requests handshakes from a relatively small number of peers.

This check in fixes this situation with the following changes:

- Remove the deadlock by running the timer in a separate go-routine so
it can run independently of the main request processing.
- Allow the discovery process matcher to match on port if no id match
can be established on initial ping. This allows subsequent node
validation to proceed and if the node proves to be valid via the
remainder of the look-up and handshake process it us used as a valid
peer.
- Completely unsolicited responses, i.e. those which come from a
completely unknown ip:port combination continue to be ignored.
-
2023-11-07 08:48:58 +00:00
Alex Sharov
6d9a4f4d94
rpcdaemon: must not create db - because doesn't know right parameters (#8445) 2023-10-12 14:11:46 +07:00
Alex Sharov
3cea1b9b9e
torrent: add --webseeds cli arg (#8176) 2023-09-12 12:18:47 +07:00
battlmonstr
bb2c2adbb6
p2p: fix RLPx disconnect message decoding (#8056)
The disconnect message could either be a plain integer, or a list with
one integer element. We were encoding it as a plain integer, but
decoding as a list. Change this to be able to decode any format.
2023-08-24 13:49:19 +02:00
Andrew Ashikhmin
6bc0ca9e85
Correctly compute fork id when timestamp fork is activated in genesis (#8046)
See https://github.com/ethereum/go-ethereum/pull/27895
2023-08-21 15:35:13 +02:00
Alex Sharov
bf8ac9a38f
mdbx_to_mdbx: to use logger (#7860) 2023-07-09 08:04:20 +01:00
ledgerwatch
c919283b0c
[devnet] separate logging p2p (#7549)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-19 23:08:45 +01:00
ledgerwatch
b0117a7c30
[devnet] separate logging - p2p (#7547)
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-19 18:41:53 +01:00
Andrew Ashikhmin
cc11462860
Prioritize eth/68 by default (#7463)
Set `DefaultConfig.ProtocolVersion` to [68, 67, 66] instead of [67, 68].
See https://github.com/ethereum/hive/pull/776
2023-05-08 16:03:59 +02:00
ledgerwatch
fdd385cef1
[Devnet tool] Side-quest to improve logging - part 1 (#7445)
This is the beginning of the series of changes to make it possible to
run multiple instances of erigon inside a single process (as devnet tool
does), with the logging from these processes going to respective log
files correctly.
This is the first part where the initial infrastructure is being
established

---------

Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-05-07 07:28:15 +01:00
Alex Sharov
e1f549ff01
cli: use SplitAndTrim instead of strgings.Split(",") (#7369) 2023-04-23 10:54:55 +07:00
Alex Sharov
999899d66d
e3: read files list from db (#6833) 2023-02-13 05:17:01 +00:00
ledgerwatch
ee83447fa1
CL lightclient to create memdb in the tmpdir which gets cleaned up on… (#6829)
… startup

---------

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro-2.local>
2023-02-11 20:44:51 +00:00
hexoscott
7dcbfbc283
reference hash, address, and chain config from lib (#6536) 2023-01-13 18:12:18 +00:00
Andrew Ashikhmin
0fc2022a0e
Extend fork ID to timestamp-based forks (#6324)
Starting from
[Shanghai](https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/shanghai.md),
forks are based on timestamps rather than block heights (see PR #6238).
This PR extends [EIP-2124](https://eips.ethereum.org/EIPS/eip-2124) Fork
ID to include timestamp-based blocks. See also
https://github.com/ethereum/go-ethereum/pull/25878.

Co-authored-by: Marius van der Wijden <m.vanderwijden@live.de>
2022-12-16 12:08:54 +01:00
Håvard Anda Estensen
099fb7c627
Grow string builders (#6192)
Strings builders have a buffer (slice) that can be pre-allocated
2022-12-04 10:51:20 +07:00
Andrew Ashikhmin
32629bdce4
Upgrade urfave/cli to v2 (#6047)
See also
https://github.com/urfave/cli/blob/main/docs/migrate-v1-to-v2.md
2022-11-14 17:33:57 +01:00
Alex Sharov
26fdf9169d
move all packages from "internal" folder - to simplify users live (#5857) 2022-10-25 09:58:25 +07:00
Alex Sharov
606ce5c99a
add logging cli flags to various cmd (#5816) 2022-10-21 12:36:17 +07:00
Håvard Anda Estensen
ea6b92b132
Enable gosec linter (#5083)
* Ignore pseudo random generators

* Enable gosec linter
2022-08-17 12:18:19 +07:00
Alex Sharov
351cd49c21
go1.19 gofmt (#4988) 2022-08-10 19:04:13 +07:00
Alex Sharov
aee4b53788
save list of snapshots in db (#4777)
* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save

* save
2022-07-28 16:57:38 +07:00
Andrew Ashikhmin
eec5fa4d41
Add support for eth/67 (#4564)
* Add eth/67

* Listen to eth/66 on a separate port

* Fix compilation error

* Fix cfg66.ListenAddr

* Update erigon ports in README

* Expose port 30304 in docker

* P2pProtocolVersionFlag instead of second sentry

* Remove "66 by default" from usage

* Small comment
2022-07-08 11:14:16 +02:00
Andrew Ashikhmin
dc5d3ffaac
Revert "Switch from eth/66 to eth/67 (#4549)" (#4562)
This reverts commit 16e57aa8a2.
2022-06-28 13:42:35 +02:00
Andrew Ashikhmin
16e57aa8a2
Switch from eth/66 to eth/67 (#4549)
* Switch from eth/66 to eth/67

* Fix a compilation error

* Update erigon-lib

* Refresh erigon-lib
2022-06-28 10:12:48 +02:00
battlmonstr
0c5d1d64a3
observer: sentry candidates intake (#4321)
* sentry client: log connected peer info

* observer: unseen sentry peers report

* observer: refactoring node.go to node_utils

* observer: sentry candidates intake
2022-06-01 22:48:24 +01:00
battlmonstr
25b14edbf6
p2p: STUN external IP detection (#4195)
--nat stun is an automatic external IP detection alternative to manual --nat extip option.
It can be used both at home or on production servers without any extra setup.
It is fast (up to 5 ms) and more reliable than alternatives (as the request goes to the public internet).

This auto-detection is useful to run multiple instances of a service in cloud environments
where the node IPs are not known in advance.
2022-05-18 16:50:50 +01:00
battlmonstr
4b3eb0ffac
Observer: fix panic on clean start (#4002) (#4017)
Problem: (nil, nil) from CountPingErrors was not handled.
This happens if the node is not in the db (a bootstrap node),
and was never crawled before.
2022-04-28 12:46:33 +02:00
Alex Sharov
ea8a347ece
Downloader: calc stat inside, add --torrent.download.slots and limit downloads inside (#3986)
* save

* save

* save

* save

* save

* save

* save

* save

* save
2022-04-28 09:10:00 +07:00
battlmonstr
92db317e06
Observer - P2P network crawler (#3928)
Observer crawls the Ethereum network and collects information about the nodes.
2022-04-22 12:02:12 +01:00