erigon-pulse/erigon-lib
Mark Holt 509a7af26a
Discovery zero refresh timer (#8661)
This fixes an issue where the mumbai testnet node struggle to find
peers. Before this fix in general test peer numbers are typically around
20 in total between eth66, eth67 and eth68. For new peers some can
struggle to find even a single peer after days of operation.

These are the numbers after 12 hours or running on a node which
previously could not find any peers: eth66=13, eth67=76, eth68=91.

The root cause of this issue is the following:

- A significant number of mumbai peers around the boot node return
network ids which are different from those currently available in the
DHT
- The available nodes are all consequently busy and return 'too many
peers' for long periods

These issues case a significant number of discovery timeouts, some of
the queries will never receive a response.

This causes the discovery read loop to enter a channel deadlock - which
means that no responses are processed, nor timeouts fired. This causes
the discovery process in the node to stop. From then on it just
re-requests handshakes from a relatively small number of peers.

This check in fixes this situation with the following changes:

- Remove the deadlock by running the timer in a separate go-routine so
it can run independently of the main request processing.
- Allow the discovery process matcher to match on port if no id match
can be established on initial ping. This allows subsequent node
validation to proceed and if the node proves to be valid via the
remainder of the look-up and handshake process it us used as a valid
peer.
- Completely unsolicited responses, i.e. those which come from a
completely unknown ip:port combination continue to be ignored.
-
2023-11-07 08:48:58 +00:00
..
.github/workflows go.mod: minimal go version 1.20 (#8495) 2023-10-17 06:52:28 +07:00
bptree Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
chain Fix Agra hf scheduling on Mumbai (#8620) 2023-10-30 15:18:47 +01:00
commitment Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
common erigon-lib: remove unused constants from protocol.go (#8644) 2023-11-03 10:20:36 +01:00
compress Block execution using embedded Silkworm (#8353) 2023-10-05 09:27:37 +07:00
crypto Sentinel refactor (#8296) 2023-10-22 01:17:18 +02:00
diagnostics Discovery zero refresh timer (#8661) 2023-11-07 08:48:58 +00:00
direct snapshots: reduce merge limit of blocks to 100K (#8614) 2023-11-01 23:22:35 +07:00
downloader downloader: demote webseed request errors (#8662) 2023-11-06 14:47:57 +01:00
etl etl: more tests (#8534) 2023-10-20 13:54:46 +07:00
gointerfaces snapshots: reduce merge limit of blocks to 100K (#8614) 2023-11-01 23:22:35 +07:00
kv Updated go.sum and go.mod for snapshots (#8664) 2023-11-06 23:39:44 +01:00
metrics Fix metrics get perf regression (#8563) 2023-10-23 15:19:52 +01:00
mmap config: avoid OOM in docker using cgroups v2 limit (#6646) (#8632) 2023-11-01 09:02:34 +07:00
patricia Upgrade libp2p (enables go 1.21 support) (#8288) 2023-09-29 22:11:13 +02:00
pedersen_hash Sentinel refactor (#8296) 2023-10-22 01:17:18 +02:00
recsplit Block execution using embedded Silkworm (#8353) 2023-10-05 09:27:37 +07:00
rlp Fix rlp.EncodeString for length 56 (#8528) 2023-10-19 11:45:19 +02:00
rlp2 rlp2 (#8643) 2023-11-04 09:22:11 +07:00
sais Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
state rpcdaemon: must not create db - because doesn't know right parameters (#8445) 2023-10-12 14:11:46 +07:00
tools Fix typo in make output (#8649) 2023-11-04 11:11:15 +00:00
txpool pool: limit broadcast tx size by 4kb (#8634) 2023-11-01 09:56:02 +07:00
types eth/68: always announce tx sizes without envelopes (#8530) 2023-10-19 14:37:55 +02:00
.gitignore Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
.golangci.yml Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
go.mod Updated go.sum and go.mod for snapshots (#8664) 2023-11-06 23:39:44 +01:00
go.sum Updated go.sum and go.mod for snapshots (#8664) 2023-11-06 23:39:44 +01:00
LICENSE Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
Makefile ci: go mod tidy check (#8263) 2023-09-22 14:04:25 +07:00
README.md make: refactor erigon-lib make tasks (#8249) 2023-09-21 13:50:59 +02:00
rules.go Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
tools.go Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00

erigon-lib

Parts of Erigon codebase, written from scratch and licensed under Apache 2.0.

License requirements

erigon-lib dependencies use various open source licenses compatible with Apache 2.0. This is checked on CI using make lint-licenses.

In order to keep license purity it is not allowed to refer to the code in the erigon root module from erigon-lib. This is ensured by the go.mod separation.

It is not allowed to copy or move code from erigon to erigon-lib unless all original authors agree to relief the code license from GPL to Apache 2.0.

Code migration policy

It is encouraged to write new erigon code inside erigon-lib.

It is encouraged to move and relicense parts of the code from erigon to erigon-lib that are safe and easy to move. For example, code written from scratch by erigon core contributors that has no significant external contributions could be refactored and moved.