erigon-pulse/erigon-lib
Alex Sharov 329d18ef6f
snapshots: reduce merge limit of blocks to 100K (#8614)
Reason: 
- produce and seed snapshots earlier on chain tip. reduce depnedency on
"good peers with history" at p2p-network.
Some networks have no much archive peers, also ConsensusLayer clients
are not-good(not-incentivised) at serving history.
- avoiding having too much files:
more files(shards) - means "more metadata", "more lookups for
non-indexed queries", "more dictionaries", "more bittorrent
connections", ...
less files - means small files will be removed after merge (no peers for
this files).


ToDo:
[x] Recent 500K - merge up to 100K 
[x] Older than 500K - merge up to 500K 
[x] Start seeding 100k files
[x] Stop seeding 100k files after merge (right before delete)

In next PR: 
[] Old version of Erigon must be able download recent hashes. To achieve
it - at first start erigon will download preverified hashes .toml from
s3 - if it's newer that what we have (build-in) - use it.
2023-11-01 23:22:35 +07:00
..
.github/workflows go.mod: minimal go version 1.20 (#8495) 2023-10-17 06:52:28 +07:00
bptree Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
chain Fix Agra hf scheduling on Mumbai (#8620) 2023-10-30 15:18:47 +01:00
commitment Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
common Numerical instead of lexicographic sorting in borKeyValueConfigHelper (#8560) 2023-10-23 14:11:32 +02:00
compress Block execution using embedded Silkworm (#8353) 2023-10-05 09:27:37 +07:00
crypto Sentinel refactor (#8296) 2023-10-22 01:17:18 +02:00
diagnostics Add code to handle close/cancel semantics (#8633) 2023-10-31 18:43:23 +00:00
direct snapshots: reduce merge limit of blocks to 100K (#8614) 2023-11-01 23:22:35 +07:00
downloader snapshots: reduce merge limit of blocks to 100K (#8614) 2023-11-01 23:22:35 +07:00
etl etl: more tests (#8534) 2023-10-20 13:54:46 +07:00
gointerfaces snapshots: reduce merge limit of blocks to 100K (#8614) 2023-11-01 23:22:35 +07:00
kv fix: batch close confict with flush (#8613) 2023-11-01 09:08:47 +07:00
metrics Fix metrics get perf regression (#8563) 2023-10-23 15:19:52 +01:00
mmap config: avoid OOM in docker using cgroups v2 limit (#6646) (#8632) 2023-11-01 09:02:34 +07:00
patricia Upgrade libp2p (enables go 1.21 support) (#8288) 2023-09-29 22:11:13 +02:00
pedersen_hash Sentinel refactor (#8296) 2023-10-22 01:17:18 +02:00
recsplit Block execution using embedded Silkworm (#8353) 2023-10-05 09:27:37 +07:00
rlp Fix rlp.EncodeString for length 56 (#8528) 2023-10-19 11:45:19 +02:00
sais Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
state rpcdaemon: must not create db - because doesn't know right parameters (#8445) 2023-10-12 14:11:46 +07:00
tools Revert "Move validator set snapshot computation to bor_heimdall stage… (#8580) 2023-10-25 14:02:31 +02:00
txpool pool: limit broadcast tx size by 4kb (#8634) 2023-11-01 09:56:02 +07:00
types eth/68: always announce tx sizes without envelopes (#8530) 2023-10-19 14:37:55 +02:00
.gitignore Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
.golangci.yml Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
go.mod snapshots: reduce merge limit of blocks to 100K (#8614) 2023-11-01 23:22:35 +07:00
go.sum snapshots: reduce merge limit of blocks to 100K (#8614) 2023-11-01 23:22:35 +07:00
LICENSE Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
Makefile ci: go mod tidy check (#8263) 2023-09-22 14:04:25 +07:00
README.md make: refactor erigon-lib make tasks (#8249) 2023-09-21 13:50:59 +02:00
rules.go Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00
tools.go Add 'erigon-lib/' from commit '93d9c9d9fe4bd8a49f7a98a6bce0f0da7094c7d3' 2023-09-20 14:50:25 +02:00

erigon-lib

Parts of Erigon codebase, written from scratch and licensed under Apache 2.0.

License requirements

erigon-lib dependencies use various open source licenses compatible with Apache 2.0. This is checked on CI using make lint-licenses.

In order to keep license purity it is not allowed to refer to the code in the erigon root module from erigon-lib. This is ensured by the go.mod separation.

It is not allowed to copy or move code from erigon to erigon-lib unless all original authors agree to relief the code license from GPL to Apache 2.0.

Code migration policy

It is encouraged to write new erigon code inside erigon-lib.

It is encouraged to move and relicense parts of the code from erigon to erigon-lib that are safe and easy to move. For example, code written from scratch by erigon core contributors that has no significant external contributions could be refactored and moved.