Reason:
- produce and seed snapshots earlier on chain tip. reduce depnedency on
"good peers with history" at p2p-network.
Some networks have no much archive peers, also ConsensusLayer clients
are not-good(not-incentivised) at serving history.
- avoiding having too much files:
more files(shards) - means "more metadata", "more lookups for
non-indexed queries", "more dictionaries", "more bittorrent
connections", ...
less files - means small files will be removed after merge (no peers for
this files).
ToDo:
[x] Recent 500K - merge up to 100K
[x] Older than 500K - merge up to 500K
[x] Start seeding 100k files
[x] Stop seeding 100k files after merge (right before delete)
In next PR:
[] Old version of Erigon must be able download recent hashes. To achieve
it - at first start erigon will download preverified hashes .toml from
s3 - if it's newer that what we have (build-in) - use it.
* fix "genesis hash does not match" when dev nodes connect
The "dev" nodes need to have the same --miner.etherbase in order to
generate the same genesis ExtraData by DeveloperGenesisBlock(). Override
DevnetEtherbase global var that's used if --miner.etherbase is not
passed. (for NonBlockProducer case)
* fix missing private key for the hardcoded DevnetEtherbase
Fixes panic if SigKey is not found. Bor non-producers will use a default
`DevnetEtherbase` while Dev nodes modify it. Save hardcoded
DevnetEtherbase/DevnetSignPrivateKey into accounts so that SigKey can
recover it.
* refactor devnet.node to contain Node config
This avoids interface{} type casts and fixes an error with
Heimdall.validatorSet == nil
* add connection retries to rpcCall and Subscribe of requestGenerator
Fixes "connection refused" errors due to node not ready to handle early
RPC requests.
* fix deadlock in Heimdall.NodeStarted
* fix GetBlockByNumber
Fixes "cannot unmarshal string into Go struct field body.transactions of
type jsonrpc.RPCTransaction"
* demote "no of blocks on childchain is less than confirmations
required" to Info (#8626)
* demote "mismatched pending subpool size" to Debug (#8615)
* revert wiggle testing code
new flag examples.
--https.enabled
--https.addr="0.0.0.0"
--https.port=443
--https.url="unix:///file.wow"
--https.cert="keyfile.cert"
--https.key="certfile.cert"
also adds support for h2c to the http handler - http2 protocol without tls.
read transaction was opened before stream.Recv(), but stream.Recv() is
blocking infinity loop. so, this read transaction never rollback -
causing unlimited db grow.
---------
Merge PR #8596 into `devel`
---------
Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
* call getEnode before NodeStarted to make sure it is ready for RPC
calls
* fix connection error detection on macOS
* use a non-default p2p port to avoid conflicts
* disable bor milestones on local heimdall
* generate node keys for static peers config
Newly introduced `t.logGaps` was being set to `nil` and still accessed
within `clearFailedLogs`. This PR changes the ordering, moving the nil
setting to `CaptureTxEnd`.
Historically we had several times when:
- erigon downloaded new version of .seg file
- or didn't finish download and start indexing
this was a "quick-fix protection" against this cases
but now we have other protections for this cases
let's try to remove this one - because it's not compatible with "copy
datadir" and "restore datadir from backup" scenarios
Calling vm to get a metric and then preregistering it to avoid
duplication is expensive.
To avoid this we can make a local lookup first which is a map this
quicker. This is a temporary fix
until vm is removed at which point we'll have a single prom_client
registry
Fixes: https://github.com/ledgerwatch/erigon/issues/8558
