If any DB method is called while Close() is waiting for db.kv.Close()
(it waits for ongoing method calls/transactions to finish)
a panic: "WaitGroup is reused before previous Wait has returned" might
happen.
Use context cancellation to ensure that new method calls immediately
return during db.kv.Close().
Mdbx now takes a logger - but this has not been pushed to all callers -
meaning it had an invalid logger
This fixes the log propagation.
It also fixed a start-up issue for http.enabled and txpool.disable
created by a previous merge
Previously "in-memory" MDBX instances for fork validation and mining
were created inside `os.TempDir()`. We should create them inside
Erigon's datadir so that the file permissions and the disk are the same
as for the main database.
Prerequisite: https://github.com/ledgerwatch/erigon-lib/pull/676.
Problem:
QuerySeeds will poke 150 random entries in the whole node DB and ignore hitting "field" entries.
In a bootstrap scenario it might hit hundreds of :lastping :lastpong entries,
and very few true "node record" entries.
After running for 15 minutes I've got totalEntryCount=1508 nodeRecordCount=114 entries.
There's a 1/16 chance of hitting a "node record" entry.
It means finding just about 10 nodes of 114 total on average from 150 attempts.
Solution:
Split "node record" entries to a separate table such that QuerySeeds doesn't do idle cycle hits.
UpdateFindFails/UpdateLastPingReceived/UpdateLastPongReceived events
are causing bursty DB commits (100 per minute).
This optimization throttles the disk writes to happen at most once in a few seconds,
because this info doesn't need to be persisted immediately.
This helps on HDD drives.
* Prevent frequent commits to the node DB in sentries
* Commit when btree goes over limit
* iterator for SeedQuery
* Fixing test
* Fix tests
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
The database panicked for invalid IPs. This is usually no problem
because all code paths leading to node DB access verify the IP, but it's
dangerous because improper validation can turn this panic into a DoS
vulnerability. The quick fix here is to just turn database accesses
using invalid IP into a noop. This isn't great, but I'm planning to
remove the node DB for discv5 long-term, so it should be fine to have
this quick fix for half a year.
Fixes#21849
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* squash
* squash
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* history_early_stop
* history_early_stop
* vmConfig with ReadOnly false
* auto_increment
* auto_increment
* rebase master
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
