Previously "in-memory" MDBX instances for fork validation and mining
were created inside `os.TempDir()`. We should create them inside
Erigon's datadir so that the file permissions and the disk are the same
as for the main database.
Prerequisite: https://github.com/ledgerwatch/erigon-lib/pull/676.
* TestTable_ReadRandomNodesGetAll: refactor to integration and examples
* TestTable_bumpNoDuplicates: refactor to integration and examples
* TestUDPv4_smallNetConvergence: speed up from 1.7s to 0.3s by applying the test config
* TestUPNP_DDWRT: move to integration tests
* TestFairMix: split in 2, do more iterations in integration tests
* TestDialSched: speed up from 1s to 0.2s by removing the unexpected dial check,
(keep the check during the integration tests)
Most places that used this method were cutting off the 1st byte.
Refactor this idea to a common place.
* better naming: MarshalPubkey matches existing UnmarshalPubkey
* "Std" suffix for the ANSI standard encoding without cut off
* docs
Problem:
QuerySeeds will poke 150 random entries in the whole node DB and ignore hitting "field" entries.
In a bootstrap scenario it might hit hundreds of :lastping :lastpong entries,
and very few true "node record" entries.
After running for 15 minutes I've got totalEntryCount=1508 nodeRecordCount=114 entries.
There's a 1/16 chance of hitting a "node record" entry.
It means finding just about 10 nodes of 114 total on average from 150 attempts.
Solution:
Split "node record" entries to a separate table such that QuerySeeds doesn't do idle cycle hits.
UpdateFindFails/UpdateLastPingReceived/UpdateLastPongReceived events
are causing bursty DB commits (100 per minute).
This optimization throttles the disk writes to happen at most once in a few seconds,
because this info doesn't need to be persisted immediately.
This helps on HDD drives.
Encode and hash logic was duplicated in multiple places.
* Move encoding to p2p/discover/v4wire
* Move hashing to p2p/enode/idscheme
* Change newRandomLookup to create a proper random key on a curve.
* Prevent frequent commits to the node DB in sentries
* Commit when btree goes over limit
* iterator for SeedQuery
* Fixing test
* Fix tests
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
* implemented crash reporting for all goroutine panics that aren't handled explicitly
* implemented crash reporting for all goroutine panics that aren't handled explicitly
* changed node defaults back to originals after testing
* implemented panic handling for all goroutines that don't explicitly handle them, outputting the stack trace to a file in crashreports
* handling panics on all goroutines gracefully
* updated missing call
* error assignment
* implemented suggestions
* path.Join added
* implemented Evgeny's suggestions
* changed path.Join to filepath.Join for cross-platform
* added err check
* updated RecoverStackTrace to LogPanic
* updated closures
* removed call of common.Go to some goroutines
* updated scope capture
* removed testing files
* reverted back to original method, I feel like its less intrusive
* update filename for clarity
The database panicked for invalid IPs. This is usually no problem
because all code paths leading to node DB access verify the IP, but it's
dangerous because improper validation can turn this panic into a DoS
vulnerability. The quick fix here is to just turn database accesses
using invalid IP into a noop. This isn't great, but I'm planning to
remove the node DB for discv5 long-term, so it should be fine to have
this quick fix for half a year.
Fixes#21849
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* change_set_dup
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* working version
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* aa
* squash
* squash
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* fix
* history_early_stop
* history_early_stop
* vmConfig with ReadOnly false
* auto_increment
* auto_increment
* rebase master
Co-authored-by: Alexey Akhunov <akhounov@gmail.com>
