Revert to older batch size logic to keep memory usage down during
execution phase and closer to the --batchSize flag size.
Spotted a "leak" in key generation as well. The unsafe pointer keeps the
byte slice around for as long as the batch is not committed, takes up a
fair chunk of memory surprisingly doing that so removed the unsafe
pointer usage giving the GC some chance to clean up along the way.
Moved the batch rollback into a defer func call rather than allowing
them to stack in the for loop. If this isn't going to work just let me
know and can change it back.
Seems simple enough, don't stop on interrupt but start a timer instead
and check on each loop if we should stop now or allow the block to
finish on its own
Previously "in-memory" MDBX instances for fork validation and mining
were created inside `os.TempDir()`. We should create them inside
Erigon's datadir so that the file permissions and the disk are the same
as for the main database.
Prerequisite: https://github.com/ledgerwatch/erigon-lib/pull/676.
Moves bodies request logic to pre OOM changes. No logging or checking
for timeouts and moving forwards with every request made. Tested on a
local node that went through the DB migration on this branch and worked
fine, caught back up to the tip. Tested on POW initial sync and looked
to be working fine writing bodies, although didn't have time to see this
run all the way through the merge.
the root cause is that when `inMemoryExecution` lambda gets created in
the `eth/backend.go`, it captures the reference of
`backend.notifications`, and so the execution of side-forks actually
adds notifications to there, and it all gets sent out to tx pool (and
RPC daemon) at the end of the stage loop (regardless of whether there
was forkchoice update or not)
so we can create a separate notification, but then somehow flush it to
the "main" nofitications when the in-memory exec state is flushed
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Alex Sharp <alexsharp@Alexs-MacBook-Pro.local>
* Fail newPayload if execution is unsuccessful
* fix linter
* fix message
* no self-hosted
* try force ci
* back to unbuntu
* Remove unnecessary unwinds
* Fix canonicalising
* Quiet logs for in-memory exec
* Remove experimental overlay flag
* Fix sync test
* Fix test
* Fix lint
* Print hash in the end of the cycle
* Not notify headers if list is empty
* Supress timings and tables logs when no change in progress
* Reduce logging
* fix
* better log for pos download
* Better newPayload and forkChoice messages
* Fix lint
* simplify
* No duplication of download messages
* Reverse log for extension
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: Igor Mandrigin <i@mandrigin.ru>
