The fixes here fix a couple of issues related to devnet start-up
1. macos threading and syscall error return where causing multi node
start to both not wait and fail
2. On windows creating DB's with the default 2 TB mapsize causes the os
to reserve about 4GB of committed memory per DB. This may not be used -
but is reserved by the OS - so a default bor node reserves around 10GB
of storage. Starting many nodes causes the OS page file to become
exhausted.
To fix this the consensus DB's now use the node's OpenDatabase function
rather than their own, which means that the consensus DB's take notice
of the config.MdbxDBSizeLimit.
This fix leaves one 4GB committed memory allocation in the TX pool which
needs its own MapSize setting.
---------
Co-authored-by: Alex Sharp <akhounov@gmail.com>
The check in catches errors in the node start-up code and makes sure
that the network is stopped if any node fails to start cleanly, and
that5 it returns an error - so that any calling code can take
appropriate action.
reasons:
- mainnet: even nodes with small FreeList - still have millions of pages
there `GC: 46446830 5.8%`. Probability of getting into state where space
re-use will be slower than free-list grow is > 0% (we now using db
version which limiting freelist-overhead, but increasing such
probability)
- polygon: size is > 8Tb
- hardware slowly moving towards bigger pageSizes (because for
OS/Hardware) maintenance of pages metadata is also not free (metadata,
lists, LRU, etc...). Macbook's default pagesize now is 16Kb. Network
disks in cloud are also likely working with 16Kb pages.
pros:
- less db fragmentation (better FS-level compression)
- less overflow pages in DB (which also reducing free-list overhead)
- smaller free-list
- bigger key-size-limit
- no 8Tb db size limit
- can setup FS - to also use bigger pagesize - it will reduce FS
overhead also
- reducing amount of page-faults during batch-reads (if FS pagesize
match)
- less write syscalls during commit (when WriteMap disabled)
cons:
- ~10% more IO: because of more RAM waste and just because need
read/write bigger pages (not all updates are co-located).
Added support tunnel to the devnet cmd. In order to get this to run I
made the following changes:
* Create a public function
* Added non root logging
I have also added commentary to the readme to explain the additional
command line arguments needed to integrate with diagnostics. In summary,
if you set the --diagnostics.url the devenet will wait for diagnostic
requests rather than exiting
---------
Co-authored-by: alex.sharov <AskAlexSharov@gmail.com>
Changes summary:
- Continue with the gasLimit check skip in ``verifyHeader`` of
``merge.go`` for unless pre-merge block and blockGasLimitContract
present
- Refactor ``aura.go`` a bit
- Have ``sysCall`` method customized to be able to call state (contract)
at a parent (or any other) header state
