From 3aeb186bd1e526caab6bbf6e209a2324d80e16c5 Mon Sep 17 00:00:00 2001 From: "alex.sharov" Date: Sun, 6 Jun 2021 20:02:15 +0700 Subject: [PATCH] db faq about RAM --- README.md | 12 +++++++++++ cmd/rpcdaemon/README.md | 2 +- .../{read_db.md => db_faq.md} | 20 ++++++++++++++++--- 3 files changed, 30 insertions(+), 4 deletions(-) rename docs/programmers_guide/{read_db.md => db_faq.md} (63%) diff --git a/README.md b/README.md index 684f2f010..8b39d3d3b 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,7 @@ Erigon is an implementation of Ethereum (aka "Ethereum client"), on the efficien + [JSON-RPC daemon](#json-rpc-daemon) + [Run all components by docker-compose](#run-all-components-by-docker-compose) + [Grafana dashboard](#grafana-dashboard) +- [FAQ](#faq) - [Getting in touch](#getting-in-touch) + [Erigon Discord Server](#erigon-discord-server) + [Reporting security issues/concerns](#reporting-security-issues-concerns) @@ -222,6 +223,17 @@ XDG_DATA_HOME=/preferred/data/folder docker-compose up `docker-compose up prometheus grafana`, [detailed docs](./cmd/prometheus/Readme.md). +FAQ +================ + +### How much RAM do I need + +- on SSD: 16Gb RAM sync takes 5 days, 32Gb - 4 days, 64Gb - 3 days +- on NVMe: 16Gb RAM sync takes 4 days, 32Gb - 4 days, 64Gb - 3 days +- File systems: Ext4 50% faster than ZFS. ZFS 50% faster than BTRfs. ZFS 50% faster than "zfs compression=on" + +Detailed explanation: [./docs/programmers_guide/db_faq.md](./docs/programmers_guide/db_faq.md) + Getting in touch ================ diff --git a/cmd/rpcdaemon/README.md b/cmd/rpcdaemon/README.md index 701130a24..305eaf648 100644 --- a/cmd/rpcdaemon/README.md +++ b/cmd/rpcdaemon/README.md @@ -384,7 +384,7 @@ Reduce `--private.api.ratelimit` ### Read DB directly without Json-RPC/Graphql -[./docs/programmers_guide/read_db.md](./docs/programmers_guide/read_db.md) +[./docs/programmers_guide/db_faq.md](./docs/programmers_guide/db_faq.md) ### Batch requests diff --git a/docs/programmers_guide/read_db.md b/docs/programmers_guide/db_faq.md similarity index 63% rename from docs/programmers_guide/read_db.md rename to docs/programmers_guide/db_faq.md index ba6f637be..74135d052 100644 --- a/docs/programmers_guide/read_db.md +++ b/docs/programmers_guide/db_faq.md @@ -1,4 +1,6 @@ -# How to read DB directly - not by Json-RPC/Graphql: +# Database FAQ + +### How to read DB directly - not by Json-RPC/Graphql: There are 2 options exist: @@ -21,8 +23,20 @@ option 2 using - `kv_mdbx.go` Erigon using MDBX database. But any articles in internet about LMDB are also valid for MDBX. -We have Go, Rust and C++ implementations of `RoKV` interface. +We have Go, Rust and C++ implementations of `RoKV` interface. Rationale and Architecture of DB interface: [./../../ethdb/Readme.md](./../../ethdb/Readme.md) -MDBX docs: [erthink.github.io/libmdbx/](https://erthink.github.io/libmdbx/) and [https://github.com/erthink/libmdbx/blob/master/mdbx.h](https://github.com/erthink/libmdbx/blob/master/mdbx.h) +MDBX docs: [erthink.github.io/libmdbx/](https://erthink.github.io/libmdbx/) +and [https://github.com/erthink/libmdbx/blob/master/mdbx.h](https://github.com/erthink/libmdbx/blob/master/mdbx.h) + + +### How RAM used + +Erigon will use all available RAM, but this RAM will not belong to Eroigon’s process. OS will own all this +memory. And OS will maintain hot part of DB in RAM. If OS will need RAM for other programs or for second Erigon instance +OS will manage all the work. This called PageCache. Erigon itself using under 2Gb. So, Erigon will benefit from more +RAM and will use all RAM without re-configuration. Same PageCache can be used by other processes if they run on same +machine by just opening same DB file. For example if RPCDaemon started with —datadir option - it will open db of +Erigon and will use same PageCache (if data A already in RAM because it’s hot and RPCDaemon read it - then it read it +from RAM not from Disk). Shared memory.