We've got a report from a user that erigon fails to run with this error:
```
/opt/erigon/releases/latest/bin/erigon: error while loading shared libraries: libsilkworm_capi.so: cannot open shared object file: No such file or directory
```
On this system erigon is executed using a service-account user which has
no permission to access the build user's $HOME where the
libsilkworm_capi.so resides (inside $GOPATH/pkg/mod).
This adds a support to silkworm-go to look for the library relative to
the executable path on Linux:
d4ec8a8bce
and a new `make DIST=<path> install` command which copies both the
library and erigon binary to any destination.
"whitelisting" mechanism (list of files - stored in DB) - which
protecting us from downloading new files after upgrade/downgrade was
broken. And seems it became over-complicated with time.
I replacing it by 1 persistent flag inside downloader:
"prohibit_new_downloads.lock"
Erigon will turn downloader into this mode after
downloading/verification of first snapshots.
```
//Corner cases:
// - Erigon generated file X with hash H1. User upgraded Erigon. New version has preverified file X with hash H2. Must ignore H2 (don't send to Downloader)
// - Erigon "download once": means restart/upgrade/downgrade must not download files (and will be fast)
// - After "download once" - Erigon will produce and seed new files
```
------
`downloader --seedbox` is never "prohibit new downloads"
Small fix to the script for the scenario where more than one matching
library can be returned.
For example, the command `/sbin/ldconfig -p | grep libstdc++ | awk '{
print $NF }'` can result in
```
/lib/x86_64-linux-gnu/libstdc++.so.6
/lib32/libstdc++.so.6
```
which then fails the check `if [[ ! -L "$link_path" ]]`
Silkworm built on Ubuntu 22 depends on glibc 2.34. In order to run on an
older OS, Silkworm needs to be built and linked with an older glibc, but
to build on an older OS we need a compatible compiler. Silkworm requires
gcc 11+ that is not available on Ubuntu 20 or Debian 11.
To simplify the deployment disable Silkworm support on versions before
Ubuntu 22, Debian 12, and glibc prior to 2.34. The check for Ubuntu and
Debian is explicit, because some Ubuntu 16 installations report glibc
2.35 with ldd, but `go build` still uses an older system one and fails.
This introduces _experimental_ RPC daemon run by embedded Silkworm
library. Same notes as in PR #8353 apply here plus the following ones:
- activated if `http` command-line option is enabled and `silkworm.path`
option is present, nothing more is required (i.e. currently, both block
execution and RPC daemon run by Silkworm when specifying
`silkworm.path`, just to keep things as simple as possible)
- only Execution API endpoints are implemented by Silkworm RPCDaemon,
whilst Engine API endpoints are still served by Erigon RPCDaemon
- some features are still missing, in particular:
- state change notification handling
- custom JSON RPC settings (i.e. Erigon RPC settings are not passed to
Silkworm yet)
This introduces _experimental_ block execution run by embedded Silkworm
API library:
- new command-line option `silkworm.path` to enable the feature by
specifying the path to the Silkworm library
- the Silkworm API shared library is dynamically loaded on-demand
- currently requires to build Silkworm library on the target machine
- available only on Linux at the moment: macOS has issue with [stack
size](https://github.com/golang/go/issues/28024) and Windows would
require [TDM-GCC-64](https://jmeubank.github.io/tdm-gcc/), both need
dedicated effort for an assessment
* Expose C Handle of lmdbTx
* LoadExecutionFunctionPointer
* silkworm ExecuteBlocks
* linter
* CLI flag for SilkwormExecutionFunc
* linter
* Call SilkwormExecutionFunc in SpawnExecuteBlocksStage
* Commit transaction after Silkworm execution
* Fix batch initialization
* Make batch_size uint64_t instead of size_t
* max_block in silkworm_execute_blocks
* More accurate logProgress
* Use CHandle exposed by lmdb-go
* Enable Silkworm on Linux only
* Update lmdb-go
* Error when attempting to use Silkworm not on Linux
* Move SilkwormFlag from cmd/utils/flags.go to turbo/cli/flags.go
* Integrate Silkworm into cmd/integration
* Check against ReaderBuilder/WriterBuilder/ChangeSetHook in Silkworm execution
* Refactor SpawnExecuteBlocksStage
* linters
* Small clean-ups
* Move ChangeSetHook inside executeBlockWithGo
* No need to tamper with batch size in executeBlocksWithSilkworm
* fix
* Fix
* Fixup
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
