erigon-pulse/cmd/headers
ledgerwatch 81ea5bab78
More updates to downloader, new p2psentry protocol (#1559)
* Initial commit

* Add sentry gRPC interface

* p2psentry directory

* Update README.md

* Update README.md

* Update README.md

* Add go package

* Correct syntax

* add external downloader interface (#2)

* Add txpool (#3)

* Add private API (#4)

* Invert control.proto, add PeerMinBlock, Separare incoming Tx message into a separate stream (#5)

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

* Separate upload messages into its own stream (#6)

Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>

* Only send changed accounts to listeners (#7)

* Txpool interface doc (#9)

* More additions

* More additions

* Fix locking

* Intermediate

* Fix separation of phases

* Intermediate

* Fix test

* More transformations

* New simplified way of downloading headers

* Fix hard-coded header sync

* Fixed syncing near the tip of the chain

* Add architecture diagram source and picture (#10)

* More fixes

* rename tip to link

* Use preverified hashes instead of preverified headers

* Fix preverified hashes generation

* more parametrisation

* Continue parametrisation

* Fix grpc data limit, interruption of headers stage

* Add ropsten preverified hashes

* Typed hashes (#11)

* Typed hashes

* Fix PeerId

* 64-bit tx nonce

* Disable penalties

* Add goerli settings, bootstrap nodes

* Try to fix goerly sync

* Remove interfaces

* Add proper golang packages, max_block into p2p sentry Status

* Prepare for proto overhaul

* Squashed 'interfaces/' content from commit ce36053c2

git-subtree-dir: interfaces
git-subtree-split: ce36053c24db2f56e48ac752808de60afa1dfb4b

* Change EtherReply to address

* Adaptations to new types

* Switch to new types

* Fixes

* Fix formatting

* Fix lint

* Lint fixes, reverse order in types

* Fix lint

* Fix lint

* Fix lint

* Fix test

* Not supporting eth/66 yet

* Fix shutdown

* Fix lint

* Fix lint

* Fix lint

* return stopped check

Co-authored-by: Artem Vorotnikov <artem@vorotnikov.me>
Co-authored-by: b00ris <b00ris@mail.ru>
Co-authored-by: Alexey Sharp <alexeysharp@Alexeys-iMac.local>
Co-authored-by: lightclient <14004106+lightclient@users.noreply.github.com>
Co-authored-by: canepat <16927169+canepat@users.noreply.github.com>
2021-03-19 21:24:49 +00:00
..
commands More updates to downloader, new p2psentry protocol (#1559) 2021-03-19 21:24:49 +00:00
download More updates to downloader, new p2psentry protocol (#1559) 2021-03-19 21:24:49 +00:00
main.go Bring in refactored p2p sentry interface and adjust downloader for it (#1497) 2021-02-15 18:54:45 +00:00
README.md Better ergonomics for new header/bodies downloader, more test friendly, README (#1507) 2021-02-22 21:59:23 +00:00

How to run the new header/body downloader

In the root of turbo-geth project, use this command to build the program:

make headers

There are two modes in which the program can be run - with external p2p sentry, or with internal p2p sentry (also called combined). Ethereum mainnet configuration is currently hard-coded.

Running with an external p2p sentry

./buid/bin/headers download --filesdir <temporary_file_dir> --chaindata <path_to_database>

The command above specifies two options - directory where temporary files will be kept (only for headers, bodies are downloaded straight into the database), and the directory where the database files will be written. These two options will need to be specified regardless of the mode the program is run. This specific command above assumes and external p2p sentry running on the same computer listening to the port 9091. In order to use a p2p sentry on a different computer, or a different port (or both), the option --sentryAddr can be used. For example:

./buid/bin/headers download --filesdir <temporary_file_dir> --chaindata <path_to_database> --sentryAddr localhost:9999

The command above will expect the p2p sentry running on the same computer, but on the port 9999

Running with an internal p2p sentry

./buid/bin/headers download --filesdir <temporary_file_dir> --chaindata <path_to_database> --combined

The command above will run p2p sentry and the header downloader in the same proccess. In this mode, p2p sentry can be configured using options --nat, --port, --staticpeers, --netrestrict, --discovery.

Running p2p sentry

We are currently testing against two implementations of the p2p sentry - one internal to turbo-geth, and another - written in Rust as a part of rust-ethereum: https://github.com/rust-ethereum/sentry In order to run the internal sentry, use the following command:

./buid/bin/headers sentry

By default, sentry is listening on the localhost:9091. In order to change that (to listen on a different network interface, different port, or both), the option --sentryAddr can be used. As with the internal p2p sentry, options --nat, --port, --staticpeers, --netrestrict, --discovery are also available.