Go to file
Michael Sproul e5bf2576f1 Optimise tree hash caching for block production (#2106)
## Proposed Changes

`@potuz` on the Eth R&D Discord observed that Lighthouse blocks on Pyrmont were always arriving at other nodes after at least 1 second. Part of this could be due to processing and slow propagation, but metrics also revealed that the Lighthouse nodes were usually taking 400-600ms to even just produce a block before broadcasting it.

I tracked the slowness down to the lack of a pre-built tree hash cache (THC) on the states being used for block production. This was due to using the head state for block production, which lacks a THC in order to keep fork choice fast (cloning a THC takes at least 30ms for 100k validators). This PR modifies block production to clone a state from the snapshot cache rather than the head, which speeds things up by 200-400ms by avoiding the tree hash cache rebuild. In practice this seems to have cut block production time down to 300ms or less. Ideally we could _remove_ the snapshot from the cache (and save the 30ms), but it is required for when we re-process the block after signing it with the validator client.

## Alternatives

I experimented with 2 alternatives to this approach, before deciding on it:

* Alternative 1: ensure the `head` has a tree hash cache. This is too slow, as it imposes a +30ms hit on fork choice, which currently takes ~5ms (with occasional spikes).
* Alternative 2: use `Arc<BeaconSnapshot>` in the snapshot cache and share snapshots between the cache and the `head`. This made fork choice blazing fast (1ms), and block production the same as in this PR, but had a negative impact on block processing which I don't think is worth it. It ended up being necessary to clone the full state from the snapshot cache during block production, imposing the +30ms penalty there _as well_ as in block production.

In contract, the approach in this PR should only impact block production, and it improves it! Yay for pareto improvements 🎉

## Additional Info

This commit (ac59dfa) is currently running on all the Lighthouse Pyrmont nodes, and I've added a dashboard to the Pyrmont grafana instance with the metrics.

In future work we should optimise the attestation packing, which consumes around 30-60ms and is now a substantial contributor to the total.
2020-12-21 06:29:39 +00:00
.github Revert "add caching to test suite (#2089)" (#2098) 2020-12-16 23:29:07 +00:00
account_manager Improve compile time (#1989) 2020-12-09 01:34:58 +00:00
beacon_node Optimise tree hash caching for block production (#2106) 2020-12-21 06:29:39 +00:00
book Fix default values and --network flag in Voluntary exits book page (#2056) 2020-12-06 22:16:05 +00:00
boot_node Version v1.0.4 (#2073) 2020-12-10 04:01:40 +00:00
common BN Fallback v2 (#2080) 2020-12-18 09:17:03 +00:00
consensus update dependencies (#2032) 2020-12-07 08:20:33 +00:00
crypto Update dependencies (#2084) 2020-12-14 02:28:19 +00:00
lcli BN Fallback v2 (#2080) 2020-12-18 09:17:03 +00:00
lighthouse Version v1.0.4 (#2073) 2020-12-10 04:01:40 +00:00
remote_signer Update dependencies (#2084) 2020-12-14 02:28:19 +00:00
scripts Release v0.3.4 (#1894) 2020-11-13 06:06:35 +00:00
slasher Add slasher broadcast (#2079) 2020-12-16 03:44:01 +00:00
testing BN Fallback v2 (#2080) 2020-12-18 09:17:03 +00:00
validator_client BN Fallback v2 (#2080) 2020-12-18 09:17:03 +00:00
.dockerignore Use OS file locks in validator client (#1958) 2020-11-26 11:25:46 +00:00
.editorconfig Add editorconfig template 2019-03-11 15:09:57 +11:00
.gitignore Delete uncompressed genesis states (#2092) 2020-12-16 03:44:05 +00:00
.gitmodules Replace EF tests submodule with a makefile 2019-09-08 04:19:54 +10:00
bors.toml Remove macos tests (#1687) 2020-09-30 01:27:36 +00:00
Cargo.lock BN Fallback v2 (#2080) 2020-12-18 09:17:03 +00:00
Cargo.toml Add slasher broadcast (#2079) 2020-12-16 03:44:01 +00:00
CONTRIBUTING.md Update CONTRIBUTING.md (#751) 2020-01-03 10:45:53 +11:00
Cross.toml Ensure RUSTFLAGS is passed through on cross compile (#1529) 2020-08-17 10:06:06 +00:00
Dockerfile Improve compile time (#1989) 2020-12-09 01:34:58 +00:00
Dockerfile.cross Multiarch docker GitHub actions (#2065) 2020-12-09 06:06:37 +00:00
LICENSE Update License to Apache 2.0 2019-04-15 16:47:35 +10:00
Makefile Remove audit ignore ws server (#2051) 2020-12-06 23:35:51 +00:00
README.md Update PGP key in README (#1986) 2020-11-30 09:28:54 +00:00

Lighthouse: Ethereum 2.0

An open-source Ethereum 2.0 client, written in Rust and maintained by Sigma Prime.

Build Status Book Status Chat Badge

Documentation

Banner

Overview

Lighthouse is:

  • Ready for use on Eth2 mainnet.
  • Fully open-source, licensed under Apache 2.0.
  • Security-focused. Fuzzing techniques have been continuously applied and several external security reviews have been performed.
  • Built in Rust, a modern language providing unique safety guarantees and excellent performance (comparable to C++).
  • Funded by various organisations, including Sigma Prime, the Ethereum Foundation, ConsenSys, the Decentralization Foundation and private individuals.
  • Actively involved in the specification and security analysis of the Ethereum 2.0 specification.

Eth2 Deposit Contract

The Lighthouse team acknowledges 0x00000000219ab540356cBB839Cbe05303d7705Fa as the canonical Eth2 deposit contract address.

Documentation

The Lighthouse Book contains information for users and developers.

The Lighthouse team maintains a blog at lighthouse.sigmaprime.io which contains periodical progress updates, roadmap insights and interesting findings.

Branches

Lighthouse maintains two permanent branches:

  • stable: Always points to the latest stable release.
    • This is ideal for most users.
  • unstable: Used for development, contains the latest PRs.
    • Developers should base thier PRs on this branch.

Contributing

Lighthouse welcomes contributors.

If you are looking to contribute, please head to the Contributing section of the Lighthouse book.

Contact

The best place for discussion is the Lighthouse Discord server. Alternatively, you may use the sigp/lighthouse gitter.

Sign up to the Lighthouse Development Updates mailing list for email notifications about releases, network status and other important information.

Encrypt sensitive messages using our PGP key.

Donations

Lighthouse is an open-source project and a public good. Funding public goods is hard and we're grateful for the donations we receive from the community via:

  • Gitcoin Grants.
  • Ethereum address: 0x25c4a76E7d118705e7Ea2e9b7d8C59930d8aCD3b (donation.sigmaprime.eth).