Go to file
Paul Hauner 4c7bb4984c Use the forwards iterator more often (#2376)
## Issue Addressed

NA

## Primary Change

When investigating memory usage, I noticed that retrieving a block from an early slot (e.g., slot 900) would cause a sharp increase in the memory footprint (from 400mb to 800mb+) which seemed to be ever-lasting.

After some investigation, I found that the reverse iteration from the head back to that slot was the likely culprit. To counter this, I've switched the `BeaconChain::block_root_at_slot` to use the forwards iterator, instead of the reverse one.

I also noticed that the networking stack is using `BeaconChain::root_at_slot` to check if a peer is relevant (`check_peer_relevance`). Perhaps the steep, seemingly-random-but-consistent increases in memory usage are caused by the use of this function.

Using the forwards iterator with the HTTP API alleviated the sharp increases in memory usage. It also made the response much faster (before it felt like to took 1-2s, now it feels instant).

## Additional Changes

In the process I also noticed that we have two functions for getting block roots:

- `BeaconChain::block_root_at_slot`: returns `None` for a skip slot.
- `BeaconChain::root_at_slot`: returns the previous root for a skip slot.

I unified these two functions into `block_root_at_slot` and added the `WhenSlotSkipped` enum. Now, the caller must be explicit about the skip-slot behaviour when requesting a root. 

Additionally, I replaced `vec![]` with `Vec::with_capacity` in `store::chunked_vector::range_query`. I stumbled across this whilst debugging and made this modification to see what effect it would have (not much). It seems like a decent change to keep around, but I'm not concerned either way.

Also, `BeaconChain::get_ancestor_block_root` is unused, so I got rid of it 🗑️.

## Additional Info

I haven't also done the same for state roots here. Whilst it's possible and a good idea, it's more work since the fwds iterators are presently block-roots-specific.

Whilst there's a few places a reverse iteration of state roots could be triggered (e.g., attestation production, HTTP API), they're no where near as common as the `check_peer_relevance` call. As such, I think we should get this PR merged first, then come back for the state root iters. I made an issue here https://github.com/sigp/lighthouse/issues/2377.
2021-05-31 04:18:20 +00:00
.github Enable Compatibility with Windows (#2333) 2021-05-19 23:05:16 +00:00
account_manager Enable Compatibility with Windows (#2333) 2021-05-19 23:05:16 +00:00
beacon_node Use the forwards iterator more often (#2376) 2021-05-31 04:18:20 +00:00
book Monitoring service api (#2251) 2021-05-26 05:58:41 +00:00
boot_node v1.3.0 (#2310) 2021-04-13 22:46:34 +00:00
common Reduce outbound requests to eth1 endpoints (#2340) 2021-05-31 04:18:18 +00:00
consensus Use the forwards iterator more often (#2376) 2021-05-31 04:18:20 +00:00
crypto VC: accept unknown fields in chain spec (#2277) 2021-03-26 04:53:57 +00:00
lcli Add SensitiveUrl to redact user secrets from endpoints (#2326) 2021-05-04 01:59:51 +00:00
lighthouse Tune GNU malloc (#2299) 2021-05-28 05:59:45 +00:00
remote_signer Enable Compatibility with Windows (#2333) 2021-05-19 23:05:16 +00:00
scripts Add NETWORK_ID variable (#2330) 2021-05-12 00:51:20 +00:00
slasher Enable Compatibility with Windows (#2333) 2021-05-19 23:05:16 +00:00
testing Enable Compatibility with Windows (#2333) 2021-05-19 23:05:16 +00:00
validator_client Monitoring service api (#2251) 2021-05-26 05:58:41 +00:00
.dockerignore Use OS file locks in validator client (#1958) 2020-11-26 11:25:46 +00:00
.editorconfig Add editorconfig template 2019-03-11 15:09:57 +11:00
.gitignore Delete uncompressed genesis states (#2092) 2020-12-16 03:44:05 +00:00
.gitmodules Replace EF tests submodule with a makefile 2019-09-08 04:19:54 +10:00
bors.toml Add Windows to Bors config (#2358) 2021-05-20 00:23:08 +00:00
Cargo.lock Correct issue with dialing peers (#2375) 2021-05-29 07:25:06 +00:00
Cargo.toml Tune GNU malloc (#2299) 2021-05-28 05:59:45 +00:00
CONTRIBUTING.md Update CONTRIBUTING.md (#751) 2020-01-03 10:45:53 +11:00
Cross.toml Ensure RUSTFLAGS is passed through on cross compile (#1529) 2020-08-17 10:06:06 +00:00
Dockerfile Network protocol upgrades (#2345) 2021-05-28 22:02:10 +00:00
Dockerfile.cross Multiarch docker GitHub actions (#2065) 2020-12-09 06:06:37 +00:00
LICENSE Update License to Apache 2.0 2019-04-15 16:47:35 +10:00
Makefile VC: accept unknown fields in chain spec (#2277) 2021-03-26 04:53:57 +00:00
README.md Fix readme typo (#2312) 2021-04-14 02:30:54 +00:00

Lighthouse: Ethereum 2.0

An open-source Ethereum 2.0 client, written in Rust and maintained by Sigma Prime.

Build Status Book Status Chat Badge

Documentation

Banner

Overview

Lighthouse is:

  • Ready for use on Eth2 mainnet.
  • Fully open-source, licensed under Apache 2.0.
  • Security-focused. Fuzzing techniques have been continuously applied and several external security reviews have been performed.
  • Built in Rust, a modern language providing unique safety guarantees and excellent performance (comparable to C++).
  • Funded by various organisations, including Sigma Prime, the Ethereum Foundation, ConsenSys, the Decentralization Foundation and private individuals.
  • Actively involved in the specification and security analysis of the Ethereum 2.0 specification.

Eth2 Deposit Contract

The Lighthouse team acknowledges 0x00000000219ab540356cBB839Cbe05303d7705Fa as the canonical Eth2 deposit contract address.

Documentation

The Lighthouse Book contains information for users and developers.

The Lighthouse team maintains a blog at lighthouse.sigmaprime.io which contains periodical progress updates, roadmap insights and interesting findings.

Branches

Lighthouse maintains two permanent branches:

  • stable: Always points to the latest stable release.
    • This is ideal for most users.
  • unstable: Used for development, contains the latest PRs.
    • Developers should base their PRs on this branch.

Contributing

Lighthouse welcomes contributors.

If you are looking to contribute, please head to the Contributing section of the Lighthouse book.

Contact

The best place for discussion is the Lighthouse Discord server. Alternatively, you may use the sigp/lighthouse gitter.

Sign up to the Lighthouse Development Updates mailing list for email notifications about releases, network status and other important information.

Encrypt sensitive messages using our PGP key.

Donations

Lighthouse is an open-source project and a public good. Funding public goods is hard and we're grateful for the donations we receive from the community via:

  • Gitcoin Grants.
  • Ethereum address: 0x25c4a76E7d118705e7Ea2e9b7d8C59930d8aCD3b (donation.sigmaprime.eth).