lighthouse-pulse/common
Michael Sproul 703c33bdc7 Fix head tracker concurrency bugs (#1771)
## Issue Addressed

Closes #1557

## Proposed Changes

Modify the pruning algorithm so that it mutates the head-tracker _before_ committing the database transaction to disk, and _only if_ all the heads to be removed are still present in the head-tracker (i.e. no concurrent mutations).

In the process of writing and testing this I also had to make a few other changes:

* Use internal mutability for all `BeaconChainHarness` functions (namely the RNG and the graffiti), in order to enable parallel calls (see testing section below).
* Disable logging in harness tests unless the `test_logger` feature is turned on

And chose to make some clean-ups:

* Delete the `NullMigrator`
* Remove type-based configuration for the migrator in favour of runtime config (simpler, less duplicated code)
* Use the non-blocking migrator unless the blocking migrator is required. In the store tests we need the blocking migrator because some tests make asserts about the state of the DB after the migration has run.
* Rename `validators_keypairs` -> `validator_keypairs` in the `BeaconChainHarness`

## Testing

To confirm that the fix worked, I wrote a test using [Hiatus](https://crates.io/crates/hiatus), which can be found here:

https://github.com/michaelsproul/lighthouse/tree/hiatus-issue-1557

That test can't be merged because it inserts random breakpoints everywhere, but if you check out that branch you can run the test with:

```
$ cd beacon_node/beacon_chain
$ cargo test --release --test parallel_tests --features test_logger
```

It should pass, and the log output should show:

```
WARN Pruning deferred because of a concurrent mutation, message: this is expected only very rarely!
```

## Additional Info

This is a backwards-compatible change with no impact on consensus.
2020-10-19 05:58:39 +00:00
..
account_utils Validator dir creation (#1746) 2020-10-08 21:01:32 +00:00
clap_utils Update external deps (#1711) 2020-10-05 08:22:19 +00:00
compare_fields Check for unused deps in CI (#1262) 2020-06-14 10:59:50 +10:00
compare_fields_derive Update external deps (#1711) 2020-10-05 08:22:19 +00:00
deposit_contract Update external deps (#1711) 2020-10-05 08:22:19 +00:00
directory Update external deps (#1711) 2020-10-05 08:22:19 +00:00
eth2 Update external deps (#1711) 2020-10-05 08:22:19 +00:00
eth2_config Bump version to v0.3.0 (#1743) 2020-10-09 02:05:30 +00:00
eth2_interop_keypairs Update external deps (#1711) 2020-10-05 08:22:19 +00:00
eth2_testnet_config Fix head tracker concurrency bugs (#1771) 2020-10-19 05:58:39 +00:00
eth2_wallet_manager Fix clippy warnings (#1385) 2020-07-23 14:18:00 +00:00
hashset_delay Add UPnP support for Lighthouse (#1587) 2020-10-03 10:07:47 +10:00
lighthouse_metrics Update external deps (#1711) 2020-10-05 08:22:19 +00:00
lighthouse_version Bump version to v0.3.0 (#1743) 2020-10-09 02:05:30 +00:00
logging Update external deps (#1711) 2020-10-05 08:22:19 +00:00
lru_cache Keep track of failed head chains and prevent re-lookups (#1534) 2020-08-18 03:54:09 +00:00
rest_types Add UPnP support for Lighthouse (#1587) 2020-10-03 10:07:47 +10:00
slot_clock Implement standard eth2.0 API (#1569) 2020-10-01 11:12:36 +10:00
task_executor Upgrade discovery and restructure task execution (#1693) 2020-10-05 18:45:54 +11:00
test_random_derive Update external deps (#1711) 2020-10-05 08:22:19 +00:00
validator_dir Update external deps (#1711) 2020-10-05 08:22:19 +00:00
warp_utils Update external deps (#1711) 2020-10-05 08:22:19 +00:00
README.md Directory Restructure (#1163) 2020-05-18 21:24:23 +10:00

eth2

Common crates containing eth2-specific logic.