Commit Graph

3376 Commits

Author SHA1 Message Date
Michael Sproul
81c9fe3817
Apply store refactor to new fork choice 2020-06-17 15:20:44 +10:00
Michael Sproul
e6f97bf466
Merge remote-tracking branch 'origin/master' into spec-v0.12 2020-06-17 12:34:11 +10:00
Paul Hauner
764cb2d32a
v0.12 fork choice update (#1229)
* Incomplete scraps

* Add progress on new fork choice impl

* Further progress

* First complete compiling version

* Remove chain reference

* Add new lmd_ghost crate

* Start integrating into beacon chain

* Update `milagro_bls` to new release (#1183)

* Update milagro_bls to new release

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* Tidy up fake cryptos

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* move SecretHash to bls and put plaintext back

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* Update state processing for v0.12

* Fix EF test runners for v0.12

* Fix some tests

* Fix broken attestation verification test

* More test fixes

* Rough beacon chain impl working

* Remove fork_choice_2

* Remove checkpoint manager

* Half finished ssz impl

* Add missed file

* Add persistence

* Tidy, fix some compile errors

* Remove RwLock from ProtoArrayForkChoice

* Fix store-based compile errors

* Add comments, tidy

* Move function out of ForkChoice struct

* Start testing

* More testing

* Fix compile error

* Tidy beacon_chain::fork_choice

* Queue attestations from the current slot

* Allow fork choice to handle prior-to-genesis start

* Improve error granularity

* Test attestation dequeuing

* Process attestations during block

* Store target root in fork choice

* Move fork choice verification into new crate

* Update tests

* Consensus updates for v0.12 (#1228)

* Update state processing for v0.12

* Fix EF test runners for v0.12

* Fix some tests

* Fix broken attestation verification test

* More test fixes

* Fix typo found in review

* Add `Block` struct to ProtoArray

* Start fixing get_ancestor

* Add rough progress on testing

* Get fork choice tests working

* Progress with testing

* Fix partialeq impl

* Move slot clock from fc_store

* Improve testing

* Add testing for best justified

* Add clone back to SystemTimeSlotClock

* Add balances test

* Start adding balances cache again

* Wire-in balances cache

* Improve tests

* Remove commented-out tests

* Remove beacon_chain::ForkChoice

* Rename crates

* Update wider codebase to new fork_choice layout

* Move advance_slot in test harness

* Tidy ForkChoice::update_time

* Fix verification tests

* Fix compile error with iter::once

* Fix fork choice tests

* Ensure block attestations are processed

* Fix failing beacon_chain tests

* Add first invalid block check

* Add finalized block check

* Progress with testing, new store builder

* Add fixes to get_ancestor

* Fix old genesis justification test

* Fix remaining fork choice tests

* Change root iteration method

* Move on_verified_block

* Remove unused method

* Start adding attestation verification tests

* Add invalid ffg target test

* Add target epoch test

* Add queued attestation test

* Remove old fork choice verification tests

* Tidy, add test

* Move fork choice lock drop

* Rename BeaconForkChoiceStore

* Add comments, tidy BeaconForkChoiceStore

* Update metrics, rename fork_choice_store.rs

* Remove genesis_block_root from ForkChoice

* Tidy

* Update fork_choice comments

* Tidy, add comments

* Tidy, simplify ForkChoice, fix compile issue

* Tidy, removed dead file

* Increase http request timeout

* Fix failing rest_api test

* Set HTTP timeout back to 5s

* Apply fix to get_ancestor

* Address Michael's comments

* Fix typo

* Revert "Fix broken attestation verification test"

This reverts commit 722cdc903b12611de27916a57eeecfa3224f2279.

Co-authored-by: Kirk Baird <baird.k@outlook.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2020-06-17 11:10:22 +10:00
Adam Szkoda
9db0c28051
Make key value storage abstractions more accurate (#1267)
* Layer do_atomically() abstractions properly

* Reduce allocs and DRY get_key_for_col()

* Parameterize HotColdDB with hot and cold item stores

* -impl Store for MemoryStore

* Replace Store uses with HotColdDB

* Ditch Store trait

* cargo fmt

* Style fix

* Readd missing dep that broke the build
2020-06-16 11:34:04 +10:00
Paul Hauner
6b8c96662f
Avoid pruning when there are no forks (#1268) 2020-06-15 19:04:27 +10:00
Michael Sproul
7818447fd2
Check for unused deps in CI (#1262)
* Check for unused deps in CI

* Bump slashing protection parking_lot version
2020-06-14 10:59:50 +10:00
Lecky Lao
d15ec9b544
Removing validator_client/src/validator_directory.rs as logic has been moved to account_manager/src/validator/deposit.rs (#1266) 2020-06-13 15:57:53 +10:00
Pawan Dhananjay
1a4de898bc
Add explicit bounds for ssz decoding in rpc (#1250)
* Update `milagro_bls` to new release (#1183)

* Update milagro_bls to new release

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* Tidy up fake cryptos

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* move SecretHash to bls and put plaintext back

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* Update v0.12.0 to v0.12.1

* Use ssz types for Request and error types

* Fix errors

* Constrain BlocksByRangeRequest count to MAX_REQUEST_BLOCKS

* Fix issues after rebasing

* Compute bounds for variable ssz containers

* Check ssz bounds before decoding

* Add clarifying comment; fix BlocksByRootRequest min/max

Co-authored-by: Kirk Baird <baird.k@outlook.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
Co-authored-by: Age Manning <Age@AgeManning.com>
2020-06-12 20:38:30 +10:00
Paul Hauner
7bbeca4fa1
Update main REAMDE (#1263)
- Remove beaconcha.in link.
- Update roadmap
2020-06-12 20:38:09 +10:00
Pawan Dhananjay
6622bf9f03
Remove interop (#1230)
* Update `milagro_bls` to new release (#1183)

* Update milagro_bls to new release

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* Tidy up fake cryptos

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* move SecretHash to bls and put plaintext back

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* Update v0.12.0 to v0.12.1

* Remove secio

* Remove ssz encoding for gossipsub

Co-authored-by: Kirk Baird <baird.k@outlook.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
Co-authored-by: Age Manning <Age@AgeManning.com>
2020-06-12 10:06:30 +10:00
Age Manning
320e72e2de
Increase RPC test timeouts (#1261) 2020-06-12 10:05:55 +10:00
Pawan Dhananjay
bb8b88edcf
Use SSZ types in rpc (#1244)
* Update `milagro_bls` to new release (#1183)

* Update milagro_bls to new release

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* Tidy up fake cryptos

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* move SecretHash to bls and put plaintext back

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* Update v0.12.0 to v0.12.1

* Use ssz types for Request and error types

* Fix errors

* Constrain BlocksByRangeRequest count to MAX_REQUEST_BLOCKS

* Fix issues after rebasing

* Address review comments

Co-authored-by: Kirk Baird <baird.k@outlook.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
Co-authored-by: Age Manning <Age@AgeManning.com>
2020-06-12 10:04:50 +10:00
Age Manning
2dfe77a8f9
Handle syncing edge case (#1258) 2020-06-11 12:06:42 +10:00
Michael Sproul
39bf05e3e5
Update v0.12.0 to v0.12.1 (#1259) 2020-06-11 10:07:10 +10:00
Paul Hauner
d2983c13df
Add raspberry pi docs (#1252) 2020-06-10 17:11:57 +10:00
Adam Szkoda
7f036a6e95
Add error handling to iterators (#1243)
* Add error handling to iterators

* Review feedback

* Leverage itertools::process_results() in few places
2020-06-10 09:55:44 +10:00
Michael Sproul
7ce9a252a4
Merge remote-tracking branch 'origin/master' into spec-v0.12 2020-06-09 18:34:44 +10:00
Age Manning
ed4b3ef471
Cleanup logs, increase test timeouts, remove gossipsub tests (#1247)
* Cleanup logs, increase test timeouts, remove gossipsub tests

* Fix compile issue

Co-authored-by: Paul Hauner <paul@paulhauner.com>
2020-06-09 07:27:06 +10:00
Boqin Qin
7baac70056
beacon_node, consensus: fix possible deadlocks when comparing with itself (#1241) 2020-06-09 07:08:54 +10:00
ethDreamer
208f1da81b
Deposit functionality now works with IPC & RPC (#1211)
* Deposit functionality now works with IPC & RPC

* Ran cargo fmt

* Second attempt. Trait objects are killin me

* Made suggested changes

* Added transaction hash log to deposit
2020-06-09 07:08:13 +10:00
Paul Hauner
d9d00cc05d
Update lru, leveldb. Run cargo update (#1249)
* Update lru, leveldb. Run cargo update

* Add cmake to docker image

* Move cmake dep in dockerfile
2020-06-06 16:39:42 +10:00
Adam Szkoda
e20a2deebd
Add first Server Sent Events API endpoint (#1107)
* Add Server Sent Events API endpoint

* Support both event handlers as a transitory measure

* Fix merge conflicts
2020-06-06 16:39:11 +10:00
realbigsean
036096ef61
add retry logic to peer discovery and an expiration time for peers (#1203)
* add retry logic to peer discovery and an expiration time for peers

* Restructure discovery

* Add mac build to CI

* Always return an error for Health when not linux

* Change macos workflow

* Rename macos tests

* Update DiscoverPeers messages to pass Instants. Implement PartialEq for AttServiceMessage

* update discover peer queueing to always check existing messages and extend min_ttl as necessary

* update method name and comment

* Correct merge issues

* Add subnet id check to partialeq, fix discover peer message dups

* fix discover peer message dups

* fix discover peer message dups for real this time

Co-authored-by: Age Manning <Age@AgeManning.com>
Co-authored-by: Paul Hauner <paul@paulhauner.com>
2020-06-05 14:55:03 +10:00
divma
0e37a16927
Super tiny RPC refactor (#1187)
* wip: mwake the request id optional

* make the request_id optional

* cleanup

* address clippy lints inside rpc

* WIP: Separate sent RPC events from received ones

* WIP: Separate sent RPC events from received ones

* cleanup

* Separate request ids from substream ids

* Make RPC's message handling independent of RequestIds

* Change behaviour RPC events to be more outside-crate friendly

* Propage changes across the network + router + processor

* Propage changes across the network + router + processor

* fmt

* "tiny" refactor

* more tiny refactors

* fmt eth2-libp2p

* wip: propagating changes

* wip: propagating changes

* cleaning up

* more cleanup

* fmt

* tests HOT fix

Co-authored-by: Age Manning <Age@AgeManning.com>
2020-06-05 13:07:59 +10:00
Michael Sproul
52d60cce1d
Update attestation gossip verification for v0.12 (#1236) 2020-06-05 11:32:46 +10:00
Pawan Dhananjay
042e80570c
Improve tokio task execution (#1181)
* Add logging on shutdown

* Replace tokio::spawn with handle.spawn

* Upgrade tokio

* Add a task executor

* Beacon chain tasks use task executor

* Validator client tasks use task executor

* Rename runtime_handle to executor

* Add duration histograms; minor fixes

* Cleanup

* Fix logs

* Fix tests

* Remove random file

* Get enr dependency instead of libp2p

* Address some review comments

* Libp2p takes a TaskExecutor

* Ugly fix libp2p tests

* Move TaskExecutor to own file

* Upgrade Dockerfile rust version

* Minor fixes

* Revert "Ugly fix libp2p tests"

This reverts commit 58d4bb690f52de28d893943b7504d2d0c6621429.

* Pretty fix libp2p tests

* Add spawn_without_exit; change Counter to Gauge

* Tidy

* Move log from RuntimeContext to TaskExecutor

* Fix errors

* Replace histogram with int_gauge for async tasks

* Fix todo

* Fix memory leak in test by exiting all spawned tasks at the end
2020-06-04 21:48:05 +10:00
Michael Sproul
fe03ff0f21
Consensus updates for v0.12 (#1228)
* Update state processing for v0.12

* Fix EF test runners for v0.12

* Fix some tests

* Fix broken attestation verification test

* More test fixes

* Fix typo found in review
2020-06-03 14:56:54 +10:00
Kirk Baird
197adeff0b
Update milagro_bls to new release (#1183)
* Update milagro_bls to new release

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* Tidy up fake cryptos

Signed-off-by: Kirk Baird <baird.k@outlook.com>

* move SecretHash to bls and put plaintext back

Signed-off-by: Kirk Baird <baird.k@outlook.com>
2020-06-03 14:56:53 +10:00
Adam Szkoda
ce10db15da
Remove code duplicating stdlib (#1239)
* Get rid of superfluous ReverseBlockRootIterator

* Get rid of superfluous ReverseStateRootIterator and ReverseChainIterator

* cargo fmt
2020-06-02 10:41:42 +10:00
Paul Hauner
a214032e1f
Update docs to suit sigp/docker-compose updates (#1235) 2020-06-01 21:18:48 +10:00
Paul Hauner
723c7cbd27
Ensure new health endpoint builds on MacOS (#1215)
* Add mac build to CI

* Always return an error for Health when not linux

* Change macos workflow

* Rename macos tests

* Disable health API test on Mac

Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2020-06-01 21:18:31 +10:00
Age Manning
cb26ddebb1
Doc updates (#1226)
* Keep wallet name and password file in sync with commands

* Fix create validator example to include required param --count

The current example fails with v0.1.2 fails with "Must supply either --count or --at-most"  This change proposes passing count=1, esp'ly since "Creating another validator is easy" a couple of paragraphs down.

* Fix broken links to wallet-create

Wallet links result in file not found.  May be caused by the lack of `.md`?

* Include node/health in TOC

* Remove unnecessary book files

Co-authored-by: J Burnett <jburnett@users.noreply.github.com>
2020-06-01 11:34:16 +10:00
Adam Szkoda
91cb14ac41
Clean up database abstractions (#1200)
* Remove redundant method

* Pull out a method out of a struct

* More precise db access abstractions

* Move fake trait method out of it

* cargo fmt

* Fix compilation error after refactoring

* Move another fake method out the Store trait

* Get rid of superfluous method

* Fix refactoring bug

* Rename: SimpleStoreItem -> StoreItem

* Get rid of the confusing DiskStore type alias

* Get rid of SimpleDiskStore type alias

* Correction: A method took both self and a ref to Self
2020-06-01 08:13:49 +10:00
Age Manning
08e6b4961d
Drive RPC streams to completion (#1219) 2020-05-29 12:04:08 +10:00
Pawan Dhananjay
d609a3f639
Upgrade to spec compliant noise protocol (#1221) 2020-05-29 12:03:30 +10:00
divma
91a28e7438
Update the RPC handler's keep alive logic (#1220) 2020-05-29 12:03:13 +10:00
Patrick Ventuzelo
812809913d
fix compilation issue using beaconfuzz/eth2fuzz (#1218) 2020-05-28 23:18:25 +10:00
Paul Hauner
5879f84d17
Fix eval order lint in ssz_derive (#1210) 2020-05-28 11:37:40 +10:00
Michael Sproul
7d897a0519
Strip new lines in validator create (#1214) 2020-05-28 11:26:24 +10:00
Rohit Narurkar
6383c95f8b
Strip off newline chars from password (#1199)
* strips off newline chars from password

* adds changes suggested in author review
2020-05-28 11:24:49 +10:00
Paul Hauner
ea4a52984c
Add /node/health endpoint (#1197)
* Start adding health endpoint

* Use psutil more

* Add get_health test

* Expose health to Prom

* Update comments

* Add /node/health to docs

* Update Prom naming
2020-05-28 11:24:24 +10:00
Pawan Dhananjay
58a9f979e0
Upgrade docker rust version (#1207) 2020-05-28 11:17:29 +10:00
Age Manning
61496d8dad
Revert runtime drop (#1198) 2020-05-27 06:38:12 +10:00
Age Manning
5122b2c13a
Allow discovery to be more lenient in enr eth2 fields (#1201)
* Allow discovery to be more lenient in enr eth2 fields

* Correct web3 dep

* Discovery bug fixes
2020-05-27 06:34:15 +10:00
Paul Hauner
8bc82c573d
Update local testnet scripts, fix eth1 sim (#1184)
* Update local testnet scripts

* Add logs when decrypting validators

* Update comment

* Update account manager

* Make random key generation explicit

* Remove unnecessary clap constraint

* Only decrypt voting keypair for eth1 deposit

* Use insecure kdf for insecure keypairs

* Simplify local testnet keygen

* Update local testnet

* Fix eth1 sim

* Add eth1 sim to CI again

* Remove old local testnet docs

* Tidy

* Remove checks for existing validators

* Tidy

* Fix typos
2020-05-26 18:30:44 +10:00
Mehdi Zerouali
d41a9f7aa6
Add cargo audit to CI (#1192) 2020-05-26 17:04:39 +10:00
Age Manning
f72094ca8d
Drop runtime after task completion (#1196)
* Drop runtime after task completion

* Update lighthouse/src/main.rs
2020-05-26 15:34:33 +10:00
Paul Hauner
ad4e5adabc
Integrate Witti testnet (#1193)
* Update for latest master

* Shift delay inside loop

* Clean up genesis service

* Tidy

* Tidy logs

* Address Michael's comments

* Add pre-genesis logging

* Remove est time till genesis

* Fix time formatting

* Tidy

* Update docs for Witti

* Update JS for Witti

* Ensure deposit data is 0x-prefixed hex

* Hard code witti testnet dir

* Add --auto-register warning

* Integrate local sigp testnet source

* Reword warning
2020-05-26 15:31:41 +10:00
Michael Sproul
9718c5db07
Update rusqlite to safer v0.23 (#1195)
Closes #1194
2020-05-26 14:41:25 +10:00
divma
103300c880
Custom net behaviour (#1122)
* expand NetworkBehaviour derive

* add handler placeholder

* add dummy custom handler wrapping the select

* cleanup behaviour's expanded impl of NetworkBehaviour

* cleanup behaviour with macro

* add missing function and clean with macros

* add custom InEvent for Behaviour's handler

* cleanup

* replace InboundProtocol with a "custom" one

* add a delegating handler to put the encapsulate the noice

* partially implement poll for handler

* partially implement poll for handler

* cleanup

* Remove warnings before merge

Co-authored-by: Age Manning <Age@AgeManning.com>
2020-05-26 14:24:38 +10:00