Michael Sproul
1d278aaa83
Implement slashing protection interchange format ( #1544 )
...
## Issue Addressed
Implements support for importing and exporting the slashing protection DB interchange format described here:
https://hackmd.io/@sproul/Bk0Y0qdGD
Also closes #1584
## Proposed Changes
* [x] Support for serializing and deserializing the format
* [x] Support for importing and exporting Lighthouse's database
* [x] CLI commands to invoke import and export
* [x] Export to minimal format (required when a minimal format has been previously imported)
* [x] Tests for export to minimal (utilising mixed importing and attestation signing?)
* [x] Tests for import/export of complete format, and import of minimal format
* [x] ~~Prevent attestations with sources less than our max source (Danny's suggestion). Required for the fake attestation that we put in for the minimal format to block attestations from source 0.~~
* [x] Add the concept of a "low watermark" for compatibility with the minimal format
Bonus!
* [x] A fix to a potentially nasty bug involving validators getting re-registered each time the validator client ran! Thankfully, the ordering of keys meant that the validator IDs used for attestations and blocks remained stable -- otherwise we could have had some slashings on our hands! 😱
* [x] Tests to confirm that this bug is indeed vanquished
2020-10-02 01:42:27 +00:00
Pawan Dhananjay
8e20176337
Directory restructure ( #1532 )
...
Closes #1487
Closes #1427
Directory restructure in accordance with #1487 . Also has temporary migration code to move the old directories into new structure.
Also extracts all default directory names and utility functions into a `directory` crate to avoid repetitio.
~Since `validator_definition.yaml` stores absolute paths, users will have to manually change the keystore paths or delete the file to get the validators picked up by the vc.~. `validator_definition.yaml` is migrated as well from the default directories.
Co-authored-by: realbigsean <seananderson33@gmail.com>
Co-authored-by: Paul Hauner <paul@paulhauner.com>
2020-10-01 11:12:35 +10:00
realbigsean
1801dd1a34
Interactive account passwords ( #1623 )
...
## Issue Addressed
#1437
## Proposed Changes
- Make the `--wallet-password` flag optional and creates an interactive prompt if not provided.
- Make the `--wallet-name` flag optional and creates an interactive prompt if not provided.
- Add a minimum password requirement of a 12 character length.
- Update the `--stdin-passwords` flag to `--stdin-inputs` because we have non-password user inputs
## Additional Info
2020-09-23 01:19:54 +00:00
Pawan Dhananjay
87181204d0
Minor documentation fixes ( #1297 )
...
## Issue Addressed
N/A
## Proposed Changes
- Fix a wrong command in the validator generation example.
- Replace occurrences of 'passphrase' with 'password'. This is mostly because I felt that there was a lot of mixing of the two phrases in the documentation and the actual commands which is a bit confusing. Picked 'password' everywhere because it felt more appropriate but I don't mind changing it to 'passphrase' as long it's consistent everywhere.
2020-09-02 04:59:22 +00:00
Paul Hauner
eaa9f9744f
Add EF launchpad import ( #1381 )
...
## Issue Addressed
NA
## Proposed Changes
Adds an integration for keys generated via https://github.com/ethereum/eth2.0-deposit (In reality keys are *actually* generated here: https://github.com/ethereum/eth2.0-deposit-cli ).
## Additional Info
NA
## TODO
- [x] Docs
- [x] Tests
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2020-07-29 04:32:50 +00:00
Paul Hauner
b73c497be2
Support multiple BLS implementations ( #1335 )
...
## Issue Addressed
NA
## Proposed Changes
- Refactor the `bls` crate to support multiple BLS "backends" (e.g., milagro, blst, etc).
- Removes some duplicate, unused code in `common/rest_types/src/validator.rs`.
- Removes the old "upgrade legacy keypairs" functionality (these were unencrypted keys that haven't been supported for a few testnets, no one should be using them anymore).
## Additional Info
Most of the files changed are just inconsequential changes to function names.
## TODO
- [x] Optimization levels
- [x] Infinity point: https://github.com/supranational/blst/issues/11
- [x] Ensure milagro *and* blst are tested via CI
- [x] What to do with unsafe code?
- [x] Test infinity point in signature sets
2020-07-25 02:03:18 +00:00
Kirk Baird
197adeff0b
Update milagro_bls to new release ( #1183 )
...
* Update milagro_bls to new release
Signed-off-by: Kirk Baird <baird.k@outlook.com>
* Tidy up fake cryptos
Signed-off-by: Kirk Baird <baird.k@outlook.com>
* move SecretHash to bls and put plaintext back
Signed-off-by: Kirk Baird <baird.k@outlook.com>
2020-06-03 14:56:53 +10:00
Paul Hauner
c93f9c351b
Improve bls::SecretKey privacy ( #1164 )
...
* Improve bls::SecretKey privacy
* Add missed file
* Remove more methods from bls::SecretKey
* Add as_bytes() to SecretKey, remove as_raw
* Remove as_raw
* Add back as_raw
* Address review comments
2020-05-19 11:23:08 +10:00
Paul Hauner
c571afb8d8
Wallet-based, encrypted key management ( #1138 )
...
* Update hashmap hashset to stable futures
* Adds panic test to hashset delay
* Port remote_beacon_node to stable futures
* Fix lcli merge conflicts
* Non rpc stuff compiles
* Remove padding
* Add error enum, zeroize more things
* Fix comment
* protocol.rs compiles
* Port websockets, timer and notifier to stable futures (#1035 )
* Fix lcli
* Port timer to stable futures
* Fix timer
* Port websocket_server to stable futures
* Port notifier to stable futures
* Add TODOS
* Port remote_beacon_node to stable futures
* Partial eth2-libp2p stable future upgrade
* Finished first round of fighting RPC types
* Further progress towards porting eth2-libp2p adds caching to discovery
* Update behaviour
* Add keystore builder
* Remove keystore stuff from val client
* Add more tests, comments
* RPC handler to stable futures
* Update RPC to master libp2p
* Add more comments, test vectors
* Network service additions
* Progress on improving JSON validation
* More JSON verification
* Start moving JSON into own mod
* Remove old code
* Add more tests, reader/writers
* Tidy
* Move keystore into own file
* Move more logic into keystore file
* Tidy
* Tidy
* Fix the fallback transport construction (#1102 )
* Allow for odd-character hex
* Correct warning
* Remove hashmap delay
* Compiling version of eth2-libp2p
* Update all crates versions
* Fix conversion function and add tests (#1113 )
* Add more json missing field checks
* Use scrypt by default
* Tidy, address comments
* Test path and uuid in vectors
* Fix comment
* Add checks for kdf params
* Enforce empty kdf message
* Port validator_client to stable futures (#1114 )
* Add PH & MS slot clock changes
* Account for genesis time
* Add progress on duties refactor
* Add simple is_aggregator bool to val subscription
* Start work on attestation_verification.rs
* Add progress on ObservedAttestations
* Progress with ObservedAttestations
* Fix tests
* Add observed attestations to the beacon chain
* Add attestation observation to processing code
* Add progress on attestation verification
* Add first draft of ObservedAttesters
* Add more tests
* Add observed attesters to beacon chain
* Add observers to attestation processing
* Add more attestation verification
* Create ObservedAggregators map
* Remove commented-out code
* Add observed aggregators into chain
* Add progress
* Finish adding features to attestation verification
* Ensure beacon chain compiles
* Link attn verification into chain
* Integrate new attn verification in chain
* Remove old attestation processing code
* Start trying to fix beacon_chain tests
* Split adding into pools into two functions
* Add aggregation to harness
* Get test harness working again
* Adjust the number of aggregators for test harness
* Fix edge-case in harness
* Integrate new attn processing in network
* Fix compile bug in validator_client
* Update validator API endpoints
* Fix aggreagation in test harness
* Fix enum thing
* Fix attestation observation bug:
* Patch failing API tests
* Start adding comments to attestation verification
* Remove unused attestation field
* Unify "is block known" logic
* Update comments
* Supress fork choice errors for network processing
* Add todos
* Tidy
* Add gossip attn tests
* Disallow test harness to produce old attns
* Comment out in-progress tests
* Partially address pruning tests
* Fix failing store test
* Add aggregate tests
* Add comments about which spec conditions we check
* Dont re-aggregate
* Split apart test harness attn production
* Fix compile error in network
* Make progress on commented-out test
* Fix skipping attestation test
* Add fork choice verification tests
* Tidy attn tests, remove dead code
* Remove some accidentally added code
* Fix clippy lint
* Rename test file
* Add block tests, add cheap block proposer check
* Rename block testing file
* Add observed_block_producers
* Tidy
* Switch around block signature verification
* Finish block testing
* Remove gossip from signature tests
* First pass of self review
* Fix deviation in spec
* Update test spec tags
* Start moving over to hashset
* Finish moving observed attesters to hashmap
* Move aggregation pool over to hashmap
* Make fc attn borrow again
* Fix rest_api compile error
* Fix missing comments
* Fix monster test
* Uncomment increasing slots test
* Address remaining comments
* Remove unsafe, use cfg test
* Remove cfg test flag
* Fix dodgy comment
* Revert "Update hashmap hashset to stable futures"
This reverts commit d432378a3cc5cd67fc29c0b15b96b886c1323554.
* Revert "Adds panic test to hashset delay"
This reverts commit 281502396fc5b90d9c421a309c2c056982c9525b.
* Ported attestation_service
* Ported duties_service
* Ported fork_service
* More ports
* Port block_service
* Minor fixes
* VC compiles
* Update TODOS
* Borrow self where possible
* Ignore aggregates that are already known.
* Unify aggregator modulo logic
* Fix typo in logs
* Refactor validator subscription logic
* Avoid reproducing selection proof
* Skip HTTP call if no subscriptions
* Rename DutyAndState -> DutyAndProof
* Tidy logs
* Print root as dbg
* Fix compile errors in tests
* Fix compile error in test
* Re-Fix attestation and duties service
* Minor fixes
Co-authored-by: Paul Hauner <paul@paulhauner.com>
* Expose json_keystore mod
* First commits on path derivation
* Progress with implementation
* More progress
* Passing intermediate test vectors
* Tidy, add comments
* Add DerivedKey structs
* Move key derivation into own crate
* Add zeroize structs
* Return error for empty seed
* Add tests
* Tidy
* First commits on path derivation
* Progress with implementation
* Move key derivation into own crate
* Start defining JSON wallet
* Add progress
* Split out encrypt/decrypt
* First commits on path derivation
* Progress with implementation
* More progress
* Passing intermediate test vectors
* Tidy, add comments
* Add DerivedKey structs
* Move key derivation into own crate
* Add zeroize structs
* Return error for empty seed
* Add tests
* Tidy
* Add progress
* Replace some password usage with slice
* First commits on path derivation
* Progress with implementation
* More progress
* Passing intermediate test vectors
* Tidy, add comments
* Add DerivedKey structs
* Move key derivation into own crate
* Add zeroize structs
* Return error for empty seed
* Add tests
* Tidy
* Add progress
* Expose PlainText struct
* First commits on path derivation
* Progress with implementation
* More progress
* Passing intermediate test vectors
* Tidy, add comments
* Add DerivedKey structs
* Move key derivation into own crate
* Add zeroize structs
* Return error for empty seed
* Add tests
* Tidy
* Add builder
* Expose consts, remove Password
* Minor progress
* Expose SALT_SIZE
* First compiling version
* Add test vectors
* Network crate update to stable futures
* Move dbg assert statement
* Port account_manager to stable futures (#1121 )
* Port account_manager to stable futures
* Run async fns in tokio environment
* Port rest_api crate to stable futures (#1118 )
* Port rest_api lib to stable futures
* Reduce tokio features
* Update notifier to stable futures
* Builder update
* Further updates
* Add mnemonic, tidy
* Convert self referential async functions
* Tidy
* Add testing
* Add first attempt at validator_dir
* Present pubkey field
* stable futures fixes (#1124 )
* Fix eth1 update functions
* Fix genesis and client
* Fix beacon node lib
* Return appropriate runtimes from environment
* Fix test rig
* Refactor eth1 service update
* Upgrade simulator to stable futures
* Lighthouse compiles on stable futures
* Add first pass of wallet manager
* Progress with CLI
* Remove println debugging statement
* Tidy output
* Tidy 600 perms
* Update libp2p service, start rpc test upgrade
* Add validator creation flow
* Update network crate for new libp2p
* Start tidying, adding comments
* Update tokio::codec to futures_codec (#1128 )
* Further work towards RPC corrections
* Correct http timeout and network service select
* Add wallet mgr testing
* Shift LockedWallet into own file
* Add comments to fs
* Start integration into VC
* Use tokio runtime for libp2p
* Revert "Update tokio::codec to futures_codec (#1128 )"
This reverts commit e57aea924acf5cbabdcea18895ac07e38a425ed7.
* Upgrade RPC libp2p tests
* Upgrade secio fallback test
* Add lcli keypair upgrade command
* Upgrade gossipsub examples
* Clean up RPC protocol
* Test fixes (#1133 )
* Correct websocket timeout and run on os thread
* Fix network test
* Add --secrets-dir to VC
* Remove --legacy-keys from VC
* Clean up PR
* Correct tokio tcp move attestation service tests
* Upgrade attestation service tests
* Fix sim
* Correct network test
* Correct genesis test
* Start docs
* Add progress for validator generation
* Tidy error messages
* Test corrections
* Log info when block is received
* Modify logs and update attester service events
* Stable futures: fixes to vc, eth1 and account manager (#1142 )
* Add local testnet scripts
* Remove whiteblock script
* Rename local testnet script
* Move spawns onto handle
* Fix VC panic
* Initial fix to block production issue
* Tidy block producer fix
* Tidy further
* Add local testnet clean script
* Run cargo fmt
* Tidy duties service
* Tidy fork service
* Tidy ForkService
* Tidy AttestationService
* Tidy notifier
* Ensure await is not suppressed in eth1
* Ensure await is not suppressed in account_manager
* Use .ok() instead of .unwrap_or(())
* RPC decoding test for proto
* Update discv5 and eth2-libp2p deps
* Run cargo fmt
* Pre-build keystores for sim
* Fix lcli double runtime issue (#1144 )
* Handle stream termination and dialing peer errors
* Correct peer_info variant types
* Add progress on new deposit flow
* Remove unnecessary warnings
* Handle subnet unsubscription removal and improve logigng
* Add logs around ping
* Upgrade discv5 and improve logging
* Handle peer connection status for multiple connections
* Improve network service logging
* Add more incomplete progress
* Improve logging around peer manager
* Upgrade swarm poll centralise peer management
* Identify clients on error
* Fix `remove_peer` in sync (#1150 )
* remove_peer removes from all chains
* Remove logs
* Fix early return from loop
* Improved logging, fix panic
* Partially correct tests
* Add deposit command
* Remove old validator directory
* Start adding AM tests
* Stable futures: Vc sync (#1149 )
* Improve syncing heuristic
* Add comments
* Use safer method for tolerance
* Fix tests
* Binary testing progress
* Progress with CLI tests
* Use constants for flags
* More account manager testing
* Improve CLI tests
* Move upgrade-legacy-keypairs into account man
* Use rayon for VC key generation
* Add comments to `validator_dir`
* Add testing to validator_dir
* Add fix to eth1-sim
* Check errors in eth1-sim
* Fix mutability issue
* Ensure password file ends in .pass
* Add more tests to wallet manager
* Tidy deposit
* Tidy account manager
* Tidy account manager
* Remove panic
* Generate keypairs earlier in sim
* Tidy eth1-sime
* Try to fix eth1 sim
* Address review comments
* Fix typo in CLI command
* Update docs
* Disable eth1 sim
* Remove eth1 sim completely
Co-authored-by: Age Manning <Age@AgeManning.com>
Co-authored-by: pawanjay176 <pawandhananjay@gmail.com>
2020-05-18 19:01:45 +10:00
