prysm-pulse/validator/keymanager/imported/import.go
Raul Jordan 92932ae58e
[Feature] - Slashing Interchange Support (#8024)
* Change LowestSignedProposal to Also Return a Boolean for Slashing Protection (#8020)

* amend to use bools

* ineff assign

* comment

* Update `LowestSignedTargetEpoch` to include exists (#8004)

* Replace highest with lowerest

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Invert equality for saveLowestSourceTargetToDB

* Add eip checks to ensure epochs cant be lower than db ones

* Should be less than equal to

* Check if epoch exists in DB getters

* Revert run time checks

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Export Attesting History for Slashing Interchange Standard (#8027)

* added in att history checks

* logic for export

* export return nil

* test for export atts

* round trip passes first try!

* rem println

* fix up tests

* pass test

* Validate Proposers Are Not Slashable With Regard to Data Within Slasher Interchange JSON (#8031)

* filter slashable blocks and atts in same json stub

* add filter blocks func

* add test for filtering out the bad public keys

* Export Slashing Protection History Via CLI (#8040)

* include cli entrypoint for history exports

* builds properly

* test to confirm we export the data as expected

* abstract helpers properly

* full test suite

* gaz

* better errors

* marshal ident

* Add the additional eip-3076 attestation checks (#7966)

* Replace highest with lowerest

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Invert equality for saveLowestSourceTargetToDB

* Add eip checks to ensure epochs cant be lower than db ones

* Should be less than equal to

* Check if epoch exists in DB getters

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Add EIP-3076 Invariants for Proposer Slashing Protection (#8067)

* add invariant for proposer protection

* write different test cases

* pass tests

* Add EIP-3076 Interchange JSON CLI command to validator (#7880)

* Import JSON CLI

* CLI impotr

* f

* Begin adding new commands in slashing protection

* Move testing helpers to separate packae

* Add command for importing slashing protection JSONs

* fix import cycle

* fix test

* Undo cleaning changes

* Improvements

* Add better prompts

* Fix prompt

* Fix

* Fix

* Fix

* Fix conflict

* Fix

* Fixes

* Fixes

* Fix exported func

* test func

* Fixes

* fix test

* simplify import and standardize with export

* add round trip test

* true integration test works

* fix up comments

* logrus

* better error

* fix build

* build fix

* Update validator/slashing-protection/cli_export.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/slashing-protection/cli_import.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Filter Slashable Attester Public Keys in Slashing Interchange Import (#8051)

* filter slashable attesters from the same JSON

* builds

* fix up initially broken test

* circular dep

* import fix

* giz

* added in attesting history package

* add test for filter slashable attester keys

* pass tests

* Save Slashable Keys to Disk in the Validator Client (#8082)

* begin db funcs

* add in test and bucket

* gaz

* rem changes to import

* ineff assign

* add godoc

* Properly Handle Duplicate Public Key Entries in Slashing Interchange Imports (#8089)

* Prevent Blacklisted Public Keys from Slashing Protection Imports from Having Duties at Runtime (#8084)

* tests on update duties

* ensure the slashable public keys are filtered out from update duties via test

* begin test

* attempt test

* rename for better context

* pass tests

* deep source

* ensure tests pass

* Check for Signing Root Mismatch When Submitting Proposals and Importing Proposals in Slashing Interchange (#8085)

* flexible signing root

* add test

* add tests

* fix test

* Preston's comments

* res tests

* ensure we consider the case for minimum proposals

* pass test

* tests passing

* rem unused code

* Set Empty Epochs in Between Attestations as FAR_FUTURE_EPOCH in Attesting History (#8113)

* set target data

* all tests passing

* ineff assign

* signing root

* Add Slashing Interchange, EIP-3076, Spec Tests to Prysm (#7858)

* Add interchange test framework

* add checks for attestations

* Import genesis root if necessary

* flexible signing root

* add test

* Sync

* fix up test build

* only 3 failing tests now

* two failing

* attempting to debug problems in conformity tests

* include latest changes

* protect test in validator/client passing

* pass tests

* imports

* spec tests passing with bazel

* gh archive link to spectests using tar.gz suffix

* rev

* rev more comment changes

* fix sha

* godoc

* add back save

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Implement Migration for Unattested Epochs in Attesting History Database (#8121)

* migrate attesting history backbone done

* begin migration logic

* implement migration logic

* migration test

* add test

* migration logic

* bazel

* migration to its own file

* Handle empty blocks and attestations in interchange json and sort interchange json by public key (#8132)

* Handle empty blocks and attestations in interchange json

* add test

* sort json

* easier empty arrays

* pass test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* builds

* more tests finally build

* Align Slashing Interchange With Optimized Slashing Protection (#8268)

* attestation history should account for multiple targets per source

* attempt at some fixes

* attempt some test fixes

* experimenting with sorting

* only one more failing test

* tests now pass

* slash protect tests passing

* only few tests now failing

* only spec tests failing now

* spec tests passing

* all tests passing

* helper function for verifying double votes

* use helper

* gaz

* deep source

* tests fixed

* expect specific number of times for domain data calls

* final comments

* Batch Save Imported EIP-3076 Attestations (#8304)

* optimize save

* test added

* add test for sad path

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* revert bad find replace

* add comment to db func

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Shay Zluf <thezluf@gmail.com>
2021-01-22 17:12:22 -06:00

140 lines
5.0 KiB
Go

package imported
import (
"context"
"encoding/hex"
"encoding/json"
"fmt"
"strings"
"github.com/k0kubun/go-ansi"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/shared/bls"
"github.com/prysmaticlabs/prysm/shared/promptutil"
"github.com/prysmaticlabs/prysm/validator/keymanager"
"github.com/schollz/progressbar/v3"
keystorev4 "github.com/wealdtech/go-eth2-wallet-encryptor-keystorev4"
)
// ImportKeystores into the imported keymanager from an external source.
func (km *Keymanager) ImportKeystores(
ctx context.Context,
keystores []*keymanager.Keystore,
importsPassword string,
) error {
decryptor := keystorev4.New()
bar := initializeProgressBar(len(keystores), "Importing accounts...")
keys := map[string]string{}
var err error
for i := 0; i < len(keystores); i++ {
var privKeyBytes []byte
var pubKeyBytes []byte
privKeyBytes, pubKeyBytes, importsPassword, err = km.attemptDecryptKeystore(decryptor, keystores[i], importsPassword)
if err != nil {
return err
}
// if key exists prior to being added then output log that duplicate key was found
if _, ok := keys[string(pubKeyBytes)]; ok {
log.Warnf("Duplicate key in import folder will be ignored: %#x", pubKeyBytes)
}
keys[string(pubKeyBytes)] = string(privKeyBytes)
if err := bar.Add(1); err != nil {
return errors.Wrap(err, "could not add to progress bar")
}
}
privKeys := make([][]byte, 0)
pubKeys := make([][]byte, 0)
for pubKey, privKey := range keys {
pubKeys = append(pubKeys, []byte(pubKey))
privKeys = append(privKeys, []byte(privKey))
}
// Write the accounts to disk into a single keystore.
accountsKeystore, err := km.CreateAccountsKeystore(ctx, privKeys, pubKeys)
if err != nil {
return err
}
encodedAccounts, err := json.MarshalIndent(accountsKeystore, "", "\t")
if err != nil {
return err
}
return km.wallet.WriteFileAtPath(ctx, AccountsPath, AccountsKeystoreFileName, encodedAccounts)
}
// ImportKeypairs directly into the keymanager.
func (km *Keymanager) ImportKeypairs(ctx context.Context, privKeys, pubKeys [][]byte) error {
// Write the accounts to disk into a single keystore.
accountsKeystore, err := km.CreateAccountsKeystore(ctx, privKeys, pubKeys)
if err != nil {
return errors.Wrap(err, "could not import account keypairs")
}
encodedAccounts, err := json.MarshalIndent(accountsKeystore, "", "\t")
if err != nil {
return errors.Wrap(err, "could not marshal accounts keystore into JSON")
}
return km.wallet.WriteFileAtPath(ctx, AccountsPath, AccountsKeystoreFileName, encodedAccounts)
}
// Retrieves the private key and public key from an EIP-2335 keystore file
// by decrypting using a specified password. If the password fails,
// it prompts the user for the correct password until it confirms.
func (km *Keymanager) attemptDecryptKeystore(
enc *keystorev4.Encryptor, keystore *keymanager.Keystore, password string,
) ([]byte, []byte, string, error) {
// Attempt to decrypt the keystore with the specifies password.
var privKeyBytes []byte
var err error
privKeyBytes, err = enc.Decrypt(keystore.Crypto, password)
doesNotDecrypt := err != nil && strings.Contains(err.Error(), "invalid checksum")
for doesNotDecrypt {
password, err = promptutil.PasswordPrompt(
fmt.Sprintf("Password incorrect for key 0x%s, input correct password", keystore.Pubkey), promptutil.NotEmpty,
)
if err != nil {
return nil, nil, "", fmt.Errorf("could not read keystore password: %w", err)
}
privKeyBytes, err = enc.Decrypt(keystore.Crypto, password)
doesNotDecrypt = err != nil && strings.Contains(err.Error(), "invalid checksum")
if err != nil && !strings.Contains(err.Error(), "invalid checksum") {
return nil, nil, "", errors.Wrap(err, "could not decrypt keystore")
}
}
if err != nil && !strings.Contains(err.Error(), "invalid checksum") {
return nil, nil, "", errors.Wrap(err, "could not decrypt keystore")
}
var pubKeyBytes []byte
// Attempt to use the pubkey present in the keystore itself as a field. If unavailable,
// then utilize the public key directly from the private key.
if keystore.Pubkey != "" {
pubKeyBytes, err = hex.DecodeString(keystore.Pubkey)
if err != nil {
return nil, nil, "", errors.Wrap(err, "could not decode pubkey from keystore")
}
} else {
privKey, err := bls.SecretKeyFromBytes(privKeyBytes)
if err != nil {
return nil, nil, "", errors.Wrap(err, "could not initialize private key from bytes")
}
pubKeyBytes = privKey.PublicKey().Marshal()
}
return privKeyBytes, pubKeyBytes, password, nil
}
func initializeProgressBar(numItems int, msg string) *progressbar.ProgressBar {
return progressbar.NewOptions(
numItems,
progressbar.OptionFullWidth(),
progressbar.OptionSetWriter(ansi.NewAnsiStdout()),
progressbar.OptionEnableColorCodes(true),
progressbar.OptionSetTheme(progressbar.Theme{
Saucer: "[green]=[reset]",
SaucerHead: "[green]>[reset]",
SaucerPadding: " ",
BarStart: "[",
BarEnd: "]",
}),
progressbar.OptionOnCompletion(func() { fmt.Println() }),
progressbar.OptionSetDescription(msg),
)
}