prysm-pulse/validator/accounts/accounts_backup.go
james-prysm aed6e13498
Web3Signer: CLI implementation (#10056)
* initial commit for cli integration of web3signer

* resolving conflicts and execution

* remove aggregation slot from proto

* rem aggregation slot

* define a sync message block root struct

* fix sync message name

* sync message block root struct

* amend where sync committee block root is used

* altered switch statement to return correct json request by type

* fixing fork data import, types, and unit tests

* reverting unwanted changes

* reverting more unwanted changes

* fixing deepsource issues

* fixing formatting

* more fixes for deepsource and code clean up

* only want to fetch once for fetch validating public keys

* adding more comments

* new unit tests for requests and fixing a mapper issue

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* adjusting comment

* adjusting comment

* fixing import organization

* including more unit tests

* adding new cli edit

* adding in checks for wallet initialize

* adding web3signer flags to main.go

* some how resolved files did not save correctly

* adding in check to make sure web flag only works with types imported and derived

* Update validator/client/sync_committee.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/main.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* reverting changes that accidently got checked in

* reverting

* reverting

* continuing to revert unintenteded changes

* reverting

* removing more unneeded changes

* addressing review comment

* initial refactor

* adding in more clarifying comments

* fixing mock

* resolving desource issues

* addressing gosec scan for helper go file

* addressing gosec

* trying to fix bazel build

* removal of interface to fix build

* fixing maligned struct

* addressing deepsource

* fixing deepsource

* addressing efficiency of type checking

* fixing bazel test failure

* fixing go linter errors

* gaz

* web changes

* add w3signer

* new kind

* proper use

* align

* adding prysm validator flags to help flags list

* addressing root comment

* ci lint

* fixing standardapi tests

* fixing accounts_test after removal of keymanager from rpc server

* fixing more unit tests

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing missed err checks

* fixing mock tests

* fixing gofmt

* unskipping minimal e2e test and removing related TODOs

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* adding some error wrapers to clarify failure point

* fixing bazel build with new error checks

* taking preston's advice to make test fail faster to understand what's going on with the test

* checking if genesis validators root is not zero hash

* adding check for genesis validators root giving zero hash

* fixing missing dependency

* adding check for wallet

* log all

* fixing errors for http responses

* switching marshal to pretty print

* adding pretty sign request test

* fixing base url setting

* adding in check for web3signer and temporary wallet instead of having to open the wallet

* refactoring web3signer to not require wallet

* bazel build fix

* fixing gazelle build

* adding content type of request

* fixing more bazel

* removing unused code

* removing unused comments

* adding skip test back in

* addressing a validation and error message

* fix parse

* body

* fixing logic for datadir

* improving error handling

* show resp

* fix

* sign resp as str

* point of pointer remove

* sign resp

* unmarshal sig resp

* read body as str

* adding more verbose logging

* removing unused result

* fixing unit test

* reconfiguring files to properly nest code and mocks

* fix build issue

* using context when using client function calls

* fixing based on suggestion

* addressing comments

* gaz

* removing defined max timeout

* reverting json print pretty

* Update validator/accounts/wallet_edit.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* removing unneeded code restrictions

* should not introduce new code that may impact existing key manager types

* adjusting comments

* adding in json validation

* running go mod tidy

* some logging

* more logs

* fixing typo

* remove logs

* testing without byte trim

* fixing order or properties

* gaz

* tidy

* reverting some logs

* removing the confusing comments

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing pr comments

* editing bytes test

* Run gazelle update-repos

* run gazelle

* improving unit test coverage

* fixing text

* fixing a potential escaped error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-31 10:44:17 -06:00

254 lines
8.4 KiB
Go

package accounts
import (
"archive/zip"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strconv"
"strings"
"github.com/logrusorgru/aurora"
"github.com/manifoldco/promptui"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/cmd/validator/flags"
fieldparams "github.com/prysmaticlabs/prysm/config/fieldparams"
"github.com/prysmaticlabs/prysm/crypto/bls"
"github.com/prysmaticlabs/prysm/encoding/bytesutil"
"github.com/prysmaticlabs/prysm/io/file"
"github.com/prysmaticlabs/prysm/io/prompt"
"github.com/prysmaticlabs/prysm/validator/accounts/iface"
"github.com/prysmaticlabs/prysm/validator/accounts/petnames"
"github.com/prysmaticlabs/prysm/validator/accounts/userprompt"
"github.com/prysmaticlabs/prysm/validator/accounts/wallet"
"github.com/prysmaticlabs/prysm/validator/keymanager"
"github.com/prysmaticlabs/prysm/validator/keymanager/derived"
"github.com/prysmaticlabs/prysm/validator/keymanager/imported"
"github.com/urfave/cli/v2"
)
var (
au = aurora.NewAurora(true)
)
const (
allAccountsText = "All accounts"
archiveFilename = "backup.zip"
backupPromptText = "Enter the directory where your backup.zip file will be written to"
)
// BackupAccountsCli allows users to select validator accounts from their wallet
// and export them as a backup.zip file containing the keys as EIP-2335 compliant
// keystore.json files, which are compatible with importing in other Ethereum consensus clients.
func BackupAccountsCli(cliCtx *cli.Context) error {
w, err := wallet.OpenWalletOrElseCli(cliCtx, func(cliCtx *cli.Context) (*wallet.Wallet, error) {
return nil, wallet.ErrNoWalletFound
})
if err != nil {
return errors.Wrap(err, "could not initialize wallet")
}
// TODO(#9883) - Remove this when we have a better way to handle this.
if w.KeymanagerKind() == keymanager.Remote || w.KeymanagerKind() == keymanager.Web3Signer {
return errors.New(
"remote and web3signer wallets cannot backup accounts",
)
}
km, err := w.InitializeKeymanager(cliCtx.Context, iface.InitKeymanagerConfig{ListenForChanges: false})
if err != nil {
return errors.Wrap(err, ErrCouldNotInitializeKeymanager)
}
pubKeys, err := km.FetchValidatingPublicKeys(cliCtx.Context)
if err != nil {
return errors.Wrap(err, "could not fetch validating public keys")
}
// Input the directory where they wish to backup their accounts.
backupDir, err := userprompt.InputDirectory(cliCtx, backupPromptText, flags.BackupDirFlag)
if err != nil {
return errors.Wrap(err, "could not parse keys directory")
}
// Allow the user to interactively select the accounts to backup or optionally
// provide them via cli flags as a string of comma-separated, hex strings.
filteredPubKeys, err := filterPublicKeysFromUserInput(
cliCtx,
flags.BackupPublicKeysFlag,
pubKeys,
userprompt.SelectAccountsBackupPromptText,
)
if err != nil {
return errors.Wrap(err, "could not filter public keys for backup")
}
// Ask the user for their desired password for their backed up accounts.
backupsPassword, err := prompt.InputPassword(
cliCtx,
flags.BackupPasswordFile,
"Enter a new password for your backed up accounts",
"Confirm new password",
true,
prompt.ValidatePasswordInput,
)
if err != nil {
return errors.Wrap(err, "could not determine password for backed up accounts")
}
var keystoresToBackup []*keymanager.Keystore
switch w.KeymanagerKind() {
case keymanager.Imported:
km, ok := km.(*imported.Keymanager)
if !ok {
return errors.New("could not assert keymanager interface to concrete type")
}
keystoresToBackup, err = km.ExtractKeystores(cliCtx.Context, filteredPubKeys, backupsPassword)
if err != nil {
return errors.Wrap(err, "could not backup accounts for imported keymanager")
}
case keymanager.Derived:
km, ok := km.(*derived.Keymanager)
if !ok {
return errors.New("could not assert keymanager interface to concrete type")
}
keystoresToBackup, err = km.ExtractKeystores(cliCtx.Context, filteredPubKeys, backupsPassword)
if err != nil {
return errors.Wrap(err, "could not backup accounts for derived keymanager")
}
case keymanager.Remote:
return errors.New("backing up keys is not supported for a remote keymanager")
case keymanager.Web3Signer:
return errors.New("backing up keys is not supported for a web3signer keymanager")
default:
return fmt.Errorf(errKeymanagerNotSupported, w.KeymanagerKind())
}
return zipKeystoresToOutputDir(keystoresToBackup, backupDir)
}
// Ask user to select accounts via an interactive userprompt.
func selectAccounts(selectionPrompt string, pubKeys [][fieldparams.BLSPubkeyLength]byte) (filteredPubKeys []bls.PublicKey, err error) {
pubKeyStrings := make([]string, len(pubKeys))
for i, pk := range pubKeys {
name := petnames.DeterministicName(pk[:], "-")
pubKeyStrings[i] = fmt.Sprintf(
"%d | %s | %#x", i, au.BrightGreen(name), au.BrightMagenta(bytesutil.Trunc(pk[:])),
)
}
templates := &promptui.SelectTemplates{
Label: "{{ . }}",
Active: "\U0001F336 {{ .Name | cyan }}",
Inactive: " {{ .Name | cyan }}",
Selected: "\U0001F336 {{ .Name | red | cyan }}",
Details: `
--------- Account ----------
{{ "Name:" | faint }} {{ .Name }}`,
}
var result string
exit := "Done selecting"
results := make([]int, 0)
au := aurora.NewAurora(true)
for result != exit {
p := promptui.Select{
Label: selectionPrompt,
HideSelected: true,
Items: append([]string{exit, allAccountsText}, pubKeyStrings...),
Templates: templates,
}
_, result, err = p.Run()
if err != nil {
return nil, err
}
if result == exit {
fmt.Printf("%s\n", au.BrightRed("Done with selections").Bold())
break
}
if result == allAccountsText {
fmt.Printf("%s\n", au.BrightRed("[Selected all accounts]").Bold())
for i := 0; i < len(pubKeys); i++ {
results = append(results, i)
}
break
}
idx := strings.Index(result, " |")
accountIndexStr := result[:idx]
accountIndex, err := strconv.Atoi(accountIndexStr)
if err != nil {
return nil, err
}
results = append(results, accountIndex)
fmt.Printf("%s %s\n", au.BrightRed("[Selected account]").Bold(), result)
}
// Deduplicate the results.
seen := make(map[int]bool)
for i := 0; i < len(results); i++ {
if _, ok := seen[results[i]]; !ok {
seen[results[i]] = true
}
}
// Filter the public keys based on user input.
filteredPubKeys = make([]bls.PublicKey, 0)
for selectedIndex := range seen {
pk, err := bls.PublicKeyFromBytes(pubKeys[selectedIndex][:])
if err != nil {
return nil, err
}
filteredPubKeys = append(filteredPubKeys, pk)
}
return filteredPubKeys, nil
}
// Zips a list of keystore into respective EIP-2335 keystore.json files and
// writes their zipped format into the specified output directory.
func zipKeystoresToOutputDir(keystoresToBackup []*keymanager.Keystore, outputDir string) error {
if len(keystoresToBackup) == 0 {
return errors.New("nothing to backup")
}
if err := file.MkdirAll(outputDir); err != nil {
return errors.Wrapf(err, "could not create directory at path: %s", outputDir)
}
// Marshal and zip all keystore files together and write the zip file
// to the specified output directory.
archivePath := filepath.Join(outputDir, archiveFilename)
if file.FileExists(archivePath) {
return errors.Errorf("Zip file already exists in directory: %s", archivePath)
}
// We create a new file to store our backup.zip.
zipfile, err := os.Create(filepath.Clean(archivePath))
if err != nil {
return errors.Wrapf(err, "could not create zip file with path: %s", archivePath)
}
defer func() {
if err := zipfile.Close(); err != nil {
log.WithError(err).Error("Could not close zipfile")
}
}()
// Using this zip file, we create a new zip writer which we write
// files to directly from our marshaled keystores.
writer := zip.NewWriter(zipfile)
defer func() {
// We close the zip writer when done.
if err := writer.Close(); err != nil {
log.WithError(err).Error("Could not close zip file after writing")
}
}()
for i, k := range keystoresToBackup {
encodedFile, err := json.MarshalIndent(k, "", "\t")
if err != nil {
return errors.Wrap(err, "could not marshal keystore to JSON file")
}
f, err := writer.Create(fmt.Sprintf("keystore-%d.json", i))
if err != nil {
return errors.Wrap(err, "could not write keystore file to zip")
}
if _, err = f.Write(encodedFile); err != nil {
return errors.Wrap(err, "could not write keystore file contents")
}
}
log.WithField(
"backup-path", archivePath,
).Infof("Successfully backed up %d accounts", len(keystoresToBackup))
return nil
}