Commit Graph

10 Commits

Author SHA1 Message Date
james-prysm
64f64f06bf
Remote Key Manager API(web3signer) (#10302)
* removing flag requirement, can run web3signer without predefined public keys

* placeholders for remote-keymanager-api

* adding proto and accountschangedfeed

* updating generated code

* fix imports

* fixing interface

* adding work in progress apimiddleware code

* started implementing functions for remote keymanager api

* fixing generted code from proto

* fixing protos

* fixing import format

* fixing proto generation again , didn't fix the first time

* fixing imports again

* continuing on implementing functions

* implementing add function

* implementing delete API function

* handling errors for API

* removing unusedcode and fixing format

* fixing bazel

* wip enable --web when running web3signer

* fixing wallet check for web3signer

* fixing apis

* adding list remote keys unit test

* import remote keys test

* delete pubkeys tests

* moving location of tests

* adding unit tests

* adding placeholder functions

* adding more unit tests

* fixing bazel

* fixing build

* fixing already slice issue with unit test

* fixing linting

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/types.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/node/node.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/types.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* adding comment on proto based on review

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* adding generated code based on review

* updating based on feedback

* fixing imports

* fixing formatting

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fixing event call

* fixing dependency

* updating bazel

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/standard_api.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing comment from review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-04-11 16:05:40 -04:00
Mike Neuder
ade7d705ec
Cleanup of Keymanager Deleter interface (#10415)
* Migrating Keymanager account list functionality into each keymanager type

* Addressing review comments

* Adding newline at end of BUILD.bazel

* bazel run //:gazelle -- fix

* account deleter cleanup

* bazel run //:gazelle -- fix

* remove stale logging statement

* adding deleter interface to mock functions

* fixing deepsource findings

* go.sum

* bazel run //:gazelle -- fix

* go mod t-dy -compat=1.17

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-03-30 16:52:54 -05:00
Mike Neuder
002253bba3
Migrating Keymanager account list functionality into each type (#10382)
* Migrating Keymanager account list functionality into each keymanager type

* Addressing review comments

* Adding newline at end of BUILD.bazel

* bazel run //:gazelle -- fix

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-03-22 03:04:09 +00:00
Michael Neuder
27082e2cd2
Adding ExtractKeystores to the Keymanager interface (#10313)
* adding ExtractKeystores to the Keymanager interface

* adding ExtractKeystore to mockKeymanager struct types

* bazel run //:gazelle -- fix

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-03-10 17:19:04 +00:00
james-prysm
59be30e9f4
Web3Signer: metrics, tracing, and code coverage (#10161)
* initial commit for metrics, tracing, and code coverage

* fixing unit test, logs, and naming conventions

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* adding required dependency

* adding more error annotations

* adding in negative test

* fixing bazel build

* gaz

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/internal/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing review comments

* Update validator/keymanager/remote-web3signer/internal/client_test.go

* reverting channel direction

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-01 18:28:41 +00:00
james-prysm
aed6e13498
Web3Signer: CLI implementation (#10056)
* initial commit for cli integration of web3signer

* resolving conflicts and execution

* remove aggregation slot from proto

* rem aggregation slot

* define a sync message block root struct

* fix sync message name

* sync message block root struct

* amend where sync committee block root is used

* altered switch statement to return correct json request by type

* fixing fork data import, types, and unit tests

* reverting unwanted changes

* reverting more unwanted changes

* fixing deepsource issues

* fixing formatting

* more fixes for deepsource and code clean up

* only want to fetch once for fetch validating public keys

* adding more comments

* new unit tests for requests and fixing a mapper issue

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* adjusting comment

* adjusting comment

* fixing import organization

* including more unit tests

* adding new cli edit

* adding in checks for wallet initialize

* adding web3signer flags to main.go

* some how resolved files did not save correctly

* adding in check to make sure web flag only works with types imported and derived

* Update validator/client/sync_committee.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/main.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* reverting changes that accidently got checked in

* reverting

* reverting

* continuing to revert unintenteded changes

* reverting

* removing more unneeded changes

* addressing review comment

* initial refactor

* adding in more clarifying comments

* fixing mock

* resolving desource issues

* addressing gosec scan for helper go file

* addressing gosec

* trying to fix bazel build

* removal of interface to fix build

* fixing maligned struct

* addressing deepsource

* fixing deepsource

* addressing efficiency of type checking

* fixing bazel test failure

* fixing go linter errors

* gaz

* web changes

* add w3signer

* new kind

* proper use

* align

* adding prysm validator flags to help flags list

* addressing root comment

* ci lint

* fixing standardapi tests

* fixing accounts_test after removal of keymanager from rpc server

* fixing more unit tests

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing missed err checks

* fixing mock tests

* fixing gofmt

* unskipping minimal e2e test and removing related TODOs

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* adding some error wrapers to clarify failure point

* fixing bazel build with new error checks

* taking preston's advice to make test fail faster to understand what's going on with the test

* checking if genesis validators root is not zero hash

* adding check for genesis validators root giving zero hash

* fixing missing dependency

* adding check for wallet

* log all

* fixing errors for http responses

* switching marshal to pretty print

* adding pretty sign request test

* fixing base url setting

* adding in check for web3signer and temporary wallet instead of having to open the wallet

* refactoring web3signer to not require wallet

* bazel build fix

* fixing gazelle build

* adding content type of request

* fixing more bazel

* removing unused code

* removing unused comments

* adding skip test back in

* addressing a validation and error message

* fix parse

* body

* fixing logic for datadir

* improving error handling

* show resp

* fix

* sign resp as str

* point of pointer remove

* sign resp

* unmarshal sig resp

* read body as str

* adding more verbose logging

* removing unused result

* fixing unit test

* reconfiguring files to properly nest code and mocks

* fix build issue

* using context when using client function calls

* fixing based on suggestion

* addressing comments

* gaz

* removing defined max timeout

* reverting json print pretty

* Update validator/accounts/wallet_edit.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* removing unneeded code restrictions

* should not introduce new code that may impact existing key manager types

* adjusting comments

* adding in json validation

* running go mod tidy

* some logging

* more logs

* fixing typo

* remove logs

* testing without byte trim

* fixing order or properties

* gaz

* tidy

* reverting some logs

* removing the confusing comments

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing pr comments

* editing bytes test

* Run gazelle update-repos

* run gazelle

* improving unit test coverage

* fixing text

* fixing a potential escaped error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-31 10:44:17 -06:00
james-prysm
8da8855ad5
Web3Signer: Sign Method Implementation (#10084)
* breaking up changes from cli pr

* reverting some changes, adding in changes from remote-web3signer

* adding raul's change

* adding fork info to signing calls

* fixing imports

* gaz

* fixing gofmt

* removing unneeded comment

* Update validator/client/aggregate.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/client/sync_committee.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* addressing comments

* revert proto changes

* proto changes

* reserve

* reserve

* switching to passing slot from passing fork, using slot to find the fork

* removing unneeded check

* fixing missed unit test

* optional

* optional

* gaz

* improving some definitions with constants

* improving some definitions with constants

* rem opt

* rem

* gaz

* moving mocks to its own folder

* adding in bazel field to fix

* fixing type check error

* fixing build

* fixing strict imports

* fixing dependencies

* changing bazel build

* changing bazel build

* changing bazel build

* removing testing only dependency

* removing dependency on testing util package

* update bazel build

* Update checktags_test.go

* Update active_balance.go

* Update sync_committee_minimal.go

* Update sync_committee_mainnet.go

* Update active_balance_disabled.go

* Update committee.go

* Update committee_disabled.go

* Update sync_committee.pb.gw.go

* Update powchain.pb.gw.go

* Update proposer_indices.go

* Update proposer_indices_disabled.go

* Update sync_committee.go

* Update mainnet.go

* Update p2p_messages.pb.gw.go

* Update finalized_block_root_container.pb.gw.go

* Update beacon_block.pb.gw.go

* Update attestation.pb.gw.go

* Update secret_key_test.go

* Update beacon_state.pb.gw.go

* Update version.pb.gw.go

* Update sync_committee.pb.gw.go

* Update sync_committee_disabled.go

* Update mainnet_test.go

* Update minimal.go

* Update signature_test.go

* Update gocast.go

* Update cgo_symbolizer.go

* Update validator.pb.gw.go

* Update beacon_state.pb.gw.go

* Update signature.go

* Update public_key_test.go

* Update minimal_test.go

* Update checktags_test.go

* Update bls_benchmark_test.go

* Update public_key.go

* Update secret_key.go

* Update aliases.go

* Update init.go

* Update stub.go

* Update journald_linux.go

* Update attestation.pb.gw.go

* Update config_utils_develop.go

* Update stub.go

* Update stub.go

* Update beacon_block.pb.gw.go

* Update validator.pb.gw.go

* Update node.pb.gw.go

* Update config_utils_prod.go

* Update journald.go

* Update beacon_block.pb.gw.go

* Update beacon_chain.pb.gw.go

* Update beacon_state.pb.gw.go

* Update events.pb.gw.go

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* addressing comments from review

* updating length of comment

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update stub.go

revert changes

* Update validator/client/aggregate_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing final comments

* fixing gofmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-18 14:31:58 -06:00
james-prysm
f52a214cce
Web3Signer: Object Mapper and Types (#10061)
* initial commit

* fixing some types and renaming file

* initial map function commit and moving to a v1 folder

* adding in remaining mapper functions and comments for types and initial commit on unit tests

* adding readme

* fixing unit tests and errors caught

* fixed a few missed properties and names

* updating error handling to be more descriptive

* Update validator/keymanager/remote-web3signer/client_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/v1/custom_mappers.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/v1/custom_mappers.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fixing comments

* fixing comments

* adding more error checks

* Update validator/keymanager/remote-web3signer/client.go

* fmt and bazel build

* fixing unit tests

* lint needed

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-11 17:15:40 -05:00
james-prysm
d28ae62d02
Web3signer keymanager: key manager implementation (#10016)
* initial commit for work on web3signer-keymanager

* WIP key manager client

* adding in more valid unit tests

* refactoring some of my code to work better with current implementations

* adding comments, some error situations and fixing unit tests

* updating go.mod

* gaz

* addressing sandbox errors on test

* fixing more sand box debug items

* missed yet another error case on my test

* reverting go.mod

* addressing comments

* missed removal of unit test

* removing some deepsource issues

* adjusting based on review comments

* addressing comments

* adding the missed types

* changing public keys to only pull once

* fixing commented code format

* gaz

* Update validator/keymanager/remote-web3signer/keymanager.go

updating comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fixing conflicts

* addressing comments

* fixing deepsource issues

* more deep source issue fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-05 14:32:46 -06:00
james-prysm
00c3a7dcaf
Web3signer: Add an HTTP Client Wrapper to Interact with Remote Server (#9991)
* initial commit for web3signer code work in progress

* adding more functions for web3signer

* more improvements to unit tests and web3signer functions

* fixing unit test

* removing path construction

* fixing failing unit test

* adding more happy path unit tests

* fixing unit tests

* temp removing keymanagerfiles being wip

* removing some comments

* Update validator/keymanager/remote-web3signer/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/keymanager/remote-web3signer/client.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* making errors lowercase

* addressing review comments

* missed resolving a conflict

* addressing deepsource issues

* bazel test and gazelle

* no lint

* deadcode

* addressing comments

* fixing comments

* small fix for readability

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-09 09:33:53 -06:00