Commit Graph

104 Commits

Author SHA1 Message Date
Radosław Kapka
e68b2821c1
REST VC: Subscribe to Beacon API events (#13354)
* Initial code for head event streaming

* handle events and error

* keepalive event

* tests

* generate new mock

* remove single case select

* cleanup

* explain eventByteLimit

* use 2 channels in test

* review

* more review

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-01-04 17:14:45 +00:00
Radosław Kapka
a0ca4a67b0
Remove API Middleware (#13243)
* remove api/gateway/apimiddleware

* fix errors in api/gateway

* remove beacon-chain/rpc/apimiddleware

* fix errors in api/client/beacon

* fix errors in validator/client/beacon-api

* fix errors in beacon-chain/node

* fix errors in validator/node

* fix errors in cmd/prysmctl/validator

* fix errors in testing/endtoend

* fix all other code

* remove comment

* fix tests

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-04 11:55:21 +00:00
james-prysm
27b4e32e1c
HTTP Validator API: /eth/v1/keystores (#13113)
* WIP

* fixing tests

* fixing bazel

* fixing api client

* fixing tests

* fixing more tests and bazel

* fixing trace and more bazel issues

* fixing router path function definitions

* fixing more tests and deep source issues

* adding delete test

* if a route is provided, reregister before the catch all on the middleware.

* fixing linting

* fixing deepsource complaint

* gaz

* more deepsource issues

* fixing missed err check

* changing how routes are registered

* radek reviews

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* fixing unit test after sammy's review

* adding radek's comments

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2023-10-31 16:33:54 +00:00
james-prysm
bfae7f3c9f
HTTP VALIDATOR API: /eth/v1/validator/{pubkey}/voluntary_exit (#13032)
* migrating set validator exit to http only and removing from api middleware

* fixing ineffassign error

* cleaning up middleware

* fixing linting

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* adding more tests based on sammy's comments

* radek's feedback

* adjusting error codes

* one more status change

* fixing unit test

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-16 22:16:20 +00:00
james-prysm
d0c740f477
Registration Cache used by default and other UX changes for Proposer settings (#12456)
* WIP

* WIP

* adding in migration function

* updating mock validator and gaz

* adding descriptive logs

* fixing mocking

* fixing tests

* fixing mock

* adding changes to handle enable builder settings

* fixing tests and edge case

* reduce cognative complexity of function

* further reducing cognative complexity on function

* WIP

* fixing unit test on migration

* adding more tests

* gaz and fix unit test

* fixing deepsource issues

* fixing more deesource issues missed previously

* removing unused reciever name

* WIP fix to migration logic

* fixing loging info

* reverting migration logic, converting logic to address issues discussed on slack, adding unit tests

* adding test for builder setting only not saved to db

* addressing comment

* fixing flag

* removing accidently missed deprecated flags

* rolling back mock on pr

* fixing fmt linting

* updating comments based on feedback

* Update config/features/flags.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* fixing based on feedback on PR

* Update config/validator/service/proposer_settings.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Update validator/client/runner.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* Update validator/db/kv/proposer_settings.go

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>

* adding additional logs to clear up some steps based on feedback

* fixing log

* deepsource

* adding comments based on review feedback

---------

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
2023-06-29 02:49:21 +00:00
james-prysm
d907cae595
persistent validator settings in validator client (#12354)
* WIP

* improving proposer settings store

* WIP persistent validator settings

* WIP persistent validator settings

* changing visibility level

* fixing some deepsource issues

* fixing more deepsource issues

* fixing json marshalling

* fix linting

* fixing tests

* fixing more tests

* fixing more tests

* fixing more tests

* fixing linting

* WIP fixing unit tests

* fixing remaining db tests

* converting json to protobuf

* fixing e2e

* k8s yaml library is used directly

* fixing linting

* fixing broken unit test

* reverting changes on e2e

* fixing linting

* fixing deepsource issue

* resolving some internal comments

* resolving some comments and adding more tests

* adding more unit tests

* gaz

* fixing flaking test

* fixing unit test contamination

* fixing deepsource issue

* resolving review item

* gaz
2023-05-16 14:08:49 -05:00
Radosław Kapka
98949d8075
Block rewards API endpoint (#12020)
Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-03-28 18:44:41 +02:00
terencechain
d17996f8b0
Update to V4 🚀 (#12134)
* Update V3 from V4

* Fix build v3 -> v4

* Update ssz

* Update beacon_chain.pb.go

* Fix formatter import

* Update update-mockgen.sh comment to v4

* Fix conflicts. Pass build and tests

* Fix test
2023-03-17 18:52:56 +00:00
Mike Neuder
cbc2153664
Wallet Create CLI manager integration (#11331)
* Wallet recover CLI Manager migration

* bazel run //:gazelle -- fix

* Wallet create CLI Manager migration

* Wallet recover CLI Manager migration (#11278)

* Wallet recover CLI Manager migration

* bazel run //:gazelle -- fix

* fix lint and build errors

* add TODO to remove duplicate code

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Wallet recover CLI Manager migration

* bazel run //:gazelle -- fix

* Wallet recover CLI Manager migration

* bazel run //:gazelle -- fix

* Wallet recover CLI Manager migration

* bazel run //:gazelle -- fix

* fix lint and build errors

* Wallet recover CLI Manager migration (#11278)

* Wallet recover CLI Manager migration

* bazel run //:gazelle -- fix

* fix lint and build errors

* add TODO to remove duplicate code

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* bazel run //:gazelle -- fix

* rename to ConstructCLIManagerOpts

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2022-09-02 14:56:47 +00:00
Raul Jordan
d077483577
Add V3 Suffix to All Prysm Packages (#11083)
* v3 import renamings

* tidy

* fmt

* rev

* Update beacon-chain/core/epoch/precompute/reward_penalty_test.go

* Update beacon-chain/core/helpers/validators_test.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/alias.go

* Update beacon-chain/db/iface/BUILD.bazel

* Update beacon-chain/db/kv/kv.go

* Update beacon-chain/db/kv/state.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

* Update beacon-chain/rpc/prysm/v1alpha1/validator/attester_test.go

* Update beacon-chain/sync/initial-sync/service.go

* fix deps

* fix bad replacements

* fix bad replacements

* change back

* gohashtree version

* fix deps

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-08-16 12:20:13 +00:00
james-prysm
bd3dfb27f3
BugFix: cli stringSlice bug (#11166)
* fixing stringslice issue and adding one more filter

* fixing linting and improving function based on feedback

* rolling back generic function to check ci

* adding function back in

* testing printing error

* reverting change

* Added gofmt detail flag

* temporary change

* Another flag added

* Revert changes

* Modifed entrypoint

* Another change

* Revert

* Change go image

* Added -e flag

* Switched to alpine image

* Switched go to strech

* Fix typo

* Yet another image added

* Added go alpine

* rolling back generic

* reverting version

* Update go.sum

reverting

* Update go.mod

reverting

* Update Dockerfile

rolling back

* Update entrypoint.sh

rolling back

* Update go.yml

reverting changes

* fixing flag for config

* reverting changes and moving change to another PR

* Update validator/node/node.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* updating based on review

* Update container/slice/slice.go

Co-authored-by: Taranpreet26311 <taran@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-08-08 16:04:34 +00:00
Radosław Kapka
b82e2e7d40
Use prysmaticlabs/fastssz as a direct dependency (#10941)
* Update dependency

* Regenerate SSZ files

* fix BUILD files

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-28 13:03:24 +00:00
Radosław Kapka
b3c3b207c9
Enable fastssz to use vectorized HTR hashing algo (#10819)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-18 12:13:36 +00:00
james-prysm
db687bf56d
Validator client builder support (#10749)
* Startinb builder service and interface

* Get header from builder

* Add get builder block

* Single validator registration

* Add mev-builder http cli flag

* Add method to verify registration signature

* Add builder registration

* Add submit validator registration

* suporting yaml

* fix yaml unmarshaling

* rolling back some changes from unmarshal from file

* adding yaml support

* adding register validator support

* added new validator requests into client/validator

* fixing gofmt

* updating flags and including gas limit, unit tests are still broken

* fixing bazel

* more name changes and fixing unit tests

* fixing unit tests and renaming functions

* fixing unit tests and renaming to match changes

* adding new test for yaml

* fixing bazel linter

* reverting change on validator service proto

* adding clarifying logs

* renaming function name to be more descriptive

* renaming variable

* rolling back some files that will be added from the builder-1 branch

* reverting more

* more reverting

* need placeholder

* need placeholder

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* fixing more unit tests

* fixing more unit tests

* rolling back mockgen

* fixing bazel

* rolling back changes

* removing duplicate function

* fixing client mock

* removing unused type

* fixing missing brace

* fixing bad field name

* fixing bazel

* updating naming

* fixing bazel

* fixing unit test

* fixing bazel linting

* unhandled err

* fixing gofmt

* simplifying name based on feedback

* using corrected function

* moving default fee recipient and gaslimit to beaconconfig

* missing a few constant changes

* fixing bazel

* fixing more missed default renames

* fixing more constants in tests

* fixing bazel

* adding update proposer setting per epoch

* refactoring to reduce complexity

* adding unit test for proposer settings

* Update validator/client/validator.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* trying out renaming based on feedback

* adjusting based on review comments

* making tests more appropriate

* fixing bazel

* updating flag description based on review feedback

* addressing review feedback

* switching to pushing at start of epoch for more time

* adding new unit test and properly throwing error

* switching keys in error to count

* fixing log variable

* resolving conflict

* resolving more conflicts

* adjusting error message

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-06-06 19:32:41 +00:00
james-prysm
184e5be9de
Fee recipient: checksum log (#10664)
* adding checksum check at validator client and beacon node

* adding validation and logs on validator client startup

* moving the log and validation

* fixing unit tests

* adding test for back checksum on validator client

* fixing bazel

* addressing comments

* fixing log display

* Update beacon-chain/node/config.go

* Apply suggestions from code review

* breaking up lines

* fixing unit test

* ugh another fix to unit test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-11 19:36:57 +00:00
james-prysm
df8da80db8
Implement Fee Recipient: CLI flags and Calling Beacon API (#10312)
* send proposer data to config

* wip implementation with file and url based config import

* improving logic on get validator index

* fix function

* optimizing function for map and address bug

* fixing log

* update cache if it doesn't exist

* updating flags

* initial unit test scaffold

* fixing validator to call rpc call, removed temporary dependency

* adding the API calls for the runner

* fixing broken build

* fixing deepsource

* fixing interface

* fixing fatal

* fixing more deepsource issues

* adding test placeholders

* updating proposer config to add validation

* changing how if statement throws error

* removing unneeded validation, validating in a different way

* wip improving tests

* more unit test work

* fixing unit test

* fixing unit tests and edge cases

* adding unit tests and adjusting how the config is created

* fixing bazel builds

* fixing proto generation

* fixing imports

* fixing unit tests

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* updating flags based on comments, fixing unit tests

* fixing bazel

* removing unneeded function

* fixing unit tests

* refactors and unit test fixes based on comments

* fixing bazel build

* refactor the cache out fo the fee recipient function

* adding usecase for multiple fee recipient

* refactor burn name

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fixing bug with validator index based on code review

* edited flag descriptions to better communicate usage

* fixing manual reference to flag name

* fixing code review comments

* fixing linting

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing comments and renaming functions

* fixing linting

* Update cmd/validator/flags/flags.go

* Update cmd/validator/flags/flags.go

* Update cmd/validator/flags/flags.go

* improving comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-03-21 13:48:02 -05:00
james-prysm
c58ce41b3b
Flagutil: utility for json file from directory or url and unmarshal to struct (#10333)
* initial commit

* adding first testcase wip

* fixing test

* adding more unit tests

* adding bazel file

* adding more unit tests and file checks

* addressing comments

* refactoring based on comments

* added bazel

* fixing build
2022-03-10 14:25:26 -06:00
james-prysm
01e9125761
web3signer: url parsing bug (#10278)
* adding in fixes for url

* fixing gazelle

* fixing wrong keymanager kind

* adding required scheme to urls

* fixing another unit test

* removing unused file

* adding new commit to retrigger deepsource ci
2022-02-24 10:24:11 -06:00
Michael Neuder
8b2b109939
Rename imported keymanager to local. (#10155)
* imported -> local

* reverting to the state of the PR at eb1e3c3d1

* use changes from develop

* del

* rem patch

* patch

* rename to local

* gazelle

* add back build

* imported rename

* gaz

* local

* merge fix + remove proto changes

* comment revert

* build

* gofmt and one new reference

* gofmt pt2

* Update validator/accounts/wallet_edit.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/rpc/accounts.go

* rename

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-02-01 19:54:19 +00:00
james-prysm
aed6e13498
Web3Signer: CLI implementation (#10056)
* initial commit for cli integration of web3signer

* resolving conflicts and execution

* remove aggregation slot from proto

* rem aggregation slot

* define a sync message block root struct

* fix sync message name

* sync message block root struct

* amend where sync committee block root is used

* altered switch statement to return correct json request by type

* fixing fork data import, types, and unit tests

* reverting unwanted changes

* reverting more unwanted changes

* fixing deepsource issues

* fixing formatting

* more fixes for deepsource and code clean up

* only want to fetch once for fetch validating public keys

* adding more comments

* new unit tests for requests and fixing a mapper issue

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* adjusting comment

* adjusting comment

* fixing import organization

* including more unit tests

* adding new cli edit

* adding in checks for wallet initialize

* adding web3signer flags to main.go

* some how resolved files did not save correctly

* adding in check to make sure web flag only works with types imported and derived

* Update validator/client/sync_committee.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/main.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* reverting changes that accidently got checked in

* reverting

* reverting

* continuing to revert unintenteded changes

* reverting

* removing more unneeded changes

* addressing review comment

* initial refactor

* adding in more clarifying comments

* fixing mock

* resolving desource issues

* addressing gosec scan for helper go file

* addressing gosec

* trying to fix bazel build

* removal of interface to fix build

* fixing maligned struct

* addressing deepsource

* fixing deepsource

* addressing efficiency of type checking

* fixing bazel test failure

* fixing go linter errors

* gaz

* web changes

* add w3signer

* new kind

* proper use

* align

* adding prysm validator flags to help flags list

* addressing root comment

* ci lint

* fixing standardapi tests

* fixing accounts_test after removal of keymanager from rpc server

* fixing more unit tests

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing missed err checks

* fixing mock tests

* fixing gofmt

* unskipping minimal e2e test and removing related TODOs

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* adding some error wrapers to clarify failure point

* fixing bazel build with new error checks

* taking preston's advice to make test fail faster to understand what's going on with the test

* checking if genesis validators root is not zero hash

* adding check for genesis validators root giving zero hash

* fixing missing dependency

* adding check for wallet

* log all

* fixing errors for http responses

* switching marshal to pretty print

* adding pretty sign request test

* fixing base url setting

* adding in check for web3signer and temporary wallet instead of having to open the wallet

* refactoring web3signer to not require wallet

* bazel build fix

* fixing gazelle build

* adding content type of request

* fixing more bazel

* removing unused code

* removing unused comments

* adding skip test back in

* addressing a validation and error message

* fix parse

* body

* fixing logic for datadir

* improving error handling

* show resp

* fix

* sign resp as str

* point of pointer remove

* sign resp

* unmarshal sig resp

* read body as str

* adding more verbose logging

* removing unused result

* fixing unit test

* reconfiguring files to properly nest code and mocks

* fix build issue

* using context when using client function calls

* fixing based on suggestion

* addressing comments

* gaz

* removing defined max timeout

* reverting json print pretty

* Update validator/accounts/wallet_edit.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* removing unneeded code restrictions

* should not introduce new code that may impact existing key manager types

* adjusting comments

* adding in json validation

* running go mod tidy

* some logging

* more logs

* fixing typo

* remove logs

* testing without byte trim

* fixing order or properties

* gaz

* tidy

* reverting some logs

* removing the confusing comments

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing pr comments

* editing bytes test

* Run gazelle update-repos

* run gazelle

* improving unit test coverage

* fixing text

* fixing a potential escaped error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-31 10:44:17 -06:00
Raul Jordan
424c8f6b46
API Middleware for Keymanager Standard API Endpoints (#9936)
* begin the middleware approach

* attempt middleware

* middleware works in tandem with web ui

* handle delete as well

* delete request

* DELETE working

* tool to perform imports

* functioning

* commentary

* build

* gaz

* smol test

* enable keymanager api use protonames

* edit

* one rule

* rem gw

* Fix custom compiler

(cherry picked from commit 3b1f65919e04ddf7e07c8f60cba1be883a736476)

* gen proto

* imports

* Update validator/node/node.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* remaining comments

* update item

* rpc

* add

* run gateway

* simplify

* rem flag

* deep source

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-07 20:26:21 +00:00
Raul Jordan
13cdb83591
Validator Changes for Optimized Slasher (#9705)
* val changes

* mock

* mocks

* gomock any

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-29 21:25:45 +00:00
Raul Jordan
a9a4bb9163
Move Shared/Testutil into Testing (#9659)
* move testutil

* util pkg

* build

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 18:53:46 +00:00
Raul Jordan
f3d6dbcc1e
Move Shared/Params Into Config/Params (#9642)
* config params into pkg

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-21 19:59:25 +00:00
Raul Jordan
eebcd52ee6
Miscellaneous Packages from Shared Into Proper Folders (#9638)
* slashutil

* builds

* interop

* viz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-21 18:11:16 +00:00
Raul Jordan
11a1f681e0
Move Shared Packages Into Async/ (#9620)
* async packages

* change pkg

* build

* working

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-18 17:26:11 +00:00
Raul Jordan
7dadc780b8
Move Shared Packages into Math/ and IO/ (#9622)
* amend

* building

* build

* userprompt

* imports

* build val

* gaz

* io file

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-17 21:55:24 +00:00
Raul Jordan
d2f74615ab
Move Miscellaneous Shared/ Packages Into Semantic Groups (#9624)
* remove shared network and ip util

* forkutil

* gaz

* build

* gaz

* nogo

* genrule

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-17 19:20:50 +00:00
Raul Jordan
8f0008c45a
Move Related Packages Into API/ Pkg (#9619)
* begin on api pkg

* build

* build

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 19:55:51 +00:00
Raul Jordan
495e92ce9f
Create Runtime Package for Moving Several Packages in Shared/ (#9588)
* add maxprocs

* add prereqs

* add tos and version pkg

* add in all runtime packages

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 09:46:29 +00:00
terence tsao
31a78ab22a
Create config package for Shared/featureconfig (#9593)
* add config/features

* Gazelle

* Gazelle

* Fix build

* Go mod tidy

* active cache

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2021-09-15 01:18:39 +00:00
Raul Jordan
9935ca3733
Move Shared/ Subpackages Into Monitoring/ Folder (#9591)
* add tracing

* monitoring pkg

* move prom

* Add client stats

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-14 20:59:51 +00:00
Raul Jordan
72cfe50eff
Move Validator Accounts Protos Into Proper Place (#9416)
* move into proper namespace

* package naming

* add strict deps

* regen protos

* add web api

* regen

* builds

* rename

* build
2021-08-18 21:24:01 +00:00
Raul Jordan
9145310647
Eliminate Proto V2 Namespace (#9297)
* get rid of v2 in prysm codebase

* replace block2

* builds

* terence comments

* gazelle
2021-07-28 21:23:44 +00:00
Raul Jordan
715aa408e3
Move Web Protos Into Prysm V2 (#9249)
* move web protos

* move web to v2

* update web protos

* gaz

* proto build

* replace mentions of validator v2

* gaz
2021-07-22 19:19:24 +00:00
Preston Van Loon
d6bd619429
Bazel: minimal test build transitions (#9122)
* Add build transitions for minimal config based tests

* respect existing gotags

* clean up import

* Add some commentary

* gaz

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2021-06-30 19:22:09 +00:00
Radosław Kapka
4f3c17cf69
Make the shared/gateway package unaware of consuming services (#9050)
* gateway refactor

* test customizations

* change variable names

* fix BUILD file

* remove unused code

* move mux registration out of gateway

* fix gateway tests

* better name for struct

* fix mock endpoint factory

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 13:35:13 +00:00
ahadda5
8dd2915d3c
Unify gRPC Gateways into Single, Shared lib (#8799)
* moved validator/rpc/gateway under shared,  validator/rpc/web is not visiable to the new shared gateway

* decided to have one lib with two methods if needed, status and stop are the same, however New and Start are not

* moved beacon's gateway to shared and moved the main func to its own /server folder

* goftm

* gofmt

* removed the extra loggin

* reduce visibility

* tighter visibilty of the shared lib for beacon and validator only for now

* fix patterns , ctx needs to be first param

* fix comments

* gofmt

* added enum for the caller id

* added unit test for gateway

* deprecated .WithDialer to .WithContextDialer

* changed the string callerId to uint8, and rearranged the Gateway struct based on the compiler

* fix 1 based on comment

* iota type

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-04 17:20:58 +02:00
Raul Jordan
cdea2debc9
Add Goland Standard "cmd" Pattern for Validator Binaries (#8541)
* validator cmd

* imports

* more imports

* e2e viz

* alias

* use native alias

* add actual

* fix macro

* work on fix e2e

* add viz

* gaz

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-03-02 12:58:40 -06:00
Radosław Kapka
4d28d5e4d2
Listen for account changes only when required (#8503)
* initial implementation

* remove listening for changes from wallet creation

* goimports

* test fix

* more goimports

* listen for changes when initializing wallet through gRPC

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-24 18:05:46 +00:00
Raul Jordan
92932ae58e
[Feature] - Slashing Interchange Support (#8024)
* Change LowestSignedProposal to Also Return a Boolean for Slashing Protection (#8020)

* amend to use bools

* ineff assign

* comment

* Update `LowestSignedTargetEpoch` to include exists (#8004)

* Replace highest with lowerest

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Invert equality for saveLowestSourceTargetToDB

* Add eip checks to ensure epochs cant be lower than db ones

* Should be less than equal to

* Check if epoch exists in DB getters

* Revert run time checks

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Export Attesting History for Slashing Interchange Standard (#8027)

* added in att history checks

* logic for export

* export return nil

* test for export atts

* round trip passes first try!

* rem println

* fix up tests

* pass test

* Validate Proposers Are Not Slashable With Regard to Data Within Slasher Interchange JSON (#8031)

* filter slashable blocks and atts in same json stub

* add filter blocks func

* add test for filtering out the bad public keys

* Export Slashing Protection History Via CLI (#8040)

* include cli entrypoint for history exports

* builds properly

* test to confirm we export the data as expected

* abstract helpers properly

* full test suite

* gaz

* better errors

* marshal ident

* Add the additional eip-3076 attestation checks (#7966)

* Replace highest with lowerest

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Invert equality for saveLowestSourceTargetToDB

* Add eip checks to ensure epochs cant be lower than db ones

* Should be less than equal to

* Check if epoch exists in DB getters

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Add EIP-3076 Invariants for Proposer Slashing Protection (#8067)

* add invariant for proposer protection

* write different test cases

* pass tests

* Add EIP-3076 Interchange JSON CLI command to validator (#7880)

* Import JSON CLI

* CLI impotr

* f

* Begin adding new commands in slashing protection

* Move testing helpers to separate packae

* Add command for importing slashing protection JSONs

* fix import cycle

* fix test

* Undo cleaning changes

* Improvements

* Add better prompts

* Fix prompt

* Fix

* Fix

* Fix

* Fix conflict

* Fix

* Fixes

* Fixes

* Fix exported func

* test func

* Fixes

* fix test

* simplify import and standardize with export

* add round trip test

* true integration test works

* fix up comments

* logrus

* better error

* fix build

* build fix

* Update validator/slashing-protection/cli_export.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/slashing-protection/cli_import.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Filter Slashable Attester Public Keys in Slashing Interchange Import (#8051)

* filter slashable attesters from the same JSON

* builds

* fix up initially broken test

* circular dep

* import fix

* giz

* added in attesting history package

* add test for filter slashable attester keys

* pass tests

* Save Slashable Keys to Disk in the Validator Client (#8082)

* begin db funcs

* add in test and bucket

* gaz

* rem changes to import

* ineff assign

* add godoc

* Properly Handle Duplicate Public Key Entries in Slashing Interchange Imports (#8089)

* Prevent Blacklisted Public Keys from Slashing Protection Imports from Having Duties at Runtime (#8084)

* tests on update duties

* ensure the slashable public keys are filtered out from update duties via test

* begin test

* attempt test

* rename for better context

* pass tests

* deep source

* ensure tests pass

* Check for Signing Root Mismatch When Submitting Proposals and Importing Proposals in Slashing Interchange (#8085)

* flexible signing root

* add test

* add tests

* fix test

* Preston's comments

* res tests

* ensure we consider the case for minimum proposals

* pass test

* tests passing

* rem unused code

* Set Empty Epochs in Between Attestations as FAR_FUTURE_EPOCH in Attesting History (#8113)

* set target data

* all tests passing

* ineff assign

* signing root

* Add Slashing Interchange, EIP-3076, Spec Tests to Prysm (#7858)

* Add interchange test framework

* add checks for attestations

* Import genesis root if necessary

* flexible signing root

* add test

* Sync

* fix up test build

* only 3 failing tests now

* two failing

* attempting to debug problems in conformity tests

* include latest changes

* protect test in validator/client passing

* pass tests

* imports

* spec tests passing with bazel

* gh archive link to spectests using tar.gz suffix

* rev

* rev more comment changes

* fix sha

* godoc

* add back save

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Implement Migration for Unattested Epochs in Attesting History Database (#8121)

* migrate attesting history backbone done

* begin migration logic

* implement migration logic

* migration test

* add test

* migration logic

* bazel

* migration to its own file

* Handle empty blocks and attestations in interchange json and sort interchange json by public key (#8132)

* Handle empty blocks and attestations in interchange json

* add test

* sort json

* easier empty arrays

* pass test

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* builds

* more tests finally build

* Align Slashing Interchange With Optimized Slashing Protection (#8268)

* attestation history should account for multiple targets per source

* attempt at some fixes

* attempt some test fixes

* experimenting with sorting

* only one more failing test

* tests now pass

* slash protect tests passing

* only few tests now failing

* only spec tests failing now

* spec tests passing

* all tests passing

* helper function for verifying double votes

* use helper

* gaz

* deep source

* tests fixed

* expect specific number of times for domain data calls

* final comments

* Batch Save Imported EIP-3076 Attestations (#8304)

* optimize save

* test added

* add test for sad path

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* revert bad find replace

* add comment to db func

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Shay Zluf <thezluf@gmail.com>
2021-01-22 17:12:22 -06:00
Radosław Kapka
5fd03f8fb0
Unify the pattern of using a package-level logger (#8245)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-01-11 20:03:28 +00:00
terence tsao
b4437e6cec
Load graffiti from file (#8041)
* Pass graffati file and   use it

* Visibility

* Parse test

* More proposal tests

* Gazelle

* Add sequential functionality

* fix length check

* Update priorities. Specified -> random -> default

* Log warn instead return err

* Comment

* E2e test

* Comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-04 23:15:12 +00:00
Raul Jordan
14e1f08208
Add Backup Webhooks to All Prysm Services With DBs (#8025)
* integrate backup webhooks

* pass slasher tests

* fix node

* cmd

* gaz

* read test passes

* radek feedback

* added comment

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-12-03 22:28:57 +00:00
Raul Jordan
7449eba612
Refactor HD Wallets for Enhanced Security (#7821)
* begin hd wallet refactor

* further simplify the new derived keymanager

* make it almost a full wrapper around an imported keymanager

* fix up the EIP test

* deprecated derived

* fixing keymanager tests

* fix up derived tests

* refactor initialize keymanager

* simplify hd

* pass some tests

* pass accounts list test

* gaz

* regenerate protos without create account privilege

* enforce account recovery on wallet create

* allow accounts delete to work

* remove mentions of accounts create

* resolve comments and go mod

* fix up tests

* build fixes

* remove insecure warning

* revert

* fix proto file

* remove create account message

* gaz

* remove account create

* update web api protos

* fix up imports

* change func sig

* tidy

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-16 22:26:04 +00:00
Shay Zluf
47daedaf11
Warn missing protection db (#7792)
* Warn user for missing protection db

* better warning message

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-12 19:22:11 +00:00
Victor Farazdagi
2f11e55869
Use t.TempDir() in tests (#7769)
* use t.TempDir()

* remove redundant delete

* simplify setupDB()

* simplify db/testing/setup_db

* fix tests
2020-11-10 22:45:17 +00:00
Shay Zluf
7d1a1643ee
Add validator datadir use (#7660)
* fix empty db prompt

* wording

* fix merge

* prompt only on datadir use

* remove move datadir function

* remove flag

* remove prompt text

* remove added prompt function

* gaz
2020-10-28 12:09:44 -05:00
Raul Jordan
8c2fff3a75
Allow for 25th Word Passphrases in Mnemonics (#7645)
* advanced functionality, enable 25th word mnemonic passphrase

* 25th word passphrase

* add test

* added test to ensure differences when using the mnemonic 25th word

* better message

* passing tests

* fix up logic
2020-10-27 20:51:29 +00:00
Raul Jordan
c59edb3358
Revert "Use validator protection datadir" (#7627)
This reverts commit ab76bdad15.
2020-10-23 17:19:38 +00:00