* send proposer data to config
* wip implementation with file and url based config import
* improving logic on get validator index
* fix function
* optimizing function for map and address bug
* fixing log
* update cache if it doesn't exist
* updating flags
* initial unit test scaffold
* fixing validator to call rpc call, removed temporary dependency
* adding the API calls for the runner
* fixing broken build
* fixing deepsource
* fixing interface
* fixing fatal
* fixing more deepsource issues
* adding test placeholders
* updating proposer config to add validation
* changing how if statement throws error
* removing unneeded validation, validating in a different way
* wip improving tests
* more unit test work
* fixing unit test
* fixing unit tests and edge cases
* adding unit tests and adjusting how the config is created
* fixing bazel builds
* fixing proto generation
* fixing imports
* fixing unit tests
* Update cmd/validator/flags/flags.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* updating flags based on comments, fixing unit tests
* fixing bazel
* removing unneeded function
* fixing unit tests
* refactors and unit test fixes based on comments
* fixing bazel build
* refactor the cache out fo the fee recipient function
* adding usecase for multiple fee recipient
* refactor burn name
* Update validator/client/validator.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/client/validator.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/client/validator.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* fixing bug with validator index based on code review
* edited flag descriptions to better communicate usage
* fixing manual reference to flag name
* fixing code review comments
* fixing linting
* Update validator/client/validator.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* addressing comments and renaming functions
* fixing linting
* Update cmd/validator/flags/flags.go
* Update cmd/validator/flags/flags.go
* Update cmd/validator/flags/flags.go
* improving comments
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* imported -> local
* reverting to the state of the PR at eb1e3c3d1
* use changes from develop
* del
* rem patch
* patch
* rename to local
* gazelle
* add back build
* imported rename
* gaz
* local
* merge fix + remove proto changes
* comment revert
* build
* gofmt and one new reference
* gofmt pt2
* Update validator/accounts/wallet_edit.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update validator/rpc/accounts.go
* rename
* gaz
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
* initial commit for cli integration of web3signer
* resolving conflicts and execution
* remove aggregation slot from proto
* rem aggregation slot
* define a sync message block root struct
* fix sync message name
* sync message block root struct
* amend where sync committee block root is used
* altered switch statement to return correct json request by type
* fixing fork data import, types, and unit tests
* reverting unwanted changes
* reverting more unwanted changes
* fixing deepsource issues
* fixing formatting
* more fixes for deepsource and code clean up
* only want to fetch once for fetch validating public keys
* adding more comments
* new unit tests for requests and fixing a mapper issue
* Update validator/client/validator.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/accounts/wallet/wallet.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* adjusting comment
* adjusting comment
* fixing import organization
* including more unit tests
* adding new cli edit
* adding in checks for wallet initialize
* adding web3signer flags to main.go
* some how resolved files did not save correctly
* adding in check to make sure web flag only works with types imported and derived
* Update validator/client/sync_committee.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/client/aggregate.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/accounts/wallet/wallet.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update cmd/validator/wallet/wallet.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update cmd/validator/wallet/wallet.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update cmd/validator/main.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update cmd/validator/flags/flags.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update cmd/validator/flags/flags.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update cmd/validator/wallet/wallet.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update cmd/validator/wallet/wallet.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* reverting changes that accidently got checked in
* reverting
* reverting
* continuing to revert unintenteded changes
* reverting
* removing more unneeded changes
* addressing review comment
* initial refactor
* adding in more clarifying comments
* fixing mock
* resolving desource issues
* addressing gosec scan for helper go file
* addressing gosec
* trying to fix bazel build
* removal of interface to fix build
* fixing maligned struct
* addressing deepsource
* fixing deepsource
* addressing efficiency of type checking
* fixing bazel test failure
* fixing go linter errors
* gaz
* web changes
* add w3signer
* new kind
* proper use
* align
* adding prysm validator flags to help flags list
* addressing root comment
* ci lint
* fixing standardapi tests
* fixing accounts_test after removal of keymanager from rpc server
* fixing more unit tests
* Update cmd/validator/flags/flags.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update cmd/validator/flags/flags.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/client/service.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/client/service.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* addressing missed err checks
* fixing mock tests
* fixing gofmt
* unskipping minimal e2e test and removing related TODOs
* Update testing/endtoend/components/validator.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update testing/endtoend/components/validator.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* adding some error wrapers to clarify failure point
* fixing bazel build with new error checks
* taking preston's advice to make test fail faster to understand what's going on with the test
* checking if genesis validators root is not zero hash
* adding check for genesis validators root giving zero hash
* fixing missing dependency
* adding check for wallet
* log all
* fixing errors for http responses
* switching marshal to pretty print
* adding pretty sign request test
* fixing base url setting
* adding in check for web3signer and temporary wallet instead of having to open the wallet
* refactoring web3signer to not require wallet
* bazel build fix
* fixing gazelle build
* adding content type of request
* fixing more bazel
* removing unused code
* removing unused comments
* adding skip test back in
* addressing a validation and error message
* fix parse
* body
* fixing logic for datadir
* improving error handling
* show resp
* fix
* sign resp as str
* point of pointer remove
* sign resp
* unmarshal sig resp
* read body as str
* adding more verbose logging
* removing unused result
* fixing unit test
* reconfiguring files to properly nest code and mocks
* fix build issue
* using context when using client function calls
* fixing based on suggestion
* addressing comments
* gaz
* removing defined max timeout
* reverting json print pretty
* Update validator/accounts/wallet_edit.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* removing unneeded code restrictions
* should not introduce new code that may impact existing key manager types
* adjusting comments
* adding in json validation
* running go mod tidy
* some logging
* more logs
* fixing typo
* remove logs
* testing without byte trim
* fixing order or properties
* gaz
* tidy
* reverting some logs
* removing the confusing comments
* Update validator/client/aggregate.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/client/aggregate.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* addressing pr comments
* editing bytes test
* Run gazelle update-repos
* run gazelle
* improving unit test coverage
* fixing text
* fixing a potential escaped error
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Bring in rest of the validator chagnes
* Go fmt
* Deepsource
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove gogoproto compiler
* Remove more gogoproto
* Improvements
* Fix gengo
* More scripts
* Gazelle, fix deps
* Fix version and errors
* Fix gocast for arrays
* Fix ethapis
* Fixes
* Fix compile errors
* fix go.mod
* //proto/... builds
* Update for protov2
* temp fix compilation to move on
* Change everything to emptypb.empty
* Add grpc to proto/slashings
* Fix almost all build failures
* Oher build problems
* FIX THIS FUCKING THING
* gaz literally every .bazel
* Final touches
* Final final touches
* Fix proto
* Begin moving proto.Marshal to native
* Fix site_data
* Fixes
* Fix duplicate gateway
* Fix gateway target
* Fix ethapis
* Fixes from review
* Update
* Fix
* Fix status test
* Fix fuzz
* Add isprotoslice to fun
* Change DeepEqual to DeepSSZEqual for proto arrays
* Fix build
* Fix gaz
* Update go
* Fixes
* Fixes
* Add case for nil validators after copy
* Fix cast
* Fix test
* Fix imports
* Go mod
* Only use extension where needed
* Fixes
* Split gateway from gengo
* gaz
* go mod
* Add back hydrated state
* fix hydrate
* Fix proto.clone
* Fies
* Revert "Split gateway from gengo"
This reverts commit 7298bb2054d446e427d9af97e13b8fabe8695085.
* Revert "gaz"
This reverts commit ca952565701a88727e22302d6c8d60ac48d97255.
* Merge all gateway into one target
* go mod
* Gaz
* Add generate v1_gateway files
* run pb again
* goimports
* gaz
* Fix comments
* Fix protos
* Fix PR
* Fix protos
* Update grpc-gateway and ethapis
* Update ethapis and gen-go-cast
* Go tidy
* Reorder
* Fix ethapis
* fix spec tests
* Fix script
* Remove unused import
* Fix fuzz
* Fix gomod
* Update version
* Error if the cloned result is nil
* Handle optional slots
* ADd more empty checks to clone
* Undo fuzz changes
* Fix build.bazel
* Gaz
* Redo fuzz changes
* Undo some eth1data changes
* Update go.mod
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Undo clone beacon state
* Remove gogo proto more and unused v1_gateway
* Add manual fix for nil vals
* Fix gaz
* tidy
* Tidy again
* Add detailed error
* Revert "Add detailed error"
This reverts commit 59bc053dcd59569a54c95b07739d5a379665ec5d.
* Undo varint changes
* Fix nil validators in deposit test
* Commit
* Undo
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Added ordered option to graffiti file
* Updated validator to use Ordered graffiti
* Track graffiti ordered index in db
* Update `ordered` to only emit each graffiti once
Co-authored-by: pinglamb <pinglambs@gmail.com>
* initial implementation
* remove listening for changes from wallet creation
* goimports
* test fix
* more goimports
* listen for changes when initializing wallet through gRPC
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Change LowestSignedProposal to Also Return a Boolean for Slashing Protection (#8020)
* amend to use bools
* ineff assign
* comment
* Update `LowestSignedTargetEpoch` to include exists (#8004)
* Replace highest with lowerest
* Update validator/db/kv/attestation_history_v2.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/db/kv/attestation_history_v2.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Invert equality for saveLowestSourceTargetToDB
* Add eip checks to ensure epochs cant be lower than db ones
* Should be less than equal to
* Check if epoch exists in DB getters
* Revert run time checks
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Export Attesting History for Slashing Interchange Standard (#8027)
* added in att history checks
* logic for export
* export return nil
* test for export atts
* round trip passes first try!
* rem println
* fix up tests
* pass test
* Validate Proposers Are Not Slashable With Regard to Data Within Slasher Interchange JSON (#8031)
* filter slashable blocks and atts in same json stub
* add filter blocks func
* add test for filtering out the bad public keys
* Export Slashing Protection History Via CLI (#8040)
* include cli entrypoint for history exports
* builds properly
* test to confirm we export the data as expected
* abstract helpers properly
* full test suite
* gaz
* better errors
* marshal ident
* Add the additional eip-3076 attestation checks (#7966)
* Replace highest with lowerest
* Update validator/db/kv/attestation_history_v2.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/db/kv/attestation_history_v2.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Invert equality for saveLowestSourceTargetToDB
* Add eip checks to ensure epochs cant be lower than db ones
* Should be less than equal to
* Check if epoch exists in DB getters
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Add EIP-3076 Invariants for Proposer Slashing Protection (#8067)
* add invariant for proposer protection
* write different test cases
* pass tests
* Add EIP-3076 Interchange JSON CLI command to validator (#7880)
* Import JSON CLI
* CLI impotr
* f
* Begin adding new commands in slashing protection
* Move testing helpers to separate packae
* Add command for importing slashing protection JSONs
* fix import cycle
* fix test
* Undo cleaning changes
* Improvements
* Add better prompts
* Fix prompt
* Fix
* Fix
* Fix
* Fix conflict
* Fix
* Fixes
* Fixes
* Fix exported func
* test func
* Fixes
* fix test
* simplify import and standardize with export
* add round trip test
* true integration test works
* fix up comments
* logrus
* better error
* fix build
* build fix
* Update validator/slashing-protection/cli_export.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update validator/slashing-protection/cli_import.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* fmt
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Filter Slashable Attester Public Keys in Slashing Interchange Import (#8051)
* filter slashable attesters from the same JSON
* builds
* fix up initially broken test
* circular dep
* import fix
* giz
* added in attesting history package
* add test for filter slashable attester keys
* pass tests
* Save Slashable Keys to Disk in the Validator Client (#8082)
* begin db funcs
* add in test and bucket
* gaz
* rem changes to import
* ineff assign
* add godoc
* Properly Handle Duplicate Public Key Entries in Slashing Interchange Imports (#8089)
* Prevent Blacklisted Public Keys from Slashing Protection Imports from Having Duties at Runtime (#8084)
* tests on update duties
* ensure the slashable public keys are filtered out from update duties via test
* begin test
* attempt test
* rename for better context
* pass tests
* deep source
* ensure tests pass
* Check for Signing Root Mismatch When Submitting Proposals and Importing Proposals in Slashing Interchange (#8085)
* flexible signing root
* add test
* add tests
* fix test
* Preston's comments
* res tests
* ensure we consider the case for minimum proposals
* pass test
* tests passing
* rem unused code
* Set Empty Epochs in Between Attestations as FAR_FUTURE_EPOCH in Attesting History (#8113)
* set target data
* all tests passing
* ineff assign
* signing root
* Add Slashing Interchange, EIP-3076, Spec Tests to Prysm (#7858)
* Add interchange test framework
* add checks for attestations
* Import genesis root if necessary
* flexible signing root
* add test
* Sync
* fix up test build
* only 3 failing tests now
* two failing
* attempting to debug problems in conformity tests
* include latest changes
* protect test in validator/client passing
* pass tests
* imports
* spec tests passing with bazel
* gh archive link to spectests using tar.gz suffix
* rev
* rev more comment changes
* fix sha
* godoc
* add back save
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Implement Migration for Unattested Epochs in Attesting History Database (#8121)
* migrate attesting history backbone done
* begin migration logic
* implement migration logic
* migration test
* add test
* migration logic
* bazel
* migration to its own file
* Handle empty blocks and attestations in interchange json and sort interchange json by public key (#8132)
* Handle empty blocks and attestations in interchange json
* add test
* sort json
* easier empty arrays
* pass test
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* builds
* more tests finally build
* Align Slashing Interchange With Optimized Slashing Protection (#8268)
* attestation history should account for multiple targets per source
* attempt at some fixes
* attempt some test fixes
* experimenting with sorting
* only one more failing test
* tests now pass
* slash protect tests passing
* only few tests now failing
* only spec tests failing now
* spec tests passing
* all tests passing
* helper function for verifying double votes
* use helper
* gaz
* deep source
* tests fixed
* expect specific number of times for domain data calls
* final comments
* Batch Save Imported EIP-3076 Attestations (#8304)
* optimize save
* test added
* add test for sad path
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* revert bad find replace
* add comment to db func
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Shay Zluf <thezluf@gmail.com>
* implement validator logs stream
* fix test
* tidy
* proto regen
* add logs stream to the beacon node
* beacon logs working
* impl
* pass test
* gaz
* rem lock
* fix space
* fix(grpcHeaders): accept values with equal signs
# What
Before this commit, it was not possible to pass in base64-encoded
content as a header value if it contained an equals sign. This commit
changes the behavior slightly. Rather than ignore key/value pairs where
the value happens to have an equals sign, we assume the first equals
sign delimits the key from the value and pass in the rest of the value
as-is.
Also, instead of printing the header name along with its value, we
print the name, so there is less risk of leaking information into logs
that shouldn't be there.
# Testing
This has not been tested in the context of the full validator client.
Instead, a small example was made to demonstrate the feasibility. The
example is shown here:
```go
package main
import (
"log"
"os"
"strings"
"github.com/davecgh/go-spew/spew"
"github.com/urfave/cli/v2"
)
type Config struct {
GrpcHeadersFlag string
}
func main() {
app := &cli.App{
Action: func(c *cli.Context) error {
for _, hdr := range strings.Split(c.String("grpc-headers"), ",") {
if hdr != "" {
ss := strings.Split(hdr, "=")
spew.Dump(ss[0])
spew.Dump(strings.Join(ss[1:], "="))
}
}
return nil
},
Flags: []cli.Flag{
&cli.StringFlag{
Name: "grpc-headers",
Usage: "A comma-separated list of key value pairs to pass as gRPC headers for all gRPC " +
"calls. Example: --grpc-headers=key=value",
},
},
}
err := app.Run(os.Args)
if err != nil {
log.Fatal(err)
}
}
```
Example invocation:
```command
❯ go run main.go --grpc-headers=key=value,Authorization="Basic $(echo -n hello:world | base64)"
(string) (len=3) "key"
(string) (len=5) "value"
(string) (len=13) "Authorization"
(string) (len=22) "Basic aGVsbG86d29ybGQ="
```
* Adds tests to new gRPC header parsing code
* bazel run //:gazelle
* begin hd wallet refactor
* further simplify the new derived keymanager
* make it almost a full wrapper around an imported keymanager
* fix up the EIP test
* deprecated derived
* fixing keymanager tests
* fix up derived tests
* refactor initialize keymanager
* simplify hd
* pass some tests
* pass accounts list test
* gaz
* regenerate protos without create account privilege
* enforce account recovery on wallet create
* allow accounts delete to work
* remove mentions of accounts create
* resolve comments and go mod
* fix up tests
* build fixes
* remove insecure warning
* revert
* fix proto file
* remove create account message
* gaz
* remove account create
* update web api protos
* fix up imports
* change func sig
* tidy
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* advanced functionality, enable 25th word mnemonic passphrase
* 25th word passphrase
* add test
* added test to ensure differences when using the mnemonic 25th word
* better message
* passing tests
* fix up logic
* rewrite of ProposeExit
# Conflicts:
# validator/accounts/v2/accounts_exit.go
* translate todos to English
* use node client to calculate current epoch
* resolve dependency cycle
* add missing dependencies
* changed required epochs error message
* fix ProposeExit tests
* remove unused metrics
* simplify public-keys flag name
* organize CLI function
* fix incorrect list of exited keys
* fix formatting of non-exited keys
* add exit root to span attributes
* use errors instead of logs
* log info when all exits failed
* do not log stack trace for non-critical errors
* modify test asserts
* use standard way of displaying errors
* remove todo
* add missing function to wallet mock
* gazelle
* gazelle after stash
* move creating clients to separate function
* gazelle
* add in cors for web ui in val client and gateway flag
* Merge branch 'master' into add-default-beacon-gateway
* gateway cors value
* Merge branch 'add-default-beacon-gateway' of github.com:prysmaticlabs/prysm into add-default-beacon-gateway
* fix tests
* Merge refs/heads/master into add-default-beacon-gateway
