* checkpoint changes
* Update beacon-chain/rpc/validator/status.go
Co-authored-by: Potuz <potuz@potuz.net>
* Update beacon-chain/rpc/validator/status.go
Co-authored-by: Potuz <potuz@potuz.net>
* add in client side tests
* add ordering
* add all new test cases
* gate feature
* handle edge case
* add one more test case
* fatal error
* zahoor's review
* Update validator/client/validator.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update validator/client/validator.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* doppelganger not doppleganger
* preston's review
* add in comment
* change comment
* Fix e2e to only run new flags on the current version
* Fix bug where zero byte public keys were always sent in the request when attestation history existed. Still some tests to fix due to another bug in attester protection AttestationHistoryForPubKey.
* go mod tidy, gazelle
* Increase test size
* fix timeout, change size back to small
Co-authored-by: Potuz <potuz@potuz.net>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Remove gogoproto compiler
* Remove more gogoproto
* Improvements
* Fix gengo
* More scripts
* Gazelle, fix deps
* Fix version and errors
* Fix gocast for arrays
* Fix ethapis
* Fixes
* Fix compile errors
* fix go.mod
* //proto/... builds
* Update for protov2
* temp fix compilation to move on
* Change everything to emptypb.empty
* Add grpc to proto/slashings
* Fix almost all build failures
* Oher build problems
* FIX THIS FUCKING THING
* gaz literally every .bazel
* Final touches
* Final final touches
* Fix proto
* Begin moving proto.Marshal to native
* Fix site_data
* Fixes
* Fix duplicate gateway
* Fix gateway target
* Fix ethapis
* Fixes from review
* Update
* Fix
* Fix status test
* Fix fuzz
* Add isprotoslice to fun
* Change DeepEqual to DeepSSZEqual for proto arrays
* Fix build
* Fix gaz
* Update go
* Fixes
* Fixes
* Add case for nil validators after copy
* Fix cast
* Fix test
* Fix imports
* Go mod
* Only use extension where needed
* Fixes
* Split gateway from gengo
* gaz
* go mod
* Add back hydrated state
* fix hydrate
* Fix proto.clone
* Fies
* Revert "Split gateway from gengo"
This reverts commit 7298bb2054d446e427d9af97e13b8fabe8695085.
* Revert "gaz"
This reverts commit ca952565701a88727e22302d6c8d60ac48d97255.
* Merge all gateway into one target
* go mod
* Gaz
* Add generate v1_gateway files
* run pb again
* goimports
* gaz
* Fix comments
* Fix protos
* Fix PR
* Fix protos
* Update grpc-gateway and ethapis
* Update ethapis and gen-go-cast
* Go tidy
* Reorder
* Fix ethapis
* fix spec tests
* Fix script
* Remove unused import
* Fix fuzz
* Fix gomod
* Update version
* Error if the cloned result is nil
* Handle optional slots
* ADd more empty checks to clone
* Undo fuzz changes
* Fix build.bazel
* Gaz
* Redo fuzz changes
* Undo some eth1data changes
* Update go.mod
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Undo clone beacon state
* Remove gogo proto more and unused v1_gateway
* Add manual fix for nil vals
* Fix gaz
* tidy
* Tidy again
* Add detailed error
* Revert "Add detailed error"
This reverts commit 59bc053dcd59569a54c95b07739d5a379665ec5d.
* Undo varint changes
* Fix nil validators in deposit test
* Commit
* Undo
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* initial implementation
* remove listening for changes from wallet creation
* goimports
* test fix
* more goimports
* listen for changes when initializing wallet through gRPC
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Change LowestSignedProposal to Also Return a Boolean for Slashing Protection (#8020)
* amend to use bools
* ineff assign
* comment
* Update `LowestSignedTargetEpoch` to include exists (#8004)
* Replace highest with lowerest
* Update validator/db/kv/attestation_history_v2.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/db/kv/attestation_history_v2.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Invert equality for saveLowestSourceTargetToDB
* Add eip checks to ensure epochs cant be lower than db ones
* Should be less than equal to
* Check if epoch exists in DB getters
* Revert run time checks
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Export Attesting History for Slashing Interchange Standard (#8027)
* added in att history checks
* logic for export
* export return nil
* test for export atts
* round trip passes first try!
* rem println
* fix up tests
* pass test
* Validate Proposers Are Not Slashable With Regard to Data Within Slasher Interchange JSON (#8031)
* filter slashable blocks and atts in same json stub
* add filter blocks func
* add test for filtering out the bad public keys
* Export Slashing Protection History Via CLI (#8040)
* include cli entrypoint for history exports
* builds properly
* test to confirm we export the data as expected
* abstract helpers properly
* full test suite
* gaz
* better errors
* marshal ident
* Add the additional eip-3076 attestation checks (#7966)
* Replace highest with lowerest
* Update validator/db/kv/attestation_history_v2.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/db/kv/attestation_history_v2.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Invert equality for saveLowestSourceTargetToDB
* Add eip checks to ensure epochs cant be lower than db ones
* Should be less than equal to
* Check if epoch exists in DB getters
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Add EIP-3076 Invariants for Proposer Slashing Protection (#8067)
* add invariant for proposer protection
* write different test cases
* pass tests
* Add EIP-3076 Interchange JSON CLI command to validator (#7880)
* Import JSON CLI
* CLI impotr
* f
* Begin adding new commands in slashing protection
* Move testing helpers to separate packae
* Add command for importing slashing protection JSONs
* fix import cycle
* fix test
* Undo cleaning changes
* Improvements
* Add better prompts
* Fix prompt
* Fix
* Fix
* Fix
* Fix conflict
* Fix
* Fixes
* Fixes
* Fix exported func
* test func
* Fixes
* fix test
* simplify import and standardize with export
* add round trip test
* true integration test works
* fix up comments
* logrus
* better error
* fix build
* build fix
* Update validator/slashing-protection/cli_export.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update validator/slashing-protection/cli_import.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* fmt
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Filter Slashable Attester Public Keys in Slashing Interchange Import (#8051)
* filter slashable attesters from the same JSON
* builds
* fix up initially broken test
* circular dep
* import fix
* giz
* added in attesting history package
* add test for filter slashable attester keys
* pass tests
* Save Slashable Keys to Disk in the Validator Client (#8082)
* begin db funcs
* add in test and bucket
* gaz
* rem changes to import
* ineff assign
* add godoc
* Properly Handle Duplicate Public Key Entries in Slashing Interchange Imports (#8089)
* Prevent Blacklisted Public Keys from Slashing Protection Imports from Having Duties at Runtime (#8084)
* tests on update duties
* ensure the slashable public keys are filtered out from update duties via test
* begin test
* attempt test
* rename for better context
* pass tests
* deep source
* ensure tests pass
* Check for Signing Root Mismatch When Submitting Proposals and Importing Proposals in Slashing Interchange (#8085)
* flexible signing root
* add test
* add tests
* fix test
* Preston's comments
* res tests
* ensure we consider the case for minimum proposals
* pass test
* tests passing
* rem unused code
* Set Empty Epochs in Between Attestations as FAR_FUTURE_EPOCH in Attesting History (#8113)
* set target data
* all tests passing
* ineff assign
* signing root
* Add Slashing Interchange, EIP-3076, Spec Tests to Prysm (#7858)
* Add interchange test framework
* add checks for attestations
* Import genesis root if necessary
* flexible signing root
* add test
* Sync
* fix up test build
* only 3 failing tests now
* two failing
* attempting to debug problems in conformity tests
* include latest changes
* protect test in validator/client passing
* pass tests
* imports
* spec tests passing with bazel
* gh archive link to spectests using tar.gz suffix
* rev
* rev more comment changes
* fix sha
* godoc
* add back save
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Implement Migration for Unattested Epochs in Attesting History Database (#8121)
* migrate attesting history backbone done
* begin migration logic
* implement migration logic
* migration test
* add test
* migration logic
* bazel
* migration to its own file
* Handle empty blocks and attestations in interchange json and sort interchange json by public key (#8132)
* Handle empty blocks and attestations in interchange json
* add test
* sort json
* easier empty arrays
* pass test
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* builds
* more tests finally build
* Align Slashing Interchange With Optimized Slashing Protection (#8268)
* attestation history should account for multiple targets per source
* attempt at some fixes
* attempt some test fixes
* experimenting with sorting
* only one more failing test
* tests now pass
* slash protect tests passing
* only few tests now failing
* only spec tests failing now
* spec tests passing
* all tests passing
* helper function for verifying double votes
* use helper
* gaz
* deep source
* tests fixed
* expect specific number of times for domain data calls
* final comments
* Batch Save Imported EIP-3076 Attestations (#8304)
* optimize save
* test added
* add test for sad path
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* revert bad find replace
* add comment to db func
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Shay Zluf <thezluf@gmail.com>
* implement validator logs stream
* fix test
* tidy
* proto regen
* add logs stream to the beacon node
* beacon logs working
* impl
* pass test
* gaz
* rem lock
* fix space
* att sign validations
* eliminate old cached methods and use a simple approach in the db
* redefined db methods
* db package builds
* add multilock to attest and propose
* gaz
* removed concurrency tests that are no longer relevant
* add cache to db functions for attesting history checks
* passing
* add in feature flag --disable-attesting-history-db-cache
* remove lock
* Revert "remove lock"
This reverts commit b1a65020e406b9fa6e4f57c5cc6a5a0b2a11a240.
* comment
* gaz
* Update waiting validator methods to be context aware
* Enable debug logging for validator in e2e
* invert logic in deferred function, remove for loop
* return early if wait is 0 or less
* overwrite ctx
* t.Stop() chan closure is hanging. Doesnt make sense, so ignoring that cleanup task. It shouldnt happen at runtime anyway
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* fix(grpcHeaders): accept values with equal signs
# What
Before this commit, it was not possible to pass in base64-encoded
content as a header value if it contained an equals sign. This commit
changes the behavior slightly. Rather than ignore key/value pairs where
the value happens to have an equals sign, we assume the first equals
sign delimits the key from the value and pass in the rest of the value
as-is.
Also, instead of printing the header name along with its value, we
print the name, so there is less risk of leaking information into logs
that shouldn't be there.
# Testing
This has not been tested in the context of the full validator client.
Instead, a small example was made to demonstrate the feasibility. The
example is shown here:
```go
package main
import (
"log"
"os"
"strings"
"github.com/davecgh/go-spew/spew"
"github.com/urfave/cli/v2"
)
type Config struct {
GrpcHeadersFlag string
}
func main() {
app := &cli.App{
Action: func(c *cli.Context) error {
for _, hdr := range strings.Split(c.String("grpc-headers"), ",") {
if hdr != "" {
ss := strings.Split(hdr, "=")
spew.Dump(ss[0])
spew.Dump(strings.Join(ss[1:], "="))
}
}
return nil
},
Flags: []cli.Flag{
&cli.StringFlag{
Name: "grpc-headers",
Usage: "A comma-separated list of key value pairs to pass as gRPC headers for all gRPC " +
"calls. Example: --grpc-headers=key=value",
},
},
}
err := app.Run(os.Args)
if err != nil {
log.Fatal(err)
}
}
```
Example invocation:
```command
❯ go run main.go --grpc-headers=key=value,Authorization="Basic $(echo -n hello:world | base64)"
(string) (len=3) "key"
(string) (len=5) "value"
(string) (len=13) "Authorization"
(string) (len=22) "Basic aGVsbG86d29ybGQ="
```
* Adds tests to new gRPC header parsing code
* bazel run //:gazelle
* begin hd wallet refactor
* further simplify the new derived keymanager
* make it almost a full wrapper around an imported keymanager
* fix up the EIP test
* deprecated derived
* fixing keymanager tests
* fix up derived tests
* refactor initialize keymanager
* simplify hd
* pass some tests
* pass accounts list test
* gaz
* regenerate protos without create account privilege
* enforce account recovery on wallet create
* allow accounts delete to work
* remove mentions of accounts create
* resolve comments and go mod
* fix up tests
* build fixes
* remove insecure warning
* revert
* fix proto file
* remove create account message
* gaz
* remove account create
* update web api protos
* fix up imports
* change func sig
* tidy
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* advanced functionality, enable 25th word mnemonic passphrase
* 25th word passphrase
* add test
* added test to ensure differences when using the mnemonic 25th word
* better message
* passing tests
* fix up logic
* rewrite of ProposeExit
# Conflicts:
# validator/accounts/v2/accounts_exit.go
* translate todos to English
* use node client to calculate current epoch
* resolve dependency cycle
* add missing dependencies
* changed required epochs error message
* fix ProposeExit tests
* remove unused metrics
* simplify public-keys flag name
* organize CLI function
* fix incorrect list of exited keys
* fix formatting of non-exited keys
* add exit root to span attributes
* use errors instead of logs
* log info when all exits failed
* do not log stack trace for non-critical errors
* modify test asserts
* use standard way of displaying errors
* remove todo
* add missing function to wallet mock
* gazelle
* gazelle after stash
* move creating clients to separate function
* gazelle
* prepare validator for web usage
* wallet exists func in rpc server
* only initialize wallet if feed exists and wallet also exists
* enabling web via a feed
* wallet creation works but needs to be able to recover from mnemonic via api
* ensure logic works
* Merge branch 'master' into validator-web
* fix up tests
* test file
* wallet rpc tests
* add all tests to create wallet
* Merge branch 'master' into validator-web
* imports
* Merge branch 'validator-web' of github.com:prysmaticlabs/prysm into validator-web
* Merge refs/heads/master into validator-web
* Merge refs/heads/master into validator-web
* begin on dynamic key rescan
* Merge branch 'master' into dynamic-rescan
* begin dynamic rescan
* fsnotify to listen for rescan dir changes
* recheck for slashing protection
* lint
* Merge branch 'master' into dynamic-rescan
* less aggressive recheck interval
* imports
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* resolve confs
* listen for file changes for accounts file
* reload accounts from keystore
* begin fixing rescan test
* add event feed
* fix confs
* fix conf
* fix broken tests
* Merge branch 'master' into dynamic-rescan
* simplify lines
* do nothing if no subscribers
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* fix tests
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* gaz
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* ident
* Update WORKSPACE
* gaz
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* add keys on service start
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* ensure debounce util works
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* complete refresh, debounce test, and ensure works at runtime
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* imports and remove log
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* Merge refs/heads/master into dynamic-rescan
* resolve confs
* fix up e2e tests
* Merge branch 'dynamic-rescan' of github.com:prysmaticlabs/prysm into dynamic-rescan
* fix up e2e
* Merge refs/heads/master into dynamic-rescan
* Revert "Revert "Update fastssz" (#7100)"
This reverts commit b954db9704.
* Preston's patch
* Merge branch 'master' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* Update fssz, add regression test case
* more HTR with fssz
* fix some tests
* only one test left
* Make it so that HTR will work
* gofmt, imports
* gofmt, imports
* fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* fix
* Merge branch 'master' into revert-7100-revert-6760-update-fssz
* Merge refs/heads/master into revert-7100-revert-6760-update-fssz
* gaz
* Merge branch 'revert-7100-revert-6760-update-fssz' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* Merge refs/heads/master into revert-7100-revert-6760-update-fssz
* fix test
* Merge branch 'revert-7100-revert-6760-update-fssz' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* Merge refs/heads/master into revert-7100-revert-6760-update-fssz
* add in configs
* ask for enable accounts v2
* begin integration of v2 keymanager
* refactor wallet opening
* include significant refactoring of how opening a wallet works, making it easy to include at runtime
* ensure build with keymanager v2
* further improving runtime integration
* default pass paths
* finally running v2 at runtime
* import spacing
* Merge branch 'master' into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* confs
* rem e2e val flag
* Merge branch 'master' into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Merge refs/heads/master into v2-accounts-feature
* Add aggregate stats struct to validator struct. Log aggregate voting summary
from beginning of node startup.
* Refactoring out to function UpdateLogAggregateStats which updates and logs aggregate statistics
* Changes to spelling. More refactoring so that the rest of aggregate stats are moved to UpdateLogAggregateStats
* Bugfix - moved startBalance initialization out of if.
* Remove a print debug. Spelling adjustment.
* Minor refactoring/grammar changes
* Renaming variables for consistency.
* Address linter feedback: omit 2nd value from range over map as values were not being used.
* Address feedback from PR re comments.
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Addressing feedback from terencechain in PR. (name changes)
* Addressing feedback from terencechain in PR.
* Create test for UpdateLogAggregateStats()
* Changed comment regarding "numberOfEpochs" log
* go fmt metrics_test.go
* bazel run //:gazelle -- fix
* fix typos
* Improved the code to detect the last iteration of the range responses loop
* Merge branch 'master' into tempIssue4987
* For both previous Epoch and total runtime statistics, do not count non-included attestations for "correctly" statistics.
* Update wording in since-launch logging
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Store first logged epoch in startEpoch and appropriately adjust calculations. Initialize startEpoch to ^uint64(0).
* Merge branch 'master' into tempIssue4987
* Update validator/client/metrics.go
* Update validator/client/metrics.go
* Update validator/client/validator.go
* Update validator/client/metrics_test.go
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Merge branch 'master' into tempIssue4987
* Get started on cleaning up slashing protection
* Merge branch 'master' of github.com:prysmaticlabs/prysm into cleanup-protection
* Move protection functions to own files
* Lint
* Merge branch 'master' of github.com:prysmaticlabs/prysm into cleanup-protection
* Begin adding test for proposal protection
* Merge branch 'master' of github.com:prysmaticlabs/prysm into cleanup-protection
* Fix build
* Fix tests
* Fix tst
* Fix tests
* Fix proposal tests
* Merge branch 'master' into cleanup-protection
* Merge branch 'master' into cleanup-protection
* Merge branch 'master' into cleanup-protection
* Merge branch 'master' of github.com:prysmaticlabs/prysm into cleanup-protection
* Reorder protections
* Change lock
* Fix test
* Merge branch 'master' into cleanup-protection
* Merge branch 'master' into cleanup-protection
* Merge branch 'master' into cleanup-protection
* Change log
* Merge branch 'cleanup-protection' of github.com:prysmaticlabs/prysm into cleanup-protection
* Merge branch 'master' into cleanup-protection
