prysm-pulse/beacon-chain/rpc/beacon/validators.go
terence tsao 046a00aa87
Modify GetValidatorParticipation to use new state service (#5409)
* Skip 2 more minimal tests

* Update readme

* gaz

* Fix import and not use

* Update workspace for new spec test

* Fix workspace

* Update workspace with new ethapi commit

* Unblock a few tests

* fixed block op test

* gaz

* Skip gen state test (test setup issue

* Updated hysteresis config

* Updated epoch processing for new hyteresis

* Updated tests

* regen proto beacon

* update state util for state root

* update state types

* update getter and setters

* update compute domain and get domain and tests

* update validators

* Add forkdata proto

* Updated compute domain api, moved it to helper pkg

* Fixed all core tests

* Fixed all the sync tests

* Fixed all the rpc tests

* Conflict fix

* visibility

* Fixed validator tests

* Fixing test util

* Fixed rest of non spec tests

* Fixed a bug proposer index wasn't included

* gaz

* Updated eth1 data voting period to epoch based

* Fixed failed tests

* fix bug

* fix error

* Fixed more misc tests

* Add new SignedAggregateAndProof to pass spec test

* Update minimalConfig.PersistentCommitteePeriod

* allow to rebuild trie

* Skip e2e tests

* Align aggregator action with v0.11 (#5146)

* Remove Head Root from Beacon Block by Range Request (#5165)

* make proto changes
* remove head root

* add back herumi's library

* Update ethapi in workspace, started fixing test. Hand off to Nishant

* fix build

* Align finalized slot check with v0.11 (#5166)

* Add DoS resistance for v0.11 (#5158)

* Add Fork Digest Helper (#5173)

* Extend DoS prevention to rest of operation objects (#5174)

* Update mapping

* Add caches

* Update seen block in validation pipeline

* Update seen att in validation pipeline

* Update seen att in validation pipeline

* Fixed rest of tests

* Gazelle

* Better writes

* Lint

* Preston's feedback

* Switched to LRU cache and fixed tests

* Gazelle

* Fix test

* Update proposer slashing

* Update proposer slashing

* Fixed a block test

* Update exit

* Update atteser slashing

* Raul's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Add remote keymanager (#5133)

* Add remote keymanager

* Add generic signRoot() helper

* Add tests for remote keymanager

* NewRemote -> NewRemoteWallet

* signRoot -> signOject, to increase reuse

* Fix end-to-end compile error

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Add Snappy Framing to the Encoder (#5172)

* change to framing

* more fixes

* fix everything

* add stricter limits

* preston feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>

* Move Subnet Functionality to its Own File (#5179)

* move subnets to their own file

* fix build fail

* build

* Update beacon-chain/p2p/discovery_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Verify proposer signature in sync (#5206)

* Fix Signed Attestation In Sync (#5207)

* Add Eth2 Fork ENR Functionality (#5181)

* add fork entry enr

* add in fork

* add the required fork entry to node

* add and retrieve fork entry

* await state initialized

* utilize new structure

* more progress, utilizing a config map instead

* send the genesis validators root via the event feed

* struct method for discovery

* fix broken builds

* fixed up more tsts using state feed initializer

* fix up most tests

* only one more failing test

* almost done with tests

* p2p tests all pass

* config fix

* fix blockchain test

* gaz

* add in todo

* lint

* add compare func

* ensure fork ENR versions match between peers

* add in test for discovery

* test name

* tests complete

* tests done

* done

* comments

* fix all flakes

* addressed comments

* build using ssz gen

* marshal record

* use custom ssz

* deduplicate import

* fix build

* add enr proto

* p2p tests done

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Verify aggregator signature in sync (#5208)

* Add Fork Digest For Gossip Topics (#5191)

* update for the day

* fix remaining failing test

* fix one more test

* change message

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

* implement fork digest'

* align digest to interface'

* passed all tests

* spawn in goroutine

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Fix Incorrect Attester Slashing Method (#5229)

* Remove keystore keymanager from validator (#5236)

* Remove keystore keymanager from validator

* Update dependency

* Update validator/flags/flags.go

* Update validator/flags/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* fix broadcaster

* update metrics with fork digest for p2p (#5251)

* update metrics with fork digest for p2p

* update p2p metrics

* update metrics using att values

* wrapped up

* fix bug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Fix incorrect domain type comments (#5250)

* Fix incorrect domain type comments

* fix broken broadcast test

* fix tests

* include protocol suffix

* lint

* fix test

* resolve broken slasher test'

* fix config override

* Remove deprecated parameters (#5249)

* Avoid div by zero in extreme balance case (#5273)

* Return effective balance increment instead of 1

* Update to new spec tests v0.11.1

* Revert "Regen historical states for `new-state-mgmt` compatibility (#5261)"

This reverts commit df9a534826.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Revert "Remove deprecated parameters (#5249)" (#5276)

This reverts commit 7d17c9ac3455ee15c67b3645485693309216bc97.

* Verify block proposer index before gossip  (#5274)

* Update pipeline

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Add in Proposer Index to Custom HTR (#5269)

* fix test

* Update beacon-chain/state/stateutil/blocks_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Resolve Flakey P2P Tests (#5285)

* double time for flakey test

* fix test flakeyness in p2p:

* flakey

* time tolerance

* greater tolerance

* release resources correctly (#5287)

* Enable NOISE Handshake by Default v0.11 (#5272)

* noise handshakes by default

* fix build

* noisy noise everywhere

* deprecated noisy noise flag with more noise

* add secio as fallback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>

* new ports

* fix broken build

* Make `new-state-mgmt` canonical  (#5289)

* Invert the flags
* Update checking messages
* Fixed all db tests
* Fixed rest of the block chain tests
* Fix chain race tests
* Fixed rpc tests
* Disable soudns better...
* Merge branch 'v0.11' into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Fix export
* Merge branch 'invert-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-new-state-mgmt
* Fix conflict tests
* Gazelle
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt

* resolve flakeyness

* Detect Proposer Slashing Implementation (#5139)

* detect blocks

* detect blocks

* use stub

* use stub

* use stub

* todo

* fix test

* add tests and utils

* fix imports

* fix imports

* fix comment

* todo

* proposerIndex

* fix broken test

* formatting and simplified if

* Update slasher/detection/service.go

* Update slasher/detection/testing/utils.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fixed up final comments

* better naming

* Update slasher/detection/service.go

* Update slasher/detection/service.go

* Update slasher/detection/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* no more named args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>

* Add Metadata And Ping RPC methods (#5271)

* add new proto files

* add flag and helper

* add initializer

* imports

* add ping method

* add receive/send ping request

* add ping test

* refactor rpc methods and add ping test

* finish adding all tests

* fix up tests

* Apply suggestions from code review

* lint

* imports

* lint

* Update beacon-chain/p2p/service.go

* Update shared/cmd/flags.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Updates for remote keymanager (#5260)

* Update to slash by slot instead of epoch (#5297)

* change to slash by slot instead of epoch

* gaz

* fix test

* fix test

* fix infinite loop on error parse

* Update proposer protection to v0.11 (#5292)

* Complete most of changes

* Fix other tests

* Test progress

* Tests

* Finish tests

* update pbs

* Fix mocked tests

* Gazelle

* pt 2

* Fix

* Fixes

* Fix tests wit hwrong copying

* Implement `SubscribeCommitteeSubnet` method (#5299)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Validator subscribe subnet to next epoch (#5312)

* Alert to subscribe to next epoch

* Fixed tests

* Comments

* Fixed tests

* Update validator/client/validator.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)

This reverts commit 47e5a2cf96f5add151bf135a5352c2dad7922615.

* Aggregate on demand for v0.11 (#5302)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Wait 1/3 on validator side

* Lint

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Fixed duplication

* Tests

* Skip e2e

* Update beacon-chain/rpc/validator/aggregator.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Refactor Dynamic Subscriptions (#5318)

* clean up

* comment

* metrics

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Fix listindexed attestations and detect historic attestations (#5321)

* fix list indexed attestations

* fix tests

* goimports

* names

* Add check for slot == 0 (#5322)

* Change attester protection to return default if DB is empty (#5323)

* Change how default values are set

* Remove unused imports

* Remove wasteful db call

* Fix db tests

* Fix db test

* fix it (#5326)

* V0.11 run time fixes to use interop config (#5324)

* Started testing
* Bunch of fixes
* use-interop
* Sync with v0.11
* Conflict
* Uncomment wait for activation
* Move pending block queue from subscriber to validator pipeline
* Merge branch 'v0.11' into use-interop-config
* passing tests
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/v0.11 into use-interop-config

* Nil Checks in Process Attestation v0.11 (#5331)

* Started testing

* Bunch of fixes

* use-interop

* Sync with v0.11

* Uncomment wait for activation

* Move pending block queue from subscriber to validator pipeline

* passing tests

* nil checks to prevent panics

* lint

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Validator batch subscribe subnets (#5332)

* Update both beacon node and validator

* Comments

* Tests

* Lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Validator smarter subscribe (#5334)

* Fix incorrect proposer index calculation (#5336)

* Use correct parent state

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* enhance error

* enhance error

* Update P2P Service to Handle Local Metadata (#5319)

* add metadata to ENR

* add new methods

* glue everything

* fix all tests and refs

* add tests

* add more tests

* Apply suggestions from code review

* fix method

* raul's review

* gaz

* fix test setup

* fix all tests

* better naming

* fix broken test

* validate nil

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Revert "Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)" (#5343)

This reverts commit e5aef1686e582fc2077767c42187c8527f3a742f.

* Wait for Genesis Event to Start P2P (#5303)

* use event feed for state initialized events

* add in handler for tests

* wait till genesis for p2p

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Avoid duplicated aggregation request (#5346)

* Avoid duplicated aggregation request

* Test and lock

* Gaz

* Fix Validate For Metadata (#5348)

* return true

* shay's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Multiple Proposer Slots Allowed Per Epoch for Validators (#5344)

* allow multiple proposer slots

* multi propose

* proposer indices to slots map

* remove deprecated comm assign

* Apply suggestions from code review

* resolve broken tests, add logic in validator client

* fix val tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Networking Fixes (#5349)

* close stream later

* add ping method

* add method

* lint

* More efficient aggregation on demand (#5354)

* Return Nil Error if Pre-Genesis in P2P Service Healthz Check (#5355)

* pregenesis healthz check:

* optimal

* right order

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* no comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Release DiscoveryV5 for Testnet Restart (#5357)

* release discv5

* fix build

* Fix Overflow in Status Check (#5361)

* fix overflow

* Apply suggestions from code review

* fix after merge

* Make Mainnet Config Default, No More Demo Config  (#5367)

* bye bye demo config

* gaz

* fix usage

* fix dep

* gaz

* Update default balance for sendDeposits

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Use FastSSZ Marshal/Unmarshal for DB Encodings in v0.11.1 (#5351)

* try

* use marshaler structure for db instead of proto

* white list types

* attempt

* revert

* testutil.NewBeaconState()

* Fully populate fields for round trip ssz marshal

* fix //beacon-chain/db/kv:go_default_test

* more passing tests

* another test target passed

* fixed stategen

* blockchain tests green

* passing sync

* more targets fixed

* more test fixes in rpc/validator

* most rpc val

* validators test fixes

* skip round robin old

* aggregate test

* whitelist done

* Update beacon-chain/rpc/validator/attester_test.go

* edit baz

* Fixed tests

* Fixed getblock test

* Add back init

* reduce test size

* fix broken build

* tests pass

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Reconnect slasher streams on beacon node shutdown (#5376)

* restart streams on beacon node shutdown

* fix comment

* remove export

* ivan feedback

* ivan feedback

* case insensitive

* Update slasher/beaconclient/receivers.go

* raul feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Amend Faucet to Offer 32.5 ETH for v0.11 (#5378)

* deposit amount in faucet

* fix eth amount

* gas cost

* unskip exec transition test

* Revert "Enable NOISE Handshake by Default v0.11 (#5272)" (#5381)

This reverts commit a8d32d504a8f923cdf7fa9dfc2684f8804fbab92.

* use string for deposit flag

* Update Bootnode to v0.11 (#5387)

* fix bootnode

* add changes

* gaz

* fix docker

* build fix

* fix flaky test

* Unskip E2E for V0.11 (#5386)

* Begin work on fixing e2e for v0.11

* Start bootnode work

* Begin implementing bootnode into e2e

* Fix E2E for v0.11

* Remove extra

* gaz

* Remove unused key gen code

* Remove trailing multiaddr code

* add skip for slashing

* Fix slashing e2e

* Fix docker image build

* Update beacon-chain/p2p/broadcaster_test.go

* Update GetValidatorParticipation

* Update tests

* Gaz

* Pass E2E Tests for v0.11 and Enable Attestation Subnets By Default (#5407)

* Update README.md

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/keystore/deposit_input.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update tools/faucet/server.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update proto/beacon/p2p/v1/BUILD.bazel

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/bls/spectest/aggregate_verify_test.go

* Addressed feedback. All test passing

* Update beacon-chain/core/blocks/block_operations_fuzz_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update shared/testutil/helpers.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update beacon-chain/core/helpers/signing_root.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Resolve Misc v0.11 Items (Raul) (#5414)

* address all comments

* set faucet

* nishant feedback

* Update beacon-chain/p2p/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Revert keymanager changes (#5416)

* Revert "Updates for remote keymanager (#5260)"

This reverts commit bbcd895db50ce5e7c0ecb64210471cf56f63b373.

* Revert "Remove keystore keymanager from validator (#5236)"

This reverts commit 46008770c162e741251e13772fd7356b43a9af87.

* Revert "Update eth2 wallet keymanager (#4984)"

This reverts commit 7f7ef43f218598a671aaeb327342d7e5130fe8b1.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Update BLS and limit visibility (#5415)

* remove duplicated BLS, add golang.org/x/mod

* Update BLS and restrict visibility

* fix build

* Fix eth1data test and fix order of ops (#5413)

* use multiaddr builder (#5419)

* Unskip benchutil and minor v0.11 fixes (#5417)

* Unskip benchutil tests

* Remove protos and gaz

* Fixes

* Networking Fixes (#5421)

* check

* fix test

* fix size

* fix test

* more fixes

* fix test again

* Update ethereum APIs with latest master

* Error handling for v0.11 tests (#5428)

* Proper err handling for tests

* Lint

* Fixed rest of the tests

* Gaz

* Fixed old master tests

* Rm old aggregate_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: Jim McDonald <Jim@mcdee.net>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
2020-04-15 11:00:39 -05:00

692 lines
24 KiB
Go

package beacon
import (
"context"
"sort"
"strconv"
ptypes "github.com/gogo/protobuf/types"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/beacon-chain/core/epoch/precompute"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/core/state"
"github.com/prysmaticlabs/prysm/beacon-chain/core/validators"
"github.com/prysmaticlabs/prysm/beacon-chain/flags"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/pagination"
"github.com/prysmaticlabs/prysm/shared/params"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
// ListValidatorBalances retrieves the validator balances for a given set of public keys.
// An optional Epoch parameter is provided to request historical validator balances from
// archived, persistent data.
func (bs *Server) ListValidatorBalances(
ctx context.Context,
req *ethpb.ListValidatorBalancesRequest) (*ethpb.ValidatorBalances, error) {
if int(req.PageSize) > flags.Get().MaxPageSize {
return nil, status.Errorf(codes.InvalidArgument, "Requested page size %d can not be greater than max size %d",
req.PageSize, flags.Get().MaxPageSize)
}
res := make([]*ethpb.ValidatorBalances_Balance, 0)
filtered := map[uint64]bool{} // Track filtered validators to prevent duplication in the response.
headState, err := bs.HeadFetcher.HeadState(ctx)
if err != nil {
return nil, status.Error(codes.Internal, "Could not get head state")
}
var requestingGenesis bool
var epoch uint64
switch q := req.QueryFilter.(type) {
case *ethpb.ListValidatorBalancesRequest_Epoch:
epoch = q.Epoch
case *ethpb.ListValidatorBalancesRequest_Genesis:
requestingGenesis = q.Genesis
default:
epoch = helpers.CurrentEpoch(headState)
}
var balances []uint64
validators := headState.Validators()
if requestingGenesis || epoch < helpers.CurrentEpoch(headState) {
balances, err = bs.BeaconDB.ArchivedBalances(ctx, epoch)
if err != nil {
return nil, status.Errorf(codes.Internal, "Could not retrieve balances for epoch %d", epoch)
}
if balances == nil {
return nil, status.Errorf(
codes.NotFound,
"Could not retrieve data for epoch %d, perhaps --archive in the running beacon node is disabled",
0,
)
}
} else if epoch == helpers.CurrentEpoch(headState) {
balances = headState.Balances()
} else {
// Otherwise, we are requesting data from the future and we return an error.
return nil, status.Errorf(
codes.InvalidArgument,
"Cannot retrieve information about an epoch in the future, current epoch %d, requesting %d",
helpers.CurrentEpoch(headState),
epoch,
)
}
balancesCount := len(balances)
for _, pubKey := range req.PublicKeys {
// Skip empty public key.
if len(pubKey) == 0 {
continue
}
pubkeyBytes := bytesutil.ToBytes48(pubKey)
index, ok := headState.ValidatorIndexByPubkey(pubkeyBytes)
if !ok {
return nil, status.Errorf(codes.NotFound, "Could not find validator index for public key %#x", pubkeyBytes)
}
filtered[index] = true
if int(index) >= len(balances) {
return nil, status.Errorf(codes.OutOfRange, "Validator index %d >= balance list %d",
index, len(balances))
}
res = append(res, &ethpb.ValidatorBalances_Balance{
PublicKey: pubKey,
Index: index,
Balance: balances[index],
})
balancesCount = len(res)
}
for _, index := range req.Indices {
if int(index) >= len(balances) {
if epoch <= helpers.CurrentEpoch(headState) {
return nil, status.Errorf(codes.OutOfRange, "Validator index %d does not exist in historical balances",
index)
}
return nil, status.Errorf(codes.OutOfRange, "Validator index %d >= balance list %d",
index, len(balances))
}
if !filtered[index] {
res = append(res, &ethpb.ValidatorBalances_Balance{
PublicKey: validators[index].PublicKey,
Index: index,
Balance: balances[index],
})
}
balancesCount = len(res)
}
// Depending on the indices and public keys given, results might not be sorted.
sort.Slice(res, func(i, j int) bool {
return res[i].Index < res[j].Index
})
// If there are no balances, we simply return a response specifying this.
// Otherwise, attempting to paginate 0 balances below would result in an error.
if balancesCount == 0 {
return &ethpb.ValidatorBalances{
Epoch: epoch,
Balances: make([]*ethpb.ValidatorBalances_Balance, 0),
TotalSize: int32(0),
NextPageToken: strconv.Itoa(0),
}, nil
}
start, end, nextPageToken, err := pagination.StartAndEndPage(req.PageToken, int(req.PageSize), balancesCount)
if err != nil {
return nil, status.Errorf(
codes.Internal,
"Could not paginate results: %v",
err,
)
}
if len(req.Indices) == 0 && len(req.PublicKeys) == 0 {
// Return everything.
for i := start; i < end; i++ {
pubkey := headState.PubkeyAtIndex(uint64(i))
res = append(res, &ethpb.ValidatorBalances_Balance{
PublicKey: pubkey[:],
Index: uint64(i),
Balance: balances[i],
})
}
return &ethpb.ValidatorBalances{
Epoch: epoch,
Balances: res,
TotalSize: int32(balancesCount),
NextPageToken: nextPageToken,
}, nil
}
return &ethpb.ValidatorBalances{
Epoch: epoch,
Balances: res[start:end],
TotalSize: int32(balancesCount),
NextPageToken: nextPageToken,
}, nil
}
// ListValidators retrieves the current list of active validators with an optional historical epoch flag to
// to retrieve validator set in time.
func (bs *Server) ListValidators(
ctx context.Context,
req *ethpb.ListValidatorsRequest,
) (*ethpb.Validators, error) {
if int(req.PageSize) > flags.Get().MaxPageSize {
return nil, status.Errorf(codes.InvalidArgument, "Requested page size %d can not be greater than max size %d",
req.PageSize, flags.Get().MaxPageSize)
}
headState, err := bs.HeadFetcher.HeadState(ctx)
if err != nil {
return nil, status.Error(codes.Internal, "Could not get head state")
}
currentEpoch := helpers.CurrentEpoch(headState)
requestedEpoch := currentEpoch
switch q := req.QueryFilter.(type) {
case *ethpb.ListValidatorsRequest_Genesis:
if q.Genesis {
requestedEpoch = 0
}
case *ethpb.ListValidatorsRequest_Epoch:
requestedEpoch = q.Epoch
}
validatorList := make([]*ethpb.Validators_ValidatorContainer, 0)
for _, index := range req.Indices {
val, err := headState.ValidatorAtIndex(index)
if err != nil {
return nil, status.Error(codes.Internal, "Could not get validator")
}
validatorList = append(validatorList, &ethpb.Validators_ValidatorContainer{
Index: index,
Validator: val,
})
}
for _, pubKey := range req.PublicKeys {
// Skip empty public key.
if len(pubKey) == 0 {
continue
}
pubkeyBytes := bytesutil.ToBytes48(pubKey)
index, ok := headState.ValidatorIndexByPubkey(pubkeyBytes)
if !ok {
continue
}
val, err := headState.ValidatorAtIndex(index)
if err != nil {
return nil, status.Error(codes.Internal, "Could not get validator")
}
validatorList = append(validatorList, &ethpb.Validators_ValidatorContainer{
Index: index,
Validator: val,
})
}
// Depending on the indices and public keys given, results might not be sorted.
sort.Slice(validatorList, func(i, j int) bool {
return validatorList[i].Index < validatorList[j].Index
})
if len(req.PublicKeys) == 0 && len(req.Indices) == 0 {
for i := 0; i < headState.NumValidators(); i++ {
val, err := headState.ValidatorAtIndex(uint64(i))
if err != nil {
return nil, status.Error(codes.Internal, "Could not get validator")
}
validatorList = append(validatorList, &ethpb.Validators_ValidatorContainer{
Index: uint64(i),
Validator: val,
})
}
}
if requestedEpoch < currentEpoch {
stopIdx := len(validatorList)
for idx, item := range validatorList {
// The first time we see a validator with an activation epoch > the requested epoch,
// we know this validator is from the future relative to what the request wants.
if item.Validator.ActivationEpoch > requestedEpoch {
stopIdx = idx
break
}
}
validatorList = validatorList[:stopIdx]
} else if requestedEpoch > currentEpoch {
// Otherwise, we are requesting data from the future and we return an error.
return nil, status.Errorf(
codes.InvalidArgument,
"Cannot retrieve information about an epoch in the future, current epoch %d, requesting %d",
currentEpoch,
requestedEpoch,
)
}
// Filter active validators if the request specifies it.
res := validatorList
if req.Active {
filteredValidators := make([]*ethpb.Validators_ValidatorContainer, 0)
for _, item := range validatorList {
if helpers.IsActiveValidator(item.Validator, requestedEpoch) {
filteredValidators = append(filteredValidators, item)
}
}
res = filteredValidators
}
validatorCount := len(res)
// If there are no items, we simply return a response specifying this.
// Otherwise, attempting to paginate 0 validators below would result in an error.
if validatorCount == 0 {
return &ethpb.Validators{
ValidatorList: make([]*ethpb.Validators_ValidatorContainer, 0),
TotalSize: int32(0),
NextPageToken: strconv.Itoa(0),
}, nil
}
start, end, nextPageToken, err := pagination.StartAndEndPage(req.PageToken, int(req.PageSize), validatorCount)
if err != nil {
return nil, status.Errorf(
codes.Internal,
"Could not paginate results: %v",
err,
)
}
return &ethpb.Validators{
ValidatorList: res[start:end],
TotalSize: int32(validatorCount),
NextPageToken: nextPageToken,
}, nil
}
// GetValidator information from any validator in the registry by index or public key.
func (bs *Server) GetValidator(
ctx context.Context, req *ethpb.GetValidatorRequest,
) (*ethpb.Validator, error) {
var requestingIndex bool
var index uint64
var pubKey []byte
switch q := req.QueryFilter.(type) {
case *ethpb.GetValidatorRequest_Index:
index = q.Index
requestingIndex = true
case *ethpb.GetValidatorRequest_PublicKey:
pubKey = q.PublicKey
default:
return nil, status.Error(
codes.InvalidArgument,
"Need to specify either validator index or public key in request",
)
}
headState, err := bs.HeadFetcher.HeadState(ctx)
if err != nil {
return nil, status.Error(codes.Internal, "Could not get head state")
}
if requestingIndex {
if index >= uint64(headState.NumValidators()) {
return nil, status.Errorf(
codes.OutOfRange,
"Requesting index %d, but there are only %d validators",
index,
headState.NumValidators(),
)
}
return headState.ValidatorAtIndex(index)
}
pk48 := bytesutil.ToBytes48(pubKey)
for i := 0; i < headState.NumValidators(); i++ {
keyFromState := headState.PubkeyAtIndex(uint64(i))
if keyFromState == pk48 {
return headState.ValidatorAtIndex(uint64(i))
}
}
return nil, status.Error(codes.NotFound, "No validator matched filter criteria")
}
// GetValidatorActiveSetChanges retrieves the active set changes for a given epoch.
//
// This data includes any activations, voluntary exits, and involuntary
// ejections.
func (bs *Server) GetValidatorActiveSetChanges(
ctx context.Context, req *ethpb.GetValidatorActiveSetChangesRequest,
) (*ethpb.ActiveSetChanges, error) {
headState, err := bs.HeadFetcher.HeadState(ctx)
if err != nil {
return nil, status.Error(codes.Internal, "Could not get head state")
}
currentEpoch := helpers.CurrentEpoch(headState)
requestedEpoch := currentEpoch
requestingGenesis := false
switch q := req.QueryFilter.(type) {
case *ethpb.GetValidatorActiveSetChangesRequest_Genesis:
requestingGenesis = q.Genesis
requestedEpoch = 0
case *ethpb.GetValidatorActiveSetChangesRequest_Epoch:
requestedEpoch = q.Epoch
}
activatedIndices := make([]uint64, 0)
exitedIndices := make([]uint64, 0)
slashedIndices := make([]uint64, 0)
ejectedIndices := make([]uint64, 0)
if requestingGenesis || requestedEpoch < currentEpoch {
archivedChanges, err := bs.BeaconDB.ArchivedActiveValidatorChanges(ctx, requestedEpoch)
if err != nil {
return nil, status.Errorf(codes.Internal, "Could not fetch archived active validator changes: %v", err)
}
if archivedChanges == nil {
return nil, status.Errorf(
codes.NotFound,
"Did not find any data for epoch %d - perhaps no active set changed occurred during the epoch",
requestedEpoch,
)
}
activatedIndices = archivedChanges.Activated
exitedIndices = archivedChanges.Exited
slashedIndices = archivedChanges.Slashed
ejectedIndices = archivedChanges.Ejected
} else if requestedEpoch == currentEpoch {
activeValidatorCount, err := helpers.ActiveValidatorCount(headState, helpers.PrevEpoch(headState))
if err != nil {
return nil, status.Errorf(codes.Internal, "Could not get active validator count: %v", err)
}
vals := headState.Validators()
activatedIndices = validators.ActivatedValidatorIndices(helpers.PrevEpoch(headState), vals)
exitedIndices, err = validators.ExitedValidatorIndices(helpers.PrevEpoch(headState), vals, activeValidatorCount)
if err != nil {
return nil, status.Errorf(codes.Internal, "Could not determine exited validator indices: %v", err)
}
slashedIndices = validators.SlashedValidatorIndices(helpers.PrevEpoch(headState), vals)
ejectedIndices, err = validators.EjectedValidatorIndices(helpers.PrevEpoch(headState), vals, activeValidatorCount)
if err != nil {
return nil, status.Errorf(codes.Internal, "Could not determine ejected validator indices: %v", err)
}
} else {
// We are requesting data from the future and we return an error.
return nil, status.Errorf(
codes.InvalidArgument,
"Cannot retrieve information about an epoch in the future, current epoch %d, requesting %d",
currentEpoch,
requestedEpoch,
)
}
// We retrieve the public keys for the indices.
activatedKeys := make([][]byte, len(activatedIndices))
exitedKeys := make([][]byte, len(exitedIndices))
slashedKeys := make([][]byte, len(slashedIndices))
ejectedKeys := make([][]byte, len(ejectedIndices))
for i, idx := range activatedIndices {
pubkey := headState.PubkeyAtIndex(idx)
activatedKeys[i] = pubkey[:]
}
for i, idx := range exitedIndices {
pubkey := headState.PubkeyAtIndex(idx)
exitedKeys[i] = pubkey[:]
}
for i, idx := range slashedIndices {
pubkey := headState.PubkeyAtIndex(idx)
slashedKeys[i] = pubkey[:]
}
for i, idx := range ejectedIndices {
pubkey := headState.PubkeyAtIndex(idx)
ejectedKeys[i] = pubkey[:]
}
return &ethpb.ActiveSetChanges{
Epoch: requestedEpoch,
ActivatedPublicKeys: activatedKeys,
ActivatedIndices: activatedIndices,
ExitedPublicKeys: exitedKeys,
ExitedIndices: exitedIndices,
SlashedPublicKeys: slashedKeys,
SlashedIndices: slashedIndices,
EjectedPublicKeys: ejectedKeys,
EjectedIndices: ejectedIndices,
}, nil
}
// GetValidatorParticipation retrieves the validator participation information for a given epoch,
// it returns the information about validator's participation rate in voting on the proof of stake
// rules based on their balance compared to the total active validator balance.
func (bs *Server) GetValidatorParticipation(
ctx context.Context, req *ethpb.GetValidatorParticipationRequest,
) (*ethpb.ValidatorParticipationResponse, error) {
currentEpoch := helpers.SlotToEpoch(bs.GenesisTimeFetcher.CurrentSlot())
var requestedEpoch uint64
switch q := req.QueryFilter.(type) {
case *ethpb.GetValidatorParticipationRequest_Genesis:
requestedEpoch = 0
case *ethpb.GetValidatorParticipationRequest_Epoch:
requestedEpoch = q.Epoch
default:
// Prevent underflow and ensure participation is always queried for previous epoch.
if currentEpoch > 1 {
requestedEpoch = currentEpoch - 1
}
}
if requestedEpoch >= currentEpoch {
return nil, status.Errorf(
codes.InvalidArgument,
"Cannot retrieve information about an epoch until older than current epoch, current epoch %d, requesting %d",
currentEpoch,
requestedEpoch,
)
}
requestedState, err := bs.StateGen.StateBySlot(ctx, helpers.StartSlot(requestedEpoch+1))
if err != nil {
return nil, status.Error(codes.Internal, "Could not get state")
}
v, b, err := precompute.New(ctx, requestedState)
if err != nil {
return nil, status.Error(codes.Internal, "Could not set up pre compute instance")
}
_, b, err = precompute.ProcessAttestations(ctx, requestedState, v, b)
if err != nil {
return nil, status.Error(codes.Internal, "Could not pre compute attestations")
}
headState, err := bs.HeadFetcher.HeadState(ctx)
if err != nil {
return nil, status.Error(codes.Internal, "Could not get head state")
}
return &ethpb.ValidatorParticipationResponse{
Epoch: requestedEpoch,
Finalized: requestedEpoch <= headState.FinalizedCheckpointEpoch(),
Participation: &ethpb.ValidatorParticipation{
GlobalParticipationRate: float32(b.PrevEpochTargetAttesters) / float32(b.PrevEpoch),
VotedEther: b.PrevEpochTargetAttesters,
EligibleEther: b.PrevEpoch,
},
}, nil
}
// GetValidatorQueue retrieves the current validator queue information.
func (bs *Server) GetValidatorQueue(
ctx context.Context, _ *ptypes.Empty,
) (*ethpb.ValidatorQueue, error) {
headState, err := bs.HeadFetcher.HeadState(ctx)
if err != nil {
return nil, status.Error(codes.Internal, "Could not get head state")
}
// Queue the validators whose eligible to activate and sort them by activation eligibility epoch number.
// Additionally, determine those validators queued to exit
awaitingExit := make([]uint64, 0)
exitEpochs := make([]uint64, 0)
activationQ := make([]uint64, 0)
vals := headState.Validators()
for idx, validator := range vals {
eligibleActivated := validator.ActivationEligibilityEpoch != params.BeaconConfig().FarFutureEpoch
canBeActive := validator.ActivationEpoch >= helpers.ActivationExitEpoch(headState.FinalizedCheckpointEpoch())
if eligibleActivated && canBeActive {
activationQ = append(activationQ, uint64(idx))
}
if validator.ExitEpoch != params.BeaconConfig().FarFutureEpoch {
exitEpochs = append(exitEpochs, validator.ExitEpoch)
awaitingExit = append(awaitingExit, uint64(idx))
}
}
sort.Slice(activationQ, func(i, j int) bool {
return vals[i].ActivationEligibilityEpoch < vals[j].ActivationEligibilityEpoch
})
sort.Slice(awaitingExit, func(i, j int) bool {
return vals[i].WithdrawableEpoch < vals[j].WithdrawableEpoch
})
// Only activate just enough validators according to the activation churn limit.
activationQueueChurn := len(activationQ)
activeValidatorCount, err := helpers.ActiveValidatorCount(headState, helpers.CurrentEpoch(headState))
if err != nil {
return nil, status.Errorf(codes.Internal, "Could not get active validator count: %v", err)
}
churnLimit, err := helpers.ValidatorChurnLimit(activeValidatorCount)
if err != nil {
return nil, status.Errorf(codes.Internal, "Could not compute churn limit: %v", err)
}
exitQueueEpoch := uint64(0)
for _, i := range exitEpochs {
if exitQueueEpoch < i {
exitQueueEpoch = i
}
}
exitQueueChurn := 0
for _, val := range vals {
if val.ExitEpoch == exitQueueEpoch {
exitQueueChurn++
}
}
// Prevent churn limit from causing index out of bound issues.
if int(churnLimit) < activationQueueChurn {
activationQueueChurn = int(churnLimit)
}
if int(churnLimit) < exitQueueChurn {
// If we are above the churn limit, we simply increase the churn by one.
exitQueueEpoch++
exitQueueChurn = int(churnLimit)
}
// We use the exit queue churn to determine if we have passed a churn limit.
minEpoch := exitQueueEpoch + params.BeaconConfig().MinValidatorWithdrawabilityDelay
exitQueueIndices := make([]uint64, 0)
for _, valIdx := range awaitingExit {
val := vals[valIdx]
// Ensure the validator has not yet exited before adding its index to the exit queue.
if val.WithdrawableEpoch < minEpoch && !validatorHasExited(val, helpers.CurrentEpoch(headState)) {
exitQueueIndices = append(exitQueueIndices, valIdx)
}
}
// Get the public keys for the validators in the queues up to the allowed churn limits.
activationQueueKeys := make([][]byte, len(activationQ))
exitQueueKeys := make([][]byte, len(exitQueueIndices))
for i, idx := range activationQ {
activationQueueKeys[i] = vals[idx].PublicKey
}
for i, idx := range exitQueueIndices {
exitQueueKeys[i] = vals[idx].PublicKey
}
return &ethpb.ValidatorQueue{
ChurnLimit: churnLimit,
ActivationPublicKeys: activationQueueKeys,
ExitPublicKeys: exitQueueKeys,
}, nil
}
// GetValidatorPerformance reports the validator's latest balance along with other important metrics on
// rewards and penalties throughout its lifecycle in the beacon chain.
func (bs *Server) GetValidatorPerformance(
ctx context.Context, req *ethpb.ValidatorPerformanceRequest,
) (*ethpb.ValidatorPerformanceResponse, error) {
validatorSummary := state.ValidatorSummary
reqPubKeysCount := len(req.PublicKeys)
beforeTransitionBalances := make([]uint64, 0, reqPubKeysCount)
afterTransitionBalances := make([]uint64, 0, reqPubKeysCount)
effectiveBalances := make([]uint64, 0, reqPubKeysCount)
inclusionSlots := make([]uint64, 0, reqPubKeysCount)
inclusionDistances := make([]uint64, 0, reqPubKeysCount)
correctlyVotedSource := make([]bool, 0, reqPubKeysCount)
correctlyVotedTarget := make([]bool, 0, reqPubKeysCount)
correctlyVotedHead := make([]bool, 0, reqPubKeysCount)
missingValidators := make([][]byte, 0, reqPubKeysCount)
headState, err := bs.HeadFetcher.HeadState(ctx)
if err != nil {
return nil, status.Error(codes.Internal, "Could not get head state")
}
// Convert the list of validator public keys to list of validator indices.
// Also track missing validators using public keys.
for _, key := range req.PublicKeys {
pubkeyBytes := bytesutil.ToBytes48(key)
idx, ok := headState.ValidatorIndexByPubkey(pubkeyBytes)
if !ok {
missingValidators = append(missingValidators, key)
continue
}
if idx >= uint64(len(validatorSummary)) {
// Not listed in validator summary yet; treat it as missing.
missingValidators = append(missingValidators, key)
continue
}
effectiveBalances = append(effectiveBalances, validatorSummary[idx].CurrentEpochEffectiveBalance)
beforeTransitionBalances = append(beforeTransitionBalances, validatorSummary[idx].BeforeEpochTransitionBalance)
afterTransitionBalances = append(afterTransitionBalances, validatorSummary[idx].AfterEpochTransitionBalance)
inclusionSlots = append(inclusionSlots, validatorSummary[idx].InclusionSlot)
inclusionDistances = append(inclusionDistances, validatorSummary[idx].InclusionDistance)
correctlyVotedSource = append(correctlyVotedSource, validatorSummary[idx].IsPrevEpochAttester)
correctlyVotedTarget = append(correctlyVotedTarget, validatorSummary[idx].IsPrevEpochTargetAttester)
correctlyVotedHead = append(correctlyVotedHead, validatorSummary[idx].IsPrevEpochHeadAttester)
}
return &ethpb.ValidatorPerformanceResponse{
InclusionSlots: inclusionSlots,
InclusionDistances: inclusionDistances,
CorrectlyVotedSource: correctlyVotedSource,
CorrectlyVotedTarget: correctlyVotedTarget,
CorrectlyVotedHead: correctlyVotedHead,
CurrentEffectiveBalances: effectiveBalances,
BalancesBeforeEpochTransition: beforeTransitionBalances,
BalancesAfterEpochTransition: afterTransitionBalances,
MissingValidators: missingValidators,
}, nil
}
// Determines whether a validator has already exited.
func validatorHasExited(validator *ethpb.Validator, currentEpoch uint64) bool {
farFutureEpoch := params.BeaconConfig().FarFutureEpoch
if currentEpoch < validator.ActivationEligibilityEpoch {
return false
}
if currentEpoch < validator.ActivationEpoch {
return false
}
if validator.ExitEpoch == farFutureEpoch {
return false
}
if currentEpoch < validator.ExitEpoch {
if validator.Slashed {
return false
}
return false
}
return true
}