prysm-pulse/beacon-chain/core/helpers/validators.go
Raul Jordan cc741ed8af
Ensure New State Type Tests Pass in Prysm (#4646)
* begin state service

* begin on the state trie idea

* created beacon state structure

* add in the full clone getter

* return by value instead

* add all setters

* new state setters are being completed

* arrays roots exposed

*  close to finishing all these headerssss

* functionality complete

* added in proto benchmark test

* test for compatibility

* add test for compat

* comments fixed

* add clone

* add clone

* remove underlying copies

* make it immutable

* integrate it into chainservice

* revert

* wrap up comments for package

* address all comments and godocs

* address all comments

* clone the pending attestation properly

* properly clone remaining items

* tests pass fixed bug

* begin using it instead of head state

* prevent nil pointer exceptions

* begin using new struct in db

* integrated new type into db package

* add proper nil checks

* using new state in archiver

* refactored much of core

* editing all the precompute functions

* done with most core refactor

* fixed up some bugs in the clone comparisons

* append current epoch atts

* add missing setters

* add new setters

* fix other core methods

* fix up transition

* main service and forkchoice

* fix rpc

* integrated to powchain

* some more changes

* fix build

* improve processing of deposits

* fix error

* prevent panic

* comment

* fix process att

* gaz

* fix up att process

* resolve existing review comments

* resolve another batch of gh comments

* resolve broken cpt state

* revise testutil to use the new state

* begin updating the state transition func to pass in more compartmentalized args

* finish editing transition function to return errors

* block operations pretty much done with refactor

* state transition fully refactored

* got epoch processing completed

* fix build in fork choice

* fixing more of the build

* fix up broken sync package

* it builds nowww it buildssss

* revert registry changes

* Recompute on Read (#4627)

* compute on read

* fix up eth1 data votes

* looking into slashings bug introduced in core/

* able to advance more slots

* add logging

* can now sync with testnet yay

* remove the leaves algorithm and other merkle imports

* expose initialize unsafe funcs

* Update beacon-chain/db/kv/state.go

* lint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* More Optimizations for New State (#4641)

* map optimization

* more optimizations

* use a custom hasher

* comment

* block operations optimizations

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* fixed up various operations to use the validator index map access

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* archiver tests pass

* fixing cache tests

* cache tests passing

* edited validator tests

* powchain tests passing

* halfway thru sync tests

* more sync test fixes

* add in tests for state/

* working through rpc tests

* assignments tests passed

* almost done with rpc/beacon tests

* resolved painful validator test

* fixed up even more tests

* resolve tests

* fix build

* reduce a randao mixes copy

* fixes under //beacon-chain/blockchain/...

* build //beacon-chain/core/...

* fixes

* Runtime Optimizations (#4648)

* parallelize shuffling

* clean up

* lint

* fix build

* use callback to read from registry

* fix array roots and size map

* new improvements

* reduce hash allocs

* improved shuffling

* terence's review

* use different method

* raul's comment

* new array roots

* remove clone in pre-compute

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul's review

* lint

* fix build issues

* fix visibility

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fix visibility

* build works for all

* fix blockchain test

* fix a few tests

* fix more tests

* update validator in slashing

* archiver passing

* fixed rpc/validator

* progress on core tests

* resolve broken rpc tests

* blockchain tests passed

* fix up some tests in core

* fix message diff

* remove unnecessary save

* Save validator after slashing

* Update validators one by one

* another update

* fix everything

* fix more precompute tests

* fix blocks tests

* more elegant fix

* more helper fixes

* change back ?

* fix test

* fix skip slot

* fix test

* reset caches

* fix testutil

* raceoff fixed

* passing

* Retrieve cached state in the beginning

* lint

* Fixed tests part 1

* Fixed rest of the tests

* Minor changes to avoid copying, small refactor to reduce deplicated code

* Handle att req for slot 0

* New beacon state: Only populate merkle layers as needed, copy merkle layers on copy/clone. (#4689)

* Only populate merkle layers as needed, copy merkle layers on copy/clone.

* use custom copy

* Make maps of correct size

* slightly fast, doesn't wait for lock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Target root can't be 0x00

* Don't use cache for current slot (may not be the right fix)

* fixed up tests

* Remove some copy for init sync. Not sure if it is safe enough for runtime though... testing...

* Align with prev logic for process slots cachedState.Slot() < slot

* Fix Initial Sync Flag (#4692)

* fixes

* fix up some test failures due to lack of nil checks

* fix up some test failures due to lack of nil checks

* fix up imports

* revert some changes

* imports

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* resolving further conflicts

* Better skip slot cache (#4694)

* Return copy of skip slot cache state, disable skip slot cache on sync

* fix

* Fix pruning

* fix up issues with broken tests

Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-31 12:57:01 -08:00

299 lines
11 KiB
Go

package helpers
import (
"github.com/pkg/errors"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
stateTrie "github.com/prysmaticlabs/prysm/beacon-chain/state"
pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
"github.com/prysmaticlabs/prysm/shared/bls"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/featureconfig"
"github.com/prysmaticlabs/prysm/shared/hashutil"
"github.com/prysmaticlabs/prysm/shared/params"
)
// IsActiveValidator returns the boolean value on whether the validator
// is active or not.
//
// Spec pseudocode definition:
// def is_active_validator(validator: Validator, epoch: Epoch) -> bool:
// """
// Check if ``validator`` is active.
// """
// return validator.activation_epoch <= epoch < validator.exit_epoch
func IsActiveValidator(validator *ethpb.Validator, epoch uint64) bool {
return checkValidatorActiveStatus(validator.ActivationEpoch, validator.ExitEpoch, epoch)
}
// IsActiveValidatorUsingTrie checks if a read only validator is active.
func IsActiveValidatorUsingTrie(validator *stateTrie.ReadOnlyValidator, epoch uint64) bool {
return checkValidatorActiveStatus(validator.ActivationEpoch(), validator.ExitEpoch(), epoch)
}
func checkValidatorActiveStatus(activationEpoch uint64, exitEpoch uint64, epoch uint64) bool {
return activationEpoch <= epoch && epoch < exitEpoch
}
// IsSlashableValidator returns the boolean value on whether the validator
// is slashable or not.
//
// Spec pseudocode definition:
// def is_slashable_validator(validator: Validator, epoch: Epoch) -> bool:
// """
// Check if ``validator`` is slashable.
// """
// return (
// validator.activation_epoch <= epoch < validator.withdrawable_epoch and
// validator.slashed is False
// )
func IsSlashableValidator(validator *ethpb.Validator, epoch uint64) bool {
active := validator.ActivationEpoch <= epoch
beforeWithdrawable := epoch < validator.WithdrawableEpoch
return beforeWithdrawable && active && !validator.Slashed
}
// ActiveValidatorIndices filters out active validators based on validator status
// and returns their indices in a list.
//
// WARNING: This method allocates a new copy of the validator index set and is
// considered to be very memory expensive. Avoid using this unless you really
// need the active validator indices for some specific reason.
//
// Spec pseudocode definition:
// def get_active_validator_indices(state: BeaconState, epoch: Epoch) -> Sequence[ValidatorIndex]:
// """
// Return the sequence of active validator indices at ``epoch``.
// """
// return [ValidatorIndex(i) for i, v in enumerate(state.validators) if is_active_validator(v, epoch)]
func ActiveValidatorIndices(state *stateTrie.BeaconState, epoch uint64) ([]uint64, error) {
seed, err := Seed(state, epoch, params.BeaconConfig().DomainBeaconAttester)
if err != nil {
return nil, errors.Wrap(err, "could not get seed")
}
activeIndices, err := committeeCache.ActiveIndices(seed)
if err != nil {
return nil, errors.Wrap(err, "could not interface with committee cache")
}
if activeIndices != nil {
return activeIndices, nil
}
var indices []uint64
state.ReadFromEveryValidator(func(idx int, val *stateTrie.ReadOnlyValidator) error {
if IsActiveValidatorUsingTrie(val, epoch) {
indices = append(indices, uint64(idx))
}
return nil
})
if err := UpdateCommitteeCache(state, epoch); err != nil {
return nil, errors.Wrap(err, "could not update committee cache")
}
return indices, nil
}
// ActiveValidatorCount returns the number of active validators in the state
// at the given epoch.
func ActiveValidatorCount(state *stateTrie.BeaconState, epoch uint64) (uint64, error) {
count := uint64(0)
state.ReadFromEveryValidator(func(idx int, val *stateTrie.ReadOnlyValidator) error {
if IsActiveValidatorUsingTrie(val, epoch) {
count++
}
return nil
})
return count, nil
}
// DelayedActivationExitEpoch takes in epoch number and returns when
// the validator is eligible for activation and exit.
//
// Spec pseudocode definition:
// def compute_activation_exit_epoch(epoch: Epoch) -> Epoch:
// """
// Return the epoch during which validator activations and exits initiated in ``epoch`` take effect.
// """
// return Epoch(epoch + 1 + ACTIVATION_EXIT_DELAY)
func DelayedActivationExitEpoch(epoch uint64) uint64 {
return epoch + 1 + params.BeaconConfig().MaxSeedLookahead
}
// ValidatorChurnLimit returns the number of validators that are allowed to
// enter and exit validator pool for an epoch.
//
// Spec pseudocode definition:
// def get_validator_churn_limit(state: BeaconState) -> uint64:
// """
// Return the validator churn limit for the current epoch.
// """
// active_validator_indices = get_active_validator_indices(state, get_current_epoch(state))
// return max(MIN_PER_EPOCH_CHURN_LIMIT, len(active_validator_indices) // CHURN_LIMIT_QUOTIENT)
func ValidatorChurnLimit(activeValidatorCount uint64) (uint64, error) {
churnLimit := activeValidatorCount / params.BeaconConfig().ChurnLimitQuotient
if churnLimit < params.BeaconConfig().MinPerEpochChurnLimit {
churnLimit = params.BeaconConfig().MinPerEpochChurnLimit
}
return churnLimit, nil
}
// BeaconProposerIndex returns proposer index of a current slot.
//
// Spec pseudocode definition:
// def get_beacon_proposer_index(state: BeaconState) -> ValidatorIndex:
// """
// Return the beacon proposer index at the current slot.
// """
// epoch = get_current_epoch(state)
// seed = hash(get_seed(state, epoch, DOMAIN_BEACON_PROPOSER) + int_to_bytes(state.slot, length=8))
// indices = get_active_validator_indices(state, epoch)
// return compute_proposer_index(state, indices, seed)
func BeaconProposerIndex(state *stateTrie.BeaconState) (uint64, error) {
e := CurrentEpoch(state)
if featureconfig.Get().EnableProposerIndexCache {
seed, err := Seed(state, e, params.BeaconConfig().DomainBeaconAttester)
if err != nil {
return 0, errors.Wrap(err, "could not generate seed")
}
proposerIndices, err := committeeCache.ProposerIndices(seed)
if err != nil {
return 0, errors.Wrap(err, "could not interface with committee cache")
}
if proposerIndices != nil {
return proposerIndices[state.Slot()%params.BeaconConfig().SlotsPerEpoch], nil
}
}
seed, err := Seed(state, e, params.BeaconConfig().DomainBeaconProposer)
if err != nil {
return 0, errors.Wrap(err, "could not generate seed")
}
seedWithSlot := append(seed[:], bytesutil.Bytes8(state.Slot())...)
seedWithSlotHash := hashutil.Hash(seedWithSlot)
indices, err := ActiveValidatorIndices(state, e)
if err != nil {
return 0, errors.Wrap(err, "could not get active indices")
}
if featureconfig.Get().EnableProposerIndexCache {
if err := UpdateProposerIndicesInCache(state, CurrentEpoch(state)); err != nil {
return 0, errors.Wrap(err, "could not update committee cache")
}
}
return ComputeProposerIndex(state.Validators(), indices, seedWithSlotHash)
}
// ComputeProposerIndex returns the index sampled by effective balance, which is used to calculate proposer.
//
// Note: This method signature deviates slightly from the spec recommended definition. The full
// state object is not required to compute the proposer index.
//
// Spec pseudocode definition:
// def compute_proposer_index(state: BeaconState, indices: Sequence[ValidatorIndex], seed: Hash) -> ValidatorIndex:
// """
// Return from ``indices`` a random index sampled by effective balance.
// """
// assert len(indices) > 0
// MAX_RANDOM_BYTE = 2**8 - 1
// i = 0
// while True:
// candidate_index = indices[compute_shuffled_index(ValidatorIndex(i % len(indices)), len(indices), seed)]
// random_byte = hash(seed + int_to_bytes(i // 32, length=8))[i % 32]
// effective_balance = state.validators[candidate_index].effective_balance
// if effective_balance * MAX_RANDOM_BYTE >= MAX_EFFECTIVE_BALANCE * random_byte:
// return ValidatorIndex(candidate_index)
// i += 1
func ComputeProposerIndex(validators []*ethpb.Validator, activeIndices []uint64, seed [32]byte) (uint64, error) {
length := uint64(len(activeIndices))
if length == 0 {
return 0, errors.New("empty active indices list")
}
maxRandomByte := uint64(1<<8 - 1)
hashFunc := hashutil.CustomSHA256Hasher()
for i := uint64(0); ; i++ {
candidateIndex, err := ComputeShuffledIndex(i%length, length, seed, true /* shuffle */)
if err != nil {
return 0, err
}
candidateIndex = activeIndices[candidateIndex]
if int(candidateIndex) >= len(validators) {
return 0, errors.New("active index out of range")
}
b := append(seed[:], bytesutil.Bytes8(i/32)...)
randomByte := hashFunc(b)[i%32]
v := validators[candidateIndex]
var effectiveBal uint64
if v != nil {
effectiveBal = v.EffectiveBalance
}
if effectiveBal*maxRandomByte >= params.BeaconConfig().MaxEffectiveBalance*uint64(randomByte) {
return candidateIndex, nil
}
}
}
// Domain returns the domain version for BLS private key to sign and verify.
//
// Spec pseudocode definition:
// def get_domain(state: BeaconState,
// domain_type: int,
// message_epoch: Epoch=None) -> int:
// """
// Return the signature domain (fork version concatenated with domain type) of a message.
// """
// epoch = get_current_epoch(state) if message_epoch is None else message_epoch
// fork_version = state.fork.previous_version if epoch < state.fork.epoch else state.fork.current_version
// return bls_domain(domain_type, fork_version)
func Domain(fork *pb.Fork, epoch uint64, domainType []byte) uint64 {
var forkVersion []byte
if epoch < fork.Epoch {
forkVersion = fork.PreviousVersion
} else {
forkVersion = fork.CurrentVersion
}
return bls.Domain(domainType, forkVersion)
}
// IsEligibleForActivationQueue checks if the validator is eligible to
// be places into the activation queue.
//
// Spec pseudocode definition:
// def is_eligible_for_activation_queue(validator: Validator) -> bool:
// """
// Check if ``validator`` is eligible to be placed into the activation queue.
// """
// return (
// validator.activation_eligibility_epoch == FAR_FUTURE_EPOCH
// and validator.effective_balance == MAX_EFFECTIVE_BALANCE
// )
func IsEligibleForActivationQueue(validator *ethpb.Validator) bool {
return validator.ActivationEligibilityEpoch == params.BeaconConfig().FarFutureEpoch &&
validator.EffectiveBalance == params.BeaconConfig().MaxEffectiveBalance
}
// IsEligibleForActivation checks if the validator is eligible for activation.
//
// Spec pseudocode definition:
// def is_eligible_for_activation(state: BeaconState, validator: Validator) -> bool:
// """
// Check if ``validator`` is eligible for activation.
// """
// return (
// # Placement in queue is finalized
// validator.activation_eligibility_epoch <= state.finalized_checkpoint.epoch
// # Has not yet been activated
// and validator.activation_epoch == FAR_FUTURE_EPOCH
// )
func IsEligibleForActivation(state *stateTrie.BeaconState, validator *ethpb.Validator) bool {
cpt := state.FinalizedCheckpoint()
if cpt == nil {
return false
}
return validator.ActivationEligibilityEpoch <= cpt.Epoch &&
validator.ActivationEpoch == params.BeaconConfig().FarFutureEpoch
}