prysm-pulse/beacon-chain/core/helpers/committee.go
terence tsao 6d2a2ebadf Update run time to v0.9.3 (#4154)
* Remove custody (#3986)

* Update proto fields

* Updated block operations

* Fixed all block operation tests

* Fixed tests part 1

* Fixed tests part 1

* All tests pass

* Clean up

* Skip spec test

* Fixed ssz test

* Skip ssz test

* Skip mainnet tests

* Update beacon-chain/operations/attestation.go

* Update beacon-chain/operations/attestation.go
* Decoy flip flop check (#3987)
* Bounce attack check (#3989)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Update beacon-chain/blockchain/forkchoice/process_block.go
* Conflict
* Unskip mainnet spec tests (#3998)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update beacon-chain/core/blocks/block_operations.go
* Unskip minimal spec tests (#3999)

* Starting

* Fixed attestation mainnet test

* Unskip ssz static and block processing tests

* Fixed workspace

* fixed workspace

* fixed workspace

* Update workspace

* Unskip all minimal spec tests

* Update workspace for general test
* Unskip test (#4001)
* Update minimal seconds per slot to 6 (#3978)
* Bounce attack tests (#3993)

* New store values

* Update process block

* Update process attestation

* Update tests

* Helper

* Fixed blockchain package tests

* Slots since epoch starts tests

* Update justified checkpt tests

* Conflict

* Fixed logic

* Update process_block.go

* Use helper
* Conflict
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.1
* Conflict
* Fixed failed tests
* Lower MinGenesisActiveValidatorCount to 16384 (#4100)
* Fork choice beacon block checks (#4107)

* Prevent future blocks check and test

* Removed old code
* Update aggregation proto (#4121)

* Update def
* Update spec test
* Conflict
* Update workspace
* patch
* Resolve conflict
* Patch
* Change workspace
* Update ethereumapis to a forked branch at commit 6eb1193e47
* Fixed all the tests
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into conflict
* fix patch
* Need to regenerate test data
* Merge branch 'master' into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Enable snappy compression for all (#4157)

* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* enable snappy compression for all
* Validate aggregate and proof subscriber (#4159)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Conflict
* Update workspace
* Conflict
* Conflict
* Conflict
* Merge branch 'master' into v0.9.2
* Merge branch 'master' into v0.9.2
* Conflict
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Remove migrate to snappy  (#4205)
* Feature flag: Deprecate --prune-states, release to all (#4204)

* Deprecated prune-states, release to all

* imports

* remote unused import

* remove unused import

* Rm prune state test

* gaz
* Refactoring for dynamic pubsub subscriptions for non-aggregated attestations (#4189)

* checkpoint progress

* chkpt

* checkpoint progress

* put pipeline in its own file

* remove unused imports

* add test, it's failing though

* fix test

* remove head state issue

* add clear db flag to e2e

* add some more error handling, debug logging

* skip processing if chain has not started

* fix test

* wrap in go routine to see if anything breaks

* remove duplicated topic

* Add a regression test. Thanks @nisdas for finding the original problem. May it never happen again *fingers crossed*

* Comments

* gofmt

* comment out with TODO
* Sync with master
* Sync with master
* RPC servers use attestation pool (#4223)
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Refactor RPC to Fully Utilize Ethereum APIs (#4243)

* include attester as a file in the validator server

* remove old proposer server impl

* include new patch and properly sync changes

* align with public pbs

* ensure matches rpc def

* fix up status tests

* resolve all broken test files in the validator rpc package

* gazelle include

* fix up the duties implementation

* fixed up all get duties functions

* all tests pass

* utilize new ethereum apis

* amend validator client to use the new beacon node validator rpc client

* fix up most of validator items

* added in mock

* fix up test

* readd test

* add chain serv mock

* fix a few more validator methods

* all validator tests passingggg

* fix broken test

* resolve even more broken tests

* all tests passsssss

* fix lint

* try PR

* fix up test

* resolve broken other tests
* Sync with master
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into v0.9.2
* Aggregate and proof subscriber (#4240)

* Added subscribers

* Fixed conflict

* Tests

* fix up patch

* Use upstream pb

* include latest patch

* Fmt

* Save state before head block
* skip tests (#4275)
* Delete block attestations from the pool (#4241)

* Added subscribers
* Clean up
* Fixed conflict
* Delete atts in pool in validate pipeline
* Moved it to subscriber
* Merge branch 'v0.9.2' of https://github.com/prysmaticlabs/prysm into use-att-pool-3
* Test
* Fixed test
* Initial work on voluntary exit (#4207)

* Initial implementation of voluntary exit: RPC call

* Update for recent merges

* Break out validation logic for voluntary exits to core module

* RequestExit -> ProposeExit

* Decrease exit package visibility

* Move to operation feed

* Wrap errors
* Fix critical proposer selection bug #4259 (#4265)

* fix critical proposer selection bug #4259

* gofmt

* add 1 more validator to make it 5

* more tests

* Fixed archivedProposerIndex

* Fixed TestFilterAttestation_OK

* Refactor ComputeProposerIndex, add regression test for potential out of range panic

* handle case of nil validator

* Update validators_test.go
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Leftover merge files, oops
* gaz
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* Fixes Duplicate Validator Bug (#4322)

* Update dict

* Test helper

* Regression test

* Comment

* Reset test cache
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fixes after PR #4328
* Complete attestation pool for run time (#4286)

* Added subscribers

* Fixed conflict

* Delete atts in pool in validate pipeline

* Moved it to subscriber

* Test

* Fixed test

* New curl for forkchoice attestations

* Starting att pool service for fork choice

* Update pool interface

* Update pool interface

* Update sync and node

* Lint

* Gazelle

* Updated servers, filled in missing functionalities

* RPC working with 1 beacon node 64 validators

* Started writing tests. Yay

* Test to aggregate and save multiple fork choice atts

* Tests for BatchAttestations for fork choice

* Fixed exisiting tests

* Minor fixes

* Fmt

* Added batch saves

* Lint

* Mo tests yay

* Delete test

* Fmt

* Update interval

* Fixed aggregation broadcast

* Clean up based on design review comment

* Fixed setupBeaconChain

* Raul's feedback. s/error/err
* resolve conflicts
* Merge branch 'v0.9.2' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Removed old protos and fixed tests (#4336)
* Merge refs/heads/master into v0.9.2
* Disallow duplicated indices and test (#4339)
* Explicit use of GENESIS_SLOT in fork choice (#4343)
* Update from 2 to 3 (#4345)
* Remove verify unaggregated attestation when aggregating (#4347)
* use slot ticker instead of run every (#4348)
* Add context check for unbounded loop work (#4346)
* Revert "Explicit use of GENESIS_SLOT in fork choice (#4343)" (#4349)

This reverts commit d3f6753c77f8f733563d00ab649c5159b2c2926f.
* Refactor Powchain Service (#4306)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* fix lint

* raul's comments

* use one method

* fix test

* raul's comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Ensure best better-justification is stored for fork choice (#4342)

* Ensure best better-justification is stored. Minor refactor
* Tests
* Merge refs/heads/v0.9.2 into better-best-justified
* Merge refs/heads/v0.9.2 into better-best-justified
* Ensure that epoch of attestation slot matches the target epoch (#4341)

* Disallow duplicated indices and test
* Add slot to target epoch check to on_attestation
* Add slot to target epoch check to process_attestation
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into no-dup-att-indices
* Fixed TestProcessAttestations_PrevEpochFFGDataMismatches
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Update beacon-chain/blockchain/forkchoice/process_attestation_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>
* Merge refs/heads/v0.9.2 into no-dup-att-indices
* Filter viable branches in fork choice (#4355)
* Only activate upon finality (#4359)

* Updated functions
* Tests
* Merge branch 'v0.9.2' of git+ssh://github.com/prysmaticlabs/prysm into queue-fix-on-finality
* Comment
* Merge refs/heads/v0.9.2 into queue-fix-on-finality
* Fixed failing test from 4359 (#4360)

* Fixed
* Skip registry spec tests
* Wait for state to be initialized at least once before running slot ticker based on genesis time (#4364)
* Sync with master
* Fix checkpoint root to  use genesis block root (#4368)
* Return an error on nil head state in fork choice (#4369)

* Return error if nil head state

* Fixed tests. Saved childen blocks state

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Update metrics every epoch (#4367)
* return empty slice if state is nil (#4365)
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* Pubsub: Broadcast attestations to committee based subnets (#4316)

* Working on un-aggregated pubsub topics

* update subscriber to call pool

* checkpointing

* fix

* untested message validation

* minor fixes

* rename slotsSinceGenesis to slotsSince

* some progress on a unit test, subscribe is not being called still...

* dont change topic

* need to set the data on the message

* restore topic

* fixes

* some helpful parameter changes for mainnet operations

* lint

* Terence feedback

* unskip e2e

* Unit test for validate committee index beacon attestation

* PR feedbacK

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into resolveConflicts
* remove condition
* Remove unused operation pool (#4361)
* Merge refs/heads/master into v0.9.2
* Aggregate attestations periodically  (#4376)
* Persist ETH1 Data to Disk (#4329)

* add data structures

* generate proto

* add in new fields

* add comments

* add new mock state

* add new mock state

* add new methods

* some more changes

* check genesis time properly

* lint

* fix refs

* fix tests

* lint

* lint

* lint

* gaz

* adding in new proto message

* remove outdated vars

* add new changes

* remove latest eth1data

* continue refactoring

* finally works

* lint

* fix test

* fix all tests

* fix all tests again

* fix build

* change back

* add full eth1 test

* fix logs and test

* add constant

* changes

* fix bug

* lint

* fix another bug

* change back

* Apply suggestions from code review

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fixed VerifyIndexedAttestation (#4382)
* rm signing root (#4381)

* rm signing root

* Fixed VerifyIndexedAttestation

* Check proposer slashed status inside ProcessBlockHeaderNoVerify

* Fixed TestUpdateJustified_CouldUpdateBest

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove Redundant Trie Generation (#4383)

* remove trie generation
* remove deposit hashes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.9.2
* fix build
* Conflict
* Implement StreamAttestations RPC Endpoint (#4390)

* started attestation stream

* stream attestations test

* on slot tick test passing

* imports

* gaz

* Update beacon-chain/rpc/beacon/attestations_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

Co-authored-by: shayzluf <thezluf@gmail.com>
* Fixed goimport (#4394)
* Use custom stateutil ssz for ssz HTR spec tests (#4396)

* Use custom stateutil ssz for ssz HTR spec tests

* gofmt
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge refs/heads/master into v0.9.2
* set mainnet to be the default for build and run (#4398)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* gracefully handle deduplicated registration of topic validators (#4399)

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* SSZ: temporarily disable roots cache until cache issues can be resolved (#4407)

* temporarily disable roots cache until cache issues can be resolved

* Also use custom ssz for spectests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Remove process block attestations as separate routine (#4408)

* Removed old save/process block atts

* Fixed tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save Deposit Cache to Disk (#4384)

* change to protos

* fix build

* glue everything together

* fix test

* raul's review

* preston's comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix activation queue sorting (#4409)

* Removed old save/process block atts

* Fixed tests

* Proper sorting by eligibility epoch then by indices

* Deleted old colde
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Merge branch 'master' into v0.9.2
* Merge refs/heads/master into v0.9.2
* stop recursive lookup if context is cancelled (#4420)
* Fix proposal bug (#4419)
* Add Pending Deposits Safely (#4422)

* safely prune cache

* use proper method

* preston's,terence's reviews and comments

* revert change to build files

* use as feature config instead
* Release custom state ssz (#4421)

* Release custom state ssz, change all HTR of beacon state to use custom method

* typo

* use mainnet config

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.9.2
* Update initial sync save justified to align with v0.9.3 (#4432)
* Merge refs/heads/master into v0.9.2
* Merge refs/heads/master into v0.9.2
* fix build
* don't blacklist on pubsub (#4435)
* Fix Flakey Slot Ticker Test (#4434)

* use interface instead for the slot ticker

* fixed up flakey tests

* add gen time

* get duties comment

* fix lifecycle test

* more fixes
* Configurable min genesis delay (#4437)

* Configurable min genesis delay based on https://github.com/ethereum/eth2.0-specs/pull/1557

* remove feature flag for genesis delay

* fix

* demo config feedback
* patch readme
* save keys unencrypted for validators (#4439)
* Add new demo configuration targeting mainnet scale (#4397)

* Add new demo configuration targeting mainnet, with 1/10th of the deposit value

* reduce quotant by 1/10th. Use 1/10th mainnet values

* only change the inactivity quotant

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Save justified checkpoint state (#4433)

* Save justified checkpoint state

* Lint

* Feedback

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update shared/testutil/deposits.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update proto/testing/ssz_regression_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/epoch/epoch_processing.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/kv/forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber_beacon_blocks_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/sync/subscriber.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/proposer.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/prepare_forkchoice.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/operations/attestations/pool.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/powchain/log_processing_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/aggregator/server.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/rpc/validator/exit_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/cache/depositcache/pending_deposits.go
* Update beacon-chain/cache/depositcache/pending_deposits_test.go
* Update beacon-chain/rpc/validator/proposer.go
* Merge refs/heads/master into v0.9.2
* Fix e2e genesis delay issues (#4442)

* fix e2e genesis delay issues

* register flag

* typo

* Update shared/featureconfig/config.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* skip demo e2e

* fix validator

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Batch Eth1 RPC Calls (#4392)

* add new methods

* get it working

* optimize past deposit logs processing

* revert change

* fix all tests

* use mock

* lint

* lint

* check for nil

* stop panics

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Terence's Review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-01-07 18:47:39 +00:00

442 lines
16 KiB
Go

// Package helpers contains helper functions outlined in ETH2.0 spec beacon chain spec
package helpers
import (
"fmt"
"sort"
"github.com/pkg/errors"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/go-bitfield"
"github.com/prysmaticlabs/prysm/beacon-chain/cache"
pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
"github.com/prysmaticlabs/prysm/shared/featureconfig"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/shared/sliceutil"
)
var committeeCache = cache.NewCommitteesCache()
// SlotCommitteeCount returns the number of crosslink committees of a slot. The
// active validator count is provided as an argument rather than a direct implementation
// from the spec definition. Having the active validator count as an argument allows for
// cheaper computation, instead of retrieving head state, one can retrieve the validator
// count.
//
//
// Spec pseudocode definition:
// def get_committee_count_at_slot(state: BeaconState, slot: Slot) -> uint64:
// """
// Return the number of committees at ``slot``.
// """
// epoch = compute_epoch_at_slot(slot)
// return max(1, min(
// MAX_COMMITTEES_PER_SLOT,
// len(get_active_validator_indices(state, epoch)) // SLOTS_PER_EPOCH // TARGET_COMMITTEE_SIZE,
// ))
func SlotCommitteeCount(activeValidatorCount uint64) uint64 {
var committeePerSlot = activeValidatorCount / params.BeaconConfig().SlotsPerEpoch / params.BeaconConfig().TargetCommitteeSize
if committeePerSlot > params.BeaconConfig().MaxCommitteesPerSlot {
return params.BeaconConfig().MaxCommitteesPerSlot
}
if committeePerSlot == 0 {
return 1
}
return committeePerSlot
}
// BeaconCommitteeFromState returns the crosslink committee of a given slot and committee index. This
// is a spec implementation where state is used as an argument. In case of state retrieval
// becomes expensive, consider using BeaconCommittee below.
//
// Spec pseudocode definition:
// def get_beacon_committee(state: BeaconState, slot: Slot, index: CommitteeIndex) -> Sequence[ValidatorIndex]:
// """
// Return the beacon committee at ``slot`` for ``index``.
// """
// epoch = compute_epoch_at_slot(slot)
// committees_per_slot = get_committee_count_at_slot(state, slot)
// epoch_offset = index + (slot % SLOTS_PER_EPOCH) * committees_per_slot
// return compute_committee(
// indices=get_active_validator_indices(state, epoch),
// seed=get_seed(state, epoch, DOMAIN_BEACON_ATTESTER),
// index=epoch_offset,
// count=committees_per_slot * SLOTS_PER_EPOCH,
// )
func BeaconCommitteeFromState(state *pb.BeaconState, slot uint64, committeeIndex uint64) ([]uint64, error) {
epoch := SlotToEpoch(slot)
seed, err := Seed(state, epoch, params.BeaconConfig().DomainBeaconAttester)
if err != nil {
return nil, errors.Wrap(err, "could not get seed")
}
if featureconfig.Get().EnableNewCache {
indices, err := committeeCache.Committee(slot, seed, committeeIndex)
if err != nil {
return nil, errors.Wrap(err, "could not interface with committee cache")
}
if indices != nil {
return indices, nil
}
}
indices, err := ActiveValidatorIndices(state, epoch)
if err != nil {
return nil, errors.Wrap(err, "could not get active indices")
}
return BeaconCommittee(indices, seed, slot, committeeIndex)
}
// BeaconCommittee returns the crosslink committee of a given slot and committee index. The
// validator indices and seed are provided as an argument rather than a direct implementation
// from the spec definition. Having them as an argument allows for cheaper computation run time.
func BeaconCommittee(validatorIndices []uint64, seed [32]byte, slot uint64, committeeIndex uint64) ([]uint64, error) {
if featureconfig.Get().EnableNewCache {
indices, err := committeeCache.Committee(slot, seed, committeeIndex)
if err != nil {
return nil, errors.Wrap(err, "could not interface with committee cache")
}
if indices != nil {
return indices, nil
}
}
committeesPerSlot := SlotCommitteeCount(uint64(len(validatorIndices)))
epochOffset := committeeIndex + (slot%params.BeaconConfig().SlotsPerEpoch)*committeesPerSlot
count := committeesPerSlot * params.BeaconConfig().SlotsPerEpoch
return ComputeCommittee(validatorIndices, seed, epochOffset, count)
}
// BeaconCommitteeWithoutCache returns the crosslink committee of a given slot and committee index without the
// usage of committee cache.
// TODO(3603): Delete this function when issue 3603 closes.
func BeaconCommitteeWithoutCache(state *pb.BeaconState, slot uint64, index uint64) ([]uint64, error) {
epoch := SlotToEpoch(slot)
activeValidatorCount, err := ActiveValidatorCount(state, epoch)
if err != nil {
return nil, err
}
committeesPerSlot := SlotCommitteeCount(activeValidatorCount)
epochOffset := index + (slot%params.BeaconConfig().SlotsPerEpoch)*committeesPerSlot
count := committeesPerSlot * params.BeaconConfig().SlotsPerEpoch
seed, err := Seed(state, epoch, params.BeaconConfig().DomainBeaconAttester)
if err != nil {
return nil, errors.Wrap(err, "could not get seed")
}
indices, err := ActiveValidatorIndices(state, epoch)
if err != nil {
return nil, errors.Wrap(err, "could not get active indices")
}
return ComputeCommittee(indices, seed, epochOffset, count)
}
// ComputeCommittee returns the requested shuffled committee out of the total committees using
// validator indices and seed.
//
// Spec pseudocode definition:
// def compute_committee(indices: Sequence[ValidatorIndex],
// seed: Hash,
// index: uint64,
// count: uint64) -> Sequence[ValidatorIndex]:
// """
// Return the committee corresponding to ``indices``, ``seed``, ``index``, and committee ``count``.
// """
// start = (len(indices) * index) // count
// end = (len(indices) * (index + 1)) // count
// return [indices[compute_shuffled_index(ValidatorIndex(i), len(indices), seed)] for i in range(start, end)
func ComputeCommittee(
indices []uint64,
seed [32]byte,
index uint64,
count uint64,
) ([]uint64, error) {
validatorCount := uint64(len(indices))
start := sliceutil.SplitOffset(validatorCount, count, index)
end := sliceutil.SplitOffset(validatorCount, count, index+1)
// Save the shuffled indices in cache, this is only needed once per epoch or once per new committee index.
shuffledIndices := make([]uint64, end-start)
for i := start; i < end; i++ {
permutedIndex, err := ShuffledIndex(i, validatorCount, seed)
if err != nil {
return []uint64{}, errors.Wrapf(err, "could not get shuffled index at index %d", i)
}
shuffledIndices[i-start] = indices[permutedIndex]
}
return shuffledIndices, nil
}
// AttestingIndices returns the attesting participants indices from the attestation data. The
// committee is provided as an argument rather than a direct implementation from the spec definition.
// Having the committee as an argument allows for re-use of beacon committees when possible.
//
// Spec pseudocode definition:
// def get_attesting_indices(state: BeaconState,
// data: AttestationData,
// bits: Bitlist[MAX_VALIDATORS_PER_COMMITTEE]) -> Set[ValidatorIndex]:
// """
// Return the set of attesting indices corresponding to ``data`` and ``bits``.
// """
// committee = get_beacon_committee(state, data.slot, data.index)
// return set(index for i, index in enumerate(committee) if bits[i])
func AttestingIndices(bf bitfield.Bitfield, committee []uint64) ([]uint64, error) {
indices := make([]uint64, 0, len(committee))
indicesSet := make(map[uint64]bool)
for i, idx := range committee {
if !indicesSet[idx] {
if bf.BitAt(uint64(i)) {
indices = append(indices, idx)
}
}
indicesSet[idx] = true
}
return indices, nil
}
// CommitteeAssignmentContainer represents a committee, index, and attester slot for a given epoch.
type CommitteeAssignmentContainer struct {
Committee []uint64
AttesterSlot uint64
CommitteeIndex uint64
}
// CommitteeAssignments is a map of validator indices pointing to the appropriate committee
// assignment for the given epoch.
//
// 1. Determine the proposer validator index for each slot.
// 2. Compute all committees.
// 3. Determine the attesting slot for each committee.
// 4. Construct a map of validator indices pointing to the respective committees.
func CommitteeAssignments(state *pb.BeaconState, epoch uint64) (map[uint64]*CommitteeAssignmentContainer, map[uint64]uint64, error) {
if epoch > NextEpoch(state) {
return nil, nil, fmt.Errorf(
"epoch %d can't be greater than next epoch %d",
epoch,
NextEpoch(state),
)
}
// Track which slot has which proposer.
startSlot := StartSlot(epoch)
proposerIndexToSlot := make(map[uint64]uint64)
for slot := startSlot; slot < startSlot+params.BeaconConfig().SlotsPerEpoch; slot++ {
state.Slot = slot
i, err := BeaconProposerIndex(state)
if err != nil {
return nil, nil, errors.Wrapf(err, "could not check proposer at slot %d", state.Slot)
}
proposerIndexToSlot[i] = slot
}
activeValidatorIndices, err := ActiveValidatorIndices(state, epoch)
if err != nil {
return nil, nil, err
}
// Each slot in an epoch has a different set of committees. This value is derived from the
// active validator set, which does not change.
numCommitteesPerSlot := SlotCommitteeCount(uint64(len(activeValidatorIndices)))
validatorIndexToCommittee := make(map[uint64]*CommitteeAssignmentContainer)
// Compute all committees for all slots.
for i := uint64(0); i < params.BeaconConfig().SlotsPerEpoch; i++ {
// Compute committees.
for j := uint64(0); j < numCommitteesPerSlot; j++ {
slot := startSlot + i
committee, err := BeaconCommitteeFromState(state, slot, j /*committee index*/)
if err != nil {
return nil, nil, err
}
cac := &CommitteeAssignmentContainer{
Committee: committee,
CommitteeIndex: j,
AttesterSlot: slot,
}
for _, vID := range committee {
validatorIndexToCommittee[vID] = cac
}
}
}
return validatorIndexToCommittee, proposerIndexToSlot, nil
}
// CommitteeAssignment is used to query committee assignment from
// current and previous epoch.
//
// Deprecated: Consider using CommitteeAssignments, especially when computing more than one
// validator assignment as this method is O(n^2) in computational complexity. This method exists to
// ensure spec definition conformance and otherwise should probably not be used.
//
// Spec pseudocode definition:
// def get_committee_assignment(state: BeaconState,
// epoch: Epoch,
// validator_index: ValidatorIndex
// ) -> Optional[Tuple[Sequence[ValidatorIndex], CommitteeIndex, Slot]]:
// """
// Return the committee assignment in the ``epoch`` for ``validator_index``.
// ``assignment`` returned is a tuple of the following form:
// * ``assignment[0]`` is the list of validators in the committee
// * ``assignment[1]`` is the index to which the committee is assigned
// * ``assignment[2]`` is the slot at which the committee is assigned
// Return None if no assignment.
// """
// next_epoch = get_current_epoch(state) + 1
// assert epoch <= next_epoch
//
// start_slot = compute_start_slot_at_epoch(epoch)
// for slot in range(start_slot, start_slot + SLOTS_PER_EPOCH):
// for index in range(get_committee_count_at_slot(state, Slot(slot))):
// committee = get_beacon_committee(state, Slot(slot), CommitteeIndex(index))
// if validator_index in committee:
// return committee, CommitteeIndex(index), Slot(slot)
// return None
func CommitteeAssignment(
state *pb.BeaconState,
epoch uint64,
validatorIndex uint64,
) ([]uint64, uint64, uint64, uint64, error) {
if epoch > NextEpoch(state) {
return nil, 0, 0, 0, fmt.Errorf(
"epoch %d can't be greater than next epoch %d",
epoch, NextEpoch(state))
}
// Track which slot has which proposer.
startSlot := StartSlot(epoch)
proposerIndexToSlot := make(map[uint64]uint64)
for slot := startSlot; slot < startSlot+params.BeaconConfig().SlotsPerEpoch; slot++ {
state.Slot = slot
i, err := BeaconProposerIndex(state)
if err != nil {
return nil, 0, 0, 0, errors.Wrapf(err, "could not check proposer at slot %d", state.Slot)
}
proposerIndexToSlot[i] = slot
}
activeValidatorIndices, err := ActiveValidatorIndices(state, epoch)
if err != nil {
return nil, 0, 0, 0, err
}
for slot := startSlot; slot < startSlot+params.BeaconConfig().SlotsPerEpoch; slot++ {
countAtSlot := SlotCommitteeCount(uint64(len(activeValidatorIndices)))
for i := uint64(0); i < countAtSlot; i++ {
committee, err := BeaconCommitteeFromState(state, slot, i)
if err != nil {
return nil, 0, 0, 0, errors.Wrapf(err, "could not get crosslink committee at slot %d", slot)
}
for _, v := range committee {
if validatorIndex == v {
proposerSlot, _ := proposerIndexToSlot[v]
return committee, i, slot, proposerSlot, nil
}
}
}
}
return []uint64{}, 0, 0, 0, fmt.Errorf("validator with index %d not found in assignments", validatorIndex)
}
// VerifyBitfieldLength verifies that a bitfield length matches the given committee size.
func VerifyBitfieldLength(bf bitfield.Bitfield, committeeSize uint64) error {
if bf.Len() != committeeSize {
return fmt.Errorf(
"wanted participants bitfield length %d, got: %d",
committeeSize,
bf.Len())
}
return nil
}
// VerifyAttestationBitfieldLengths verifies that an attestations aggregation bitfields is
// a valid length matching the size of the committee.
func VerifyAttestationBitfieldLengths(state *pb.BeaconState, att *ethpb.Attestation) error {
committee, err := BeaconCommitteeFromState(state, att.Data.Slot, att.Data.CommitteeIndex)
if err != nil {
return errors.Wrap(err, "could not retrieve beacon committees")
}
if committee == nil {
return errors.New("no committee exist for this attestation")
}
if err := VerifyBitfieldLength(att.AggregationBits, uint64(len(committee))); err != nil {
return errors.Wrap(err, "failed to verify aggregation bitfield")
}
return nil
}
// ShuffledIndices uses input beacon state and returns the shuffled indices of the input epoch,
// the shuffled indices then can be used to break up into committees.
func ShuffledIndices(state *pb.BeaconState, epoch uint64) ([]uint64, error) {
seed, err := Seed(state, epoch, params.BeaconConfig().DomainBeaconAttester)
if err != nil {
return nil, errors.Wrapf(err, "could not get seed for epoch %d", epoch)
}
indices := make([]uint64, 0, len(state.Validators))
for i, v := range state.Validators {
if IsActiveValidator(v, epoch) {
indices = append(indices, uint64(i))
}
}
validatorCount := uint64(len(indices))
shuffledIndices := make([]uint64, validatorCount)
for i := 0; i < len(shuffledIndices); i++ {
permutedIndex, err := ShuffledIndex(uint64(i), validatorCount, seed)
if err != nil {
return []uint64{}, errors.Wrapf(err, "could not get shuffled index at index %d", i)
}
shuffledIndices[i] = indices[permutedIndex]
}
return shuffledIndices, nil
}
// UpdateCommitteeCache gets called at the beginning of every epoch to cache the committee shuffled indices
// list with committee index and epoch number. It caches the shuffled indices for current epoch and next epoch.
func UpdateCommitteeCache(state *pb.BeaconState, epoch uint64) error {
for _, epoch := range []uint64{epoch, epoch + 1} {
shuffledIndices, err := ShuffledIndices(state, epoch)
if err != nil {
return err
}
count := SlotCommitteeCount(uint64(len(shuffledIndices)))
seed, err := Seed(state, epoch, params.BeaconConfig().DomainBeaconAttester)
if err != nil {
return err
}
// Store the sorted indices as well as shuffled indices. In current spec,
// sorted indices is required to retrieve proposer index. This is also
// used for failing verify signature fallback.
sortedIndices := make([]uint64, len(shuffledIndices))
copy(sortedIndices, shuffledIndices)
sort.Slice(sortedIndices, func(i, j int) bool {
return sortedIndices[i] < sortedIndices[j]
})
if err := committeeCache.AddCommitteeShuffledList(&cache.Committees{
ShuffledIndices: shuffledIndices,
CommitteeCount: count * params.BeaconConfig().SlotsPerEpoch,
Seed: seed,
SortedIndices: sortedIndices,
}); err != nil {
return err
}
}
return nil
}