prysm-pulse/beacon-chain/core/state/state.go
Raul Jordan cc741ed8af
Ensure New State Type Tests Pass in Prysm (#4646)
* begin state service

* begin on the state trie idea

* created beacon state structure

* add in the full clone getter

* return by value instead

* add all setters

* new state setters are being completed

* arrays roots exposed

*  close to finishing all these headerssss

* functionality complete

* added in proto benchmark test

* test for compatibility

* add test for compat

* comments fixed

* add clone

* add clone

* remove underlying copies

* make it immutable

* integrate it into chainservice

* revert

* wrap up comments for package

* address all comments and godocs

* address all comments

* clone the pending attestation properly

* properly clone remaining items

* tests pass fixed bug

* begin using it instead of head state

* prevent nil pointer exceptions

* begin using new struct in db

* integrated new type into db package

* add proper nil checks

* using new state in archiver

* refactored much of core

* editing all the precompute functions

* done with most core refactor

* fixed up some bugs in the clone comparisons

* append current epoch atts

* add missing setters

* add new setters

* fix other core methods

* fix up transition

* main service and forkchoice

* fix rpc

* integrated to powchain

* some more changes

* fix build

* improve processing of deposits

* fix error

* prevent panic

* comment

* fix process att

* gaz

* fix up att process

* resolve existing review comments

* resolve another batch of gh comments

* resolve broken cpt state

* revise testutil to use the new state

* begin updating the state transition func to pass in more compartmentalized args

* finish editing transition function to return errors

* block operations pretty much done with refactor

* state transition fully refactored

* got epoch processing completed

* fix build in fork choice

* fixing more of the build

* fix up broken sync package

* it builds nowww it buildssss

* revert registry changes

* Recompute on Read (#4627)

* compute on read

* fix up eth1 data votes

* looking into slashings bug introduced in core/

* able to advance more slots

* add logging

* can now sync with testnet yay

* remove the leaves algorithm and other merkle imports

* expose initialize unsafe funcs

* Update beacon-chain/db/kv/state.go

* lint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* More Optimizations for New State (#4641)

* map optimization

* more optimizations

* use a custom hasher

* comment

* block operations optimizations

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* fixed up various operations to use the validator index map access

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* archiver tests pass

* fixing cache tests

* cache tests passing

* edited validator tests

* powchain tests passing

* halfway thru sync tests

* more sync test fixes

* add in tests for state/

* working through rpc tests

* assignments tests passed

* almost done with rpc/beacon tests

* resolved painful validator test

* fixed up even more tests

* resolve tests

* fix build

* reduce a randao mixes copy

* fixes under //beacon-chain/blockchain/...

* build //beacon-chain/core/...

* fixes

* Runtime Optimizations (#4648)

* parallelize shuffling

* clean up

* lint

* fix build

* use callback to read from registry

* fix array roots and size map

* new improvements

* reduce hash allocs

* improved shuffling

* terence's review

* use different method

* raul's comment

* new array roots

* remove clone in pre-compute

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul's review

* lint

* fix build issues

* fix visibility

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fix visibility

* build works for all

* fix blockchain test

* fix a few tests

* fix more tests

* update validator in slashing

* archiver passing

* fixed rpc/validator

* progress on core tests

* resolve broken rpc tests

* blockchain tests passed

* fix up some tests in core

* fix message diff

* remove unnecessary save

* Save validator after slashing

* Update validators one by one

* another update

* fix everything

* fix more precompute tests

* fix blocks tests

* more elegant fix

* more helper fixes

* change back ?

* fix test

* fix skip slot

* fix test

* reset caches

* fix testutil

* raceoff fixed

* passing

* Retrieve cached state in the beginning

* lint

* Fixed tests part 1

* Fixed rest of the tests

* Minor changes to avoid copying, small refactor to reduce deplicated code

* Handle att req for slot 0

* New beacon state: Only populate merkle layers as needed, copy merkle layers on copy/clone. (#4689)

* Only populate merkle layers as needed, copy merkle layers on copy/clone.

* use custom copy

* Make maps of correct size

* slightly fast, doesn't wait for lock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Target root can't be 0x00

* Don't use cache for current slot (may not be the right fix)

* fixed up tests

* Remove some copy for init sync. Not sure if it is safe enough for runtime though... testing...

* Align with prev logic for process slots cachedState.Slot() < slot

* Fix Initial Sync Flag (#4692)

* fixes

* fix up some test failures due to lack of nil checks

* fix up some test failures due to lack of nil checks

* fix up imports

* revert some changes

* imports

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* resolving further conflicts

* Better skip slot cache (#4694)

* Return copy of skip slot cache state, disable skip slot cache on sync

* fix

* Fix pruning

* fix up issues with broken tests

Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-31 12:57:01 -08:00

245 lines
8.3 KiB
Go

// Package state implements the whole state transition
// function which consists of per slot, per-epoch transitions.
// It also bootstraps the genesis beacon state for slot 0.
package state
import (
"context"
"github.com/pkg/errors"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/go-ssz"
b "github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
stateTrie "github.com/prysmaticlabs/prysm/beacon-chain/state"
pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/prysmaticlabs/prysm/shared/trieutil"
)
// GenesisBeaconState gets called when MinGenesisActiveValidatorCount count of
// full deposits were made to the deposit contract and the ChainStart log gets emitted.
//
// Spec pseudocode definition:
// def initialize_beacon_state_from_eth1(eth1_block_hash: Hash,
// eth1_timestamp: uint64,
// deposits: Sequence[Deposit]) -> BeaconState:
// state = BeaconState(
// genesis_time=eth1_timestamp - eth1_timestamp % SECONDS_PER_DAY + 2 * SECONDS_PER_DAY,
// eth1_data=Eth1Data(block_hash=eth1_block_hash, deposit_count=len(deposits)),
// latest_block_header=BeaconBlockHeader(body_root=hash_tree_root(BeaconBlockBody())),
// randao_mixes=[eth1_block_hash] * EPOCHS_PER_HISTORICAL_VECTOR, # Seed RANDAO with Eth1 entropy
// )
//
// # Process deposits
// leaves = list(map(lambda deposit: deposit.data, deposits))
// for index, deposit in enumerate(deposits):
// deposit_data_list = List[DepositData, 2**DEPOSIT_CONTRACT_TREE_DEPTH](*leaves[:index + 1])
// state.eth1_data.deposit_root = hash_tree_root(deposit_data_list)
// process_deposit(state, deposit)
//
// # Process activations
// for index, validator in enumerate(state.validators):
// balance = state.balances[index]
// validator.effective_balance = min(balance - balance % EFFECTIVE_BALANCE_INCREMENT, MAX_EFFECTIVE_BALANCE)
// if validator.effective_balance == MAX_EFFECTIVE_BALANCE:
// validator.activation_eligibility_epoch = GENESIS_EPOCH
// validator.activation_epoch = GENESIS_EPOCH
//
// # Populate active_index_roots and compact_committees_roots
// indices_list = List[ValidatorIndex, VALIDATOR_REGISTRY_LIMIT](get_active_validator_indices(state, GENESIS_EPOCH))
// active_index_root = hash_tree_root(indices_list)
// committee_root = get_compact_committees_root(state, GENESIS_EPOCH)
// for index in range(EPOCHS_PER_HISTORICAL_VECTOR):
// state.active_index_roots[index] = active_index_root
// state.compact_committees_roots[index] = committee_root
// return state
// This method differs from the spec so as to process deposits beforehand instead of the end of the function.
func GenesisBeaconState(deposits []*ethpb.Deposit, genesisTime uint64, eth1Data *ethpb.Eth1Data) (*stateTrie.BeaconState, error) {
if eth1Data == nil {
return nil, errors.New("no eth1data provided for genesis state")
}
state, err := EmptyGenesisState()
if err != nil {
return nil, err
}
// Process initial deposits.
leaves := [][]byte{}
for _, deposit := range deposits {
hash, err := ssz.HashTreeRoot(deposit.Data)
if err != nil {
return nil, err
}
leaves = append(leaves, hash[:])
}
var trie *trieutil.SparseMerkleTrie
if len(leaves) > 0 {
trie, err = trieutil.GenerateTrieFromItems(leaves, int(params.BeaconConfig().DepositContractTreeDepth))
if err != nil {
return nil, err
}
} else {
trie, err = trieutil.NewTrie(int(params.BeaconConfig().DepositContractTreeDepth))
if err != nil {
return nil, err
}
}
depositRoot := trie.Root()
eth1Data.DepositRoot = depositRoot[:]
err = state.SetEth1Data(eth1Data)
if err != nil {
return nil, err
}
for i, deposit := range deposits {
state, err = b.ProcessPreGenesisDeposit(context.Background(), state, deposit)
if err != nil {
return nil, errors.Wrapf(err, "could not process validator deposit %d", i)
}
}
return OptimizedGenesisBeaconState(genesisTime, state, state.Eth1Data())
}
// OptimizedGenesisBeaconState is used to create a state that has already processed deposits. This is to efficiently
// create a mainnet state at chainstart.
func OptimizedGenesisBeaconState(genesisTime uint64, preState *stateTrie.BeaconState, eth1Data *ethpb.Eth1Data) (*stateTrie.BeaconState, error) {
if eth1Data == nil {
return nil, errors.New("no eth1data provided for genesis state")
}
randaoMixes := make([][]byte, params.BeaconConfig().EpochsPerHistoricalVector)
for i := 0; i < len(randaoMixes); i++ {
h := make([]byte, 32)
copy(h, eth1Data.BlockHash)
randaoMixes[i] = h
}
zeroHash := params.BeaconConfig().ZeroHash[:]
activeIndexRoots := make([][]byte, params.BeaconConfig().EpochsPerHistoricalVector)
for i := 0; i < len(activeIndexRoots); i++ {
activeIndexRoots[i] = zeroHash
}
blockRoots := make([][]byte, params.BeaconConfig().SlotsPerHistoricalRoot)
for i := 0; i < len(blockRoots); i++ {
blockRoots[i] = zeroHash
}
stateRoots := make([][]byte, params.BeaconConfig().SlotsPerHistoricalRoot)
for i := 0; i < len(stateRoots); i++ {
stateRoots[i] = zeroHash
}
slashings := make([]uint64, params.BeaconConfig().EpochsPerSlashingsVector)
state := &pb.BeaconState{
// Misc fields.
Slot: 0,
GenesisTime: genesisTime,
Fork: &pb.Fork{
PreviousVersion: params.BeaconConfig().GenesisForkVersion,
CurrentVersion: params.BeaconConfig().GenesisForkVersion,
Epoch: 0,
},
// Validator registry fields.
Validators: preState.Validators(),
Balances: preState.Balances(),
// Randomness and committees.
RandaoMixes: randaoMixes,
// Finality.
PreviousJustifiedCheckpoint: &ethpb.Checkpoint{
Epoch: 0,
Root: params.BeaconConfig().ZeroHash[:],
},
CurrentJustifiedCheckpoint: &ethpb.Checkpoint{
Epoch: 0,
Root: params.BeaconConfig().ZeroHash[:],
},
JustificationBits: []byte{0},
FinalizedCheckpoint: &ethpb.Checkpoint{
Epoch: 0,
Root: params.BeaconConfig().ZeroHash[:],
},
HistoricalRoots: [][]byte{},
BlockRoots: blockRoots,
StateRoots: stateRoots,
Slashings: slashings,
CurrentEpochAttestations: []*pb.PendingAttestation{},
PreviousEpochAttestations: []*pb.PendingAttestation{},
// Eth1 data.
Eth1Data: eth1Data,
Eth1DataVotes: []*ethpb.Eth1Data{},
Eth1DepositIndex: preState.Eth1DepositIndex(),
}
bodyRoot, err := ssz.HashTreeRoot(&ethpb.BeaconBlockBody{})
if err != nil {
return nil, errors.Wrapf(err, "could not hash tree root %v", bodyRoot)
}
state.LatestBlockHeader = &ethpb.BeaconBlockHeader{
ParentRoot: zeroHash,
StateRoot: zeroHash,
BodyRoot: bodyRoot[:],
}
return stateTrie.InitializeFromProto(state)
}
// EmptyGenesisState returns an empty beacon state object.
func EmptyGenesisState() (*stateTrie.BeaconState, error) {
state := &pb.BeaconState{
// Misc fields.
Slot: 0,
Fork: &pb.Fork{
PreviousVersion: params.BeaconConfig().GenesisForkVersion,
CurrentVersion: params.BeaconConfig().GenesisForkVersion,
Epoch: 0,
},
// Validator registry fields.
Validators: []*ethpb.Validator{},
Balances: []uint64{},
JustificationBits: []byte{0},
HistoricalRoots: [][]byte{},
CurrentEpochAttestations: []*pb.PendingAttestation{},
PreviousEpochAttestations: []*pb.PendingAttestation{},
// Eth1 data.
Eth1Data: &ethpb.Eth1Data{},
Eth1DataVotes: []*ethpb.Eth1Data{},
Eth1DepositIndex: 0,
}
return stateTrie.InitializeFromProto(state)
}
// IsValidGenesisState gets called whenever there's a deposit event,
// it checks whether there's enough effective balance to trigger and
// if the minimum genesis time arrived already.
//
// Spec pseudocode definition:
// def is_valid_genesis_state(state: BeaconState) -> bool:
// if state.genesis_time < MIN_GENESIS_TIME:
// return False
// if len(get_active_validator_indices(state, GENESIS_EPOCH)) < MIN_GENESIS_ACTIVE_VALIDATOR_COUNT:
// return False
// return True
// This method has been modified from the spec to allow whole states not to be saved
// but instead only cache the relevant information.
func IsValidGenesisState(chainStartDepositCount uint64, currentTime uint64) bool {
if currentTime < params.BeaconConfig().MinGenesisTime {
return false
}
if chainStartDepositCount < params.BeaconConfig().MinGenesisActiveValidatorCount {
return false
}
return true
}