prysm-pulse/beacon-chain/blockchain/forkchoice/service_test.go
Raul Jordan cc741ed8af
Ensure New State Type Tests Pass in Prysm (#4646)
* begin state service

* begin on the state trie idea

* created beacon state structure

* add in the full clone getter

* return by value instead

* add all setters

* new state setters are being completed

* arrays roots exposed

*  close to finishing all these headerssss

* functionality complete

* added in proto benchmark test

* test for compatibility

* add test for compat

* comments fixed

* add clone

* add clone

* remove underlying copies

* make it immutable

* integrate it into chainservice

* revert

* wrap up comments for package

* address all comments and godocs

* address all comments

* clone the pending attestation properly

* properly clone remaining items

* tests pass fixed bug

* begin using it instead of head state

* prevent nil pointer exceptions

* begin using new struct in db

* integrated new type into db package

* add proper nil checks

* using new state in archiver

* refactored much of core

* editing all the precompute functions

* done with most core refactor

* fixed up some bugs in the clone comparisons

* append current epoch atts

* add missing setters

* add new setters

* fix other core methods

* fix up transition

* main service and forkchoice

* fix rpc

* integrated to powchain

* some more changes

* fix build

* improve processing of deposits

* fix error

* prevent panic

* comment

* fix process att

* gaz

* fix up att process

* resolve existing review comments

* resolve another batch of gh comments

* resolve broken cpt state

* revise testutil to use the new state

* begin updating the state transition func to pass in more compartmentalized args

* finish editing transition function to return errors

* block operations pretty much done with refactor

* state transition fully refactored

* got epoch processing completed

* fix build in fork choice

* fixing more of the build

* fix up broken sync package

* it builds nowww it buildssss

* revert registry changes

* Recompute on Read (#4627)

* compute on read

* fix up eth1 data votes

* looking into slashings bug introduced in core/

* able to advance more slots

* add logging

* can now sync with testnet yay

* remove the leaves algorithm and other merkle imports

* expose initialize unsafe funcs

* Update beacon-chain/db/kv/state.go

* lint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* More Optimizations for New State (#4641)

* map optimization

* more optimizations

* use a custom hasher

* comment

* block operations optimizations

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* fixed up various operations to use the validator index map access

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* archiver tests pass

* fixing cache tests

* cache tests passing

* edited validator tests

* powchain tests passing

* halfway thru sync tests

* more sync test fixes

* add in tests for state/

* working through rpc tests

* assignments tests passed

* almost done with rpc/beacon tests

* resolved painful validator test

* fixed up even more tests

* resolve tests

* fix build

* reduce a randao mixes copy

* fixes under //beacon-chain/blockchain/...

* build //beacon-chain/core/...

* fixes

* Runtime Optimizations (#4648)

* parallelize shuffling

* clean up

* lint

* fix build

* use callback to read from registry

* fix array roots and size map

* new improvements

* reduce hash allocs

* improved shuffling

* terence's review

* use different method

* raul's comment

* new array roots

* remove clone in pre-compute

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul's review

* lint

* fix build issues

* fix visibility

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fix visibility

* build works for all

* fix blockchain test

* fix a few tests

* fix more tests

* update validator in slashing

* archiver passing

* fixed rpc/validator

* progress on core tests

* resolve broken rpc tests

* blockchain tests passed

* fix up some tests in core

* fix message diff

* remove unnecessary save

* Save validator after slashing

* Update validators one by one

* another update

* fix everything

* fix more precompute tests

* fix blocks tests

* more elegant fix

* more helper fixes

* change back ?

* fix test

* fix skip slot

* fix test

* reset caches

* fix testutil

* raceoff fixed

* passing

* Retrieve cached state in the beginning

* lint

* Fixed tests part 1

* Fixed rest of the tests

* Minor changes to avoid copying, small refactor to reduce deplicated code

* Handle att req for slot 0

* New beacon state: Only populate merkle layers as needed, copy merkle layers on copy/clone. (#4689)

* Only populate merkle layers as needed, copy merkle layers on copy/clone.

* use custom copy

* Make maps of correct size

* slightly fast, doesn't wait for lock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Target root can't be 0x00

* Don't use cache for current slot (may not be the right fix)

* fixed up tests

* Remove some copy for init sync. Not sure if it is safe enough for runtime though... testing...

* Align with prev logic for process slots cachedState.Slot() < slot

* Fix Initial Sync Flag (#4692)

* fixes

* fix up some test failures due to lack of nil checks

* fix up some test failures due to lack of nil checks

* fix up imports

* revert some changes

* imports

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* resolving further conflicts

* Better skip slot cache (#4694)

* Return copy of skip slot cache state, disable skip slot cache on sync

* fix

* Fix pruning

* fix up issues with broken tests

Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-31 12:57:01 -08:00

544 lines
14 KiB
Go

package forkchoice
import (
"bytes"
"context"
"reflect"
"testing"
"time"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/go-ssz"
"github.com/prysmaticlabs/prysm/beacon-chain/cache"
"github.com/prysmaticlabs/prysm/beacon-chain/core/blocks"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/db/filters"
testDB "github.com/prysmaticlabs/prysm/beacon-chain/db/testing"
beaconstate "github.com/prysmaticlabs/prysm/beacon-chain/state"
pb "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
"github.com/prysmaticlabs/prysm/shared/bytesutil"
"github.com/prysmaticlabs/prysm/shared/featureconfig"
"github.com/prysmaticlabs/prysm/shared/params"
)
func TestStore_GenesisStoreOk(t *testing.T) {
ctx := context.Background()
db := testDB.SetupDB(t)
defer testDB.TeardownDB(t, db)
store := NewForkChoiceService(ctx, db)
genesisTime := time.Unix(9999, 0)
genesisState, err := beaconstate.InitializeFromProto(&pb.BeaconState{GenesisTime: uint64(genesisTime.Unix())})
if err != nil {
t.Fatal(err)
}
genesisStateRoot, err := genesisState.HashTreeRoot()
if err != nil {
t.Fatal(err)
}
genesisBlk := blocks.NewGenesisBlock(genesisStateRoot[:])
genesisBlkRoot, err := ssz.HashTreeRoot(genesisBlk.Block)
if err != nil {
t.Fatal(err)
}
if err := db.SaveState(ctx, genesisState, genesisBlkRoot); err != nil {
t.Fatal(err)
}
if err := db.SaveGenesisBlockRoot(ctx, genesisBlkRoot); err != nil {
t.Fatal(err)
}
checkPoint := &ethpb.Checkpoint{Root: genesisBlkRoot[:]}
if err := store.GenesisStore(ctx, checkPoint, checkPoint); err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(store.justifiedCheckpt, checkPoint) {
t.Error("Justified check point from genesis store did not match")
}
if !reflect.DeepEqual(store.finalizedCheckpt, checkPoint) {
t.Error("Finalized check point from genesis store did not match")
}
cachedState, err := store.checkpointState.StateByCheckpoint(checkPoint)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(cachedState.InnerStateUnsafe(), genesisState.InnerStateUnsafe()) {
t.Error("Incorrect genesis state cached")
}
}
func TestStore_AncestorOk(t *testing.T) {
ctx := context.Background()
db := testDB.SetupDB(t)
defer testDB.TeardownDB(t, db)
store := NewForkChoiceService(ctx, db)
roots, err := blockTree1(db, []byte{'g'})
if err != nil {
t.Fatal(err)
}
type args struct {
root []byte
slot uint64
}
// /- B1
// B0 /- B5 - B7
// \- B3 - B4 - B6 - B8
tests := []struct {
args *args
want []byte
}{
{args: &args{roots[1], 0}, want: roots[0]},
{args: &args{roots[8], 0}, want: roots[0]},
{args: &args{roots[8], 4}, want: roots[4]},
{args: &args{roots[7], 4}, want: roots[4]},
{args: &args{roots[7], 0}, want: roots[0]},
}
for _, tt := range tests {
got, err := store.ancestor(ctx, tt.args.root, tt.args.slot)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(got, tt.want) {
t.Errorf("Store.ancestor(ctx, ) = %v, want %v", got, tt.want)
}
}
}
func TestStore_AncestorNotPartOfTheChain(t *testing.T) {
ctx := context.Background()
db := testDB.SetupDB(t)
defer testDB.TeardownDB(t, db)
store := NewForkChoiceService(ctx, db)
roots, err := blockTree1(db, []byte{'g'})
if err != nil {
t.Fatal(err)
}
// /- B1
// B0 /- B5 - B7
// \- B3 - B4 - B6 - B8
root, err := store.ancestor(ctx, roots[8], 1)
if err != nil {
t.Fatal(err)
}
if root != nil {
t.Error("block at slot 1 is not part of the chain")
}
root, err = store.ancestor(ctx, roots[8], 2)
if err != nil {
t.Fatal(err)
}
if root != nil {
t.Error("block at slot 2 is not part of the chain")
}
}
func TestStore_LatestAttestingBalance(t *testing.T) {
helpers.ClearCache()
ctx := context.Background()
db := testDB.SetupDB(t)
defer testDB.TeardownDB(t, db)
store := NewForkChoiceService(ctx, db)
roots, err := blockTree1(db, []byte{'g'})
if err != nil {
t.Fatal(err)
}
validators := make([]*ethpb.Validator, 100)
for i := 0; i < len(validators); i++ {
validators[i] = &ethpb.Validator{ExitEpoch: 2, EffectiveBalance: 1e9}
}
s, err := beaconstate.InitializeFromProto(&pb.BeaconState{Validators: validators, RandaoMixes: make([][]byte, params.BeaconConfig().EpochsPerHistoricalVector)})
if err != nil {
t.Fatal(err)
}
stateRoot, err := s.HashTreeRoot()
if err != nil {
t.Fatal(err)
}
b := blocks.NewGenesisBlock(stateRoot[:])
blkRoot, err := ssz.HashTreeRoot(b.Block)
if err != nil {
t.Fatal(err)
}
if err := db.SaveState(ctx, s, blkRoot); err != nil {
t.Fatal(err)
}
if err := db.SaveGenesisBlockRoot(ctx, blkRoot); err != nil {
t.Fatal(err)
}
checkPoint := &ethpb.Checkpoint{Root: blkRoot[:]}
if err := store.GenesisStore(ctx, checkPoint, checkPoint); err != nil {
t.Fatal(err)
}
// /- B1 (33 votes)
// B0 /- B5 - B7 (33 votes)
// \- B3 - B4 - B6 - B8 (34 votes)
for i := 0; i < len(validators); i++ {
switch {
case i < 33:
store.latestVoteMap[uint64(i)] = &pb.ValidatorLatestVote{Root: roots[1]}
case i > 66:
store.latestVoteMap[uint64(i)] = &pb.ValidatorLatestVote{Root: roots[7]}
default:
store.latestVoteMap[uint64(i)] = &pb.ValidatorLatestVote{Root: roots[8]}
}
}
tests := []struct {
root []byte
want uint64
}{
{root: roots[0], want: 100 * 1e9},
{root: roots[1], want: 33 * 1e9},
{root: roots[3], want: 67 * 1e9},
{root: roots[4], want: 67 * 1e9},
{root: roots[7], want: 33 * 1e9},
{root: roots[8], want: 34 * 1e9},
}
for _, tt := range tests {
got, err := store.latestAttestingBalance(ctx, tt.root)
if err != nil {
t.Fatal(err)
}
if got != tt.want {
t.Errorf("Store.latestAttestingBalance(ctx, ) = %v, want %v", got, tt.want)
}
}
}
func TestStore_ChildrenBlocksFromParentRoot(t *testing.T) {
ctx := context.Background()
db := testDB.SetupDB(t)
defer testDB.TeardownDB(t, db)
store := NewForkChoiceService(ctx, db)
roots, err := blockTree1(db, []byte{'g'})
if err != nil {
t.Fatal(err)
}
filter := filters.NewFilter().SetParentRoot(roots[0]).SetStartSlot(0)
children, err := store.db.BlockRoots(ctx, filter)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(children, [][32]byte{bytesutil.ToBytes32(roots[1]), bytesutil.ToBytes32(roots[3])}) {
t.Error("Did not receive correct children roots")
}
filter = filters.NewFilter().SetParentRoot(roots[0]).SetStartSlot(2)
children, err = store.db.BlockRoots(ctx, filter)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(children, [][32]byte{bytesutil.ToBytes32(roots[3])}) {
t.Error("Did not receive correct children roots")
}
}
func TestStore_GetHead(t *testing.T) {
helpers.ClearCache()
ctx := context.Background()
db := testDB.SetupDB(t)
defer testDB.TeardownDB(t, db)
store := NewForkChoiceService(ctx, db)
roots, err := blockTree1(db, []byte{'g'})
if err != nil {
t.Fatal(err)
}
validators := make([]*ethpb.Validator, 100)
for i := 0; i < len(validators); i++ {
validators[i] = &ethpb.Validator{ExitEpoch: 2, EffectiveBalance: 1e9}
}
s, err := beaconstate.InitializeFromProto(&pb.BeaconState{Validators: validators, RandaoMixes: make([][]byte, params.BeaconConfig().EpochsPerHistoricalVector)})
if err != nil {
t.Fatal(err)
}
stateRoot, err := s.HashTreeRoot()
if err != nil {
t.Fatal(err)
}
b := blocks.NewGenesisBlock(stateRoot[:])
blkRoot, err := ssz.HashTreeRoot(b.Block)
if err != nil {
t.Fatal(err)
}
if err := store.db.SaveState(ctx, s, blkRoot); err != nil {
t.Fatal(err)
}
if err := store.db.SaveGenesisBlockRoot(ctx, blkRoot); err != nil {
t.Fatal(err)
}
checkPoint := &ethpb.Checkpoint{Root: blkRoot[:]}
if err := store.GenesisStore(ctx, checkPoint, checkPoint); err != nil {
t.Fatal(err)
}
if err := store.db.SaveState(ctx, s, bytesutil.ToBytes32(roots[0])); err != nil {
t.Fatal(err)
}
store.justifiedCheckpt.Root = roots[0]
if err := store.checkpointState.AddCheckpointState(&cache.CheckpointState{
Checkpoint: store.justifiedCheckpt,
State: s,
}); err != nil {
t.Fatal(err)
}
// /- B1 (33 votes)
// B0 /- B5 - B7 (33 votes)
// \- B3 - B4 - B6 - B8 (34 votes)
for i := 0; i < len(validators); i++ {
switch {
case i < 33:
store.latestVoteMap[uint64(i)] = &pb.ValidatorLatestVote{Root: roots[1]}
case i > 66:
store.latestVoteMap[uint64(i)] = &pb.ValidatorLatestVote{Root: roots[7]}
default:
store.latestVoteMap[uint64(i)] = &pb.ValidatorLatestVote{Root: roots[8]}
}
}
// Default head is B8
head, err := store.Head(ctx)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(head, roots[8]) {
t.Error("Incorrect head")
}
// 1 validator switches vote to B7 to gain 34%, enough to switch head
store.latestVoteMap[uint64(50)] = &pb.ValidatorLatestVote{Root: roots[7]}
head, err = store.Head(ctx)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(head, roots[7]) {
t.Error("Incorrect head")
}
// 18 validators switches vote to B1 to gain 51%, enough to switch head
for i := 0; i < 18; i++ {
idx := 50 + uint64(i)
store.latestVoteMap[idx] = &pb.ValidatorLatestVote{Root: roots[1]}
}
head, err = store.Head(ctx)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(head, roots[1]) {
t.Log(head)
t.Error("Incorrect head")
}
}
func TestCacheGenesisState_Correct(t *testing.T) {
ctx := context.Background()
db := testDB.SetupDB(t)
defer testDB.TeardownDB(t, db)
store := NewForkChoiceService(ctx, db)
config := &featureconfig.Flags{
InitSyncCacheState: true,
}
featureconfig.Init(config)
b := &ethpb.BeaconBlock{Slot: 1}
r, _ := ssz.HashTreeRoot(b)
s, err := beaconstate.InitializeFromProto(&pb.BeaconState{GenesisTime: 99})
if err != nil {
t.Fatal(err)
}
store.db.SaveState(ctx, s, r)
store.db.SaveGenesisBlockRoot(ctx, r)
if err := store.cacheGenesisState(ctx); err != nil {
t.Fatal(err)
}
for _, state := range store.initSyncState {
if !reflect.DeepEqual(s.InnerStateUnsafe(), state.InnerStateUnsafe()) {
t.Error("Did not get wanted state")
}
}
}
func TestStore_GetFilterBlockTree_CorrectLeaf(t *testing.T) {
ctx := context.Background()
db := testDB.SetupDB(t)
defer testDB.TeardownDB(t, db)
store := NewForkChoiceService(ctx, db)
roots, err := blockTree1(db, []byte{'g'})
if err != nil {
t.Fatal(err)
}
s, err := beaconstate.InitializeFromProto(&pb.BeaconState{})
if err != nil {
t.Fatal(err)
}
stateRoot, err := s.HashTreeRoot()
if err != nil {
t.Fatal(err)
}
b := blocks.NewGenesisBlock(stateRoot[:])
blkRoot, err := ssz.HashTreeRoot(b.Block)
if err != nil {
t.Fatal(err)
}
if err := store.db.SaveState(ctx, s, blkRoot); err != nil {
t.Fatal(err)
}
if err := store.db.SaveGenesisBlockRoot(ctx, blkRoot); err != nil {
t.Fatal(err)
}
checkPoint := &ethpb.Checkpoint{Root: blkRoot[:]}
if err := store.GenesisStore(ctx, checkPoint, checkPoint); err != nil {
t.Fatal(err)
}
if err := store.db.SaveState(ctx, s, bytesutil.ToBytes32(roots[0])); err != nil {
t.Fatal(err)
}
store.justifiedCheckpt.Root = roots[0]
if err := store.checkpointState.AddCheckpointState(&cache.CheckpointState{
Checkpoint: store.justifiedCheckpt,
State: s,
}); err != nil {
t.Fatal(err)
}
tree, err := store.getFilterBlockTree(ctx)
if err != nil {
t.Fatal(err)
}
wanted := make(map[[32]byte]*ethpb.BeaconBlock)
for _, root := range roots {
root32 := bytesutil.ToBytes32(root)
b, _ := store.db.Block(ctx, root32)
if b != nil {
wanted[root32] = b.Block
}
}
if !reflect.DeepEqual(tree, wanted) {
t.Error("Did not filter tree correctly")
}
}
func TestStore_GetFilterBlockTree_IncorrectLeaf(t *testing.T) {
ctx := context.Background()
db := testDB.SetupDB(t)
defer testDB.TeardownDB(t, db)
store := NewForkChoiceService(ctx, db)
roots, err := blockTree1(db, []byte{'g'})
if err != nil {
t.Fatal(err)
}
s, err := beaconstate.InitializeFromProto(&pb.BeaconState{})
if err != nil {
t.Fatal(err)
}
stateRoot, err := s.HashTreeRoot()
if err != nil {
t.Fatal(err)
}
b := blocks.NewGenesisBlock(stateRoot[:])
blkRoot, err := ssz.HashTreeRoot(b.Block)
if err != nil {
t.Fatal(err)
}
if err := store.db.SaveState(ctx, s, blkRoot); err != nil {
t.Fatal(err)
}
if err := store.db.SaveGenesisBlockRoot(ctx, blkRoot); err != nil {
t.Fatal(err)
}
checkPoint := &ethpb.Checkpoint{Root: blkRoot[:]}
if err := store.GenesisStore(ctx, checkPoint, checkPoint); err != nil {
t.Fatal(err)
}
if err := store.db.SaveState(ctx, s, bytesutil.ToBytes32(roots[0])); err != nil {
t.Fatal(err)
}
store.justifiedCheckpt.Root = roots[0]
if err := store.checkpointState.AddCheckpointState(&cache.CheckpointState{
Checkpoint: store.justifiedCheckpt,
State: s,
}); err != nil {
t.Fatal(err)
}
// Filter for incorrect leaves for 1, 7 and 8
store.db.SaveState(ctx, &beaconstate.BeaconState{}, bytesutil.ToBytes32(roots[1]))
store.db.SaveState(ctx, &beaconstate.BeaconState{}, bytesutil.ToBytes32(roots[7]))
store.db.SaveState(ctx, &beaconstate.BeaconState{}, bytesutil.ToBytes32(roots[8]))
store.justifiedCheckpt.Epoch = 1
tree, err := store.getFilterBlockTree(ctx)
if err != nil {
t.Fatal(err)
}
if len(tree) != 0 {
t.Error("filtered tree should be 0 length")
}
// Set leave 1 as correct
s, err = beaconstate.InitializeFromProto(&pb.BeaconState{CurrentJustifiedCheckpoint: &ethpb.Checkpoint{Epoch: 1, Root: store.justifiedCheckpt.Root}})
if err != nil {
t.Fatal(err)
}
store.db.SaveState(ctx, s, bytesutil.ToBytes32(roots[1]))
tree, err = store.getFilterBlockTree(ctx)
if err != nil {
t.Fatal(err)
}
wanted := make(map[[32]byte]*ethpb.BeaconBlock)
root32 := bytesutil.ToBytes32(roots[0])
b, err = store.db.Block(ctx, root32)
if err != nil {
t.Fatal(err)
}
wanted[root32] = b.Block
root32 = bytesutil.ToBytes32(roots[1])
b, err = store.db.Block(ctx, root32)
if err != nil {
t.Fatal(err)
}
wanted[root32] = b.Block
if !reflect.DeepEqual(tree, wanted) {
t.Error("Did not filter tree correctly")
}
}