prysm-pulse/beacon-chain/db/kafka/passthrough.go
Raul Jordan cc741ed8af
Ensure New State Type Tests Pass in Prysm (#4646)
* begin state service

* begin on the state trie idea

* created beacon state structure

* add in the full clone getter

* return by value instead

* add all setters

* new state setters are being completed

* arrays roots exposed

*  close to finishing all these headerssss

* functionality complete

* added in proto benchmark test

* test for compatibility

* add test for compat

* comments fixed

* add clone

* add clone

* remove underlying copies

* make it immutable

* integrate it into chainservice

* revert

* wrap up comments for package

* address all comments and godocs

* address all comments

* clone the pending attestation properly

* properly clone remaining items

* tests pass fixed bug

* begin using it instead of head state

* prevent nil pointer exceptions

* begin using new struct in db

* integrated new type into db package

* add proper nil checks

* using new state in archiver

* refactored much of core

* editing all the precompute functions

* done with most core refactor

* fixed up some bugs in the clone comparisons

* append current epoch atts

* add missing setters

* add new setters

* fix other core methods

* fix up transition

* main service and forkchoice

* fix rpc

* integrated to powchain

* some more changes

* fix build

* improve processing of deposits

* fix error

* prevent panic

* comment

* fix process att

* gaz

* fix up att process

* resolve existing review comments

* resolve another batch of gh comments

* resolve broken cpt state

* revise testutil to use the new state

* begin updating the state transition func to pass in more compartmentalized args

* finish editing transition function to return errors

* block operations pretty much done with refactor

* state transition fully refactored

* got epoch processing completed

* fix build in fork choice

* fixing more of the build

* fix up broken sync package

* it builds nowww it buildssss

* revert registry changes

* Recompute on Read (#4627)

* compute on read

* fix up eth1 data votes

* looking into slashings bug introduced in core/

* able to advance more slots

* add logging

* can now sync with testnet yay

* remove the leaves algorithm and other merkle imports

* expose initialize unsafe funcs

* Update beacon-chain/db/kv/state.go

* lint

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* More Optimizations for New State (#4641)

* map optimization

* more optimizations

* use a custom hasher

* comment

* block operations optimizations

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* fixed up various operations to use the validator index map access

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* archiver tests pass

* fixing cache tests

* cache tests passing

* edited validator tests

* powchain tests passing

* halfway thru sync tests

* more sync test fixes

* add in tests for state/

* working through rpc tests

* assignments tests passed

* almost done with rpc/beacon tests

* resolved painful validator test

* fixed up even more tests

* resolve tests

* fix build

* reduce a randao mixes copy

* fixes under //beacon-chain/blockchain/...

* build //beacon-chain/core/...

* fixes

* Runtime Optimizations (#4648)

* parallelize shuffling

* clean up

* lint

* fix build

* use callback to read from registry

* fix array roots and size map

* new improvements

* reduce hash allocs

* improved shuffling

* terence's review

* use different method

* raul's comment

* new array roots

* remove clone in pre-compute

* Update beacon-chain/state/types.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul's review

* lint

* fix build issues

* fix visibility

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fix visibility

* build works for all

* fix blockchain test

* fix a few tests

* fix more tests

* update validator in slashing

* archiver passing

* fixed rpc/validator

* progress on core tests

* resolve broken rpc tests

* blockchain tests passed

* fix up some tests in core

* fix message diff

* remove unnecessary save

* Save validator after slashing

* Update validators one by one

* another update

* fix everything

* fix more precompute tests

* fix blocks tests

* more elegant fix

* more helper fixes

* change back ?

* fix test

* fix skip slot

* fix test

* reset caches

* fix testutil

* raceoff fixed

* passing

* Retrieve cached state in the beginning

* lint

* Fixed tests part 1

* Fixed rest of the tests

* Minor changes to avoid copying, small refactor to reduce deplicated code

* Handle att req for slot 0

* New beacon state: Only populate merkle layers as needed, copy merkle layers on copy/clone. (#4689)

* Only populate merkle layers as needed, copy merkle layers on copy/clone.

* use custom copy

* Make maps of correct size

* slightly fast, doesn't wait for lock

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Target root can't be 0x00

* Don't use cache for current slot (may not be the right fix)

* fixed up tests

* Remove some copy for init sync. Not sure if it is safe enough for runtime though... testing...

* Align with prev logic for process slots cachedState.Slot() < slot

* Fix Initial Sync Flag (#4692)

* fixes

* fix up some test failures due to lack of nil checks

* fix up some test failures due to lack of nil checks

* fix up imports

* revert some changes

* imports

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* resolving further conflicts

* Better skip slot cache (#4694)

* Return copy of skip slot cache state, disable skip slot cache on sync

* fix

* Fix pruning

* fix up issues with broken tests

Co-authored-by: Nishant Das <nish1993@hotmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-01-31 12:57:01 -08:00

309 lines
11 KiB
Go

package kafka
import (
"context"
"github.com/ethereum/go-ethereum/common"
eth "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
ethpb "github.com/prysmaticlabs/ethereumapis/eth/v1alpha1"
"github.com/prysmaticlabs/prysm/beacon-chain/db/filters"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/proto/beacon/db"
ethereum_beacon_p2p_v1 "github.com/prysmaticlabs/prysm/proto/beacon/p2p/v1"
)
// DatabasePath -- passthrough.
func (e Exporter) DatabasePath() string {
return e.db.DatabasePath()
}
// ClearDB -- passthrough.
func (e Exporter) ClearDB() error {
return e.db.ClearDB()
}
// Backup -- passthrough.
func (e Exporter) Backup(ctx context.Context) error {
return e.db.Backup(ctx)
}
// AttestationsByDataRoot -- passthrough.
func (e Exporter) AttestationsByDataRoot(ctx context.Context, attDataRoot [32]byte) ([]*eth.Attestation, error) {
return e.db.AttestationsByDataRoot(ctx, attDataRoot)
}
// Attestations -- passthrough.
func (e Exporter) Attestations(ctx context.Context, f *filters.QueryFilter) ([]*eth.Attestation, error) {
return e.db.Attestations(ctx, f)
}
// HasAttestation -- passthrough.
func (e Exporter) HasAttestation(ctx context.Context, attDataRoot [32]byte) bool {
return e.db.HasAttestation(ctx, attDataRoot)
}
// DeleteAttestation -- passthrough.
func (e Exporter) DeleteAttestation(ctx context.Context, attDataRoot [32]byte) error {
return e.db.DeleteAttestation(ctx, attDataRoot)
}
// DeleteAttestations -- passthrough.
func (e Exporter) DeleteAttestations(ctx context.Context, attDataRoots [][32]byte) error {
return e.db.DeleteAttestations(ctx, attDataRoots)
}
// Block -- passthrough.
func (e Exporter) Block(ctx context.Context, blockRoot [32]byte) (*eth.SignedBeaconBlock, error) {
return e.db.Block(ctx, blockRoot)
}
// HeadBlock -- passthrough.
func (e Exporter) HeadBlock(ctx context.Context) (*eth.SignedBeaconBlock, error) {
return e.db.HeadBlock(ctx)
}
// Blocks -- passthrough.
func (e Exporter) Blocks(ctx context.Context, f *filters.QueryFilter) ([]*eth.SignedBeaconBlock, error) {
return e.db.Blocks(ctx, f)
}
// BlockRoots -- passthrough.
func (e Exporter) BlockRoots(ctx context.Context, f *filters.QueryFilter) ([][32]byte, error) {
return e.db.BlockRoots(ctx, f)
}
// HasBlock -- passthrough.
func (e Exporter) HasBlock(ctx context.Context, blockRoot [32]byte) bool {
return e.db.HasBlock(ctx, blockRoot)
}
// DeleteBlock -- passthrough.
func (e Exporter) DeleteBlock(ctx context.Context, blockRoot [32]byte) error {
return e.db.DeleteBlock(ctx, blockRoot)
}
// DeleteBlocks -- passthrough.
func (e Exporter) DeleteBlocks(ctx context.Context, blockRoots [][32]byte) error {
return e.db.DeleteBlocks(ctx, blockRoots)
}
// ValidatorIndex -- passthrough.
func (e Exporter) ValidatorIndex(ctx context.Context, publicKey []byte) (uint64, bool, error) {
return e.db.ValidatorIndex(ctx, publicKey)
}
// HasValidatorIndex -- passthrough.
func (e Exporter) HasValidatorIndex(ctx context.Context, publicKey []byte) bool {
return e.db.HasValidatorIndex(ctx, publicKey)
}
// DeleteValidatorIndex -- passthrough.
func (e Exporter) DeleteValidatorIndex(ctx context.Context, publicKey []byte) error {
return e.db.DeleteValidatorIndex(ctx, publicKey)
}
// State -- passthrough.
func (e Exporter) State(ctx context.Context, blockRoot [32]byte) (*state.BeaconState, error) {
return e.db.State(ctx, blockRoot)
}
// HeadState -- passthrough.
func (e Exporter) HeadState(ctx context.Context) (*state.BeaconState, error) {
return e.db.HeadState(ctx)
}
// GenesisState -- passthrough.
func (e Exporter) GenesisState(ctx context.Context) (*state.BeaconState, error) {
return e.db.GenesisState(ctx)
}
// ProposerSlashing -- passthrough.
func (e Exporter) ProposerSlashing(ctx context.Context, slashingRoot [32]byte) (*eth.ProposerSlashing, error) {
return e.db.ProposerSlashing(ctx, slashingRoot)
}
// AttesterSlashing -- passthrough.
func (e Exporter) AttesterSlashing(ctx context.Context, slashingRoot [32]byte) (*eth.AttesterSlashing, error) {
return e.db.AttesterSlashing(ctx, slashingRoot)
}
// HasProposerSlashing -- passthrough.
func (e Exporter) HasProposerSlashing(ctx context.Context, slashingRoot [32]byte) bool {
return e.db.HasProposerSlashing(ctx, slashingRoot)
}
// HasAttesterSlashing -- passthrough.
func (e Exporter) HasAttesterSlashing(ctx context.Context, slashingRoot [32]byte) bool {
return e.db.HasAttesterSlashing(ctx, slashingRoot)
}
// DeleteProposerSlashing -- passthrough.
func (e Exporter) DeleteProposerSlashing(ctx context.Context, slashingRoot [32]byte) error {
return e.db.DeleteProposerSlashing(ctx, slashingRoot)
}
// DeleteAttesterSlashing -- passthrough.
func (e Exporter) DeleteAttesterSlashing(ctx context.Context, slashingRoot [32]byte) error {
return e.db.DeleteAttesterSlashing(ctx, slashingRoot)
}
// VoluntaryExit -- passthrough.
func (e Exporter) VoluntaryExit(ctx context.Context, exitRoot [32]byte) (*eth.VoluntaryExit, error) {
return e.db.VoluntaryExit(ctx, exitRoot)
}
// HasVoluntaryExit -- passthrough.
func (e Exporter) HasVoluntaryExit(ctx context.Context, exitRoot [32]byte) bool {
return e.db.HasVoluntaryExit(ctx, exitRoot)
}
// DeleteVoluntaryExit -- passthrough.
func (e Exporter) DeleteVoluntaryExit(ctx context.Context, exitRoot [32]byte) error {
return e.db.DeleteVoluntaryExit(ctx, exitRoot)
}
// JustifiedCheckpoint -- passthrough.
func (e Exporter) JustifiedCheckpoint(ctx context.Context) (*eth.Checkpoint, error) {
return e.db.JustifiedCheckpoint(ctx)
}
// FinalizedCheckpoint -- passthrough.
func (e Exporter) FinalizedCheckpoint(ctx context.Context) (*eth.Checkpoint, error) {
return e.db.FinalizedCheckpoint(ctx)
}
// ArchivedActiveValidatorChanges -- passthrough.
func (e Exporter) ArchivedActiveValidatorChanges(ctx context.Context, epoch uint64) (*ethereum_beacon_p2p_v1.ArchivedActiveSetChanges, error) {
return e.db.ArchivedActiveValidatorChanges(ctx, epoch)
}
// ArchivedCommitteeInfo -- passthrough.
func (e Exporter) ArchivedCommitteeInfo(ctx context.Context, epoch uint64) (*ethereum_beacon_p2p_v1.ArchivedCommitteeInfo, error) {
return e.db.ArchivedCommitteeInfo(ctx, epoch)
}
// ArchivedBalances -- passthrough.
func (e Exporter) ArchivedBalances(ctx context.Context, epoch uint64) ([]uint64, error) {
return e.db.ArchivedBalances(ctx, epoch)
}
// ArchivedValidatorParticipation -- passthrough.
func (e Exporter) ArchivedValidatorParticipation(ctx context.Context, epoch uint64) (*eth.ValidatorParticipation, error) {
return e.db.ArchivedValidatorParticipation(ctx, epoch)
}
// DepositContractAddress -- passthrough.
func (e Exporter) DepositContractAddress(ctx context.Context) ([]byte, error) {
return e.db.DepositContractAddress(ctx)
}
// SaveHeadBlockRoot -- passthrough.
func (e Exporter) SaveHeadBlockRoot(ctx context.Context, blockRoot [32]byte) error {
return e.db.SaveHeadBlockRoot(ctx, blockRoot)
}
// GenesisBlock -- passthrough.
func (e Exporter) GenesisBlock(ctx context.Context) (*ethpb.SignedBeaconBlock, error) {
return e.db.GenesisBlock(ctx)
}
// SaveGenesisBlockRoot -- passthrough.
func (e Exporter) SaveGenesisBlockRoot(ctx context.Context, blockRoot [32]byte) error {
return e.db.SaveGenesisBlockRoot(ctx, blockRoot)
}
// SaveValidatorIndex -- passthrough.
func (e Exporter) SaveValidatorIndex(ctx context.Context, publicKey []byte, validatorIdx uint64) error {
return e.db.SaveValidatorIndex(ctx, publicKey, validatorIdx)
}
// SaveValidatorIndices -- passthrough.
func (e Exporter) SaveValidatorIndices(ctx context.Context, publicKeys [][48]byte, validatorIndices []uint64) error {
return e.db.SaveValidatorIndices(ctx, publicKeys, validatorIndices)
}
// SaveState -- passthrough.
func (e Exporter) SaveState(ctx context.Context, state *state.BeaconState, blockRoot [32]byte) error {
return e.db.SaveState(ctx, state, blockRoot)
}
// SaveProposerSlashing -- passthrough.
func (e Exporter) SaveProposerSlashing(ctx context.Context, slashing *eth.ProposerSlashing) error {
return e.db.SaveProposerSlashing(ctx, slashing)
}
// SaveAttesterSlashing -- passthrough.
func (e Exporter) SaveAttesterSlashing(ctx context.Context, slashing *eth.AttesterSlashing) error {
return e.db.SaveAttesterSlashing(ctx, slashing)
}
// SaveVoluntaryExit -- passthrough.
func (e Exporter) SaveVoluntaryExit(ctx context.Context, exit *eth.VoluntaryExit) error {
return e.db.SaveVoluntaryExit(ctx, exit)
}
// SaveJustifiedCheckpoint -- passthrough.
func (e Exporter) SaveJustifiedCheckpoint(ctx context.Context, checkpoint *eth.Checkpoint) error {
return e.db.SaveJustifiedCheckpoint(ctx, checkpoint)
}
// SaveFinalizedCheckpoint -- passthrough.
func (e Exporter) SaveFinalizedCheckpoint(ctx context.Context, checkpoint *eth.Checkpoint) error {
return e.db.SaveFinalizedCheckpoint(ctx, checkpoint)
}
// SaveArchivedActiveValidatorChanges -- passthrough.
func (e Exporter) SaveArchivedActiveValidatorChanges(ctx context.Context, epoch uint64, changes *ethereum_beacon_p2p_v1.ArchivedActiveSetChanges) error {
return e.db.SaveArchivedActiveValidatorChanges(ctx, epoch, changes)
}
// SaveArchivedCommitteeInfo -- passthrough.
func (e Exporter) SaveArchivedCommitteeInfo(ctx context.Context, epoch uint64, info *ethereum_beacon_p2p_v1.ArchivedCommitteeInfo) error {
return e.db.SaveArchivedCommitteeInfo(ctx, epoch, info)
}
// SaveArchivedBalances -- passthrough.
func (e Exporter) SaveArchivedBalances(ctx context.Context, epoch uint64, balances []uint64) error {
return e.db.SaveArchivedBalances(ctx, epoch, balances)
}
// SaveArchivedValidatorParticipation -- passthrough.
func (e Exporter) SaveArchivedValidatorParticipation(ctx context.Context, epoch uint64, part *eth.ValidatorParticipation) error {
return e.db.SaveArchivedValidatorParticipation(ctx, epoch, part)
}
// SaveDepositContractAddress -- passthrough.
func (e Exporter) SaveDepositContractAddress(ctx context.Context, addr common.Address) error {
return e.db.SaveDepositContractAddress(ctx, addr)
}
// DeleteState -- passthrough.
func (e Exporter) DeleteState(ctx context.Context, blockRoot [32]byte) error {
return e.db.DeleteState(ctx, blockRoot)
}
// DeleteStates -- passthrough.
func (e Exporter) DeleteStates(ctx context.Context, blockRoots [][32]byte) error {
return e.db.DeleteStates(ctx, blockRoots)
}
// HasState -- passthrough.
func (e Exporter) HasState(ctx context.Context, blockRoot [32]byte) bool {
return e.db.HasState(ctx, blockRoot)
}
// IsFinalizedBlock -- passthrough.
func (e Exporter) IsFinalizedBlock(ctx context.Context, blockRoot [32]byte) bool {
return e.db.IsFinalizedBlock(ctx, blockRoot)
}
// PowchainData -- passthrough
func (e Exporter) PowchainData(ctx context.Context) (*db.ETH1ChainData, error) {
return e.db.PowchainData(ctx)
}
// SavePowchainData -- passthrough
func (e Exporter) SavePowchainData(ctx context.Context, data *db.ETH1ChainData) error {
return e.db.SavePowchainData(ctx, data)
}