prysm-pulse/beacon-chain/state/stategen/service.go

94 lines
3.2 KiB
Go
Raw Normal View History

// Package stategen defines functions to regenerate beacon chain states
// by replaying blocks from a stored state checkpoint, useful for
// optimization and reducing a beacon node's resource consumption.
package stategen
import (
2020-03-15 16:47:49 +00:00
"context"
Hot states use no DB (#6488) * Add cache to service struct * Update hot getters/setters to use cache * Update migration * Update other services to adapt * Fix initial sync get state * Update getter related tests * Update hot related tests * Update migrate related tests * New awesome tests for migration * Clean up rest of the tests * Merge refs/heads/master into hot-state-no-db * Fix block chain head tests * Fix block chain processor tests * Fixed RPC tests * Update cold getter and test * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Fix sync tests * Short cut if state is already in DB * Remove uneeded saves * Update beacon-chain/state/stategen/hot_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/getter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/getter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/service.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/setter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Preston's feedback * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Return a copy of cache states * Remove hot state caches check in StateByRoot * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Merge refs/heads/master into hot-state-no-db * Raul's feedback * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
2020-07-06 17:22:12 +00:00
"sync"
"github.com/prysmaticlabs/prysm/beacon-chain/cache"
"github.com/prysmaticlabs/prysm/beacon-chain/db"
2020-03-15 16:47:49 +00:00
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/shared/params"
2020-03-15 16:47:49 +00:00
"go.opencensus.io/trace"
)
// State represents a management object that handles the internal
// logic of maintaining both hot and cold states in DB.
type State struct {
Hot states use no DB (#6488) * Add cache to service struct * Update hot getters/setters to use cache * Update migration * Update other services to adapt * Fix initial sync get state * Update getter related tests * Update hot related tests * Update migrate related tests * New awesome tests for migration * Clean up rest of the tests * Merge refs/heads/master into hot-state-no-db * Fix block chain head tests * Fix block chain processor tests * Fixed RPC tests * Update cold getter and test * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Fix sync tests * Short cut if state is already in DB * Remove uneeded saves * Update beacon-chain/state/stategen/hot_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/getter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/getter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/service.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/setter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Preston's feedback * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Return a copy of cache states * Remove hot state caches check in StateByRoot * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Merge refs/heads/master into hot-state-no-db * Raul's feedback * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
2020-07-06 17:22:12 +00:00
beaconDB db.NoHeadAccessDatabase
slotsPerArchivedPoint uint64
hotStateCache *cache.HotStateCache
finalizedInfo *finalizedInfo
stateSummaryCache *cache.StateSummaryCache
epochBoundaryStateCache *epochBoundaryState
}
// This tracks the finalized point. It's also the point where slot and the block root of
// cold and hot sections of the DB splits.
Hot states use no DB (#6488) * Add cache to service struct * Update hot getters/setters to use cache * Update migration * Update other services to adapt * Fix initial sync get state * Update getter related tests * Update hot related tests * Update migrate related tests * New awesome tests for migration * Clean up rest of the tests * Merge refs/heads/master into hot-state-no-db * Fix block chain head tests * Fix block chain processor tests * Fixed RPC tests * Update cold getter and test * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Fix sync tests * Short cut if state is already in DB * Remove uneeded saves * Update beacon-chain/state/stategen/hot_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/getter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/getter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/service.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/setter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Preston's feedback * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Return a copy of cache states * Remove hot state caches check in StateByRoot * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Merge refs/heads/master into hot-state-no-db * Raul's feedback * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
2020-07-06 17:22:12 +00:00
type finalizedInfo struct {
slot uint64
root [32]byte
state *state.BeaconState
lock sync.RWMutex
}
// New returns a new state management object.
func New(db db.NoHeadAccessDatabase, stateSummaryCache *cache.StateSummaryCache) *State {
return &State{
Hot states use no DB (#6488) * Add cache to service struct * Update hot getters/setters to use cache * Update migration * Update other services to adapt * Fix initial sync get state * Update getter related tests * Update hot related tests * Update migrate related tests * New awesome tests for migration * Clean up rest of the tests * Merge refs/heads/master into hot-state-no-db * Fix block chain head tests * Fix block chain processor tests * Fixed RPC tests * Update cold getter and test * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Fix sync tests * Short cut if state is already in DB * Remove uneeded saves * Update beacon-chain/state/stategen/hot_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/getter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/getter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/service.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/setter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Preston's feedback * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Return a copy of cache states * Remove hot state caches check in StateByRoot * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Merge refs/heads/master into hot-state-no-db * Raul's feedback * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
2020-07-06 17:22:12 +00:00
beaconDB: db,
hotStateCache: cache.NewHotStateCache(),
finalizedInfo: &finalizedInfo{slot: 0, root: params.BeaconConfig().ZeroHash},
slotsPerArchivedPoint: params.BeaconConfig().SlotsPerArchivedPoint,
stateSummaryCache: stateSummaryCache,
epochBoundaryStateCache: newBoundaryStateCache(),
}
}
2020-03-15 16:47:49 +00:00
// Resume resumes a new state management object from previously saved finalized check point in DB.
func (s *State) Resume(ctx context.Context) (*state.BeaconState, error) {
2020-03-15 16:47:49 +00:00
ctx, span := trace.StartSpan(ctx, "stateGen.Resume")
defer span.End()
Improved cold state checkpoints: migrate database index and usage (#6461) * Add database migrations, still need to update the API usage... * gofmt goimports * progress * Merge branch 'master' of github.com:prysmaticlabs/prysm into index-migration * use slot instead of index * rename LastArchivedIndex to LastArchivedSlot * rename LastArchivedIndexRoot to LastArchivedRoot * remove unused HighestSlotStates method * deprecate old key, include in migration * deprecate old key, include in migration * remove blocks index in migration * rename bucket variable * fix code to pass tests * Merge branch 'master' of github.com:prysmaticlabs/prysm into index-migration * gofmt, goimports * fix * Add state slot index * progress * lint * fix build * Merge branch 'master' of github.com:prysmaticlabs/prysm into index-migration * kafka * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * remove SaveArchivedPointRoot, a few other big changes * Merge branch 'index-migration' of github.com:prysmaticlabs/prysm into index-migration * fix tests and lint * lint again * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * block migration, some renaming * gaz, gofmt * add tests * change index to uint bytes * Merge branch 'index-migration' of github.com:prysmaticlabs/prysm into index-migration * rm method notes * stop if the bucket doesn't exist * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * @rauljordan pr feedback * Simplify * Merge refs/heads/master into index-migration * Remove unused method, add roundtrip test * gofmt * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge refs/heads/master into index-migration * Merge branch 'master' of github.com:prysmaticlabs/prysm into index-migration
2020-07-18 18:05:04 +00:00
lastArchivedRoot := s.beaconDB.LastArchivedRoot(ctx)
lastArchivedState, err := s.beaconDB.State(ctx, lastArchivedRoot)
2020-03-15 16:47:49 +00:00
if err != nil {
return nil, err
}
// Resume as genesis state if there's no last archived state.
if lastArchivedState == nil {
return s.beaconDB.GenesisState(ctx)
}
2020-07-07 15:04:55 +00:00
s.finalizedInfo = &finalizedInfo{slot: lastArchivedState.Slot(), root: lastArchivedRoot, state: lastArchivedState.Copy()}
return lastArchivedState, nil
2020-03-15 16:47:49 +00:00
}
Hot states use no DB (#6488) * Add cache to service struct * Update hot getters/setters to use cache * Update migration * Update other services to adapt * Fix initial sync get state * Update getter related tests * Update hot related tests * Update migrate related tests * New awesome tests for migration * Clean up rest of the tests * Merge refs/heads/master into hot-state-no-db * Fix block chain head tests * Fix block chain processor tests * Fixed RPC tests * Update cold getter and test * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Fix sync tests * Short cut if state is already in DB * Remove uneeded saves * Update beacon-chain/state/stategen/hot_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/getter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/getter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/service.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Update beacon-chain/state/stategen/setter_test.go Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> * Preston's feedback * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Return a copy of cache states * Remove hot state caches check in StateByRoot * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db * Merge refs/heads/master into hot-state-no-db * Raul's feedback * Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
2020-07-06 17:22:12 +00:00
// SaveFinalizedState saves the finalized slot, root and state into memory to be used by state gen service.
// This used for migration at the correct start slot and used for hot state play back to ensure
// lower bound to start is always at the last finalized state.
func (s *State) SaveFinalizedState(fSlot uint64, fRoot [32]byte, fState *state.BeaconState) {
s.finalizedInfo.lock.Lock()
defer s.finalizedInfo.lock.Unlock()
s.finalizedInfo.root = fRoot
s.finalizedInfo.state = fState.Copy()
s.finalizedInfo.slot = fSlot
}
// Returns true if input root equals to cached finalized root.
func (s *State) isFinalizedRoot(r [32]byte) bool {
s.finalizedInfo.lock.RLock()
defer s.finalizedInfo.lock.RUnlock()
return r == s.finalizedInfo.root
}
// Returns the cached and copied finalized state.
func (s *State) finalizedState() *state.BeaconState {
s.finalizedInfo.lock.RLock()
defer s.finalizedInfo.lock.RUnlock()
return s.finalizedInfo.state.Copy()
}