prysm-pulse/beacon-chain/blockchain/state_balance_cache.go

82 lines
2.5 KiB
Go
Raw Normal View History

Switch to lazy state balance cache (#9822) * quick lazy balance cache proof of concept * WIP refactoring to use lazy cache * updating tests to use functional opts * updating the rest of the tests, all passing * use mock stategen where possible reduces the number of test cases that require db setup * rename test opt method for clear link * Update beacon-chain/blockchain/process_block.go Co-authored-by: terence tsao <terence@prysmaticlabs.com> * test assumption that zerohash is in db * remove unused MockDB (mocking stategen instead) * fix cache bug, switch to sync.Mutex * improve test coverage for the state cache * uncomment failing genesis test for discussion * gofmt * remove unused Service struct member * cleanup unused func input * combining type declaration in signature * don't export the state cache constructor * work around blockchain deps w/ new file service_test brings in a ton of dependencies that make bazel rules for blockchain complex, so just sticking these mocks in their own file simplifies things. * gofmt * remove intentionally failing test this test established that the zero root can't be used to look up the state, resulting in a change in another PR to update stategen to use the GenesisState db method instead when the zero root is detected. * fixed error introduced by develop refresh * fix import ordering * appease deepsource * remove unused function * godoc comments on new requires/assert * defensive constructor per terence's PR comment * more differentiated balance cache metric names Co-authored-by: kasey <kasey@users.noreply.github.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com> Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-19 15:59:26 +00:00
package blockchain
import (
"context"
"errors"
"sync"
"github.com/prysmaticlabs/prysm/beacon-chain/core/helpers"
"github.com/prysmaticlabs/prysm/beacon-chain/core/time"
"github.com/prysmaticlabs/prysm/beacon-chain/state"
"github.com/prysmaticlabs/prysm/beacon-chain/state/stategen"
)
type stateBalanceCache struct {
sync.Mutex
balances []uint64
root [32]byte
stateGen stateByRooter
}
type stateByRooter interface {
StateByRoot(context.Context, [32]byte) (state.BeaconState, error)
}
// newStateBalanceCache exists to remind us that stateBalanceCache needs a state gen
Switch to lazy state balance cache (#9822) * quick lazy balance cache proof of concept * WIP refactoring to use lazy cache * updating tests to use functional opts * updating the rest of the tests, all passing * use mock stategen where possible reduces the number of test cases that require db setup * rename test opt method for clear link * Update beacon-chain/blockchain/process_block.go Co-authored-by: terence tsao <terence@prysmaticlabs.com> * test assumption that zerohash is in db * remove unused MockDB (mocking stategen instead) * fix cache bug, switch to sync.Mutex * improve test coverage for the state cache * uncomment failing genesis test for discussion * gofmt * remove unused Service struct member * cleanup unused func input * combining type declaration in signature * don't export the state cache constructor * work around blockchain deps w/ new file service_test brings in a ton of dependencies that make bazel rules for blockchain complex, so just sticking these mocks in their own file simplifies things. * gofmt * remove intentionally failing test this test established that the zero root can't be used to look up the state, resulting in a change in another PR to update stategen to use the GenesisState db method instead when the zero root is detected. * fixed error introduced by develop refresh * fix import ordering * appease deepsource * remove unused function * godoc comments on new requires/assert * defensive constructor per terence's PR comment * more differentiated balance cache metric names Co-authored-by: kasey <kasey@users.noreply.github.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com> Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-19 15:59:26 +00:00
// to avoid nil pointer bugs when updating the cache in the read path (get())
func newStateBalanceCache(sg *stategen.State) (*stateBalanceCache, error) {
if sg == nil {
return nil, errors.New("can't initialize state balance cache without stategen")
Switch to lazy state balance cache (#9822) * quick lazy balance cache proof of concept * WIP refactoring to use lazy cache * updating tests to use functional opts * updating the rest of the tests, all passing * use mock stategen where possible reduces the number of test cases that require db setup * rename test opt method for clear link * Update beacon-chain/blockchain/process_block.go Co-authored-by: terence tsao <terence@prysmaticlabs.com> * test assumption that zerohash is in db * remove unused MockDB (mocking stategen instead) * fix cache bug, switch to sync.Mutex * improve test coverage for the state cache * uncomment failing genesis test for discussion * gofmt * remove unused Service struct member * cleanup unused func input * combining type declaration in signature * don't export the state cache constructor * work around blockchain deps w/ new file service_test brings in a ton of dependencies that make bazel rules for blockchain complex, so just sticking these mocks in their own file simplifies things. * gofmt * remove intentionally failing test this test established that the zero root can't be used to look up the state, resulting in a change in another PR to update stategen to use the GenesisState db method instead when the zero root is detected. * fixed error introduced by develop refresh * fix import ordering * appease deepsource * remove unused function * godoc comments on new requires/assert * defensive constructor per terence's PR comment * more differentiated balance cache metric names Co-authored-by: kasey <kasey@users.noreply.github.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com> Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-19 15:59:26 +00:00
}
return &stateBalanceCache{stateGen: sg}, nil
}
// update is called by get() when the requested root doesn't match
// the previously read value. This cache assumes we only want to cache one
// set of balances for a single root (the current justified root).
//
// WARNING: this is not thread-safe on its own, relies on get() for locking
Switch to lazy state balance cache (#9822) * quick lazy balance cache proof of concept * WIP refactoring to use lazy cache * updating tests to use functional opts * updating the rest of the tests, all passing * use mock stategen where possible reduces the number of test cases that require db setup * rename test opt method for clear link * Update beacon-chain/blockchain/process_block.go Co-authored-by: terence tsao <terence@prysmaticlabs.com> * test assumption that zerohash is in db * remove unused MockDB (mocking stategen instead) * fix cache bug, switch to sync.Mutex * improve test coverage for the state cache * uncomment failing genesis test for discussion * gofmt * remove unused Service struct member * cleanup unused func input * combining type declaration in signature * don't export the state cache constructor * work around blockchain deps w/ new file service_test brings in a ton of dependencies that make bazel rules for blockchain complex, so just sticking these mocks in their own file simplifies things. * gofmt * remove intentionally failing test this test established that the zero root can't be used to look up the state, resulting in a change in another PR to update stategen to use the GenesisState db method instead when the zero root is detected. * fixed error introduced by develop refresh * fix import ordering * appease deepsource * remove unused function * godoc comments on new requires/assert * defensive constructor per terence's PR comment * more differentiated balance cache metric names Co-authored-by: kasey <kasey@users.noreply.github.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com> Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-19 15:59:26 +00:00
func (c *stateBalanceCache) update(ctx context.Context, justifiedRoot [32]byte) ([]uint64, error) {
stateBalanceCacheMiss.Inc()
justifiedState, err := c.stateGen.StateByRoot(ctx, justifiedRoot)
if err != nil {
return nil, err
}
if justifiedState == nil || justifiedState.IsNil() {
return nil, errNilStateFromStategen
}
epoch := time.CurrentEpoch(justifiedState)
justifiedBalances := make([]uint64, justifiedState.NumValidators())
var balanceAccumulator = func(idx int, val state.ReadOnlyValidator) error {
if helpers.IsActiveValidatorUsingTrie(val, epoch) {
justifiedBalances[idx] = val.EffectiveBalance()
} else {
justifiedBalances[idx] = 0
}
return nil
}
if err := justifiedState.ReadFromEveryValidator(balanceAccumulator); err != nil {
return nil, err
}
c.balances = justifiedBalances
c.root = justifiedRoot
return c.balances, nil
}
// getBalances takes an explicit justifiedRoot so it can invalidate the singleton cache key
// when the justified root changes, and takes a context so that the long-running stategen
// read path can connect to the upstream cancellation/timeout chain.
func (c *stateBalanceCache) get(ctx context.Context, justifiedRoot [32]byte) ([]uint64, error) {
c.Lock()
defer c.Unlock()
if justifiedRoot != [32]byte{} && justifiedRoot == c.root {
Switch to lazy state balance cache (#9822) * quick lazy balance cache proof of concept * WIP refactoring to use lazy cache * updating tests to use functional opts * updating the rest of the tests, all passing * use mock stategen where possible reduces the number of test cases that require db setup * rename test opt method for clear link * Update beacon-chain/blockchain/process_block.go Co-authored-by: terence tsao <terence@prysmaticlabs.com> * test assumption that zerohash is in db * remove unused MockDB (mocking stategen instead) * fix cache bug, switch to sync.Mutex * improve test coverage for the state cache * uncomment failing genesis test for discussion * gofmt * remove unused Service struct member * cleanup unused func input * combining type declaration in signature * don't export the state cache constructor * work around blockchain deps w/ new file service_test brings in a ton of dependencies that make bazel rules for blockchain complex, so just sticking these mocks in their own file simplifies things. * gofmt * remove intentionally failing test this test established that the zero root can't be used to look up the state, resulting in a change in another PR to update stategen to use the GenesisState db method instead when the zero root is detected. * fixed error introduced by develop refresh * fix import ordering * appease deepsource * remove unused function * godoc comments on new requires/assert * defensive constructor per terence's PR comment * more differentiated balance cache metric names Co-authored-by: kasey <kasey@users.noreply.github.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com> Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-11-19 15:59:26 +00:00
stateBalanceCacheHit.Inc()
return c.balances, nil
}
return c.update(ctx, justifiedRoot)
}