prysm-pulse/beacon-chain/blockchain/init_sync_process_block.go

84 lines
2.6 KiB
Go
Raw Normal View History

Bound Initial Sync Cache Size (#4844) * bound initial sync * fix lint * Revert "Better block attestation inclusion (#4838)" This reverts commit 090d9627feed47281f9ab9660bb4d9854c491a7a. * add memory pool * more fixes * revert changes * add hack * revert hack * push halving * bring back hack * increase cache size * more fixes * more changes * new fixes * add test * add reverse test * more tests and clean up * add helper * more cleanup and tests * fix test * remove code * set gc percent flag * lint * lint * Fix comment formatting * Fix some formatting * inverse if statement * remove debug log * Apply suggestions from code review Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com> * Update beacon-chain/state/getters.go Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com> * Update beacon-chain/db/kv/state.go * integrate state generator * gaz * fixes * terence's review * reduce bound further * fix test * separate into new files * gaz * mod build file * add test * revert changes * fix test * Update beacon-chain/core/helpers/slot_epoch.go Co-Authored-By: terence tsao <terence@prysmaticlabs.com> * handle edge case * add back test * fix test again * handle edge case * Update beacon-chain/blockchain/init_sync_process_block.go * Update beacon-chain/blockchain/init_sync_process_block.go * Update beacon-chain/stategen/service_test.go Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com> * Update beacon-chain/blockchain/init_sync_process_block.go Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com> * Update beacon-chain/stategen/service.go Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com> * Update beacon-chain/stategen/service.go Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com> * raul's review * raul's review * fix refs * terence's review * one more fix * Update beacon-chain/blockchain/init_sync_process_block.go Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com> Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-02-18 21:10:54 +00:00
package blockchain
import (
"context"
"github.com/pkg/errors"
"github.com/prysmaticlabs/prysm/v3/consensus-types/blocks"
"github.com/prysmaticlabs/prysm/v3/consensus-types/interfaces"
Bound Initial Sync Cache Size (#4844) * bound initial sync * fix lint * Revert "Better block attestation inclusion (#4838)" This reverts commit 090d9627feed47281f9ab9660bb4d9854c491a7a. * add memory pool * more fixes * revert changes * add hack * revert hack * push halving * bring back hack * increase cache size * more fixes * more changes * new fixes * add test * add reverse test * more tests and clean up * add helper * more cleanup and tests * fix test * remove code * set gc percent flag * lint * lint * Fix comment formatting * Fix some formatting * inverse if statement * remove debug log * Apply suggestions from code review Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com> * Update beacon-chain/state/getters.go Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com> * Update beacon-chain/db/kv/state.go * integrate state generator * gaz * fixes * terence's review * reduce bound further * fix test * separate into new files * gaz * mod build file * add test * revert changes * fix test * Update beacon-chain/core/helpers/slot_epoch.go Co-Authored-By: terence tsao <terence@prysmaticlabs.com> * handle edge case * add back test * fix test again * handle edge case * Update beacon-chain/blockchain/init_sync_process_block.go * Update beacon-chain/blockchain/init_sync_process_block.go * Update beacon-chain/stategen/service_test.go Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com> * Update beacon-chain/blockchain/init_sync_process_block.go Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com> * Update beacon-chain/stategen/service.go Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com> * Update beacon-chain/stategen/service.go Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com> * raul's review * raul's review * fix refs * terence's review * one more fix * Update beacon-chain/blockchain/init_sync_process_block.go Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com> Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-02-18 21:10:54 +00:00
)
Deprecate store in blockchain pkg (#10903) * Deprecate store WIP * fix spectests build * mock right interface * sync tests build * more tests builds * blockchain tests - TestFinalizedCheckpt_GenesisRootOk - TestCurrentJustifiedCheckpt_CanRetrieve - TestJustifiedCheckpt_GenesisRootOk - TestHeadRoot_CanRetrieve - TestHeadRoot_UseDB - TestService_ProcessAttestationsAndUpdateHead - TestService_VerifyWeakSubjectivityRoot - TestVerifyFinalizedConsistency_InconsistentRoot_ProtoArray - TestVerifyFinalizedConsistency_InconsistentRoot_DoublyLinkedTree - TestVerifyFinalizedConsistency_Ok - TestStore_OnBlock_ProtoArray - TestStore_OnBlock_DoublyLinkedTree - TestStore_OnBlockBatch_ProtoArray - TestStore_OnBlockBatch_DoublyLinkedTree - TestStore_OnBlockBatch_NotifyNewPayload - TestCachedPreState_CanGetFromStateSummary_ProtoArray - TestCachedPreState_CanGetFromStateSummary_DoublyLinkedTree * more blockchain tests - TestStore_OnBlockBatch_PruneOK_Protoarray - TestFillForkChoiceMissingBlocks_CanSave_ProtoArray - TestFillForkChoiceMissingBlocks_CanSave_DoublyLinkedTree - TestFillForkChoiceMissingBlocks_RootsMatch_Protoarray - TestFillForkChoiceMissingBlocks_RootsMatch_DoublyLinkedTree - TestFillForkChoiceMissingBlocks_FilterFinalized_ProtoArray - TestFillForkChoiceMissingBlocks_FilterFinalized_DoublyLinkedTree - TestVerifyBlkDescendant - Test_verifyBlkFinalizedSlot_invalidBlock - TestChainService_SaveHeadNoDB * update best justified from genesis * deal with nil head on saveHead * initialize prev justified checkpoint * update finalization correctly * update finalization logic * update finalization logic * track the wall clock on forkchoice spectests * export copies of checkpoints from blockchain package * do not use forkchoice's head on HeadRoot * Remove debug remain Co-authored-by: terencechain <terence@prysmaticlabs.com> * terence's review * add forkchoice types deps * wtf * debugging * init-sync: update justified and finalized checkpoints on db * call updateFinalized instead of only DB * remove debug symbols * safe copy headroot Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-25 03:57:52 +00:00
// This saves a beacon block to the initial sync blocks cache. It rate limits how many blocks
// the cache keeps in memory (2 epochs worth of blocks) and saves them to DB when it hits this limit.
2023-02-09 09:23:32 +00:00
func (s *Service) saveInitSyncBlock(ctx context.Context, r [32]byte, b interfaces.ReadOnlySignedBeaconBlock) error {
s.initSyncBlocksLock.Lock()
s.initSyncBlocks[r] = b
Deprecate store in blockchain pkg (#10903) * Deprecate store WIP * fix spectests build * mock right interface * sync tests build * more tests builds * blockchain tests - TestFinalizedCheckpt_GenesisRootOk - TestCurrentJustifiedCheckpt_CanRetrieve - TestJustifiedCheckpt_GenesisRootOk - TestHeadRoot_CanRetrieve - TestHeadRoot_UseDB - TestService_ProcessAttestationsAndUpdateHead - TestService_VerifyWeakSubjectivityRoot - TestVerifyFinalizedConsistency_InconsistentRoot_ProtoArray - TestVerifyFinalizedConsistency_InconsistentRoot_DoublyLinkedTree - TestVerifyFinalizedConsistency_Ok - TestStore_OnBlock_ProtoArray - TestStore_OnBlock_DoublyLinkedTree - TestStore_OnBlockBatch_ProtoArray - TestStore_OnBlockBatch_DoublyLinkedTree - TestStore_OnBlockBatch_NotifyNewPayload - TestCachedPreState_CanGetFromStateSummary_ProtoArray - TestCachedPreState_CanGetFromStateSummary_DoublyLinkedTree * more blockchain tests - TestStore_OnBlockBatch_PruneOK_Protoarray - TestFillForkChoiceMissingBlocks_CanSave_ProtoArray - TestFillForkChoiceMissingBlocks_CanSave_DoublyLinkedTree - TestFillForkChoiceMissingBlocks_RootsMatch_Protoarray - TestFillForkChoiceMissingBlocks_RootsMatch_DoublyLinkedTree - TestFillForkChoiceMissingBlocks_FilterFinalized_ProtoArray - TestFillForkChoiceMissingBlocks_FilterFinalized_DoublyLinkedTree - TestVerifyBlkDescendant - Test_verifyBlkFinalizedSlot_invalidBlock - TestChainService_SaveHeadNoDB * update best justified from genesis * deal with nil head on saveHead * initialize prev justified checkpoint * update finalization correctly * update finalization logic * update finalization logic * track the wall clock on forkchoice spectests * export copies of checkpoints from blockchain package * do not use forkchoice's head on HeadRoot * Remove debug remain Co-authored-by: terencechain <terence@prysmaticlabs.com> * terence's review * add forkchoice types deps * wtf * debugging * init-sync: update justified and finalized checkpoints on db * call updateFinalized instead of only DB * remove debug symbols * safe copy headroot Co-authored-by: terencechain <terence@prysmaticlabs.com>
2022-06-25 03:57:52 +00:00
numBlocks := len(s.initSyncBlocks)
s.initSyncBlocksLock.Unlock()
if uint64(numBlocks) > initialSyncBlockCacheSize {
if err := s.cfg.BeaconDB.SaveBlocks(ctx, s.getInitSyncBlocks()); err != nil {
return err
}
s.clearInitSyncBlocks()
}
return nil
}
// This checks if a beacon block exists in the initial sync blocks cache using the root
// of the block.
func (s *Service) hasInitSyncBlock(r [32]byte) bool {
s.initSyncBlocksLock.RLock()
defer s.initSyncBlocksLock.RUnlock()
_, ok := s.initSyncBlocks[r]
return ok
}
// Returns true if a block for root `r` exists in the initial sync blocks cache or the DB.
func (s *Service) hasBlockInInitSyncOrDB(ctx context.Context, r [32]byte) bool {
if s.hasInitSyncBlock(r) {
return true
}
return s.cfg.BeaconDB.HasBlock(ctx, r)
}
// Returns block for a given root `r` from either the initial sync blocks cache or the DB.
// Error is returned if the block is not found in either cache or DB.
2023-02-09 09:23:32 +00:00
func (s *Service) getBlock(ctx context.Context, r [32]byte) (interfaces.ReadOnlySignedBeaconBlock, error) {
s.initSyncBlocksLock.RLock()
// Check cache first because it's faster.
b, ok := s.initSyncBlocks[r]
s.initSyncBlocksLock.RUnlock()
var err error
if !ok {
b, err = s.cfg.BeaconDB.Block(ctx, r)
if err != nil {
return nil, errors.Wrap(err, "could not retrieve block from db")
}
}
if err := blocks.BeaconBlockIsNil(b); err != nil {
return nil, errBlockNotFoundInCacheOrDB
}
return b, nil
}
// This retrieves all the beacon blocks from the initial sync blocks cache, the returned
// blocks are unordered.
2023-02-09 09:23:32 +00:00
func (s *Service) getInitSyncBlocks() []interfaces.ReadOnlySignedBeaconBlock {
s.initSyncBlocksLock.RLock()
defer s.initSyncBlocksLock.RUnlock()
2023-02-09 09:23:32 +00:00
blks := make([]interfaces.ReadOnlySignedBeaconBlock, 0, len(s.initSyncBlocks))
for _, b := range s.initSyncBlocks {
blks = append(blks, b)
}
return blks
}
// This clears out the initial sync blocks cache.
func (s *Service) clearInitSyncBlocks() {
s.initSyncBlocksLock.Lock()
defer s.initSyncBlocksLock.Unlock()
2023-02-09 09:23:32 +00:00
s.initSyncBlocks = make(map[[32]byte]interfaces.ReadOnlySignedBeaconBlock)
}