prysm-pulse/validator/client/runner.go

115 lines
3.7 KiB
Go
Raw Normal View History

package client
import (
"context"
Allow 8 Validator Multinode Cluster to Run Indefinitely (#2050) * plug forkchoice to blockchain service's block processing * fixed tests * more fixes... * clean ups * fixed test * Update beacon-chain/blockchain/block_processing.go * merged with 2006 and started fixing tests * remove prints * fixed tests * lint * include ops service * if there's a skip slot, slot-- * fixed typo * started working on test * no fork choice in propose * bleh, need to fix state generator first * state gen takes input slot * feedback * fixed tests * preston's feedback * fmt * removed extra logging * add more logs * fixed validator attest * builds * fixed save block * children fix * removed verbose logs * fix fork choice * right logs * Add Prometheus Counter for Reorg (#2051) * fetch every slot (#2052) * test Fixes * lint * only regenerate state if there was a reorg * better logging * fixed seed * better logging * process skip slots in assignment requests * fix lint * disable state root computation * filter attestations in regular sync * log important items * better info logs * added spans to stategen * span in stategen * set validator deadline * randao stuff * disable sig verify * lint * lint * save only using historical states * use new goroutine for handling sync messages * change default buffer sizes * better p2p * rem some useless logs * lint * sync tests complete * complete tests * tests fixed * lint * fix flakey att service * PR feedback * undo k8s changes * Update beacon-chain/blockchain/block_processing.go * Update beacon-chain/sync/regular_sync.go * Add feature flag to enable compute state root * add comment * gazelle lint fix
2019-03-25 15:21:21 +00:00
"time"
pb "github.com/prysmaticlabs/prysm/proto/beacon/rpc/v1"
"github.com/prysmaticlabs/prysm/shared/params"
"github.com/sirupsen/logrus"
2019-02-28 03:55:47 +00:00
"go.opencensus.io/trace"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
// Validator interface defines the primary methods of a validator client.
type Validator interface {
Done()
WaitForChainStart(ctx context.Context) error
WaitForActivation(ctx context.Context) error
CanonicalHeadSlot(ctx context.Context) (uint64, error)
NextSlot() <-chan uint64
Allow 8 Validator Multinode Cluster to Run Indefinitely (#2050) * plug forkchoice to blockchain service's block processing * fixed tests * more fixes... * clean ups * fixed test * Update beacon-chain/blockchain/block_processing.go * merged with 2006 and started fixing tests * remove prints * fixed tests * lint * include ops service * if there's a skip slot, slot-- * fixed typo * started working on test * no fork choice in propose * bleh, need to fix state generator first * state gen takes input slot * feedback * fixed tests * preston's feedback * fmt * removed extra logging * add more logs * fixed validator attest * builds * fixed save block * children fix * removed verbose logs * fix fork choice * right logs * Add Prometheus Counter for Reorg (#2051) * fetch every slot (#2052) * test Fixes * lint * only regenerate state if there was a reorg * better logging * fixed seed * better logging * process skip slots in assignment requests * fix lint * disable state root computation * filter attestations in regular sync * log important items * better info logs * added spans to stategen * span in stategen * set validator deadline * randao stuff * disable sig verify * lint * lint * save only using historical states * use new goroutine for handling sync messages * change default buffer sizes * better p2p * rem some useless logs * lint * sync tests complete * complete tests * tests fixed * lint * fix flakey att service * PR feedback * undo k8s changes * Update beacon-chain/blockchain/block_processing.go * Update beacon-chain/sync/regular_sync.go * Add feature flag to enable compute state root * add comment * gazelle lint fix
2019-03-25 15:21:21 +00:00
SlotDeadline(slot uint64) time.Time
LogValidatorGainsAndLosses(ctx context.Context, slot uint64) error
UpdateAssignments(ctx context.Context, slot uint64) error
Validator-multiple key (#2069) * first version - broken * working proto changes * resolve review remarks * fix goimport issues * fix service issues * first logic version-broken * first running version - no new tests * fix validator client test * add wait group to goroutines * remove unused var in function call * fix review remarks and tests * merge master changes and fix conflicts * gazzele fix * fix prestonvanloon requested changes * merge and some of terenc3t remarks addressed * _,pk bug fix in log * fix account file name suffix and filter not active validator out * merge with master and fix missing parameters * run over all public keys in hasvalidators * add test for error when no all the validators has index in the db and hasvalidators is called * fix runner tests fail due to timing issues * goimports * smaller sleep time in proposer tests * fix UpdateAssignments loging * fix goimports * added && false commented TestUpdateAssignments_DoesNothingWhenNotEpochStartAndAlreadyExistingAssignments * hasvalidators without missing publickeys list * fix some of prestone review remarks * fixes for prestone comments * review changes applied * expect context call in TestWaitForActivation_ValidatorOriginallyExists * changed hasvalidators to return true if one validator exists * fix init problem to getkeys * hasvalidators requiers all validators to be in db * validator attest assignments update * fix ap var name * Change name to hasallvalidators * fix tests * update script, fix any vs all validator calls * fix wait for activation * filter validator * reuse the reply block * fix imports * Remove dup * better lookup of active validators * better filter active vlaidators, still need to fix committee assignment tests * lint * use activated keys * fix for postchainstart * fix logging * move state transitions * hasanyvalidator and hasallvalidators * fix tests with updatechainhead missing * add tests * fix TestCommitteeAssignment_OK * fix test * fix validator tests * fix TestCommitteeAssignment_multipleKeys_OK and TestWaitForActivation_ValidatorOriginallyExists * fix goimports * removed unused param from assignment * change string(pk) to hex.EncodeString(pk) fix change requests * add inactive validator status to assignments * fix logging mess due to multi validator setup * set no assignment to debug level * log assignments every epoch * logging fixes * fixed runtime by using the right assignments * correct activation request * fix the validator panic * correct assignment * fix test fail and waitforactivation * performance log issue fix * fix goimports * add log message with truncated pk for attest * add truncated pk to attest and propose logs * Add comment to script, change 9 to 8 * Update assignment log * Add comment, report number of assignments * Use WithError, add validator as field, merge block proposal log * Update validator_propose.go * fix * use entry.String() * fix fmt
2019-04-18 17:23:38 +00:00
RolesAt(slot uint64) map[string]pb.ValidatorRole // validatorIndex -> role
AttestToBlockHead(ctx context.Context, slot uint64, idx string)
ProposeBlock(ctx context.Context, slot uint64, idx string)
}
// Run the main validator routine. This routine exits if the context is
// cancelled.
//
// Order of operations:
// 1 - Initialize validator data
// 2 - Wait for validator activation
// 3 - Wait for the next slot start
// 4 - Update assignments
// 5 - Determine role at current slot
// 6 - Perform assigned role, if any
func run(ctx context.Context, v Validator) {
defer v.Done()
if err := v.WaitForChainStart(ctx); err != nil {
log.Fatalf("Could not determine if beacon chain started: %v", err)
}
if err := v.WaitForActivation(ctx); err != nil {
log.Fatalf("Could not wait for validator activation: %v", err)
}
headSlot, err := v.CanonicalHeadSlot(ctx)
if err != nil {
log.Fatalf("Could not get current canonical head slot: %v", err)
}
if err := v.UpdateAssignments(ctx, headSlot); err != nil {
handleAssignmentError(err, headSlot)
}
for {
2019-02-28 03:55:47 +00:00
ctx, span := trace.StartSpan(ctx, "processSlot")
defer span.End()
select {
case <-ctx.Done():
log.Info("Context canceled, stopping validator")
return // Exit if context is canceled.
case slot := <-v.NextSlot():
2019-02-28 03:55:47 +00:00
span.AddAttributes(trace.Int64Attribute("slot", int64(slot)))
Allow 8 Validator Multinode Cluster to Run Indefinitely (#2050) * plug forkchoice to blockchain service's block processing * fixed tests * more fixes... * clean ups * fixed test * Update beacon-chain/blockchain/block_processing.go * merged with 2006 and started fixing tests * remove prints * fixed tests * lint * include ops service * if there's a skip slot, slot-- * fixed typo * started working on test * no fork choice in propose * bleh, need to fix state generator first * state gen takes input slot * feedback * fixed tests * preston's feedback * fmt * removed extra logging * add more logs * fixed validator attest * builds * fixed save block * children fix * removed verbose logs * fix fork choice * right logs * Add Prometheus Counter for Reorg (#2051) * fetch every slot (#2052) * test Fixes * lint * only regenerate state if there was a reorg * better logging * fixed seed * better logging * process skip slots in assignment requests * fix lint * disable state root computation * filter attestations in regular sync * log important items * better info logs * added spans to stategen * span in stategen * set validator deadline * randao stuff * disable sig verify * lint * lint * save only using historical states * use new goroutine for handling sync messages * change default buffer sizes * better p2p * rem some useless logs * lint * sync tests complete * complete tests * tests fixed * lint * fix flakey att service * PR feedback * undo k8s changes * Update beacon-chain/blockchain/block_processing.go * Update beacon-chain/sync/regular_sync.go * Add feature flag to enable compute state root * add comment * gazelle lint fix
2019-03-25 15:21:21 +00:00
slotCtx, _ := context.WithDeadline(ctx, v.SlotDeadline(slot))
// Report this validator client's rewards and penalties throughout its lifecycle.
Allow 8 Validator Multinode Cluster to Run Indefinitely (#2050) * plug forkchoice to blockchain service's block processing * fixed tests * more fixes... * clean ups * fixed test * Update beacon-chain/blockchain/block_processing.go * merged with 2006 and started fixing tests * remove prints * fixed tests * lint * include ops service * if there's a skip slot, slot-- * fixed typo * started working on test * no fork choice in propose * bleh, need to fix state generator first * state gen takes input slot * feedback * fixed tests * preston's feedback * fmt * removed extra logging * add more logs * fixed validator attest * builds * fixed save block * children fix * removed verbose logs * fix fork choice * right logs * Add Prometheus Counter for Reorg (#2051) * fetch every slot (#2052) * test Fixes * lint * only regenerate state if there was a reorg * better logging * fixed seed * better logging * process skip slots in assignment requests * fix lint * disable state root computation * filter attestations in regular sync * log important items * better info logs * added spans to stategen * span in stategen * set validator deadline * randao stuff * disable sig verify * lint * lint * save only using historical states * use new goroutine for handling sync messages * change default buffer sizes * better p2p * rem some useless logs * lint * sync tests complete * complete tests * tests fixed * lint * fix flakey att service * PR feedback * undo k8s changes * Update beacon-chain/blockchain/block_processing.go * Update beacon-chain/sync/regular_sync.go * Add feature flag to enable compute state root * add comment * gazelle lint fix
2019-03-25 15:21:21 +00:00
if err := v.LogValidatorGainsAndLosses(slotCtx, slot); err != nil {
log.Errorf("Could not report validator's rewards/penalties for slot %d: %v",
slot-params.BeaconConfig().GenesisSlot, err)
}
2019-02-28 03:55:47 +00:00
// Keep trying to update assignments if they are nil or if we are past an
// epoch transition in the beacon node's state.
Allow 8 Validator Multinode Cluster to Run Indefinitely (#2050) * plug forkchoice to blockchain service's block processing * fixed tests * more fixes... * clean ups * fixed test * Update beacon-chain/blockchain/block_processing.go * merged with 2006 and started fixing tests * remove prints * fixed tests * lint * include ops service * if there's a skip slot, slot-- * fixed typo * started working on test * no fork choice in propose * bleh, need to fix state generator first * state gen takes input slot * feedback * fixed tests * preston's feedback * fmt * removed extra logging * add more logs * fixed validator attest * builds * fixed save block * children fix * removed verbose logs * fix fork choice * right logs * Add Prometheus Counter for Reorg (#2051) * fetch every slot (#2052) * test Fixes * lint * only regenerate state if there was a reorg * better logging * fixed seed * better logging * process skip slots in assignment requests * fix lint * disable state root computation * filter attestations in regular sync * log important items * better info logs * added spans to stategen * span in stategen * set validator deadline * randao stuff * disable sig verify * lint * lint * save only using historical states * use new goroutine for handling sync messages * change default buffer sizes * better p2p * rem some useless logs * lint * sync tests complete * complete tests * tests fixed * lint * fix flakey att service * PR feedback * undo k8s changes * Update beacon-chain/blockchain/block_processing.go * Update beacon-chain/sync/regular_sync.go * Add feature flag to enable compute state root * add comment * gazelle lint fix
2019-03-25 15:21:21 +00:00
if err := v.UpdateAssignments(slotCtx, slot); err != nil {
handleAssignmentError(err, slot)
continue
}
Validator-multiple key (#2069) * first version - broken * working proto changes * resolve review remarks * fix goimport issues * fix service issues * first logic version-broken * first running version - no new tests * fix validator client test * add wait group to goroutines * remove unused var in function call * fix review remarks and tests * merge master changes and fix conflicts * gazzele fix * fix prestonvanloon requested changes * merge and some of terenc3t remarks addressed * _,pk bug fix in log * fix account file name suffix and filter not active validator out * merge with master and fix missing parameters * run over all public keys in hasvalidators * add test for error when no all the validators has index in the db and hasvalidators is called * fix runner tests fail due to timing issues * goimports * smaller sleep time in proposer tests * fix UpdateAssignments loging * fix goimports * added && false commented TestUpdateAssignments_DoesNothingWhenNotEpochStartAndAlreadyExistingAssignments * hasvalidators without missing publickeys list * fix some of prestone review remarks * fixes for prestone comments * review changes applied * expect context call in TestWaitForActivation_ValidatorOriginallyExists * changed hasvalidators to return true if one validator exists * fix init problem to getkeys * hasvalidators requiers all validators to be in db * validator attest assignments update * fix ap var name * Change name to hasallvalidators * fix tests * update script, fix any vs all validator calls * fix wait for activation * filter validator * reuse the reply block * fix imports * Remove dup * better lookup of active validators * better filter active vlaidators, still need to fix committee assignment tests * lint * use activated keys * fix for postchainstart * fix logging * move state transitions * hasanyvalidator and hasallvalidators * fix tests with updatechainhead missing * add tests * fix TestCommitteeAssignment_OK * fix test * fix validator tests * fix TestCommitteeAssignment_multipleKeys_OK and TestWaitForActivation_ValidatorOriginallyExists * fix goimports * removed unused param from assignment * change string(pk) to hex.EncodeString(pk) fix change requests * add inactive validator status to assignments * fix logging mess due to multi validator setup * set no assignment to debug level * log assignments every epoch * logging fixes * fixed runtime by using the right assignments * correct activation request * fix the validator panic * correct assignment * fix test fail and waitforactivation * performance log issue fix * fix goimports * add log message with truncated pk for attest * add truncated pk to attest and propose logs * Add comment to script, change 9 to 8 * Update assignment log * Add comment, report number of assignments * Use WithError, add validator as field, merge block proposal log * Update validator_propose.go * fix * use entry.String() * fix fmt
2019-04-18 17:23:38 +00:00
for id, role := range v.RolesAt(slot) {
go func(role pb.ValidatorRole, id string) {
switch role {
case pb.ValidatorRole_ATTESTER:
v.AttestToBlockHead(slotCtx, slot, id)
case pb.ValidatorRole_PROPOSER:
v.ProposeBlock(slotCtx, slot, id)
v.AttestToBlockHead(slotCtx, slot, id)
case pb.ValidatorRole_UNKNOWN:
pk12Char := id
if len(id) > 12 {
pk12Char = id[:12]
}
log.WithFields(logrus.Fields{
"public_key": pk12Char,
"slot": slot - params.BeaconConfig().GenesisSlot,
"role": role,
}).Debug("No active assignment, doing nothing")
default:
// Do nothing :)
}
Validator-multiple key (#2069) * first version - broken * working proto changes * resolve review remarks * fix goimport issues * fix service issues * first logic version-broken * first running version - no new tests * fix validator client test * add wait group to goroutines * remove unused var in function call * fix review remarks and tests * merge master changes and fix conflicts * gazzele fix * fix prestonvanloon requested changes * merge and some of terenc3t remarks addressed * _,pk bug fix in log * fix account file name suffix and filter not active validator out * merge with master and fix missing parameters * run over all public keys in hasvalidators * add test for error when no all the validators has index in the db and hasvalidators is called * fix runner tests fail due to timing issues * goimports * smaller sleep time in proposer tests * fix UpdateAssignments loging * fix goimports * added && false commented TestUpdateAssignments_DoesNothingWhenNotEpochStartAndAlreadyExistingAssignments * hasvalidators without missing publickeys list * fix some of prestone review remarks * fixes for prestone comments * review changes applied * expect context call in TestWaitForActivation_ValidatorOriginallyExists * changed hasvalidators to return true if one validator exists * fix init problem to getkeys * hasvalidators requiers all validators to be in db * validator attest assignments update * fix ap var name * Change name to hasallvalidators * fix tests * update script, fix any vs all validator calls * fix wait for activation * filter validator * reuse the reply block * fix imports * Remove dup * better lookup of active validators * better filter active vlaidators, still need to fix committee assignment tests * lint * use activated keys * fix for postchainstart * fix logging * move state transitions * hasanyvalidator and hasallvalidators * fix tests with updatechainhead missing * add tests * fix TestCommitteeAssignment_OK * fix test * fix validator tests * fix TestCommitteeAssignment_multipleKeys_OK and TestWaitForActivation_ValidatorOriginallyExists * fix goimports * removed unused param from assignment * change string(pk) to hex.EncodeString(pk) fix change requests * add inactive validator status to assignments * fix logging mess due to multi validator setup * set no assignment to debug level * log assignments every epoch * logging fixes * fixed runtime by using the right assignments * correct activation request * fix the validator panic * correct assignment * fix test fail and waitforactivation * performance log issue fix * fix goimports * add log message with truncated pk for attest * add truncated pk to attest and propose logs * Add comment to script, change 9 to 8 * Update assignment log * Add comment, report number of assignments * Use WithError, add validator as field, merge block proposal log * Update validator_propose.go * fix * use entry.String() * fix fmt
2019-04-18 17:23:38 +00:00
}(role, id)
}
}
}
}
func handleAssignmentError(err error, slot uint64) {
if errCode, ok := status.FromError(err); ok && errCode.Code() == codes.NotFound {
log.WithField(
"epoch", (slot/params.BeaconConfig().SlotsPerEpoch)-params.BeaconConfig().GenesisEpoch,
).Warn("Validator not yet assigned to epoch")
} else {
log.WithField("error", err).Error("Failed to update assignments")
}
}