* `EpochFromString`: Use already defined `Uint64FromString` function.
* `Test_uint64FromString` => `Test_FromString`
This test function tests more functions than `Uint64FromString`.
* Slashing protection history: Remove unreachable code.
The function `NewKVStore` creates, via `kv.UpdatePublicKeysBuckets`,
a new item in the `proposal-history-bucket-interchange`.
IMO there is no real reason to prefer `proposal` than `attestation`
as a prefix for this bucket, but this is the way it is done right now
and renaming the bucket will probably be backward incompatible.
An `attestedPublicKey` cannot exist without
the corresponding `proposedPublicKey`.
Thus, the `else` portion of code removed in this commit is not reachable.
We raise an error if we get there.
This is also probably the reason why the removed `else` portion was not
tested.
* `NewKVStore`: Switch items in `createBuckets`.
So the order corresponds to `schema.go`
* `slashableAttestationCheck`: Fix comments and logs.
* `ValidatorClient.db`: Use `iface.ValidatorDB`.
* BoltDB database: Implement `GraffitiFileHash`.
* Filesystem database: Creates `db.go`.
This file defines the following structs:
- `Store`
- `Graffiti`
- `Configuration`
- `ValidatorSlashingProtection`
This files implements the following public functions:
- `NewStore`
- `Close`
- `Backup`
- `DatabasePath`
- `ClearDB`
- `UpdatePublicKeysBuckets`
This files implements the following private functions:
- `slashingProtectionDirPath`
- `configurationFilePath`
- `configuration`
- `saveConfiguration`
- `validatorSlashingProtection`
- `saveValidatorSlashingProtection`
- `publicKeys`
* Filesystem database: Creates `genesis.go`.
This file defines the following public functions:
- `GenesisValidatorsRoot`
- `SaveGenesisValidatorsRoot`
* Filesystem database: Creates `graffiti.go`.
This file defines the following public functions:
- `SaveGraffitiOrderedIndex`
- `GraffitiOrderedIndex`
* Filesystem database: Creates `migration.go`.
This file defines the following public functions:
- `RunUpMigrations`
- `RunDownMigrations`
* Filesystem database: Creates proposer_settings.go.
This file defines the following public functions:
- `ProposerSettings`
- `ProposerSettingsExists`
- `SaveProposerSettings`
* Filesystem database: Creates `attester_protection.go`.
This file defines the following public functions:
- `EIPImportBlacklistedPublicKeys`
- `SaveEIPImportBlacklistedPublicKeys`
- `SigningRootAtTargetEpoch`
- `LowestSignedTargetEpoch`
- `LowestSignedSourceEpoch`
- `AttestedPublicKeys`
- `CheckSlashableAttestation`
- `SaveAttestationForPubKey`
- `SaveAttestationsForPubKey`
- `AttestationHistoryForPubKey`
* Filesystem database: Creates `proposer_protection.go`.
This file defines the following public functions:
- `HighestSignedProposal`
- `LowestSignedProposal`
- `ProposalHistoryForPubKey`
- `ProposalHistoryForSlot`
- `ProposedPublicKeys`
* Ensure that the filesystem store implements the `ValidatorDB` interface.
* `slashableAttestationCheck`: Check the database type.
* `slashableProposalCheck`: Check the database type.
* `slashableAttestationCheck`: Allow usage of minimal slashing protection.
* `slashableProposalCheck`: Allow usage of minimal slashing protection.
* `ImportStandardProtectionJSON`: Check the database type.
* `ImportStandardProtectionJSON`: Allow usage of min slashing protection.
* Implement `RecursiveDirFind`.
* Implement minimal<->complete DB conversion.
3 public functions are implemented:
- `IsCompleteDatabaseExisting`
- `IsMinimalDatabaseExisting`
- `ConvertDatabase`
* `setupDB`: Add `isSlashingProtectionMinimal` argument.
The feature addition is located in `validator/node/node_test.go`.
The rest of this commit consists in minimal slashing protection testing.
* `setupWithKey`: Add `isSlashingProtectionMinimal` argument.
The feature addition is located in `validator/client/propose_test.go`.
The rest of this commit consists in tests wrapping.
* `setup`: Add `isSlashingProtectionMinimal` argument.
The added feature is located in the `validator/client/propose_test.go`
file.
The rest of this commit consists in tests wrapping.
* `initializeFromCLI` and `initializeForWeb`: Factorize db init.
* Add `convert-complete-to-minimal` command.
* Creates `--enable-minimal-slashing-protection` flag.
* `importSlashingProtectionJSON`: Check database type.
* `exportSlashingProtectionJSON`: Check database type.
* `TestClearDB`: Test with minimal slashing protection.
* KeyManager: Test with minimal slashing protection.
* RPC: KeyManager: Test with minimal slashing protection.
* `convert-complete-to-minimal`: Change option names.
Options were:
- `--source` (for source data directory), and
- `--target` (for target data directory)
However, since this command deals with slashing protection, which has
source (epochs) and target (epochs), the initial option names may confuse
the user.
In this commit:
`--source` ==> `--source-data-dir`
`--target` ==> `--target-data-dir`
* Set `SlashableAttestationCheck` as an iface method.
And delete `CheckSlashableAttestation` from iface.
* Move helpers functions in a more general directory.
No functional change.
* Extract common structs out of `kv`.
==> `filesystem` does not depend anymore on `kv`.
==> `iface` does not depend anymore on `kv`.
==> `slashing-protection` does not depend anymore on `kv`.
* Move `ValidateMetadata` in `validator/helpers`.
* `ValidateMetadata`: Test with mock.
This way, we can:
- Avoid any circular import for tests.
- Implement once for all `iface.ValidatorDB` implementations
the `ValidateMetadata`function.
- Have tests (and coverage) of `ValidateMetadata`in
its own package.
The ideal solution would have been to implement `ValidateMetadata` as
a method with the `iface.ValidatorDB`receiver.
Unfortunately, golang does not allow that.
* `iface.ValidatorDB`: Implement ImportStandardProtectionJSON.
The whole purpose of this commit is to avoid the `switch validatorDB.(type)`
in `ImportStandardProtectionJSON`.
* `iface.ValidatorDB`: Implement `SlashableProposalCheck`.
* Remove now useless `slashableProposalCheck`.
* Delete useless `ImportStandardProtectionJSON`.
* `file.Exists`: Detect directories and return an error.
Before, `Exists` was only able to detect if a file exists.
Now, this function takes an extra `File` or `Directory` argument.
It detects either if a file or a directory exists.
Before, if an error was returned by `os.Stat`, the the file was
considered as non existing.
Now, it is treated as a real error.
* Replace `os.Stat` by `file.Exists`.
* Remove `Is{Complete,Minimal}DatabaseExisting`.
* `publicKeys`: Add log if unexpected file found.
* Move `{Source,Target}DataDirFlag`in `db.go`.
* `failedAttLocalProtectionErr`: `var`==> `const`
* `signingRoot`: `32`==> `fieldparams.RootLength`.
* `validatorClientData`==> `validator-client-data`.
To be consistent with `slashing-protection`.
* Add progress bars for `import` and `convert`.
* `parseBlocksForUniquePublicKeys`: Move in `db/kv`.
* helpers: Remove unused `initializeProgressBar` function.
* First take at updating everything to v5
* Patch gRPC gateway to use prysm v5
Fix patch
* Update go ssz
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Define `cli.App` without mutation.
No functional change.
* `usage.go`: Clean `appHelpTemplate`.
No functional change is added.
Modifications consist in adding prefix/suffix `-` to improve readability of
the template without adding new lines in template inference.
We now see some inconsistencies of the template:
- `if .App.Version` is around the `AUTHOR` section.
- `if .App.Copyright` is around both `COPYRIGHT` and `VERSION` sections.
- `if len .App.Authors` is around nothing.
* `usage.go`: Surround version and author correctly.
* `usage.go`: `AUTHOR` ==> `AUTHORS`
* `usage.go`: `GLOBAL` --> `global`.
* `--grpc-max-msg-size`: Remove double default.
* VC: Standardize help message.
- Flags help begin with a capital letter and end with a period.
- If a flag help begins with a verb, it is conjugated.
- Expermitemtal, danger etc... mentions are between parenthesis.
* VC help message: Wrap too long lines.
* scaffolding for verification package
* WIP blob verification methods
* lock wrapper for safer forkchoice sharing
* more solid cache and verification designs; adding tests
* more test coverage, adding missing cache files
* clearer func name
* remove forkchoice borrower (it's in another PR)
* revert temporary interface experiment
* lint
* nishant feedback
* add comments with spec text to all verifications
* some comments on public methods
* invert confusing verification name
* deep source
* remove cache from ProposerCache + gaz
* more consistently early return on error paths
* messed up the test with the wrong config value
* terence naming feedback
* tests on BeginsAt
* lint
* deep source...
* name errors after failure, not expectation
* deep sooource
* check len()==0 instead of nil so empty lists work
* update test for EIP-7044
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* Fix exported names that start with a package name
* A few more renames
* Fix exported names that start with a package name
* A few more renames
* Radek's feedback
* Fix conflict
* fix keymanager test
* Fix comments
---------
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* in progress...
* in progress...
* remove log
* log root
* Revert "Auxiliary commit to revert individual files from f12a609ea2a2f1e87e97321f3a717cd324b5ae97"
This reverts commit 5ae35edb6477d8d0ea4e94b273efc6590484da85.
* cleanup
* remove log
* remove whitespace
* remove logs
* more stuff
* copy
* always rebuild trie
* revert
* add state
* init state
* fix all
* uintptr
* move slice to new package
* lock in `Detach`
* remove constraint
* reorder
* blockroots and stateroots
* fill roots in empty()
* fix hasher
* implement slice for balances and inactivity scores
* detach in setters
* Revert "implement slice for balances and inactivity scores"
This reverts commit 59eb9df8d766cb1c44a7eb5b3f5e3c042249943d.
# Conflicts:
# beacon-chain/state/state-native/setters_validator.go
* use counter to track states
* typos
* rename interface
* balances
* gauge
* some improvements
* first try with map
* fix
* inactivity scores in progress
* fix test
# Conflicts:
# beacon-chain/state/state-native/helpers_test.go
* test fixes
* ToProto fix
* copy roots
* validators
* build fixes
* fix bug in `ToProto`
* fix fuzz test
* fix bug in slice getters
* fix state equality checks
* make tests pass
* make tests pass
* more test updates
* Revert "Auxiliary commit to revert individual files from 34e7344bff08a589e6341bb1829e3cb74159e878"
This reverts commit ecd64efa8917f37ca41460e0356ff007fe55dd9d.
* Revert "make tests pass"
This reverts commit 0cf00f19eecf4678cd2b866dd107f3179d0426ef.
* Revert "make tests pass"
This reverts commit 521b65e1d2e13be3d720f333008b6838a8e78878.
* pass tests
* deepequal identifiable types
* Deflake `cloners_test.go`
* feature flag for block roots
* feature flag
* remove recursive locks
* reduce complexity of rootSelector
* fix randao mixes root
* some fixes
* revisit tests
* revert change to field trie helpers
* initialize field map for tests
* remove whitespace
* initialize roots with proper length
* more fixes
* out of bounds message fix
* optimize length calculation
* remove call to Len in PubkeyAtIndex
* don't log deposits
* unit tests
* unit tests
* fix
* comments
* test fixes
* id
* remove Enumerator interface
* review feedback
* simplify field trie
* bring back fieldtrie package
* fix bazel file
* use handle32ByteArrays for root computation
* fix locks
* metrics
* bzl
* simplify some things
* use htr in state test
* remove code from require package
* gzl
* more htr
* Fuzzing of the multi-value slice
* assert values
* getter optimizations
* use At when reading from validators
* Nishant's review
* restore safe copy
* remove empty line
* build fix
* restore how we get root at index for deafult mode
* more review comments
* optimize default behavior
* simplify Slice calls
* test fix
* remove unnecessary package
* remove unused setter
* make fieldMap unexported
* some improvements in state package
* call `Slice` instead of manually copying
* unlock in ReadFromEveryValidator
* Potuz's comments
* lock the state when reading from all validators
# Conflicts:
# beacon-chain/state/state-native/getters_validator.go
* add back preston's changes
* add index
---------
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* update github actions
* use quotes or it is go 1.2
lol
* Update gosec
* Update gosec
* Update go lint
* fix gosec violations
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* refactor initialization to blocking startup method
* require genesisSetter in blockchain, fix tests
* work-around gazelle weirdness
* fix dep gazelle ignores
* only call SetGenesis once
* fix typo
* validator test setup and fix to return right error
* move waitForChainStart to Start
* wire up sync Service.genesisWaiter
* fix p2p genesisWaiter plumbing
* remove extra clock type, integrate into genesis
and rename
* use time.Now when no Nower is specified
* remove unused ClockSetter
* simplify rpc context checking
* fix typo
* use clock everywhere in sync; [32]byte val root
* don't use DeepEqual to compare [32]byte and []byte
* don't use clock in init sync, not wired up yet
* use clock waiter in blockchain as well
* use cancelable contexts in tests with goroutines
* missed a reference to WithClockSetter
* Update beacon-chain/startup/genesis.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Update beacon-chain/blockchain/service_test.go
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* more clear docs
* doc for NewClock
* move clock typedef to more logical file name
* adding documentation
* gaz
* fixes for capella
* reducing test raciness
* fix races in committee cache tests
* lint
* add tests on Duration slot math helper
* startup package test coverage
* fix bad merge
* set non-zero genesis time in tests that call Start
* happy deepsource, happy me-epsource
* replace Synced event with channel
* remove unused error
* remove accidental wip commit
* gaz!
* remove unused event constants
* remove sync statefeed subscription to fix deadlock
* remove state notifier
* fix build
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
* prysmctl using the same genesis func as e2e
* add whitespace to genesis.json for readability
* fix typo in fork name
* don't require validator count if deposits given
* add gosec exception
* the other nosec :(
* appease deepsource
* fix comments on renamed public value/func
---------
Co-authored-by: kasey <kasey@users.noreply.github.com>
* e2e using dynamic configs; runtime bellatrix setup
* starts from phase0 but fee recipient eval fails
* lower TTD to cross before bellatrix epoch in e2e
* fix debug printf
* fix it
* dynamic ttd and more robust fee_recipient eval
* let multiplication set ttd=0 if bellatrix epoch=0
* refactoring fee recipient test after cognit errors
* lint
* appease deepsource
* deep sourcin
* gazelle
* missed a usage of this when refactoring
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
* WIP trying to start from bellatrix state
* env var to control log path with unique paths
due to flaky test re-run behavior, logs from a failed test run are
overwritten by subsequent retries. This makes it difficult to retrieve
logs after the first failed run. It also takes some squinting through
output to find the location of the log file in the first place. This
flag enables logs to be placed in an arbitrary path. Note that bazel
sandboxing generally will force this path to be in the /tmp tree.
* WIP - grabbing changes from rm-pre-genesis branch
* combine bellatrix state w/ rm-pre-genesis branch
* WIP
* use encoding/detect for genesis state bytes
* WIP more fixes towards start from bellatrix
* remove debug wrapping
* WIP
* multiple bugfixes
* fix fork ordering bug and bellatrix genesis blocks
* send deposits, spam tx to advance, fix miner alloc
* WIP
* WIP mess
* WIP
* Print process ID information for purposes of attaching a debugger
* bugs: genesis body_root and deposit index mismatch
* fix voting period start, skip altair check
* add changes
* make it better
* rm startup FCU, rm logs
* cleanup import grouping&ordering
* restore FCU log, get rid of tmp var
* rm newline
* restore newline
* restore wrapped error
* rm newline
* removing boot node version override
this doesn't seem to matter?
* add issue number to todo comment
* rm commented code
* rm vmdebug geth flag
* unexport values only used with genesis test pkg
and add comments where missing from exported values.
* adding comments to special cases for testnets
* migrate comments from PR to actual code :)
* rm unused test param
* mark e2e spawns exempt from gosec warning
* Fix DeepSource errors in `proposer_bellatrix.go` (#11739)
* Fix DeepSource errors in
* Omit receiver name
* Address PR comments
* Remove unused variable
* Fix more DeepSource errors
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Remove `Test_IsExecutionEnabledCapella` (#11752)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* Add REST implementation for Validator's `ProposeBeaconBlock` (#11731)
* WIP
* WIP
* WIP
* Add tests
* WIP
* Add more tests
* Address DeepSource errors
* Remove unused param
* Add more tests
* Address PR comments
* Address PR comments
* Fix formatting
* Remove unused parameter
* Fix TestLittleEndianBytesToBigInt
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
* fix validator client (#11755)
* fix validator client
(cherry picked from commit deb138959a2ffcb89cd2e3eb8304477526f4a168)
* Use signed changes in middleware block
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
* API `finalized` metadata field - update protos (#11749)
* API `finalized` metadata field - update protos
* change nums
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* log breaks unit tests that don't do full arg setup
easiest to just remove it for now
* restore prior behavior of phase0 block for altair
* update unit tests to account for special case
* loosen condition for fork version to match config
we don't know which fork version genesis will start from, so we
shouldn't force it to be a phase0 genesis.
* skip until we can mod configs at runtime
* NewGenesisBlockForState computes state root itself
* rm noisy log
* this log would be noisy in mainnet
* fix format specifier, []byte -> string
* core.Genesis UnmarshalJson has a value receiver :)
* no longer needs to be exported
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: Patrice Vignola <vignola.patrice@gmail.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: terencechain <terence@prysmaticlabs.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* support generating non-phase0 genesis.ssz
* make default (Value) work for EnumValue + lint
* remove messy punctuation
* Ran gazelle for @kasey
* Fix deps viz
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
* rewrite/refactor deposit testing code
keep track of sent deposits so that they can be compared in detail with
the validator set retreived from the API.
* fix bugs in evaluator and retry
* lint + deepsource appeasement
* typo s/Sprintf/Printf/
* gosec, more like nosec
* fix gosec number - 204->304
* type switch to get signed block from container
* improve comments
* centralizing constants and adding comments
* lock around Depositor to avoid future races
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
* Update go to 1.19.3
* Update other items to 1.19
* Update golangci-lint to latest release
* Run gofmt -s with go1.19
* Huge gofmt changes
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* moving voluntary exit command to prysmctl
* fixing linting
* fixing imports
* removing unused import:
* refactoring and adding a new unit test
* rolling back some changes
* fixing parameters
* fixing tests
* adding to main prysm ctl
* refactoring how wallet function works and ux to validator exit
* adding interop support
* fixing bazel build
* fixing if statement
* fixing beaconcha.in printout
* fixing web3signer bug, missing signing slot info
* fixing deep source issues
* fixing bazel package visibility for prysmctl
* gaz
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Logging: strip new lines and other control characters
* Use strings.Map and only sanitize the field value
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add a copy of github.com/x-cray/logrus-prefixed-formatter with fixes for our static analysis
* gazelle and add failing test
* fix it
* fix link
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* v0
* getters/setters
* init and copy
* hasher
* all the nice stuff
* make bazel happy
* remove tests for smaller PR
* remove old states
* move files
* import fixes
* custom MarshalSSZ
* fixed deadlock
* copy version when copying state
* correct issues in state_trie
* fix Copy()
* better e2e comment
* add code to minimal state
* spectest test
* Revert "Auxiliary commit to revert individual files from 84154423464e8372f7e0a03367403656ac5cd78e"
This reverts commit 9602599d183081291dfa0ba4f1036430f63a7822.
* native state assert
* always error
* always log
* more native state usage
* cleanup
* remove empty line
* Revert "spectests"
This reverts commit 1c49bed5d1cf6224afaf21e18562bf72fae5d2b6.
# Conflicts:
# beacon-chain/powchain/service.go
# beacon-chain/state/v1/state_trie.go
# beacon-chain/state/v2/state_trie.go
# beacon-chain/state/v3/state_trie.go
# testing/spectest/shared/phase0/finality/BUILD.bazel
# testing/spectest/shared/phase0/finality/runner.go
* dedup field trie
* fix test issues
* cleanup
* use correct field num in FinalizedRootProof
* use existing version constant
* halfway there
* "working" version
* some fixes
* fix field nums in tests
* rename v0types to nativetypes
* Revert "Auxiliary commit to revert individual files from dc549b1cf8e724bd08cee1ecc760ff3771d5592d"
This reverts commit 7254d3070d8693b283fc686a2e01a822ecbac1b3.
* uncomment code
* remove map size
* Revert "Revert "spectests""
This reverts commit 39c271ae6b57d6a3737e2c202cd8407857475e56.
* use reverse map
* Revert "Revert "Revert "spectests"""
This reverts commit 19ba8cf95cdca689357c8234a262e08cccbafef4.
* finally found the bug
(cherry picked from commit a5414c4be1bdb61a50b391ea5301895e772cc5e9)
* simplify populateFieldIndexes
* fix copy
(cherry picked from commit 7da4fb8cf51557ef931bb781872ea52fc6731af5)
* remove native state from e2e
* remove index map
* unsupported functions
* Use ProtobufBeaconState() from native state
* tests
* typo
* reduce complexity of `SaveStatesEfficient`
* remove unused receiver name
* update doc.go
* fix test assertion
* fix test assertion 2
* Phase0 justification bits
* bring back state tests
* rename fieldIndexRev
* versioning of ToProto
* remove version check from unexported function
* hasher tests
* don't return error from JustificationBits
* extract fieldConvertersNative
* helper error function
* use fieldConvertersNative
* Introduce RealPosition method on FieldIndex
* use RealPosition in hasher
* remove unused fields
* remove TestAppendBeyondIndicesLimit
(cherry picked from commit 3017e700282969c30006b64c95c21ffe6b166f8b)
* simplify RealPosition
* rename field interface
* use helper in proofs.go
* Update beacon-chain/core/altair/upgrade.go
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
