* include migration and logic for stopping early in slashing protection checks
* remove commented code
* extract methods
* migration logic tested up
* migration up and down tests
* Update validator/db/kv/attester_protection.go
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* added in pruning and batched migrations
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Add a debug log to show duration
* merge from dev
* use safe pending attestation records struct
* fix build, use atomic bool
* Add deadline checks to CheckSlashableAttestation
* Go fmt
* Add test for in-progress log
* GoDocs
* Rename pending attestation records to queued attestation records
* rename and add commentary on log
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add a debug log to show duration
* Autofix issues in 1 file
Resolved issues in validator/db/kv/attester_protection.go via DeepSource Autofix
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
* add in batch method
* add in new proposal history methods for efficiency and progress bars
* tests fixed to use the new methods
* add back get slot proposing history method
* add gaz
* Change LowestSignedProposal to Also Return a Boolean for Slashing Protection (#8020)
* amend to use bools
* ineff assign
* comment
* Update `LowestSignedTargetEpoch` to include exists (#8004)
* Replace highest with lowerest
* Update validator/db/kv/attestation_history_v2.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/db/kv/attestation_history_v2.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Invert equality for saveLowestSourceTargetToDB
* Add eip checks to ensure epochs cant be lower than db ones
* Should be less than equal to
* Check if epoch exists in DB getters
* Revert run time checks
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Export Attesting History for Slashing Interchange Standard (#8027)
* added in att history checks
* logic for export
* export return nil
* test for export atts
* round trip passes first try!
* rem println
* fix up tests
* pass test
* Validate Proposers Are Not Slashable With Regard to Data Within Slasher Interchange JSON (#8031)
* filter slashable blocks and atts in same json stub
* add filter blocks func
* add test for filtering out the bad public keys
* Export Slashing Protection History Via CLI (#8040)
* include cli entrypoint for history exports
* builds properly
* test to confirm we export the data as expected
* abstract helpers properly
* full test suite
* gaz
* better errors
* marshal ident
* Add the additional eip-3076 attestation checks (#7966)
* Replace highest with lowerest
* Update validator/db/kv/attestation_history_v2.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update validator/db/kv/attestation_history_v2.go
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Invert equality for saveLowestSourceTargetToDB
* Add eip checks to ensure epochs cant be lower than db ones
* Should be less than equal to
* Check if epoch exists in DB getters
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Add EIP-3076 Invariants for Proposer Slashing Protection (#8067)
* add invariant for proposer protection
* write different test cases
* pass tests
* Add EIP-3076 Interchange JSON CLI command to validator (#7880)
* Import JSON CLI
* CLI impotr
* f
* Begin adding new commands in slashing protection
* Move testing helpers to separate packae
* Add command for importing slashing protection JSONs
* fix import cycle
* fix test
* Undo cleaning changes
* Improvements
* Add better prompts
* Fix prompt
* Fix
* Fix
* Fix
* Fix conflict
* Fix
* Fixes
* Fixes
* Fix exported func
* test func
* Fixes
* fix test
* simplify import and standardize with export
* add round trip test
* true integration test works
* fix up comments
* logrus
* better error
* fix build
* build fix
* Update validator/slashing-protection/cli_export.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update validator/slashing-protection/cli_import.go
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* fmt
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Filter Slashable Attester Public Keys in Slashing Interchange Import (#8051)
* filter slashable attesters from the same JSON
* builds
* fix up initially broken test
* circular dep
* import fix
* giz
* added in attesting history package
* add test for filter slashable attester keys
* pass tests
* Save Slashable Keys to Disk in the Validator Client (#8082)
* begin db funcs
* add in test and bucket
* gaz
* rem changes to import
* ineff assign
* add godoc
* Properly Handle Duplicate Public Key Entries in Slashing Interchange Imports (#8089)
* Prevent Blacklisted Public Keys from Slashing Protection Imports from Having Duties at Runtime (#8084)
* tests on update duties
* ensure the slashable public keys are filtered out from update duties via test
* begin test
* attempt test
* rename for better context
* pass tests
* deep source
* ensure tests pass
* Check for Signing Root Mismatch When Submitting Proposals and Importing Proposals in Slashing Interchange (#8085)
* flexible signing root
* add test
* add tests
* fix test
* Preston's comments
* res tests
* ensure we consider the case for minimum proposals
* pass test
* tests passing
* rem unused code
* Set Empty Epochs in Between Attestations as FAR_FUTURE_EPOCH in Attesting History (#8113)
* set target data
* all tests passing
* ineff assign
* signing root
* Add Slashing Interchange, EIP-3076, Spec Tests to Prysm (#7858)
* Add interchange test framework
* add checks for attestations
* Import genesis root if necessary
* flexible signing root
* add test
* Sync
* fix up test build
* only 3 failing tests now
* two failing
* attempting to debug problems in conformity tests
* include latest changes
* protect test in validator/client passing
* pass tests
* imports
* spec tests passing with bazel
* gh archive link to spectests using tar.gz suffix
* rev
* rev more comment changes
* fix sha
* godoc
* add back save
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Implement Migration for Unattested Epochs in Attesting History Database (#8121)
* migrate attesting history backbone done
* begin migration logic
* implement migration logic
* migration test
* add test
* migration logic
* bazel
* migration to its own file
* Handle empty blocks and attestations in interchange json and sort interchange json by public key (#8132)
* Handle empty blocks and attestations in interchange json
* add test
* sort json
* easier empty arrays
* pass test
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* builds
* more tests finally build
* Align Slashing Interchange With Optimized Slashing Protection (#8268)
* attestation history should account for multiple targets per source
* attempt at some fixes
* attempt some test fixes
* experimenting with sorting
* only one more failing test
* tests now pass
* slash protect tests passing
* only few tests now failing
* only spec tests failing now
* spec tests passing
* all tests passing
* helper function for verifying double votes
* use helper
* gaz
* deep source
* tests fixed
* expect specific number of times for domain data calls
* final comments
* Batch Save Imported EIP-3076 Attestations (#8304)
* optimize save
* test added
* add test for sad path
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* revert bad find replace
* add comment to db func
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Shay Zluf <thezluf@gmail.com>
* Revert "Revert New Attester Protection DB Logic (#8237)"
This reverts commit 6738fa3493.
* Batch Attestation Records and Flush All at Once in Validator DB (#8243)
* begin flushing logic
* finalize logic before starting tests
* make code DRY
* better log fields
* gaz
* tweak parameter
* rename
* clarifying comment on error handling in event feed
* comprehensive tests
* more comments
* explain parameters in comments
* renamed consts
* Apply suggestions from code review
* gaz
* simplify
* typo
* comments
* Revert "Optimize Migration for new Attester Protection DB (#8231)"
This reverts commit c4ab67832f.
* Revert "Integrate New Slashing Protection DB Methods at Runtime (#8219)"
This reverts commit 3858068201.
* Revert "DB Migration for Optimal Local Slashing Protection (#8212)"
This reverts commit dd3ac6c2ed.
* integrate at runtime and revamp tests
* historical att
* Update validator/db/iface/interface.go
* deepsource
* import
* log the slashing kind
* gaz
* create a slashutils
* integrate new slashutil
* imports
* pruning and begin test
* comprehensive pruning tests
* add pruning on startup
* also prune source epochs bucket
* more testing
* greatly simplify pruning function
* pruning logic and comprehensive tests in
* att protection test
* gaz
* fix sneaky change
* rev
* documented and tested helper func
* begin migration logic
* wrote migration logic
* begin test file
* test for migration working
* gaz
* progressutil
* migration works even if partial data was written
* att sign validations
* eliminate old cached methods and use a simple approach in the db
* redefined db methods
* db package builds
* add multilock to attest and propose
* gaz
* removed concurrency tests that are no longer relevant
* add cache to db functions for attesting history checks
* passing
* add in feature flag --disable-attesting-history-db-cache
* remove lock
* Revert "remove lock"
This reverts commit b1a65020e406b9fa6e4f57c5cc6a5a0b2a11a240.
* comment
* gaz
* restore beacon node db
* revert image name
* move restore out of the kv folder
* remove files from kv folder
* go mod tidy
* Remove usage of prometheus testutil
* add yes/no to prompt text
* restore slasher db
* organize imports
* go mod tidy
* restore validator db
* close slasher db
* defer close backup db in tests
* simplify function literal
* Add getters and setters for source and target epochs
* Use the correct ImportStandardProtectionJSON
* Add tests
* Fix buckets
* Fix source to target
* Update validator/slashing-protection/local/standard-protection-format/import.go
* Fix bytesutil.BytesToUint64BigEndian will return 0 if input is less than 8 bytes
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* add in highest and lowest signed proposal
* begin adding tests
* add in db tests
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
