* First take at updating everything to v5
* Patch gRPC gateway to use prysm v5
Fix patch
* Update go ssz
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* retrieve and save blobs during backfill
* Update beacon-chain/sync/backfill/batch.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Update beacon-chain/sync/backfill/blobs.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Update beacon-chain/sync/backfill/metrics.go
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* make blobSync initialization path a little safer
* use bytes.Equal to avoid extra allocation
* stop using log.Fatal and other little cleanups
* wrap up post blob sync actions in batch mutator
* unit test coverage on verifier
---------
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* `LastEpochWrittenForValidators`: Use golang if.
* `SaveLastEpochsWrittenForValidators`: Refactor.
* `SaveLastEpochsWrittenForValidators`: Fix when `epochByValIndex` > `batchSize`.
Before this commit, `TestStore_LastEpochWrittenForValidators` works if `validatorsCount <= 10000`
and stops working if `validatorsCount > 10000`.
* Slasher: Detect surrounded votes in multiple batches.
Fixes https://github.com/prysmaticlabs/prysm/issues/13591.
* `Test_processAttestations`: Remove duplicated tests.
* Sort indexed attestations by data root.
* `processAttestations`: Don't return duplicate slashings anymore.
Fix https://github.com/prysmaticlabs/prysm/issues/13592.
* `AttesterDoubleVote`: Rename fields.
* Detect double votes in different batches.
In order to do that:
1. Each attestation of the batch is tested against the other attestations of the batch.
2. Each attestation of the batch is tested against the content of the database.
2. Attestations are saved into the database.
Fixes https://github.com/prysmaticlabs/prysm/issues/13590.
* `helpers.go`: Improve naming consistency.
* `detect_attestations.go`: Improve readability.
* `receive.go`: Add `attsQueueSize` in log message.
* `checkSlashableAttestations`: Improve logging.
`avgBatchProcessingTime` is not displayed any more if not batch is
processed.
* `loadChunks`: Use explicit `chunkKind` and `chunkIndices`.
* `getChunk`: Use specific `chunkIndex` and `chunkKind`.
* `validatorIndicesInChunk` -> `validatorIndexesInChunk`.
* `epochUpdateForValidator`: Use explicit arguments.
* `getChunk`: Change order of arguments.
* `latestEpochWrittenForValidator`: Use `ok` parameter.
So the default value is not any more considered as the absence of
value.
* `applyAttestationForValidator`: Use explicit arguments.
* `updateSpans`: Use explicit arguments.
* `saveUpdatedChunks`: Use explicit arguments.
* `checkSurrounds`: Use explicit arguments.
We see here that, previously, in `checkSlashableAttestations`,
`checkSurrounds` was called with the default value of `slashertypes`: `MinSpan`.
Now, we set it expliciterly at `MinSpan`, which may explicit a bug.
* `epochUpdateForValidator`: Set modified by the function argument first.
* `applyAttestationForValidator`: Set mutated argument `chunksByChunkIdx`first.
* `applyAttestationForValidator`: Rename variables.
* `Test_processQueuedAttestations`: Fix test.
Two tests were actually exactly the same.
* `updateSpans`: Keep happy path in the outer scope.
Even if in this case the "happy" path means slashing.
* `checkSurrounds`: Rename variable.
* `getChunk`: Avoid side effects.
It adds a few lines for callers, but it does not modify any more
arguments and it does what it says: getting a chunk.
* `CheckSlashable`: Flatten.
* `detect_attestations_test.go`: Simplify.
* `CheckSlashable`: Add error log in case of missing attestation.
* `processQueuedAttestations`: Extract a sub function.
So testing will be easier.
* `processAttesterSlashings` and `processProposerSlashings`: Improve.
* `processAttesterSlashings`: Return processed slashings.
* `createAttestationWrapper`: Rename variables.
* `signingRoot` ==> `headerRoot` or `dataRoot`.
Before this commit, there is two typse of `signing root`s floating around.
- The first one is a real signing root, aka a hash tree root computed from an object root and
a domain. This real signing root is the object ready to be signed.
- The second one is a "false" signing root, which is actually just the hash tree root of an object. This object is either the `Data` field of an attestation, or the `Header` field of a block.
Having 2 differents objects with the same name `signing root` is quite confusing.
This commit renames wrongly named `signing root` objects.
* `createAttestationWrapper` => `createAttestationWrapperEmptySig`.
So it's clear for the user that the created attestation wrapper has an empty signature.
* Implement `createAttestationWrapper`.
* `processAttestations`: Return processed attester slashings.
* Test `processAttestations` instead of `processQueuedAttestations`.
By testing `processAttestations` instead of `processQueuedAttestations`, we get rid of a lot of tests fixtures, including the 200 ms sleep.
The whole testing duration is shorter.
* `Test_processAttestations`: Allow multiple steps.
* `Test_processAttestations`: Add double steps tests.
Some new failing tests are commented with a corresponding github issue.
* `NextChunkStartEpoch`: Fix function comment.
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* `chunks.go`: Avoid templating log messages.
* `checkSlashableAttestations`: Simplify duration computation.
---------
Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
* Add coreutils to docker images
* add coreutils dependencies
* Add a prysmaticlabs.com/uploads backup of the deb files
* Run gazelle and fix issues
* Remove broken tar, change http_archive deps to debian_archive, remove http mirrors in favor of snapshot
* Add comments about which deps are required by other deps
* Slasher: Ensure all gorouting are stopped before running `Stop` actions.
Fixes#13550.
In tests, `exitChan` are now useless since waitgroup are used to wait
for all goroutines to be stopped.
* `slasher.go`: Add comments and rename some variables. - NFC
* `detect_blocks.go`: Improve. - NFC
- Rename some variables.
- Add comments.
- Use second element of `range` when possible.
* `chunks.go`: Remove `_`receivers. - NFC
* `validateAttestationIntegrity`: Improve documentation. - NFC
* `filterAttestations`: Avoid `else`and rename variable. - NFC
* `slasher.go`: Fix and add comments.
* `SaveAttestationRecordsForValidators`: Remove unused code.
* `LastEpochWrittenForValidators`: Name variables consistently. - NFC
Avoid mixes between `indice(s)`and `index(es)`.
* `SaveLastEpochsWrittenForValidators`: Name variables consistently. - NFC
* `CheckAttesterDoubleVotes`: Rename variables and add comments. - NFC
* `schema.go`: Add comments. - NFC
* `processQueuedAttestations`: Add comments. - NFC
* `checkDoubleVotes`: Rename variable. - NFC
* `Test_processQueuedAttestations`: Ensure there is no error log.
* `shouldNotBeSlashable` => `shouldBeSlashable`
* `Test_processQueuedAttestations`: Add 2 test cases:
- Same target with different signing roots
- Same target with same signing roots
* `checkDoubleVotesOnDisk` ==> `checkDoubleVotes`.
Before this commit, `checkDoubleVotes` did two tasks:
- Checking if there are any slashable double votes in the input
list of attestations with respect to each other.
- Checking if there are any slashable double votes in the input
list of attestations with respect to our database.
However, `checkDoubleVotes` is called only in
`checkSlashableAttestations`.
And `checkSlashableAttestations` is called only in:
- `processQueuedAttestations`, and in
- `IsSlashableAttestation`
Study of case `processQueuedAttestations`:
---------------------------------------------
In `processQueuedAttestations`, `checkSlashableAttestations`
is ALWAYS called after
`Database.SaveAttestationRecordsForValidators`.
It means that, when calling `checkSlashableAttestations`,
`validAtts` are ALREADY stored in the DB.
Each attestation of `validAtts` will be checked twice:
- Against the other attestations of `validAtts` (the portion of
deleted code)
- Against the content of the database.
One of those two checks is redundent.
==> We can remove the check against other attestations in `validAtts`.
Study of case `Database.SaveAttestationRecordsForValidators`:
----------------------------------------------------------------
In `Database.SaveAttestationRecordsForValidators`,
`checkSlashableAttestations` is ALWAYS called with a list of
attestations containing only ONE attestation.
This only attestaion will be checked twice:
- Against itself, and an attestation cannot conflict with itself.
- Against the content of the database.
==> We can remove the check against other attestations in `validAtts`.
=========================
In both cases, we showed that we can remove the check of attestation
against the content of `validAtts`, and the corresponding test
`Test_checkDoubleVotes_SlashableInputAttestations`.
* `Test_processQueuedBlocks_DetectsDoubleProposals`: Wrap proposals.
So we can add new proposals later.
* Fix slasher multiple proposals false negative.
If a first batch of blocks is sent with:
- validator 1 - slot 4 - signing root 1
- validator 1 - slot 5 - signing root 1
Then, if a second batch of blocks is sent with:
- validator 1 - slot 4 - signing root 2
Because we have two blocks proposed by the same validator (1) and for
the same slot (4), but with two different signing roots (1 and 2), the
validator 1 should be slashed.
This is not the case before this commit.
A new test case has been added as well to check this.
Fixes#13551
* `params.go`: Change comments. - NFC
* `CheckSlashable`: Keep the happy path without indentation.
* `detectAllAttesterSlashings` => `checkSurrounds`.
* Update beacon-chain/db/slasherkv/slasher.go
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
* Update beacon-chain/db/slasherkv/slasher.go
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
* `CheckAttesterDoubleVotes`: Keep happy path without indentation.
Well, even if, in our case, "happy path" mean slashing.
* 'SaveAttestationRecordsForValidators': Save the first attestation.
In case of multiple votes, arbitrarily save the first attestation.
Saving the first one in particular has no functional impact,
since in any case all attestations will be tested against
the content of the database. So all but the first one will be
detected as slashable.
However, saving the first one and not an other one let us not
to modify the end to end tests, since they expect the first one
to be saved in the database.
* Rename `min` => `minimum`.
Not to conflict with the new `min` built-in function.
* `couldNotSaveSlashableAtt` ==> `couldNotCheckSlashableAtt`
---------
Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
