Commit Graph

694 Commits

Author SHA1 Message Date
Radosław Kapka
e3ce1bde45
Move API structs to api module (#13577) 2024-02-03 11:57:01 +00:00
james-prysm
4afb379f8d
cleanup duties naming (#13451)
* updating some naming to reflect changes to duties

* fixing unit tests

* fixing more tests
2024-01-22 16:58:25 +00:00
terence
852db1f3eb
Remove debug setting highest slot log (#13488) 2024-01-19 04:25:15 +00:00
Radosław Kapka
204de13c86
REST VC: Subscribe to Beacon API events (#13453)
* Revert "Revert "REST VC: Subscribe to Beacon API events  (#13354)" (#13428)"

This reverts commit 8d092a1113.

* change logic

* review

* test fix

* fix critical error

* merge flag check

* change error msg

* return on errors

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-01-18 14:27:41 +00:00
james-prysm
790a09f9b1
Improve wait for activation (#13448)
* removing timeout on wait for activation, instead switched to an event driven approach

* fixing unit tests

* linting

* simplifying return

* adding sleep for the remaining slot to avoid cpu spikes

* removing ifstatement on log

* removing ifstatement on log

* improving switch statement

* removing the loop entirely

* fixing unit test

* fixing manu's reported issue with deletion of json file

* missed change around writefile at path

* gofmt

* fixing deepsource issue with reading file

* trying to clean file to avoid deepsource issue

* still getting error trying a different approach

* fixing stream loop

* fixing unit test

* Update validator/keymanager/local/keymanager.go

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>

* fixing linting

---------

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
2024-01-16 17:04:54 +00:00
Manu NALEPA
99a8d0bac6
Validator client - Improve readability - NO FUNCTIONAL CHANGE (#13468)
* Improve `NewServiceRegistry` documentation.

* Improve `README.md`.

* Improve readability of `registerValidatorService`.

* Move `log` in `main.go`.

Since `log` is only used in `main.go`.

* Clean Tos.

* `DefaultDataDir`: Use `switch` instead of `if/elif`.

* `ReadPassword`: Remove unused receiver.

* `validator/main.go`: Clean.

* `WarnIfPlatformNotSupported`: Add Mac OSX ARM64.

* `runner.go`: Use idiomatic `err` handling.

* `waitForChainStart`: Avoid `chainStartResponse`mutation.

* `WaitForChainStart`: Reduce cognitive complexity.

* Logs: `powchain` ==> `execution`.
2024-01-15 14:46:54 +00:00
Radosław Kapka
2875ce6ee1
Use a single rest handler (#13446) 2024-01-11 16:03:35 +00:00
Brandon Liu
c41a54be9d
fix metric for exited validator (#13379)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-01-09 22:15:53 +00:00
Justin Traglia
703cfc5819
Initialize exec payload fields and enforce order (#13372)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-01-09 21:49:35 +00:00
Radosław Kapka
8d092a1113
Revert "REST VC: Subscribe to Beacon API events (#13354)" (#13428)
This reverts commit e68b2821c1.
2024-01-06 21:36:42 +00:00
Radosław Kapka
e68b2821c1
REST VC: Subscribe to Beacon API events (#13354)
* Initial code for head event streaming

* handle events and error

* keepalive event

* tests

* generate new mock

* remove single case select

* cleanup

* explain eventByteLimit

* use 2 channels in test

* review

* more review

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-01-04 17:14:45 +00:00
Manu NALEPA
82b2840d68
--validatorS-registration-batch-size (add s) (#13396) 2024-01-02 09:52:14 +00:00
Radosław Kapka
d0bf03e863
Simplify error handling for JsonRestHandler (#13369)
* Simplify error handling for `JsonRestHandler`

* POST

* reduce complexity

* review feedback

* uncomment route

* fix rest of tests
2023-12-22 22:39:20 +00:00
Potuz
b7e0819f00
refactor Payload Id caches (#12987)
* init

- getLocalPayload does not use the proposer ID from the cache but takes
  it from the block

- Fixed tests in blockchain package
- Fixed tests in the RPC package
- Fixed spectests

EpochProposers takes 256 bytes that can be avoided to be copied, but
this optimization is not clear to be worth it.

assginmentStatus can be optimized to use the cached version from the
TrackedValidatorsCache

We shouldn't cache the proposer duties when calling getDuties but when
we update the epoch boundary instead

* track validators on prepare proposers

* more rpc tests

* more rpc tests

* initialize grpc caches

* Add back fcu log

Also fix two existing bugs wrong parent hash on pre Capella and wrong
blockhashes on altair

* use beacon default fee recipient if there is none in the vc

* fix validator test

* radek's review

* push always proposer settings even if no flag is specified in the VC

* Only register with the builder if the VC flag is set

Great find by @terencechain

* add regression test

* Radek's review

* change signature of registration builder
2023-12-22 18:47:51 +00:00
james-prysm
856907d760
Small encoding fixes on logs and http error code change (#13345)
* fixing some bad encodings

* changing http error to align with other clients

* fixing unit test
2023-12-20 18:18:55 +00:00
Preston Van Loon
db096488b0
fixing sa4006 (#13350) 2023-12-15 16:49:27 +00:00
Justin Traglia
4a374435c0
Enable errname linter and fix findings (#13341) 2023-12-15 03:26:48 +00:00
Potuz
96df81d5c5
Hook to slot stream instead of block stream on the VC (#13327)
* Hook to slot stream instead of block stream on the VC

* Implement StreamSlots in the BN

* mock update

* fix tests

* don't return from stream

* Terence's review

* deepsource second complain

---------

Co-authored-by: rkapka <rkapka@wp.pl>
2023-12-13 23:13:56 +00:00
Radosław Kapka
8387088a52
Handle HTTP 404 Not Found in SubmitAggregateAndProof (#13320)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-12-12 19:46:19 +00:00
Manu NALEPA
a94f2b93e3
filterAndCacheActiveKeys: Stop filtering out exiting validators (#13305)
* `filterAndCacheActiveKeys`: Add test cases

- Validator is in unknown status (to be fitered out)
- Validator is in pending status,
   with activation period > current period (to be filtered out)
- Validator is in pending status,
   with activation period == current period (to be kept)

* `filterAndCacheActiveKeys`: Keep exiting keys

Initially:
-------
If a validator is in exiting (so, with status==EXITING != ACTIVE) state,
it will be filtered out by the `filterAndCacheActiveKeys` function.
The validator won't be registered to the beacon node.

If this exiting validator has to propose a block:
- the block will be proposed using local block building only.
- the fee recipient will be the one set in the beacon node.

(Additionally, if the beacon node Prysm without any
fee recipient defined at the beacon node level, the fee recipient
will default on the `0x00000...` burn address.)

This commit modifies the `filterAndCacheActiveKeys` function
by stopping filtering out exiting validators.
2023-12-09 07:53:08 +00:00
Radosław Kapka
4c47756aed
HTTP endpoints cleanup (#13251)
* remove validation package

* structs cleanup

* merge with apimiddleware removal

* more validation and Bls capitalization

* builder test fix

* use strconv for uint->str conversions

* use DecodeHexWithLength

* use exact param names

* rename http package to httputil

* change conversions to fmt.Sprintf

* handle query paramsd and route variables

* spans and receiver name

* split structs, move bytes helper

* missing ok check

* fix reference to indexed failure

* errors fixup

* add godoc to helper

* fix BLS casing and chainhead ref

* review

* fix import in tests

* gzl
2023-12-08 20:37:20 +00:00
Radosław Kapka
cee38660c7
Gracefully handle unknown validator index in the REST VC (#13296)
* Gracefully handle unknown validaor index in the REST VC

* add apostrophes
2023-12-08 04:30:50 +00:00
james-prysm
83af9a5694
Beacon API: update Deneb endpoints after removing blob signing (#13235)
* making needed changes to beacon API based on removal of blobsidecar from block contents

* fixing tests and reverting some changes to be addressed later

* fixing generated code from protos

* gaz

* fixing get blob handler and adding blob storage to the blob service

* updating unit tests

* WIP

* wip tests

* got tests passing but needs cleanup

* removing gomod and gosum changes

* fixing more tests

* fixing more tests

* fixing more tests

* gaz

* moving some proto types around

* removing unneeded unit test

* fixing proposer paths

* adding more tests

* fixing more tests

* improving more unit tests

* updating one blob only unit test

* changing arguments of buildBlobSidecar

* reverting a change based on feedback

* terence's review items

* fixing test based on new develop changes

* radek's comments

* addressed more comments from radek

* adding in blobs to test data

* fixing casing in test

* removing extra line

* fixing issue from bad merge

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/blob/handlers.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* moving core getblob business logic to blocker based on radek's comment

* fixing mock blocker

* gaz

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-07 17:37:11 +00:00
Radosław Kapka
28c3330375
Don't fetch duties for unknown keys (#13269)
* Don't fetch duties for unknown keys

* test
2023-12-06 14:18:45 +00:00
Nishant Das
cdd7958739
Fix Domain Data Caching (#13263)
* fix domain data caching

* fix locking of domain data

* preston's review
2023-12-06 12:04:19 +00:00
Manu NALEPA
1112e01c06
Make Prysm VC compatible with the version v5.3.0 of the slashing protections interchange tests. (#13232)
* `TestStore_GenesisValidatorsRoot_ReadAndWrite`: Make all test cases independents.

In a test with multiple test cases, each test case should be independents.
(aka: Removing test case `A` should not impact test case `B`)

* `SaveGenesisValidatorsRoot`: Allow to overwrite the genesis validator root if the root is the same.

* `ProposalHistoryForSlot`: Add `signingRootExists`

Currently, it is not possible with `ProposalHistoryForSlot` to know if a
proposal is stored with and `0x00000....` signing root or with an empty
signing root. Both cases result to `proposalExists == true` and
`signingRoot == 0x00000`.

This commit adds a new return boolean: `signingRootExists`.

If a proposal has been saved with a `0x00000...` signing root, then:
- `proposalExists` is set to `true`, and
- `signingRootExists` is set to `true`, and
- `signingRoot` is set to `0x00000...`

If a proposal has been saved with an empty signing root, then:
- `proposalExists` is set to `true`, and
- `signingRootExists` is set to `false`, and
- (`signingRoot` is set to `0x00000...`)

* `ImportStandardProtectionJSON`: When importing EIP-3076 Slashing Protection Interchange Format, do not filter any more slashable keys.
Note: Those keys are still saved into the black-listed public keys list.

There is two reason not to do so:
- The EIP-3076 test cases do not know about Prysm's internal black-listed public keys list.
  Tests will expect, without looking into this internal black-listed public keys list,
  to deny a further signature. If we filter these keys from the DB (even if we keep them
  into the black-listed keys list), then some tests will fail.
- If we import a interchange file containing slashable keys and we filter them, then,
  if we re-export the DB, those slashing offences won't appear in the exported interchange
  file.

* `transformSignedBlocks`: Store an 0-len byte slice

When importing an EIP-3076 interchange format, and when no
signing root is specified into the file, we currently store a
`0x00000.....` signing root.

In such a case, instead storing `0x00000...`, this commit stores
a 0-len byte array, so we can differentiate real `0x000.....` signing
root and no signing-root at all.

* `slashableProposalCheck`: Manage lack of sign root

Currently, `slashableProposalCheck` does not really make a difference
between a `0x0000.....` signing root and a missing signing root.

(Signing roots can be missing when importing an EIP-3076 interchange
file.)

This commit differentiate, for  `slashableProposalCheck`, `0x0000....`
signing root and a missing signing root.

* `AttestationRecord.SigningRoot`: ==> `[]byte`

When importing attestations from EIP-3076 interchange format,
the signing root of an attestation may be missing.

Currently, Prysm consider any missing attestation signing root as
`0x000...`.
However, it may conflict with signing root which really are equal to
`0x000...`.

This commit transforms `AttestationRecord.SigningRoot` from `[32]byte` to
`[]byte`, and change the minimal set of functions (sic) to support this
new type.

* `CheckSlashableAttestation`: Empty signing root

Regarding slashing roots, 2 attestations are slashable, if:
- both signing roots are defined and differs, or
- one attestation exists, but without a signing root

* `filterSlashablePubKeysFromAttestations`: Err sort

Rergarding `CheckSlashableAttestation`, we consider that:
- If slashable == NotSlashable and err != nil, then CheckSlashableAttestation
failed.
- If slashable != NotSlashable, then err contains the reason why the attestation
is slashable.

* `setupEIP3076SpecTests`: Update to `v5.3.0`

This commit:
- Updates the version of EIP-3076 tests to `v.5.2.1`.
- Setups on anti-slashing DB per test case, instead per step.

* `ImportStandardProtectionJSON`: Reduce cycl cmplxt

* `AttestationHistoryForPubKey`: copy signing root

BoltDB documentation specifies:
| Byte slices returned from Bolt are only valid during a transaction.
| Once the transaction has been committed or rolled back then the memory
| they point to can be reused by a new page or can be unmapped
| from virtual memory and you'll see an unexpected fault address panic
| when accessing it.
2023-12-04 17:10:32 +00:00
Radosław Kapka
a0ca4a67b0
Remove API Middleware (#13243)
* remove api/gateway/apimiddleware

* fix errors in api/gateway

* remove beacon-chain/rpc/apimiddleware

* fix errors in api/client/beacon

* fix errors in validator/client/beacon-api

* fix errors in beacon-chain/node

* fix errors in validator/node

* fix errors in cmd/prysmctl/validator

* fix errors in testing/endtoend

* fix all other code

* remove comment

* fix tests

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-04 11:55:21 +00:00
Radosław Kapka
5ecb4d62a9
REST VC: Use POST to fetch validators (#13239) 2023-11-29 18:53:26 +01:00
james-prysm
bc107a61e3
builder API: remove blinded blob sidecar (#13202) 2023-11-29 06:28:37 -08:00
Manu NALEPA
da2212f6cc
Allow validators registration batching on Builder API /eth/v1/builder/validators (#13178)
* builder `NewClient`: Simplify + fix some typos.

* Validator client: Implement `validator-registration-batch-size` option

* Address Potuz comments

* Address Potuz's comments

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-28 00:23:48 +00:00
terence
7cc05401ca
Update proposer RPC to new blob sidecar format (#13189) 2023-11-27 15:44:52 -08:00
Radosław Kapka
2fbda536b0
Fix handling POST requests in the REST VC (#13215)
* Fix handling POST requests in the REST VC

* tests for decodeResp
2023-11-25 23:25:14 +00:00
Radosław Kapka
a7edec9b98
Better error handling in REST VC (#13203) 2023-11-21 17:42:55 +01:00
Terence
12f7143c4f
Validator client: remove blob signing (#13169) 2023-11-03 12:10:15 -07:00
Terence
57eda1de63
Add RO blob sidecar (#13144) 2023-11-01 10:03:49 -07:00
james-prysm
27b4e32e1c
HTTP Validator API: /eth/v1/keystores (#13113)
* WIP

* fixing tests

* fixing bazel

* fixing api client

* fixing tests

* fixing more tests and bazel

* fixing trace and more bazel issues

* fixing router path function definitions

* fixing more tests and deep source issues

* adding delete test

* if a route is provided, reregister before the catch all on the middleware.

* fixing linting

* fixing deepsource complaint

* gaz

* more deepsource issues

* fixing missed err check

* changing how routes are registered

* radek reviews

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* Update validator/rpc/handlers_keymanager.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* fixing unit test after sammy's review

* adding radek's comments

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2023-10-31 16:33:54 +00:00
Radosław Kapka
2ef0b3526d
Fix block proposals in the REST validator client (#13116)
* Fix block proposals in the REST validator client

* fix graffiti test

* return empty graffiti

* fallback to old endpoints

* logs

* handle 404

* everything passes

* review from James

* log undecoded value

* test fixes and additions

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-30 18:15:45 +00:00
hyunchel
470ea6d717
Remove no-op cancel func (#13069)
This cancel function is currently a no-op due to the blank identifier.
One might argue that the cancel func should be restored from no-op by
replacing the blank identifier with the proper variable. When the parent
context is cancelled, however, all the functions down the call tree with
the context will be notified of the cancellation anyway. Removing the
cancel function would not change any outcome under the current
implementation.

Co-authored-by: Preston Van Loon <pvanloon@offchainlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-30 14:43:25 +00:00
Radosław Kapka
f925aded66
Allow unknown fields in Beacon API responses (#13131) 2023-10-27 16:47:50 +00:00
terencechain
76fec1799e
Replace Empty Slice Literals with Nil Slices (#13093)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-23 16:36:11 +00:00
terencechain
9387a36b66
Refactor Exported Names to Follow Golang Best Practices (#13075)
* Fix exported names that start with a package name

* A few more renames

* Fix exported names that start with a package name

* A few more renames

* Radek's feedback

* Fix conflict

* fix keymanager test

* Fix comments

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-20 16:45:33 +00:00
Radosław Kapka
a536612c39
HTTP Beacon APIs for node (#13010) 2023-10-11 22:18:49 +02:00
james-prysm
55e4c6e1db
HTTP API: /eth/v1/beacon/pool/bls_to_execution_changes (#12963)
* wip

* wip changes for beacon handlers

* wip

* updating protos

* fixing conflict

* fixing protos

* fixing more tests

* migrating tests and removing one that is irrelevant

* fixing e2e

* fixing unit tests

* missed adding routes

* gaz

* fixing more tests

* Update beacon-chain/rpc/eth/beacon/handlers_pool.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/shared/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/shared/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/shared/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/shared/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* radek's comments

* fixing protos after conflict

* accidently removed a proto type

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-11 14:08:06 +00:00
Dhruv Bodani
2806326155
integrate validator count endpoint in validator client (#12912)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-10-11 15:23:02 +02:00
Preston Van Loon
8eb82dd378
Update rules_go and gazelle to 0.42 & 0.33 (latest releases) (#13021)
* Provide @go_googleapis, update gazelle

* fix pb build failures

Fix build issues with grpc-gateway
2023-10-10 12:50:29 +08:00
Radosław Kapka
f37301c0c0
Remove remote slashing protection feature (#12989)
* Remove remote slashing protection feature

* test fix

* remove mock from tests

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-10-04 04:15:06 +00:00
james-prysm
c8b39e08ef
cleanup on api types (#12961)
* removing code duplication using shared types

* gaz

* fixing test

* fixing tests
2023-09-27 09:26:32 -05:00
Radosław Kapka
e22258caa9
HTTP validator APIs (#12887)
* GetValidators in progress

* in progress

* completed implementation of GetValidators + some tests

* typo

* completed tests for GetValidators

* GetValidator

* GetValidatorBalances

* register

* more tests and e2e fix

* middleware cleanup

* remove struct

* handle all ignored

* test fixes

* more test fixes

* even more test fixes

* remove unused structs

* docs

* fix validator count test

* build fix

* add length checks

* lint fix

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-09-21 18:16:59 +00:00
Radosław Kapka
3bc9ac37f6
Add Content-Type header to VC POST requests (#12942) 2023-09-21 17:21:06 +00:00
Radosław Kapka
e76aedf1ae
HTTP API: GetLiveness and GetBlockHeader (#12916)
* GetLiveness

* GetBlockHeader

* simplify request objects

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-09-20 19:06:11 +00:00