Commit Graph

1385 Commits

Author SHA1 Message Date
Radosław Kapka
e3ce1bde45
Move API structs to api module (#13577) 2024-02-03 11:57:01 +00:00
james-prysm
2dad245bc8
handle slice out of range (#13568)
* handle slice out of range

* adding some tests
2024-02-01 16:59:40 +00:00
james-prysm
2cddb5ca86
fixing jwt auth checks (#13565) 2024-02-01 15:13:52 +00:00
james-prysm
258123341e
add a log and update size for promptui (#13542) 2024-01-30 17:19:31 +00:00
james-prysm
8dd5e96b29
re-enabling jwt on keymanager API (#13492)
* re-enabling jwt on keymanager API

* adding tests

* Update validator/rpc/intercepter.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* handling error in test

* remove debugging logs

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
2024-01-22 22:16:10 +00:00
james-prysm
4afb379f8d
cleanup duties naming (#13451)
* updating some naming to reflect changes to duties

* fixing unit tests

* fixing more tests
2024-01-22 16:58:25 +00:00
terence
852db1f3eb
Remove debug setting highest slot log (#13488) 2024-01-19 04:25:15 +00:00
Radosław Kapka
204de13c86
REST VC: Subscribe to Beacon API events (#13453)
* Revert "Revert "REST VC: Subscribe to Beacon API events  (#13354)" (#13428)"

This reverts commit 8d092a1113.

* change logic

* review

* test fix

* fix critical error

* merge flag check

* change error msg

* return on errors

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-01-18 14:27:41 +00:00
james-prysm
790a09f9b1
Improve wait for activation (#13448)
* removing timeout on wait for activation, instead switched to an event driven approach

* fixing unit tests

* linting

* simplifying return

* adding sleep for the remaining slot to avoid cpu spikes

* removing ifstatement on log

* removing ifstatement on log

* improving switch statement

* removing the loop entirely

* fixing unit test

* fixing manu's reported issue with deletion of json file

* missed change around writefile at path

* gofmt

* fixing deepsource issue with reading file

* trying to clean file to avoid deepsource issue

* still getting error trying a different approach

* fixing stream loop

* fixing unit test

* Update validator/keymanager/local/keymanager.go

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>

* fixing linting

---------

Co-authored-by: Manu NALEPA <enalepa@offchainlabs.com>
2024-01-16 17:04:54 +00:00
Manu NALEPA
46387a903a
getLegacyDatabaseLocation: Change message. (#13471)
* `getLegacyDatabaseLocation`: Change message.

* Update validator/node/node.go

Co-authored-by: Nishant Das <nishdas93@gmail.com>

---------

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2024-01-16 11:29:36 +00:00
Manu NALEPA
99a8d0bac6
Validator client - Improve readability - NO FUNCTIONAL CHANGE (#13468)
* Improve `NewServiceRegistry` documentation.

* Improve `README.md`.

* Improve readability of `registerValidatorService`.

* Move `log` in `main.go`.

Since `log` is only used in `main.go`.

* Clean Tos.

* `DefaultDataDir`: Use `switch` instead of `if/elif`.

* `ReadPassword`: Remove unused receiver.

* `validator/main.go`: Clean.

* `WarnIfPlatformNotSupported`: Add Mac OSX ARM64.

* `runner.go`: Use idiomatic `err` handling.

* `waitForChainStart`: Avoid `chainStartResponse`mutation.

* `WaitForChainStart`: Reduce cognitive complexity.

* Logs: `powchain` ==> `execution`.
2024-01-15 14:46:54 +00:00
Manu NALEPA
22a484c45e
Fixes issues when running validator client with the --web flag and non existing validator.db file AND/OR prysm-wallet-v2 directory. (#13460)
* `getLegacyDatabaseLocation`: Add tests.

* `getLegacyDatabaseLocation`: Handle `c.wallet == nil`.

* `saveAuthToken`: Create parent directory if needed.
2024-01-12 15:53:27 +00:00
Radosław Kapka
2875ce6ee1
Use a single rest handler (#13446) 2024-01-11 16:03:35 +00:00
psr
4fe6834ba5
http endpoint cleanup (#13432)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2024-01-09 23:48:43 +00:00
Brandon Liu
c41a54be9d
fix metric for exited validator (#13379)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-01-09 22:15:53 +00:00
Justin Traglia
703cfc5819
Initialize exec payload fields and enforce order (#13372)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-01-09 21:49:35 +00:00
Radosław Kapka
8d092a1113
Revert "REST VC: Subscribe to Beacon API events (#13354)" (#13428)
This reverts commit e68b2821c1.
2024-01-06 21:36:42 +00:00
Radosław Kapka
e68b2821c1
REST VC: Subscribe to Beacon API events (#13354)
* Initial code for head event streaming

* handle events and error

* keepalive event

* tests

* generate new mock

* remove single case select

* cleanup

* explain eventByteLimit

* use 2 channels in test

* review

* more review

---------

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2024-01-04 17:14:45 +00:00
Manu NALEPA
82b2840d68
--validatorS-registration-batch-size (add s) (#13396) 2024-01-02 09:52:14 +00:00
Manu NALEPA
cf221d0f4c
Validator client: Always use the --datadir value. (#13392)
Fix https://github.com/prysmaticlabs/prysm/issues/13391
2024-01-02 09:24:24 +00:00
Radosław Kapka
d0bf03e863
Simplify error handling for JsonRestHandler (#13369)
* Simplify error handling for `JsonRestHandler`

* POST

* reduce complexity

* review feedback

* uncomment route

* fix rest of tests
2023-12-22 22:39:20 +00:00
Potuz
b7e0819f00
refactor Payload Id caches (#12987)
* init

- getLocalPayload does not use the proposer ID from the cache but takes
  it from the block

- Fixed tests in blockchain package
- Fixed tests in the RPC package
- Fixed spectests

EpochProposers takes 256 bytes that can be avoided to be copied, but
this optimization is not clear to be worth it.

assginmentStatus can be optimized to use the cached version from the
TrackedValidatorsCache

We shouldn't cache the proposer duties when calling getDuties but when
we update the epoch boundary instead

* track validators on prepare proposers

* more rpc tests

* more rpc tests

* initialize grpc caches

* Add back fcu log

Also fix two existing bugs wrong parent hash on pre Capella and wrong
blockhashes on altair

* use beacon default fee recipient if there is none in the vc

* fix validator test

* radek's review

* push always proposer settings even if no flag is specified in the VC

* Only register with the builder if the VC flag is set

Great find by @terencechain

* add regression test

* Radek's review

* change signature of registration builder
2023-12-22 18:47:51 +00:00
james-prysm
856907d760
Small encoding fixes on logs and http error code change (#13345)
* fixing some bad encodings

* changing http error to align with other clients

* fixing unit test
2023-12-20 18:18:55 +00:00
Justin Traglia
ffe2f6b732
Enable mirror linter and fix findings (#13342)
* Enable mirror linter and fix findings

* Use latest version of golangci-lint

* Use v1.55.2 instead of latest

---------

Co-authored-by: terence <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-18 12:16:26 +00:00
Justin Traglia
c56abfb840
Enable usestdlibvars linter and fix findings (#13339)
Co-authored-by: terence <terence@prysmaticlabs.com>
2023-12-15 19:21:54 +00:00
Preston Van Loon
db096488b0
fixing sa4006 (#13350) 2023-12-15 16:49:27 +00:00
Justin Traglia
4a374435c0
Enable errname linter and fix findings (#13341) 2023-12-15 03:26:48 +00:00
Potuz
96df81d5c5
Hook to slot stream instead of block stream on the VC (#13327)
* Hook to slot stream instead of block stream on the VC

* Implement StreamSlots in the BN

* mock update

* fix tests

* don't return from stream

* Terence's review

* deepsource second complain

---------

Co-authored-by: rkapka <rkapka@wp.pl>
2023-12-13 23:13:56 +00:00
james-prysm
4cbe144a6c
CLI: fixing account import ux bugs (#13328)
* fixing account import checking wallet twice, and adding sub folder search with a depth of 2

* removing uneeded check

* fixing unit test

* adding reset cache to fix potential flake

* improving test based on feedback
2023-12-13 17:11:32 +00:00
Radosław Kapka
8387088a52
Handle HTTP 404 Not Found in SubmitAggregateAndProof (#13320)
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2023-12-12 19:46:19 +00:00
Manu NALEPA
a94f2b93e3
filterAndCacheActiveKeys: Stop filtering out exiting validators (#13305)
* `filterAndCacheActiveKeys`: Add test cases

- Validator is in unknown status (to be fitered out)
- Validator is in pending status,
   with activation period > current period (to be filtered out)
- Validator is in pending status,
   with activation period == current period (to be kept)

* `filterAndCacheActiveKeys`: Keep exiting keys

Initially:
-------
If a validator is in exiting (so, with status==EXITING != ACTIVE) state,
it will be filtered out by the `filterAndCacheActiveKeys` function.
The validator won't be registered to the beacon node.

If this exiting validator has to propose a block:
- the block will be proposed using local block building only.
- the fee recipient will be the one set in the beacon node.

(Additionally, if the beacon node Prysm without any
fee recipient defined at the beacon node level, the fee recipient
will default on the `0x00000...` burn address.)

This commit modifies the `filterAndCacheActiveKeys` function
by stopping filtering out exiting validators.
2023-12-09 07:53:08 +00:00
Radosław Kapka
4c47756aed
HTTP endpoints cleanup (#13251)
* remove validation package

* structs cleanup

* merge with apimiddleware removal

* more validation and Bls capitalization

* builder test fix

* use strconv for uint->str conversions

* use DecodeHexWithLength

* use exact param names

* rename http package to httputil

* change conversions to fmt.Sprintf

* handle query paramsd and route variables

* spans and receiver name

* split structs, move bytes helper

* missing ok check

* fix reference to indexed failure

* errors fixup

* add godoc to helper

* fix BLS casing and chainhead ref

* review

* fix import in tests

* gzl
2023-12-08 20:37:20 +00:00
Preston Van Loon
ff99616833
Fix staticcheck violations (#13301)
* Fix violations of sa2002

* Fix violations of sa4005

* Fix violations of sa4010

* Fix violations for sa4023

* Comment on commented static checks
2023-12-08 13:07:52 +00:00
Radosław Kapka
cee38660c7
Gracefully handle unknown validator index in the REST VC (#13296)
* Gracefully handle unknown validaor index in the REST VC

* add apostrophes
2023-12-08 04:30:50 +00:00
james-prysm
481d77bfde
APIs: reusing grpc cors middleware for rest (#13284)
* reusing grpc cors middleware for rest

* addressing radek's comments

* Update api/server/middleware.go

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>

* fixing to recommended name

* fixing naming

* fixing rename on test

---------

Co-authored-by: Sammy Rosso <15244892+saolyn@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-08 04:24:18 +00:00
Nishant Das
590317553c
Support New Subnet Backbone (#13179)
* add in changes

* fix it up

* fix test

* gaz

* lint

* add back

* fix tests

* fix it

* fix tests

* add lib

* fix it
2023-12-08 04:07:48 +00:00
james-prysm
83af9a5694
Beacon API: update Deneb endpoints after removing blob signing (#13235)
* making needed changes to beacon API based on removal of blobsidecar from block contents

* fixing tests and reverting some changes to be addressed later

* fixing generated code from protos

* gaz

* fixing get blob handler and adding blob storage to the blob service

* updating unit tests

* WIP

* wip tests

* got tests passing but needs cleanup

* removing gomod and gosum changes

* fixing more tests

* fixing more tests

* fixing more tests

* gaz

* moving some proto types around

* removing unneeded unit test

* fixing proposer paths

* adding more tests

* fixing more tests

* improving more unit tests

* updating one blob only unit test

* changing arguments of buildBlobSidecar

* reverting a change based on feedback

* terence's review items

* fixing test based on new develop changes

* radek's comments

* addressed more comments from radek

* adding in blobs to test data

* fixing casing in test

* removing extra line

* fixing issue from bad merge

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/beacon/handlers_test.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update beacon-chain/rpc/eth/blob/handlers.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* moving core getblob business logic to blocker based on radek's comment

* fixing mock blocker

* gaz

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-12-07 17:37:11 +00:00
Radosław Kapka
28c3330375
Don't fetch duties for unknown keys (#13269)
* Don't fetch duties for unknown keys

* test
2023-12-06 14:18:45 +00:00
Nishant Das
cdd7958739
Fix Domain Data Caching (#13263)
* fix domain data caching

* fix locking of domain data

* preston's review
2023-12-06 12:04:19 +00:00
Manu NALEPA
1112e01c06
Make Prysm VC compatible with the version v5.3.0 of the slashing protections interchange tests. (#13232)
* `TestStore_GenesisValidatorsRoot_ReadAndWrite`: Make all test cases independents.

In a test with multiple test cases, each test case should be independents.
(aka: Removing test case `A` should not impact test case `B`)

* `SaveGenesisValidatorsRoot`: Allow to overwrite the genesis validator root if the root is the same.

* `ProposalHistoryForSlot`: Add `signingRootExists`

Currently, it is not possible with `ProposalHistoryForSlot` to know if a
proposal is stored with and `0x00000....` signing root or with an empty
signing root. Both cases result to `proposalExists == true` and
`signingRoot == 0x00000`.

This commit adds a new return boolean: `signingRootExists`.

If a proposal has been saved with a `0x00000...` signing root, then:
- `proposalExists` is set to `true`, and
- `signingRootExists` is set to `true`, and
- `signingRoot` is set to `0x00000...`

If a proposal has been saved with an empty signing root, then:
- `proposalExists` is set to `true`, and
- `signingRootExists` is set to `false`, and
- (`signingRoot` is set to `0x00000...`)

* `ImportStandardProtectionJSON`: When importing EIP-3076 Slashing Protection Interchange Format, do not filter any more slashable keys.
Note: Those keys are still saved into the black-listed public keys list.

There is two reason not to do so:
- The EIP-3076 test cases do not know about Prysm's internal black-listed public keys list.
  Tests will expect, without looking into this internal black-listed public keys list,
  to deny a further signature. If we filter these keys from the DB (even if we keep them
  into the black-listed keys list), then some tests will fail.
- If we import a interchange file containing slashable keys and we filter them, then,
  if we re-export the DB, those slashing offences won't appear in the exported interchange
  file.

* `transformSignedBlocks`: Store an 0-len byte slice

When importing an EIP-3076 interchange format, and when no
signing root is specified into the file, we currently store a
`0x00000.....` signing root.

In such a case, instead storing `0x00000...`, this commit stores
a 0-len byte array, so we can differentiate real `0x000.....` signing
root and no signing-root at all.

* `slashableProposalCheck`: Manage lack of sign root

Currently, `slashableProposalCheck` does not really make a difference
between a `0x0000.....` signing root and a missing signing root.

(Signing roots can be missing when importing an EIP-3076 interchange
file.)

This commit differentiate, for  `slashableProposalCheck`, `0x0000....`
signing root and a missing signing root.

* `AttestationRecord.SigningRoot`: ==> `[]byte`

When importing attestations from EIP-3076 interchange format,
the signing root of an attestation may be missing.

Currently, Prysm consider any missing attestation signing root as
`0x000...`.
However, it may conflict with signing root which really are equal to
`0x000...`.

This commit transforms `AttestationRecord.SigningRoot` from `[32]byte` to
`[]byte`, and change the minimal set of functions (sic) to support this
new type.

* `CheckSlashableAttestation`: Empty signing root

Regarding slashing roots, 2 attestations are slashable, if:
- both signing roots are defined and differs, or
- one attestation exists, but without a signing root

* `filterSlashablePubKeysFromAttestations`: Err sort

Rergarding `CheckSlashableAttestation`, we consider that:
- If slashable == NotSlashable and err != nil, then CheckSlashableAttestation
failed.
- If slashable != NotSlashable, then err contains the reason why the attestation
is slashable.

* `setupEIP3076SpecTests`: Update to `v5.3.0`

This commit:
- Updates the version of EIP-3076 tests to `v.5.2.1`.
- Setups on anti-slashing DB per test case, instead per step.

* `ImportStandardProtectionJSON`: Reduce cycl cmplxt

* `AttestationHistoryForPubKey`: copy signing root

BoltDB documentation specifies:
| Byte slices returned from Bolt are only valid during a transaction.
| Once the transaction has been committed or rolled back then the memory
| they point to can be reused by a new page or can be unmapped
| from virtual memory and you'll see an unexpected fault address panic
| when accessing it.
2023-12-04 17:10:32 +00:00
Radosław Kapka
a0ca4a67b0
Remove API Middleware (#13243)
* remove api/gateway/apimiddleware

* fix errors in api/gateway

* remove beacon-chain/rpc/apimiddleware

* fix errors in api/client/beacon

* fix errors in validator/client/beacon-api

* fix errors in beacon-chain/node

* fix errors in validator/node

* fix errors in cmd/prysmctl/validator

* fix errors in testing/endtoend

* fix all other code

* remove comment

* fix tests

---------

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-12-04 11:55:21 +00:00
james-prysm
394bd1786a
HTTP validator API: beacon and account endpoints (#13191)
* fixing squashing changes, migrates beacon , account, and auth endpoints on validator client

* adding accounts endpoints

* fixing tests and query endpoints

* adding auth endpoint and fixing unit tests

* removing unused files and updating node file to skip gRPC

* ineffectual assignment fix

* rolling back a change to fix e2e

* fixing issues with ui

* updating with webui version 2.0.5

* updating package name flag in readme

* removing restore assets functions

* adding nomemcopy flag to see if vulenerability scan passes

* making data non compressed to avoid copy vulnerability

* Update beacon-chain/rpc/eth/shared/structs_validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* updating site_data, and skipping static analysis on file

* adding back deprecation comment notice

* updating workflows to ignore generated

* addressing radek comments

* missed a conversion

---------

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2023-12-01 20:40:09 +00:00
Radosław Kapka
5ecb4d62a9
REST VC: Use POST to fetch validators (#13239) 2023-11-29 18:53:26 +01:00
james-prysm
bc107a61e3
builder API: remove blinded blob sidecar (#13202) 2023-11-29 06:28:37 -08:00
Manu NALEPA
da2212f6cc
Allow validators registration batching on Builder API /eth/v1/builder/validators (#13178)
* builder `NewClient`: Simplify + fix some typos.

* Validator client: Implement `validator-registration-batch-size` option

* Address Potuz comments

* Address Potuz's comments

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2023-11-28 00:23:48 +00:00
terence
7cc05401ca
Update proposer RPC to new blob sidecar format (#13189) 2023-11-27 15:44:52 -08:00
Radosław Kapka
2fbda536b0
Fix handling POST requests in the REST VC (#13215)
* Fix handling POST requests in the REST VC

* tests for decodeResp
2023-11-25 23:25:14 +00:00
Radosław Kapka
a7edec9b98
Better error handling in REST VC (#13203) 2023-11-21 17:42:55 +01:00
james-prysm
4c381938e1
HTTP validator API: wallet endpoints (#13171)
* converting wallet calls to pure http

* fixing proto and gaz

* adding routes and fixing test

* fixing error handling

* fixing protos after conflict with develop

* adding deprecation notice

* fixing route test

* review feedback

* addressing more comments

* updating comment to be more clear

* fixing web_api proto
2023-11-15 19:40:14 +00:00
james-prysm
d4726f2866
HTTP Validator API: slashing protection import and export (#13165)
* adding migration for import and export slashing protection

* Update validator/rpc/handle_slashing.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/handle_slashing.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/handle_slashing.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Update validator/rpc/handle_slashing.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* addressing comments

* fixing unit test errors after view comments

---------

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2023-11-15 17:35:22 +00:00