Commit Graph

461 Commits

Author SHA1 Message Date
Radosław Kapka
1936f991eb
Prepare protos for Ethereum API v2.0.0 (#9384)
* Move proto services to a different package

# Conflicts:
#	beacon-chain/rpc/service.go

* fix tests

* goimports

* fix java class name

* without aggregation_bits

* aggregation_bits

* v2 endpoints

* generate new code and define function stubs

* build files

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-14 16:41:03 +00:00
Radosław Kapka
de907fc362
Fix java classname in beacon_debug_service.proto (#9380)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-13 17:59:55 +00:00
Radosław Kapka
f516e71167
Move proto services to a different package (#9379)
* Move proto services to a different package

# Conflicts:
#	beacon-chain/rpc/service.go

* fix tests

* goimports

* fix java class name
2021-08-13 10:55:24 +00:00
Radosław Kapka
ede85c21e2
Clean up v1 protos (#9377)
* Clean up v1 protos

# Conflicts:
#	proto/eth/v1/generated.ssz.go

* A little more cleanup

# Conflicts:
#	proto/eth/v1/generated.ssz.go

* fix java class names
2021-08-13 09:12:18 +00:00
terence tsao
d77616f705
Fixes Issues Found by Goland IDE Code Inspect (#9368)
* Ran code inspect

* Update shared/testutil/altair.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 20:12:22 +00:00
Raul Jordan
a217d71d08
Sync Protobufs Between Hard Fork Branch and Develop (#9362)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-11 19:14:49 +00:00
Radosław Kapka
456b4d463e
Implement PrepareBeaconCommitteeSubnet in the validator API (#9318)
* validator migration

* status helpers

* implementation

* fix existing tests

* build file

* tests

* run endpoint through API Middleware

* reduce helper package visibility

* empty caches before each test

* assign all validators to subnets

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-08-05 07:23:52 +00:00
Radosław Kapka
c5b68b4f03
Implement PublishAggregateAndProofs in the validator API (#9294)
* grpc implementation

* build file

* full implementation

* remove unused function

* tests

* wrap top-level array

* Update beacon-chain/rpc/eth/v1/validator/validator.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-08-04 09:01:38 +00:00
terence tsao
012d279663
Fix duplicated imports (#9304)
* Fix duplicated imports

* Fix metrics test
2021-07-29 16:45:17 -05:00
Raul Jordan
9145310647
Eliminate Proto V2 Namespace (#9297)
* get rid of v2 in prysm codebase

* replace block2

* builds

* terence comments

* gazelle
2021-07-28 21:23:44 +00:00
Raul Jordan
98f7f82b2f
Copy Altair Changes Into v1alpha1 Namespace (#9296)
* move the v2 changes into v1alpha1

* v1alpha1 builds

* regenerate pbs

* regenerate ssz

* top level builds
2021-07-28 18:05:07 +00:00
Raul Jordan
2d9915d2d9
Define Unimplemented V2 Services (#9286)
* generate ssz bindings for protos

* define services and unimplemented

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-27 20:42:14 +00:00
Raul Jordan
f700808c8b
Generate SSZ Bindings for Latest Protos (#9285) 2021-07-27 17:57:11 +00:00
Raul Jordan
1ea1c69174
Copy V1Alpha1 Protos and API Endpoints To V2 Namespace (#9284)
* v2 builds

* generate go code

* fix build
2021-07-27 16:54:49 +00:00
terence tsao
8db0c9a0e6
Move proto/interfaces -> proto/prysm/v2 (#9270)
* Move proto/interfaces -> proto/prysm

* Update paths

* Fix block2

* Update blocks_fetcher_utils.go

* Update BUILD.bazel

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-23 20:10:15 +00:00
Raul Jordan
30e93f5763
Move Slashing Protos to V2 (#9256)
* proto slashing

* move slashing related protos

* gaz

* include slasher pbs consolidated

* slasher pb fmt imports

* imports

* beacon builds

* build validator

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-23 18:01:01 +00:00
Radosław Kapka
a8363405f8
Implement ProduceBlock in the validator API (#9259)
* server-side implementation

* build file

* run through api middleware

* tests

* build file

* gofmt

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-07-23 16:29:38 +02:00
Raul Jordan
2d10bcf179
Move State Protos Into V2 Namespace (#9257)
* move state protos

* regen ssz

* edit v1 code

* fix imports

* building

* beacon chain builds

* validator and shared builds

* fuzz builds

* changes

* spectest builds

* tools build

* remove import cycle

* generate ssz

* pcli

* gaz

* kafka

* gaz
2021-07-23 08:07:40 -05:00
Raul Jordan
5864795ca5
Move RPC Protos Into V2 Namespace (#9254)
* moved rpc protos

* gazelle

* change pb

* validator health

* edit name

* naming
2021-07-22 21:00:28 +00:00
Raul Jordan
b7919b3115
Move DB Protos Into V2 Namespace (#9253)
* move db protos into prysm v2

* build

* build

* imports gaz

* v2

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-22 20:05:06 +00:00
Raul Jordan
715aa408e3
Move Web Protos Into Prysm V2 (#9249)
* move web protos

* move web to v2

* update web protos

* gaz

* proto build

* replace mentions of validator v2

* gaz
2021-07-22 19:19:24 +00:00
Raul Jordan
ceb1ec451a
Remove Faucet Protos (#9252)
* rem faucet protos

* tidy
2021-07-22 18:27:31 +00:00
Raul Jordan
0fb43aec34
Remove Unused Cluster Protos (#9251)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-22 16:39:27 +00:00
Raul Jordan
6dadb80cc4
Move v1alpha1 into Prysm API namespace (#9245) 2021-07-21 21:34:07 +00:00
terence tsao
e7bdf35721
Fix unused variables (#9234)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-21 16:06:57 +00:00
Preston Van Loon
3a35a953bf
proto/prysm/v2: Introduce Altair pb changes (#9222)
* Copy v2 protos from hf1

* Remove some extra Altair naming

* Build with github.com/ferranbt/fastssz/pull/50

* Regen pb.go

* Update fastssz to master
2021-07-19 20:42:06 +00:00
Preston Van Loon
15bfcf8ff6
Add tests for wrapped Altair objects (#9213)
* start tests for wrapped methods

* Add a few tests and skeletons for interface impl methods

* Add latest interfaces update from hf1 and stub the appropriate tests

* Add a few more tests

* More progress

* Migitate nil pointer issues in construct. Complete tests

* Copy changes from hf1 to v1alpha1 wrapper

* Use testutils from #9201

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-18 19:32:52 +00:00
terence tsao
97e6eb14e2
Altair block wrapper: update return to interfaces (#9212) 2021-07-16 23:09:45 +00:00
Radosław Kapka
223d5309a0
Implement GetAttesterDuties in the validator API (#9207)
* remove irrelevant comment

* fix proto documentation

* first implementation

* register validator server

* first working version

* add sync checker and test file

* first test

* gzl

* final version + tests

* gzl

* run duties through API Middleware

* extract dependent root getter

* dependentRoot docs

* wrap indices in array

* fix static analysis issues

* modify err nil check

* create local variables in slot processing test

* Update proto/eth/v1/validator_service.proto

* review

* simplify index loops

* better calculation of committees at slot

* comment about impossible comndition

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-16 20:55:22 +00:00
Preston Van Loon
31d2482f9c
Add proto wrappers for Altair (#9209)
* Starting attestation test

* Commiting checkpoint progress before dinner

* Add remainder of tests

* Copy hf1 cloners and add tests

* Introduce wrapper changes from hf1

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-16 20:30:26 +00:00
terence tsao
7f0749c398
Clean up shared interface package (#9172)
* Remove shared interface folder

* Rename file names

* Gazelle

* Move interface to /beacon/p2p/

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-07-13 18:58:43 +00:00
Radosław Kapka
4ebb008bc9
Protos and stubs for validator API endpoints (#9174)
* match proto definitions with spec

* stubs for grpc server's functions
2021-07-12 10:38:55 -05:00
terence tsao
1d3a9983cc
Move block interface next to generated pb (#9146)
* Move block interface next to pb

* Update fuzz build bazel

* Move interface to /proto/interface and wrapper next to generated pb

* Fix fuzz build bazel

* Add //proto/eth/v1alpha1/wrapper

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-06 15:34:05 +00:00
terence tsao
466546bb40
Add sync committee aggregation_bits tags for minimal and mainnet (#9147)
* Add aggregation_bits tags for minimal and mainnet

* Update go-bitfield

* Go mod tidy
2021-07-03 18:59:35 -05:00
Nishant Das
5d65ace970
Implement Doppelganger Check (#9120)
* checkpoint changes

* Update beacon-chain/rpc/validator/status.go

Co-authored-by: Potuz <potuz@potuz.net>

* Update beacon-chain/rpc/validator/status.go

Co-authored-by: Potuz <potuz@potuz.net>

* add in client side tests

* add ordering

* add all new test cases

* gate feature

* handle edge case

* add one more test case

* fatal error

* zahoor's review

* Update validator/client/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* doppelganger not doppleganger

* preston's review

* add in comment

* change comment

* Fix e2e to only run new flags on the current version

* Fix bug where zero byte public keys were always sent in the request when attestation history existed. Still some tests to fix due to another bug in attester protection AttestationHistoryForPubKey.

* go mod tidy, gazelle

* Increase test size

* fix timeout, change size back to small

Co-authored-by: Potuz <potuz@potuz.net>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-07-02 12:11:09 +08:00
Preston Van Loon
d6bd619429
Bazel: minimal test build transitions (#9122)
* Add build transitions for minimal config based tests

* respect existing gotags

* clean up import

* Add some commentary

* gaz

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2021-06-30 19:22:09 +00:00
Raul Jordan
b23f63a064
Beacon State V2 Interface Definition With Semantic Version Paths (#9125)
* v2 state initialize and semantic paths

* ensure build

* gaz changes to ignored build files

* gaz
2021-06-30 15:06:19 +00:00
terence tsao
442f8d1d3c
Remove old protos (#9107) 2021-06-28 07:41:37 +00:00
Raul Jordan
a860648960
Reduce Usage of Eth2 Terminology in Prysm (#9104)
* remove all mentions

* more changes

* folder by folder

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-26 19:00:33 +00:00
Raul Jordan
8f5c0833ff
Block Stream V2 Protos (#9096) 2021-06-25 00:05:38 +00:00
Raul Jordan
860716666b
Add Altair Protobuf Changes to Prysm (#9087)
* all altair protobuf additions

* add altair changes to deps
2021-06-24 10:32:30 -05:00
terence tsao
7d4418e2fe
Go imports (#9045)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-16 14:28:04 +00:00
Radosław Kapka
12403d249f
[Feature] - API Middleware (#8926)
* HTTP proxy server for Eth2 APIs (#8904)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* Run several new endpoints through API middleware (#8922)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* /eth/v1/beacon/states/{state_id}/validators

* /eth/v1/beacon/states/{state_id}/validators/{validator_id}

* /eth/v1/beacon/states/{state_id}/validator_balances

* /eth/v1/beacon/states/{state_id}/committees

* allow skipping base64-encoding for query params

* /eth/v1/beacon/pool/attestations

* replace break with continue

* Remove unused functions (#8924)

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd71e2c1e3a14be85abd29a59b57ae9b5, reversing
changes made to 2d0f8e012ecb006888ed8e826b45625a3edc2eeb.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd71e2c1e3a14be85abd29a59b57ae9b5, reversing
changes made to 2d0f8e012ecb006888ed8e826b45625a3edc2eeb.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* unused import

* Return correct status codes from beacon endpoints (#8960)

* Various API Middleware fixes (#8963)

* Return correct status codes from `/states` endpoints

* better error messages in debug and node

* better error messages in state

* returning correct error codes from validator endpoints

* correct error codes for getting a block header

* gzl

* fix err variable name

* fix nil block comparison

* test fixes

* make status enum test better

* fix ineffectual assignment

* make PR unstuck

* return proper status codes

* return uppercase keys from /config/spec

* return lowercase validator status

* convert requested enum values to uppercase

* validator fixes

* Implement `/beacon/headers` endpoint (#8966)

* Refactor API Middleware into more manageable code  (#8984)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* API Middleware unit tests (#8998)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* api_middleware_processing

* endpoints

* gzl

* remove gazelle:ignore

* merge

* Implement SSZ version of `/blocks/{block_id}` (#8970)

* Implement SSZ version of `/blocks/{block_id}`

* add dependencies back

* fix indentation in deps.bzl

* parameterize ssz functions

* get block ssz

* update ethereumapis dependency

* gzl

* Do not reuse `Endpoint` structs between API calls (#9007)

* code refactor

* implement endpoint factory

* fix test

* fmt

* include pbs

* gaz

* test naming fixes

* remove unused code

* radek comments

* revert endpoint test

* bring back bytes test case

* move `signedBeaconBlock` to `migration` package

* change `fmt.Errorf` to `errors.Wrap`

* capitalize SSZ

* capitalize URL

* more review feedback

* rename `handleGetBlockSSZ` to `handleGetBeaconBlockSSZ`

* rename `IndexOutOfRangeError` to `ValidatorIndexOutOfRangeError`

* simplify parameter names

* test header

* more corrections

* properly allocate array capacity

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-06-15 10:28:49 -05:00
Raul Jordan
4551bb47fc
Add Events Streaming Endpoint for Eth2 Standard API (#9001)
* begin event source support

* add in events server and proto definitions

* fmt

* add in stream support and proper building

* add in the protos

* tidy

* add structure and comments

* fmt

* revert proto changes

* only return events we explicitly request

* capture missing events needed for eth2 apis

* handle underflow

* no hash tree roots

* gazelle

* static

* race tests passing

* simplify

* handle all required events

* begin on stream tests

* correctness and abstracting method for forwarding to state feed

* genesis root

* include missing pieces

* logging

* rem and and handle return

* added mock

* add in mock

* add tests for block events

* added comprehensive tests

* radek comments

* add missing unit tests

* updatemockgen file

* viz

* done

* Update beacon-chain/rpc/eventsv1/events.go

* Update beacon-chain/rpc/eventsv1/events.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-14 16:58:08 +00:00
Preston Van Loon
8f90e91e99
Update jwt-go library to fix CVE-2020-26160 (#9017)
* Replace github.com/dgrijalva/jwt-go with github.com/form3tech-oss/jwt-go, including security fix

* Fix to include go cast in deps.bzl and go mod

* Add gocast stub for go.mod / deps.bzl trick

* revert some changes

* Remove ignore tag

* gaz

* go mod tidy

* add build exclusion

* Go mod?

* go mod tidy
2021-06-10 13:59:34 -05:00
Raul Jordan
bb6f00fd87
New Proto Messages Required for Event Streams (#9010)
* add event messages

* define all required event messages

* add new proto event messages

* formatting
2021-06-09 16:49:03 +00:00
Raul Jordan
a81c4ed6a0
Protobuf Changes for Eth2 APIs EventSource Support (#8999)
* protobuf changes for event source support

* deps update

* tidy

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-07 19:29:36 +00:00
terence tsao
aa4c21d6df
Run go imports (#8995) 2021-06-06 07:39:16 +02:00
Raul Jordan
5aac06f04e
Move EthereumAPIs Into Prysm (#8968)
* begin move

* use same import path

* imports

* regen protos

* regen

* no rename

* generate ssz

* gaz

* fmt

* edit build file

* imports

* modify

* remove generated files

* remove protos

* edit imports in prysm

* beacon chain all builds

* edit script

* add generated pbs

* add replace rules

* license for ethereumapis protos

* change visibility

* fmt

* update build files to gaz ignore

* use proper form

* edit imports

* wrap block

* revert scripts

* revert go mod
2021-06-02 18:49:52 -05:00
Raul Jordan
1d2fc60ba7
Checkin Generated Protobufs and Proto Files for EthereumAPIs (#8973)
* generated

* add ignore

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-02 20:20:11 +00:00