Commit Graph

1240 Commits

Author SHA1 Message Date
Preston Van Loon
c0076cc7a2
State: move state interfaces into state package (#9268)
* Move interfaces from github.com/prysmaticlabs/prysm/beacon-chain/state/interface to github.com/prysmaticlabs/prysm/beacon-chain/state

* remove/rename state2
2021-07-23 16:11:21 +00:00
Raul Jordan
2d10bcf179
Move State Protos Into V2 Namespace (#9257)
* move state protos

* regen ssz

* edit v1 code

* fix imports

* building

* beacon chain builds

* validator and shared builds

* fuzz builds

* changes

* spectest builds

* tools build

* remove import cycle

* generate ssz

* pcli

* gaz

* kafka

* gaz
2021-07-23 08:07:40 -05:00
Raul Jordan
5864795ca5
Move RPC Protos Into V2 Namespace (#9254)
* moved rpc protos

* gazelle

* change pb

* validator health

* edit name

* naming
2021-07-22 21:00:28 +00:00
Raul Jordan
b7919b3115
Move DB Protos Into V2 Namespace (#9253)
* move db protos into prysm v2

* build

* build

* imports gaz

* v2

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-22 20:05:06 +00:00
Raul Jordan
715aa408e3
Move Web Protos Into Prysm V2 (#9249)
* move web protos

* move web to v2

* update web protos

* gaz

* proto build

* replace mentions of validator v2

* gaz
2021-07-22 19:19:24 +00:00
terence tsao
c8685d256c
Lint: add v1 as import alias (#9250)
* Add v1 as import alias

* Update helpers.go

* More

* Update skip_slot_cache_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-22 17:13:18 +00:00
Raul Jordan
6dadb80cc4
Move v1alpha1 into Prysm API namespace (#9245) 2021-07-21 21:34:07 +00:00
Nishant Das
2ffe8336fc
Fix Parsing of Nested Subcommands (#9236) 2021-07-21 13:01:33 +02:00
Radosław Kapka
53e02b1601
Log correct warning when disabling flags (#9230)
* Log correct warning when disabling flags

* use interface

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2021-07-20 16:54:08 +00:00
Nishant Das
40a96bc2b6
Invert Update Head Timely Flag (#9224)
* invert

* Update shared/featureconfig/flags.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-19 17:23:43 +00:00
Nishant Das
dad205bd04
Invert Optimized Balance Update Feature (#9225) 2021-07-19 14:27:20 +00:00
terence tsao
15704053e1
Add Altair configs (#9150)
* Add altair configs

* Update minimal_config.go

* Update workspace and config tests

* Revert workspace changes

* Update WORKSPACE

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-07-18 16:18:43 +00:00
terence tsao
5be1ccd3cf
Hydrate altair block (#9201)
* Hydrate altair blocks

* Update BUILD.bazel

* HTR test

* Update BUILD.bazel

* Comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-16 23:35:37 +00:00
Radosław Kapka
223d5309a0
Implement GetAttesterDuties in the validator API (#9207)
* remove irrelevant comment

* fix proto documentation

* first implementation

* register validator server

* first working version

* add sync checker and test file

* first test

* gzl

* final version + tests

* gzl

* run duties through API Middleware

* extract dependent root getter

* dependentRoot docs

* wrap indices in array

* fix static analysis issues

* modify err nil check

* create local variables in slot processing test

* Update proto/eth/v1/validator_service.proto

* review

* simplify index loops

* better calculation of committees at slot

* comment about impossible comndition

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-16 20:55:22 +00:00
Preston Van Loon
187a1ca53d
shared/copyutil: Add copy methods for Altair (#9208)
* Starting attestation test

* Commiting checkpoint progress before dinner

* Add remainder of tests

* Copy hf1 cloners and add tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-07-16 19:30:48 +00:00
Preston Van Loon
17d7fc492e
shared/copyutil: Add tests for copy methods (#9204)
* Starting attestation test

* Commiting checkpoint progress before dinner

* Add remainder of tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-07-16 16:56:09 +00:00
Preston Van Loon
44d266313e
testutil: Add NotEmpty method (#9202) 2021-07-15 22:32:10 +00:00
terence tsao
1a9a46d042
Block util: block header from block interface (#9196)
* Add beacon block header from block interface

* Update BUILD.bazel

* Update shared/blockutil/block_utils.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Go fmt

* Remove unused bodyroot

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-07-15 16:03:46 +00:00
Preston Van Loon
71c8c75bc3
shared/queue: Add RetrieveByKey method (#9195)
* Checkout shared/queue changes from hf1 branch

* Remove logic that removes and reinserts an item into the queue at the back

* Add test to ensure the correct thing was returned

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-07-15 10:31:19 -05:00
Nishant Das
e6bd5f31b1
Cleanup Fast Aggregate Verify (#9193) 2021-07-14 11:12:50 +00:00
terence tsao
7f0749c398
Clean up shared interface package (#9172)
* Remove shared interface folder

* Rename file names

* Gazelle

* Move interface to /beacon/p2p/

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-07-13 18:58:43 +00:00
Preston Van Loon
9dc3dd04c7
Update go-bitfield (#9162) 2021-07-08 08:31:40 -07:00
terence tsao
1d3a9983cc
Move block interface next to generated pb (#9146)
* Move block interface next to pb

* Update fuzz build bazel

* Move interface to /proto/interface and wrapper next to generated pb

* Fix fuzz build bazel

* Add //proto/eth/v1alpha1/wrapper

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-06 15:34:05 +00:00
terence tsao
1ff4605398
Add priority queue tests (#9142)
* Add priority tests

* Gazelle

* bazel

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-02 21:54:28 +00:00
terence tsao
2ae6452cb5
Add Eth2FastAggregateVerify (#9144)
* Add `Eth2FastAggregateVerify`

* Update propose_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-07-02 19:27:08 +00:00
terence tsao
877cb9f6e6
Add priority queue from hf1 (#9138)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-02 15:31:32 +00:00
Nishant Das
5d65ace970
Implement Doppelganger Check (#9120)
* checkpoint changes

* Update beacon-chain/rpc/validator/status.go

Co-authored-by: Potuz <potuz@potuz.net>

* Update beacon-chain/rpc/validator/status.go

Co-authored-by: Potuz <potuz@potuz.net>

* add in client side tests

* add ordering

* add all new test cases

* gate feature

* handle edge case

* add one more test case

* fatal error

* zahoor's review

* Update validator/client/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* doppelganger not doppleganger

* preston's review

* add in comment

* change comment

* Fix e2e to only run new flags on the current version

* Fix bug where zero byte public keys were always sent in the request when attestation history existed. Still some tests to fix due to another bug in attester protection AttestationHistoryForPubKey.

* go mod tidy, gazelle

* Increase test size

* fix timeout, change size back to small

Co-authored-by: Potuz <potuz@potuz.net>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-07-02 12:11:09 +08:00
Preston Van Loon
d6bd619429
Bazel: minimal test build transitions (#9122)
* Add build transitions for minimal config based tests

* respect existing gotags

* clean up import

* Add some commentary

* gaz

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2021-06-30 19:22:09 +00:00
Preston Van Loon
24d17a536c
Return an error on unrecognized arguments when running default commands in Prysm (#9129)
* Return an error on unrecognized arguments when running default commands in Prysm

* fix cases for subcommands

* Deepsource suggestion
2021-06-30 16:24:04 +00:00
Raul Jordan
b23f63a064
Beacon State V2 Interface Definition With Semantic Version Paths (#9125)
* v2 state initialize and semantic paths

* ensure build

* gaz changes to ignored build files

* gaz
2021-06-30 15:06:19 +00:00
terence tsao
cd3a2e87fe
Naive aggregation for SyncCommitteeContribution (#9114)
* Copy CopySyncCommitteeContribution

* Update BUILD.bazel

* Add naive aggregation for sync contribution

* Gazelle

* Update deps.bzl

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-06-29 16:11:11 +00:00
Radosław Kapka
43623e1089
Handle comma-separated query parameters in API requests (#9102)
* return empty array when no peers found

* add missing query params to endpoint factory

* fix validator endpoints in endpoint factory

* peers test

* Move API param handling to separate file

* Handle comma-separated query parameters
2021-06-28 15:37:14 +00:00
Radosław Kapka
71f1f19334
Small API Middleware improvements (#9111)
* extract creating DefaultErrorJson objects

* do not use default http client for proxying

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-28 11:36:12 +00:00
Nishant Das
f75548ad1e
Protect Against Unsafe Coefficients (#9109) 2021-06-28 11:53:03 +02:00
ahadda5
349f832bd3
No length check in AggregatePublicKeys (#9105)
* check of len and nil

* minor edit

* minor edit

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-06-27 16:13:55 +00:00
Raul Jordan
a860648960
Reduce Usage of Eth2 Terminology in Prysm (#9104)
* remove all mentions

* more changes

* folder by folder

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-26 19:00:33 +00:00
Radosław Kapka
dd0ae1bbef
Move API param handling to separate file (#9101)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-26 18:42:54 +00:00
Radosław Kapka
4f3c17cf69
Make the shared/gateway package unaware of consuming services (#9050)
* gateway refactor

* test customizations

* change variable names

* fix BUILD file

* remove unused code

* move mux registration out of gateway

* fix gateway tests

* better name for struct

* fix mock endpoint factory

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 13:35:13 +00:00
terence tsao
a4be7c4a46
Testutil: Reset deposit cache internally (#9059)
* Reset testutil cache internally

* Fix import

* Fix test to correct error

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 04:05:40 +00:00
Alexander Mollohan
ae7e2764e0
Adding ability to override directory permissions for db backup (#8888)
* Adding ability to override directory permissions for db backup

* adding new flag to usage.go

* Backup now has its own directory handling method to reduce footprint of prospective change

* removing unneeded test

* fixing build error

* switching to url param for override

* stripping flag

* adding in unit tests

* fixing a test

* minor changes for testing

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-18 01:37:58 +00:00
Radosław Kapka
fbed11b380
Do not register API Middleware with validator gateway (#9055)
* Do not register API Middleware with validator gateway

* add nil checker

* revert bool

* nil check

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 00:38:15 +00:00
Mohamed Zahoor
e953804239
Process future Blocks and Attestations (#8975)
* queue future blocks

* process early attestations

* fix formatting

* fix testcases

* fix signature testcase and move verify slot time to original place

* Add testcases to test early blocks and slot processing

* added test case to cover failure case while inserting in block pending queue

* satisfy deepsource

* fix review comments

* remove invalid testcase

* do not queue blocks which are within MAXIMUM_GOSSIP_CLOCK_DISPARITY

* format fix

* added a helper to check of the block slot is within MAXIMUM_GOSSIP_CLOCK_DISPARITY

* add helper to check clock disparity and test case for the same

* deepsource fix

* satisfy deepsoruce

* sysc/service.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-17 00:17:52 +00:00
Preston Van Loon
6971245e05
Revert "Invert attest-timely flag" (#9044)
This reverts commit 5fe98ab260.
2021-06-15 20:48:47 +00:00
Radosław Kapka
12403d249f
[Feature] - API Middleware (#8926)
* HTTP proxy server for Eth2 APIs (#8904)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* Run several new endpoints through API middleware (#8922)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* /eth/v1/beacon/states/{state_id}/validators

* /eth/v1/beacon/states/{state_id}/validators/{validator_id}

* /eth/v1/beacon/states/{state_id}/validator_balances

* /eth/v1/beacon/states/{state_id}/committees

* allow skipping base64-encoding for query params

* /eth/v1/beacon/pool/attestations

* replace break with continue

* Remove unused functions (#8924)

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd71e2c1e3a14be85abd29a59b57ae9b5, reversing
changes made to 2d0f8e012ecb006888ed8e826b45625a3edc2eeb.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd71e2c1e3a14be85abd29a59b57ae9b5, reversing
changes made to 2d0f8e012ecb006888ed8e826b45625a3edc2eeb.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* unused import

* Return correct status codes from beacon endpoints (#8960)

* Various API Middleware fixes (#8963)

* Return correct status codes from `/states` endpoints

* better error messages in debug and node

* better error messages in state

* returning correct error codes from validator endpoints

* correct error codes for getting a block header

* gzl

* fix err variable name

* fix nil block comparison

* test fixes

* make status enum test better

* fix ineffectual assignment

* make PR unstuck

* return proper status codes

* return uppercase keys from /config/spec

* return lowercase validator status

* convert requested enum values to uppercase

* validator fixes

* Implement `/beacon/headers` endpoint (#8966)

* Refactor API Middleware into more manageable code  (#8984)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* API Middleware unit tests (#8998)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* api_middleware_processing

* endpoints

* gzl

* remove gazelle:ignore

* merge

* Implement SSZ version of `/blocks/{block_id}` (#8970)

* Implement SSZ version of `/blocks/{block_id}`

* add dependencies back

* fix indentation in deps.bzl

* parameterize ssz functions

* get block ssz

* update ethereumapis dependency

* gzl

* Do not reuse `Endpoint` structs between API calls (#9007)

* code refactor

* implement endpoint factory

* fix test

* fmt

* include pbs

* gaz

* test naming fixes

* remove unused code

* radek comments

* revert endpoint test

* bring back bytes test case

* move `signedBeaconBlock` to `migration` package

* change `fmt.Errorf` to `errors.Wrap`

* capitalize SSZ

* capitalize URL

* more review feedback

* rename `handleGetBlockSSZ` to `handleGetBeaconBlockSSZ`

* rename `IndexOutOfRangeError` to `ValidatorIndexOutOfRangeError`

* simplify parameter names

* test header

* more corrections

* properly allocate array capacity

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-06-15 10:28:49 -05:00
kasey
19ff25f171
fixes 9029 + a few related client-stats fixes/improvements (#9039)
- fix missing scrape logic for these metrics:
      - sync_eth1_connected
      - sync_eth1_fallback_configured
      - sync_eth1_fallback_connected
    - update readme to add more detailed source info
      and indicate unsupported metrics.
    - fix documentation bug in prometheus help text
      for powchain_sync_eth1_connected

Co-authored-by: kasey <kasey@users.noreply.github.com>
2021-06-14 23:59:15 +00:00
Raul Jordan
4551bb47fc
Add Events Streaming Endpoint for Eth2 Standard API (#9001)
* begin event source support

* add in events server and proto definitions

* fmt

* add in stream support and proper building

* add in the protos

* tidy

* add structure and comments

* fmt

* revert proto changes

* only return events we explicitly request

* capture missing events needed for eth2 apis

* handle underflow

* no hash tree roots

* gazelle

* static

* race tests passing

* simplify

* handle all required events

* begin on stream tests

* correctness and abstracting method for forwarding to state feed

* genesis root

* include missing pieces

* logging

* rem and and handle return

* added mock

* add in mock

* add tests for block events

* added comprehensive tests

* radek comments

* add missing unit tests

* updatemockgen file

* viz

* done

* Update beacon-chain/rpc/eventsv1/events.go

* Update beacon-chain/rpc/eventsv1/events.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-14 16:58:08 +00:00
Nishant Das
fb08014146
Add Better Zero Key Checks (#9022)
* add better checks

* preston's review
2021-06-11 13:48:57 -05:00
Raul Jordan
6eb006175a
Capture ChainReorg, FinalizedCheckpoint, and NewHead Events in Beacon Node (#9011)
* capture missing events needed for eth2 apis

* handle underflow

* no hash tree roots

* gazelle

* static

* race tests passing

* simplify

* correctness and abstracting method for forwarding to state feed

* genesis root

* add in slotutil
2021-06-10 16:13:15 -05:00
Preston Van Loon
f2768c1414
Client-stats: Fail gracefully when metrics are missing (#8976)
* Client-stauts: Fail gracefully when metrics are missing

* gazelle
2021-06-07 20:37:14 +00:00
Preston Van Loon
d4aa4bc258
Remove unused ClosestPowerOf2 method (#8990) 2021-06-04 15:03:18 +00:00