Commit Graph

3183 Commits

Author SHA1 Message Date
terence tsao
6c8fd745a4
Better import alias names (#9149) 2021-07-06 09:37:33 -05:00
terence tsao
00907fd7d1
Update log test and ChainInfoFetcher interface (#9143) 2021-07-02 14:01:37 -05:00
terence tsao
1edf1f4c87
Restructure RPC packages to be future compatible (#9136)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-02 15:54:52 +00:00
Nishant Das
5d65ace970
Implement Doppelganger Check (#9120)
* checkpoint changes

* Update beacon-chain/rpc/validator/status.go

Co-authored-by: Potuz <potuz@potuz.net>

* Update beacon-chain/rpc/validator/status.go

Co-authored-by: Potuz <potuz@potuz.net>

* add in client side tests

* add ordering

* add all new test cases

* gate feature

* handle edge case

* add one more test case

* fatal error

* zahoor's review

* Update validator/client/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* doppelganger not doppleganger

* preston's review

* add in comment

* change comment

* Fix e2e to only run new flags on the current version

* Fix bug where zero byte public keys were always sent in the request when attestation history existed. Still some tests to fix due to another bug in attester protection AttestationHistoryForPubKey.

* go mod tidy, gazelle

* Increase test size

* fix timeout, change size back to small

Co-authored-by: Potuz <potuz@potuz.net>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-07-02 12:11:09 +08:00
Preston Van Loon
d6bd619429
Bazel: minimal test build transitions (#9122)
* Add build transitions for minimal config based tests

* respect existing gotags

* clean up import

* Add some commentary

* gaz

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2021-06-30 19:22:09 +00:00
Raul Jordan
b23f63a064
Beacon State V2 Interface Definition With Semantic Version Paths (#9125)
* v2 state initialize and semantic paths

* ensure build

* gaz changes to ignored build files

* gaz
2021-06-30 15:06:19 +00:00
terence tsao
4fb3e05124
Add search for most profitable sync contribution (#9121)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-29 17:26:01 +00:00
Radosław Kapka
0347b4b4af
Retrieve validators through v1 API in e2e (#9117)
* Retrieve validators through v1 API in e2e

* fix comment

* simplify boolean comparison
2021-06-29 09:47:59 -05:00
terence tsao
ea34af4abf
Add proposer sync contribution type (#9106)
* Add `proposerSyncContributions`

* Update BUILD.bazel

* Add dedup and tests

* Update BUILD.bazel

* Revert grpc changes
2021-06-28 13:54:33 -05:00
Radosław Kapka
940ce0c292
Make Eth APIs flag name implementation agnostic (#9112)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-28 15:12:39 +00:00
Radosław Kapka
71f1f19334
Small API Middleware improvements (#9111)
* extract creating DefaultErrorJson objects

* do not use default http client for proxying

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-28 11:36:12 +00:00
Raul Jordan
a860648960
Reduce Usage of Eth2 Terminology in Prysm (#9104)
* remove all mentions

* more changes

* folder by folder

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-26 19:00:33 +00:00
Radosław Kapka
9eb3ff6bdf
Fix a few bugs in the API (#9099) 2021-06-26 11:17:45 -07:00
Radosław Kapka
b114247836
Move common gateway registration code to new package (#9092)
* Move common gateway registration code to new package

* no shared inside of beaconchain

* beacon gateway

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-25 11:02:11 +00:00
Radosław Kapka
ff99a2ed40
Run text/event-stream values returned from grpc-gateway through API Middleware (#9080)
* register events handler in v1

* begin `status.Error` messages with uppercase

* something works

* handle errors

* topic comments

* handle the rest of event types

* gzl

* make code more testable + test

* better error handling

* tests

* gzl

* fix deps

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-24 16:18:28 +00:00
Raul Jordan
1e99ec30cb
Revert "Improve performance during CheckAttesterDoubleVotes" (#9085)
This reverts commit a958dd246b.
2021-06-23 21:19:22 +00:00
Nishant Das
9ee086feaf
Disable No Freelist Sync In Backups (#9069) 2021-06-21 15:14:55 +02:00
Nishant Das
6e42f1bc5d
add lock (#9068) 2021-06-21 16:33:06 +08:00
Radosław Kapka
4f3c17cf69
Make the shared/gateway package unaware of consuming services (#9050)
* gateway refactor

* test customizations

* change variable names

* fix BUILD file

* remove unused code

* move mux registration out of gateway

* fix gateway tests

* better name for struct

* fix mock endpoint factory

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 13:35:13 +00:00
terence tsao
a4be7c4a46
Testutil: Reset deposit cache internally (#9059)
* Reset testutil cache internally

* Fix import

* Fix test to correct error

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 04:05:40 +00:00
Alexander Mollohan
ae7e2764e0
Adding ability to override directory permissions for db backup (#8888)
* Adding ability to override directory permissions for db backup

* adding new flag to usage.go

* Backup now has its own directory handling method to reduce footprint of prospective change

* removing unneeded test

* fixing build error

* switching to url param for override

* stripping flag

* adding in unit tests

* fixing a test

* minor changes for testing

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-18 01:37:58 +00:00
Radosław Kapka
fbed11b380
Do not register API Middleware with validator gateway (#9055)
* Do not register API Middleware with validator gateway

* add nil checker

* revert bool

* nil check

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 00:38:15 +00:00
Mohamed Zahoor
e953804239
Process future Blocks and Attestations (#8975)
* queue future blocks

* process early attestations

* fix formatting

* fix testcases

* fix signature testcase and move verify slot time to original place

* Add testcases to test early blocks and slot processing

* added test case to cover failure case while inserting in block pending queue

* satisfy deepsource

* fix review comments

* remove invalid testcase

* do not queue blocks which are within MAXIMUM_GOSSIP_CLOCK_DISPARITY

* format fix

* added a helper to check of the block slot is within MAXIMUM_GOSSIP_CLOCK_DISPARITY

* add helper to check clock disparity and test case for the same

* deepsource fix

* satisfy deepsoruce

* sysc/service.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-17 00:17:52 +00:00
terence tsao
3c0a9455a2
Rename field ops service to att service (#9046)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-16 21:48:56 +00:00
Radosław Kapka
12403d249f
[Feature] - API Middleware (#8926)
* HTTP proxy server for Eth2 APIs (#8904)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* Run several new endpoints through API middleware (#8922)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* /eth/v1/beacon/states/{state_id}/validators

* /eth/v1/beacon/states/{state_id}/validators/{validator_id}

* /eth/v1/beacon/states/{state_id}/validator_balances

* /eth/v1/beacon/states/{state_id}/committees

* allow skipping base64-encoding for query params

* /eth/v1/beacon/pool/attestations

* replace break with continue

* Remove unused functions (#8924)

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd71e2c1e3a14be85abd29a59b57ae9b5, reversing
changes made to 2d0f8e012ecb006888ed8e826b45625a3edc2eeb.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd71e2c1e3a14be85abd29a59b57ae9b5, reversing
changes made to 2d0f8e012ecb006888ed8e826b45625a3edc2eeb.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* unused import

* Return correct status codes from beacon endpoints (#8960)

* Various API Middleware fixes (#8963)

* Return correct status codes from `/states` endpoints

* better error messages in debug and node

* better error messages in state

* returning correct error codes from validator endpoints

* correct error codes for getting a block header

* gzl

* fix err variable name

* fix nil block comparison

* test fixes

* make status enum test better

* fix ineffectual assignment

* make PR unstuck

* return proper status codes

* return uppercase keys from /config/spec

* return lowercase validator status

* convert requested enum values to uppercase

* validator fixes

* Implement `/beacon/headers` endpoint (#8966)

* Refactor API Middleware into more manageable code  (#8984)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* API Middleware unit tests (#8998)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* api_middleware_processing

* endpoints

* gzl

* remove gazelle:ignore

* merge

* Implement SSZ version of `/blocks/{block_id}` (#8970)

* Implement SSZ version of `/blocks/{block_id}`

* add dependencies back

* fix indentation in deps.bzl

* parameterize ssz functions

* get block ssz

* update ethereumapis dependency

* gzl

* Do not reuse `Endpoint` structs between API calls (#9007)

* code refactor

* implement endpoint factory

* fix test

* fmt

* include pbs

* gaz

* test naming fixes

* remove unused code

* radek comments

* revert endpoint test

* bring back bytes test case

* move `signedBeaconBlock` to `migration` package

* change `fmt.Errorf` to `errors.Wrap`

* capitalize SSZ

* capitalize URL

* more review feedback

* rename `handleGetBlockSSZ` to `handleGetBeaconBlockSSZ`

* rename `IndexOutOfRangeError` to `ValidatorIndexOutOfRangeError`

* simplify parameter names

* test header

* more corrections

* properly allocate array capacity

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-06-15 10:28:49 -05:00
kasey
19ff25f171
fixes 9029 + a few related client-stats fixes/improvements (#9039)
- fix missing scrape logic for these metrics:
      - sync_eth1_connected
      - sync_eth1_fallback_configured
      - sync_eth1_fallback_connected
    - update readme to add more detailed source info
      and indicate unsupported metrics.
    - fix documentation bug in prometheus help text
      for powchain_sync_eth1_connected

Co-authored-by: kasey <kasey@users.noreply.github.com>
2021-06-14 23:59:15 +00:00
Raul Jordan
4551bb47fc
Add Events Streaming Endpoint for Eth2 Standard API (#9001)
* begin event source support

* add in events server and proto definitions

* fmt

* add in stream support and proper building

* add in the protos

* tidy

* add structure and comments

* fmt

* revert proto changes

* only return events we explicitly request

* capture missing events needed for eth2 apis

* handle underflow

* no hash tree roots

* gazelle

* static

* race tests passing

* simplify

* handle all required events

* begin on stream tests

* correctness and abstracting method for forwarding to state feed

* genesis root

* include missing pieces

* logging

* rem and and handle return

* added mock

* add in mock

* add tests for block events

* added comprehensive tests

* radek comments

* add missing unit tests

* updatemockgen file

* viz

* done

* Update beacon-chain/rpc/eventsv1/events.go

* Update beacon-chain/rpc/eventsv1/events.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-14 16:58:08 +00:00
Radosław Kapka
77bf4bd9f0
Create ValidatorIndexOutOfRangeError (#9035) 2021-06-14 16:38:48 +00:00
terence tsao
dfd2cc751c
Refactor helpers for Altair (#9026)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-11 19:29:50 +00:00
Preston Van Loon
6981b16ecd
RPC: Surface underlying error when failing to return a state. (#9024) 2021-06-11 13:50:02 +00:00
Nishant Das
7a6ce27497
Change To LRU from FIFO Cache (#8996)
* change to lru

* fix tests

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-10 21:53:34 +00:00
Raul Jordan
6eb006175a
Capture ChainReorg, FinalizedCheckpoint, and NewHead Events in Beacon Node (#9011)
* capture missing events needed for eth2 apis

* handle underflow

* no hash tree roots

* gazelle

* static

* race tests passing

* simplify

* correctness and abstracting method for forwarding to state feed

* genesis root

* add in slotutil
2021-06-10 16:13:15 -05:00
terence tsao
8433a313f0
Enhance att sig set checks (#8978)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-03 15:30:46 +08:00
Raul Jordan
5aac06f04e
Move EthereumAPIs Into Prysm (#8968)
* begin move

* use same import path

* imports

* regen protos

* regen

* no rename

* generate ssz

* gaz

* fmt

* edit build file

* imports

* modify

* remove generated files

* remove protos

* edit imports in prysm

* beacon chain all builds

* edit script

* add generated pbs

* add replace rules

* license for ethereumapis protos

* change visibility

* fmt

* update build files to gaz ignore

* use proper form

* edit imports

* wrap block

* revert scripts

* revert go mod
2021-06-02 18:49:52 -05:00
Nishant Das
91bd477f4f
Add Metadata V2 Object and Interface (#8962)
* checkpoint changes

* gaz

* add error check

* gaz

* fix tests

* fix tests

* terence's review

* terence's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-02 13:44:34 +08:00
Nishant Das
8a6e2a33a2
Fix Interop Mode (#8961) 2021-06-02 02:45:10 +00:00
aliask
eb1d122aec
Feature/debian packages (#8872)
* Basic build targets for debian packages

* Add comments to config files

* Harden beacon-chain systemd service

* Add install scripts to set users and folders up

* Rename bazel target, fix service files

* Extra service hardening, cleanup install scripts

* Fix linting issues

* //shared:version_file run local only

* gazelle

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2021-06-01 15:44:24 +00:00
terence tsao
638e76cbd7
Sort imports (#8959) 2021-05-31 20:47:30 +02:00
Ivan Martinez
bb8625ede5
Remove stack trace from eth1 connection errors (#8957) 2021-05-28 21:25:54 +00:00
Ivan Martinez
40b4079924
Update deprecated function usages (#8953)
* Clear deprecated usages within Prysm

* Remove deprecatd usages where possible

* Update
2021-05-28 08:57:47 +02:00
Nishant Das
6bb3cc45ef
Add Nested Nil Checks To Wrapper (#8951)
* add checks

* fix
2021-05-27 16:07:51 +00:00
Ivan Martinez
e97f6cfdbf
ETH2 APIs: Handle invalid validator status as error (#8946)
* Handle invalid validator status as error

* Change text

* Fix review comments

* Remove nil status check

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-05-27 06:44:36 +00:00
terence tsao
276d03553c
Block interface clean up (#8947) 2021-05-26 11:33:46 -07:00
Nishant Das
caf9bdbc6f
Use Block Interface Across Prysm (#8918)
* commit initial work

* checkpoint current work

* gaz

* checkpoint

* req/resp changes

* initial-sync

* finally works

* fix error

* fix bugs

* fix issue

* fix issues

* fix refs

* tests

* more text fixes

* more text fixes

* more text fixes

* fix tests

* fix tests

* tests

* finally fix builds

* finally

* comments

* fix fuzz

* share common library

* fix

* fix

* add in more defensive nil checks

* add in more defensive nil checks

* imports

* Apply suggestions from code review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Apply suggestions from code review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_interface.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_wrapper.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_interface.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* imports

* fix bad changes

* fix

* terence's review

* terence's review

* fmt

* Update beacon-chain/rpc/beacon/blocks.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fix tests

* fix

* fix all tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-05-26 16:19:54 +00:00
Preston Van Loon
7b55213e6d
Refactor attestationDelta to reuse the sqrt value of active current epoch (#8945)
* Refactor attestationDelta to reuse the sqrt value of active current epoch

* use baseRewardsPerEpoch
2021-05-26 09:37:35 -05:00
Ivan Martinez
d5ad15ed80
ETH2 APIs: Implement Validator Status support (#8940)
* Update protos for new changes

* Begin implementing validator status

* Add support for validator status to state/validators

* Fix nil pointer
2021-05-26 00:24:29 -04:00
Zahoor Mohamed
a958dd246b
Improve performance during CheckAttesterDoubleVotes (#8927)
* de-normalize attestationDataRootsBucket to improve performace in CheckAttesterDoubleVotes

* parallize processing of every attestation

* fix double vote test case to take care of jumbled double votes due to parallel processing

* remove attestationRecordsBucket totally and take care of pruning

* missed out build file

* fix review comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-25 13:02:18 +05:30
Ivan Martinez
aa0fb058c6
ETH2 APIs: Update protos for new ethapis changes (#8920)
* Update protos for new changes

* Fix

* Fix deps

* Fix build

* Include newer changes to ethapis

* Fix

* Fix import

* Do not return unconnected peers
2021-05-24 20:02:01 -05:00
Nishant Das
503ec126e1
Fix Panic When Accessing State Frequently (#8928)
* fix issue

* dont allocate for existence checks

* radek's review
2021-05-24 06:52:33 +00:00
Nishant Das
76cdbaca22
Interface Analyzer (#8923) 2021-05-23 21:55:42 -07:00