prysm-pulse/shared/grpcutils/grpcutils.go
Radosław Kapka 12403d249f
[Feature] - API Middleware (#8926)
* HTTP proxy server for Eth2 APIs (#8904)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* Run several new endpoints through API middleware (#8922)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* /eth/v1/beacon/states/{state_id}/validators

* /eth/v1/beacon/states/{state_id}/validators/{validator_id}

* /eth/v1/beacon/states/{state_id}/validator_balances

* /eth/v1/beacon/states/{state_id}/committees

* allow skipping base64-encoding for query params

* /eth/v1/beacon/pool/attestations

* replace break with continue

* Remove unused functions (#8924)

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd71e2c1e3a14be85abd29a59b57ae9b5, reversing
changes made to 2d0f8e012ecb006888ed8e826b45625a3edc2eeb.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd71e2c1e3a14be85abd29a59b57ae9b5, reversing
changes made to 2d0f8e012ecb006888ed8e826b45625a3edc2eeb.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* unused import

* Return correct status codes from beacon endpoints (#8960)

* Various API Middleware fixes (#8963)

* Return correct status codes from `/states` endpoints

* better error messages in debug and node

* better error messages in state

* returning correct error codes from validator endpoints

* correct error codes for getting a block header

* gzl

* fix err variable name

* fix nil block comparison

* test fixes

* make status enum test better

* fix ineffectual assignment

* make PR unstuck

* return proper status codes

* return uppercase keys from /config/spec

* return lowercase validator status

* convert requested enum values to uppercase

* validator fixes

* Implement `/beacon/headers` endpoint (#8966)

* Refactor API Middleware into more manageable code  (#8984)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* API Middleware unit tests (#8998)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* api_middleware_processing

* endpoints

* gzl

* remove gazelle:ignore

* merge

* Implement SSZ version of `/blocks/{block_id}` (#8970)

* Implement SSZ version of `/blocks/{block_id}`

* add dependencies back

* fix indentation in deps.bzl

* parameterize ssz functions

* get block ssz

* update ethereumapis dependency

* gzl

* Do not reuse `Endpoint` structs between API calls (#9007)

* code refactor

* implement endpoint factory

* fix test

* fmt

* include pbs

* gaz

* test naming fixes

* remove unused code

* radek comments

* revert endpoint test

* bring back bytes test case

* move `signedBeaconBlock` to `migration` package

* change `fmt.Errorf` to `errors.Wrap`

* capitalize SSZ

* capitalize URL

* more review feedback

* rename `handleGetBlockSSZ` to `handleGetBeaconBlockSSZ`

* rename `IndexOutOfRangeError` to `ValidatorIndexOutOfRangeError`

* simplify parameter names

* test header

* more corrections

* properly allocate array capacity

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-06-15 10:28:49 -05:00

97 lines
2.6 KiB
Go

package grpcutils
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"github.com/sirupsen/logrus"
"google.golang.org/grpc"
"google.golang.org/grpc/metadata"
)
// LogRequests logs the gRPC backend as well as request duration when the log level is set to debug
// or higher.
func LogRequests(
ctx context.Context,
method string, req,
reply interface{},
cc *grpc.ClientConn,
invoker grpc.UnaryInvoker,
opts ...grpc.CallOption,
) error {
// Shortcut when debug logging is not enabled.
if logrus.GetLevel() < logrus.DebugLevel {
return invoker(ctx, method, req, reply, cc, opts...)
}
var header metadata.MD
opts = append(
opts,
grpc.Header(&header),
)
start := time.Now()
err := invoker(ctx, method, req, reply, cc, opts...)
logrus.WithField("backend", header["x-backend"]).
WithField("method", method).WithField("duration", time.Since(start)).
Debug("gRPC request finished.")
return err
}
// LogStream prints the method at DEBUG level at the start of the stream.
func LogStream(
ctx context.Context,
sd *grpc.StreamDesc,
conn *grpc.ClientConn,
method string,
streamer grpc.Streamer,
opts ...grpc.CallOption,
) (grpc.ClientStream, error) {
// Shortcut when debug logging is not enabled.
if logrus.GetLevel() < logrus.DebugLevel {
return streamer(ctx, sd, conn, method, opts...)
}
var header metadata.MD
opts = append(
opts,
grpc.Header(&header),
)
strm, err := streamer(ctx, sd, conn, method, opts...)
logrus.WithField("backend", header["x-backend"]).
WithField("method", method).
Debug("gRPC stream started.")
return strm, err
}
// AppendHeaders parses the provided GRPC headers
// and attaches them to the provided context.
func AppendHeaders(parent context.Context, headers []string) context.Context {
for _, h := range headers {
if h != "" {
keyValue := strings.Split(h, "=")
if len(keyValue) < 2 {
logrus.Warnf("Incorrect gRPC header flag format. Skipping %v", keyValue[0])
continue
}
parent = metadata.AppendToOutgoingContext(parent, keyValue[0], strings.Join(keyValue[1:], "="))
}
}
return parent
}
// AppendCustomErrorHeader sets a CustomErrorMetadataKey gRPC header on the passed in context,
// using the passed in error data as the header's value. The data is serialized as JSON.
func AppendCustomErrorHeader(ctx context.Context, errorData interface{}) error {
j, err := json.Marshal(errorData)
if err != nil {
return fmt.Errorf("could not marshal error data into JSON: %w", err)
}
if err := grpc.SetHeader(ctx, metadata.Pairs(CustomErrorMetadataKey, string(j))); err != nil {
return fmt.Errorf("could not set custom error header: %w", err)
}
return nil
}