Commit Graph

785 Commits

Author SHA1 Message Date
Raul Jordan
4c6e0c5f46
Update Prysm Web UI to Beta.3 (#8163)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-18 23:25:48 +00:00
terence tsao
ff64fdcfb5
Flag to enable duty count down (#8161)
* Add a flag to enable count down

* Add tests
2020-12-18 21:12:57 +00:00
Raul Jordan
72dc43989f
Stream Validator and Beacon Logs via gRPC Streams (#8150)
* implement validator logs stream

* fix test

* tidy

* proto regen

* add logs stream to the beacon node

* beacon logs working

* impl

* pass test

* gaz

* rem lock

* fix space
2020-12-18 18:03:24 +00:00
Raul Jordan
d19c57cdb6
Remove Logout from Validator RPC Authenticated Paths (#8151)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-17 19:46:27 +00:00
terence tsao
756ccbe5e4
Update Attestation schedule... log for clarify (#8148)
* Don't use fraction, use different fields

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Go fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-17 19:12:41 +00:00
Radosław Kapka
25b151ab78
Make TLS mandatory by default when unmarshalling remote wallet options (#8133)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2020-12-17 16:59:27 +00:00
Raul Jordan
f75b8a3be1
Implement Unified Validator API Endpoints for Beacon Chain Information (#8139)
* define all endpoints

* wrapper

* implement required validator endpoints

* begin impl

* implement remaining endpoint

* imports

* add in list validators

* intercepter

* test added
2020-12-17 16:26:32 +00:00
Radosław Kapka
d5bf8376c2
Clean up account logs and errors (#8142)
* small fixes in logs and errors

* fix error message in test

* found a period hiding in an error message

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-17 13:37:06 +00:00
Raul Jordan
d650034734
Lock Only When Needed When Fetching Attesting History (#8140)
* lock when needed in attesting history

* rw
2020-12-17 00:18:38 +00:00
Raul Jordan
148e7fcd59
Consolidate Required Web UI Endpoints Into Single Protobuf File (#8127)
* define required endpoints for unifying backends

* proto definitions

* impl

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-16 17:30:48 +00:00
Preston Van Loon
4326cbbf08
Validator database: use snappy compression on encoded attestation history (#8129)
* Remove old buckets, rename new buckets to be canonical

* Add migration and test

* Do the migrations

* Gazelle

* Clearify code, code review

* gofmt
2020-12-16 03:33:04 +00:00
Victor Farazdagi
f5c87075f2
Fix issue with roaming data dir on Wins (#8095)
* patch beacon node

* make sure that check is executed before anything else

* fix format

* make sure that check is reusable by other executables

* debug infor

* fix typo

* more debug info

* use copydir

* cleanup

* better explanation

* gazelle

* go fmt

* debug tos permissions

* upadte copydir

* gazelle

* better check of tos acceptance

* expand path

* update validator

* move fixing func

* move fixing method

* make sure that updater works both on main and subcommands

* remove from startnode

* add copydir test

* add DirFiles method and tests

* fix test

* add and test HashDir

* update tests

* fix test

* add datadir removal

* update messages

* further update messages
2020-12-15 14:01:51 -06:00
Preston Van Loon
ad7d3c74cc
Validator DB cleanup: remove obsolete buckets (#8122) 2020-12-15 18:24:56 +00:00
terence tsao
3e9d721280
Validator code health improvements (#8106) 2020-12-14 21:10:02 +00:00
Nishant Das
579335f81a
Remove Saving Of Target And Source Epoch (#8102)
* remove

* skip
2020-12-13 04:26:18 +00:00
Radosław Kapka
4ec396c025
Refetch validating keys if no keys are fetched (#8000)
* refetch validating keys every 30 seconds

* deduplicate error/log messages

* remove redundant break statement

* comment about execution flow

* move code to wait_for_activation.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-11 20:55:52 +01:00
Raul Jordan
1fbfd52e52
Simpler and Safer Attester Slashing Protection (#8086)
* att sign validations

* eliminate old cached methods and use a simple approach in the db

* redefined db methods

* db package builds

* add multilock to attest and propose

* gaz

* removed concurrency tests that are no longer relevant

* add cache to db functions for attesting history checks

* passing

* add in feature flag --disable-attesting-history-db-cache

* remove lock

* Revert "remove lock"

This reverts commit b1a65020e406b9fa6e4f57c5cc6a5a0b2a11a240.

* comment

* gaz
2020-12-11 12:31:35 -06:00
Mohamed Mansour
2e18df642d
Expand Validator Input Errors (#8090)
The reason of why a validator input was failing was very generic,
this allows the actual implementation to state why it failed by
passing the exception forward.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-11 10:57:47 -06:00
Preston Van Loon
923e4d3a5e
Attempt to reconnect when waiting for activation (#8057)
* Attempt to reconnect when waiting for activation

* rm fuzz/attestation_fuzz.go

* math.Min, not math.Max

* Gofmt

* resolve RVV-B0003
2020-12-10 19:26:31 -06:00
terence tsao
bb9e2ba12c
Validator client logs time left to next duty (#8088) 2020-12-10 10:56:18 -08:00
Preston Van Loon
f44b2a35e4
Update waiting validator methods to be context aware (#8078)
* Update waiting validator methods to be context aware

* Enable debug logging for validator in e2e

* invert logic in deferred function, remove for loop

* return early if wait is 0 or less

* overwrite ctx

* t.Stop() chan closure is hanging. Doesnt make sense, so ignoring that cleanup task. It shouldnt happen at runtime anyway

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-12-10 10:00:48 -06:00
terence tsao
00dacbd00d
Remove custom block body root and block root methods (#8069)
* Remove custom block body root and block root methods

* Add nil checks and fix tests

* Fmt

* Typo
2020-12-09 12:11:42 -06:00
Radosław Kapka
92736d0188
warn when creating an imported wallet (#8081) 2020-12-09 16:41:26 +01:00
Preston Van Loon
c96db1a122
Add spans to pre and post att signing updates (#8079)
* Add spans to pre and post att signing updates

* span on db method

* Add span to isNewAttSlashable
2020-12-08 23:47:02 -06:00
terence tsao
c5770a2e56
Add beacon block nil body checks (#8077)
* Add nil checks and fix tests

* Gaz

* Revert one minor typo

* Update test

* Remove extra space
2020-12-08 16:27:58 -06:00
Radosław Kapka
9d7052796b
Restore database CLI command (#8061)
* restore beacon node db

* revert image name

* move restore out of the kv folder

* remove files from kv folder

* go mod tidy

* Remove usage of prometheus testutil

* add yes/no to prompt text

* restore slasher db

* organize imports

* go mod tidy

* restore validator db

* close slasher db

* defer close backup db in tests

* simplify function literal
2020-12-07 21:36:43 +01:00
terence tsao
b4437e6cec
Load graffiti from file (#8041)
* Pass graffati file and   use it

* Visibility

* Parse test

* More proposal tests

* Gazelle

* Add sequential functionality

* fix length check

* Update priorities. Specified -> random -> default

* Log warn instead return err

* Comment

* E2e test

* Comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-04 23:15:12 +00:00
Arthur Burkart
8ad328d9b3
fix(grpcHeaders): accept values with "=" symbols (#8047)
* fix(grpcHeaders): accept values with equal signs

# What

Before this commit, it was not possible to pass in base64-encoded
content as a header value if it contained an equals sign. This commit
changes the behavior slightly. Rather than ignore key/value pairs where
the value happens to have an equals sign, we assume the first equals
sign delimits the key from the value and pass in the rest of the value
as-is.

Also, instead of printing the header name along with its value, we
print the name, so there is less risk of leaking information into logs
that shouldn't be there.

# Testing

This has not been tested in the context of the full validator client.
Instead, a small example was made to demonstrate the feasibility. The
example is shown here:

```go
package main

import (
	"log"
	"os"
	"strings"

	"github.com/davecgh/go-spew/spew"
	"github.com/urfave/cli/v2"
)

type Config struct {
	GrpcHeadersFlag string
}

func main() {
	app := &cli.App{
		Action: func(c *cli.Context) error {
			for _, hdr := range strings.Split(c.String("grpc-headers"), ",") {
				if hdr != "" {
					ss := strings.Split(hdr, "=")
					spew.Dump(ss[0])
					spew.Dump(strings.Join(ss[1:], "="))
				}
			}
			return nil
		},
		Flags: []cli.Flag{
			&cli.StringFlag{
				Name: "grpc-headers",
				Usage: "A comma-separated list of key value pairs to pass as gRPC headers for all gRPC " +
					"calls. Example: --grpc-headers=key=value",
			},
		},
	}

	err := app.Run(os.Args)
	if err != nil {
		log.Fatal(err)
	}
}
```

Example invocation:

```command
❯ go run main.go --grpc-headers=key=value,Authorization="Basic $(echo -n hello:world | base64)"
(string) (len=3) "key"
(string) (len=5) "value"
(string) (len=13) "Authorization"
(string) (len=22) "Basic aGVsbG86d29ybGQ="
```

* Adds tests to new gRPC header parsing code

* bazel run //:gazelle
2020-12-04 21:31:15 +00:00
terence tsao
3ce96701de
Update attestation schedule log with total attester count (#8013)
* Log attesting total

* Use the right library

* Go fmt

* Use fmt

* No space is better

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-04 07:02:58 +00:00
Raul Jordan
14e1f08208
Add Backup Webhooks to All Prysm Services With DBs (#8025)
* integrate backup webhooks

* pass slasher tests

* fix node

* cmd

* gaz

* read test passes

* radek feedback

* added comment

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-12-03 22:28:57 +00:00
terence tsao
c090c6a1c5
Revert "Logging with PadLevelText to true" (#8036)
This reverts commit c51754fa8a.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 20:38:00 +00:00
Shay Zluf
821620c520
Add test for local protection genesis attestation (#7977)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 01:02:51 +00:00
Radosław Kapka
323769bf1a
Make TLS connections to a remote wallet non-mandatory (#7953)
* disable-remote-signer-tls flag

* use flag in edit-config

* send requests without TLS

* change warning message

* fix account list output test

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 00:18:15 +00:00
terence tsao
c51754fa8a
Logging with PadLevelText to true (#8003)
* Use logrus formatter with pad

* Validator

* Fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-02 23:49:13 +00:00
terence tsao
2153a2d7c3
Remove logging deposit inclusion slot (#8023)
* Remove logging deposit inclusion slot

* Remove old tests

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-02 21:17:47 +00:00
Mohamed Mansour
9d2fe80140
Fix missing space in error message in account list (#8009)
When validating account list, there was a missing space. This fixes that nitpick.

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-12-02 16:02:59 +03:00
Raul Jordan
3bd5e58a5c Merge branch 'master' into develop 2020-12-01 08:06:16 -06:00
Raul Jordan
fc7c6776f6
Create Bucket If Not Exists When Fetching Proposal History (#8011)
* create bucket if not exist when fetching proposal history

* adding unit test
2020-12-01 08:00:03 -06:00
Raul Jordan
01bf97293f
Fix Miscellaneous Deep Source Issues (#8007)
* deep source fixes

* sh fixes

* fix import

* test err

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-11-30 19:55:30 -06:00
terence tsao
ea10784a4a
Validator logging: return early if no att included (#7979)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-30 23:28:12 +00:00
Raul Jordan
9d174d5927
Add Support for Prysm Web V1.0.0-beta.1 (#7983)
* new web release

* site data update, add /logs to logs endpoints

* tests fixed

* test

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-27 20:30:12 +00:00
terence tsao
1b1b36497f
Add validator bolt db metric collector (#7982)
* Validator db: add metrics collector

* Use the right library

* Go fmt

* Fix tests
2020-11-27 20:03:44 +00:00
Raul Jordan
04615cb97b
Add Validator RPC Endpoint to Retrieve Beacon + Validator Logs Websocket Endpoints (#7981)
* add logs endpoint

* commands to retrieve logs endpoints

* ensure works at runtime

* add auth
2020-11-27 18:28:45 +00:00
terence tsao
b243665d3e
Save lowest source and target epochs in post (#7973)
* Replace highest with lowerest

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Invert equality for saveLowestSourceTargetToDB

* Save lowest epcohs at post signature update

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-26 19:58:20 +00:00
terence tsao
c456dcdce8
Replace highest with lowest for signed epoch DB methods (#7965)
* Replace highest with lowerest

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/db/kv/attestation_history_v2.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Invert equality for saveLowestSourceTargetToDB

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-26 17:35:36 +00:00
Raul Jordan
10857223d0
Delete Dead Code in the Validator Client's Database Package (#7970)
* delete deprecated

* mod

* deep source

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-26 16:50:55 +00:00
Raul Jordan
af0977b8d0
Implement Export Block Proposals (#7964)
* export funcs for proposals

* add failing test

* round trip test passes

* progress

* deepsource

* Update validator/slashing-protection/local/standard-protection-format/export.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/slashing-protection/local/standard-protection-format/import.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* revert

* empty root

* deep source

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-26 04:09:35 +00:00
terence tsao
528272fc66
Add Getters/Setters in ValidatorDB For Highest Signed Source and Target Epochs (#7961)
* Add getters and setters for source and target epochs

* Use the correct ImportStandardProtectionJSON

* Add tests

* Fix buckets

* Fix source to target

* Update validator/slashing-protection/local/standard-protection-format/import.go

* Fix bytesutil.BytesToUint64BigEndian will return 0 if input is less than 8 bytes

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-26 02:39:23 +00:00
Raul Jordan
c5c868d0a6
Use Separate Buckets to Store Special Data for Proposal Slashing Protection (#7963)
* ignore keys in the schema

* add test

* use separate buckets for special values where pubkey is the index

* test fix
2020-11-25 23:58:01 +00:00
Raul Jordan
ae8a619775
Allow Optional Signing Roots in Proposal History (#7960) 2020-11-25 22:24:07 +00:00