Commit Graph

105 Commits

Author SHA1 Message Date
terencechain
0b50ab7832
Consolidate ExecutionPayloadHeader Protobuf definitions (#10917) 2022-06-22 13:08:06 +02:00
terencechain
7e64a3963d
Change submit validator registration to registration(s) (#10907)
* Change submit validator registration to registration(s)

* Fixed a few tests

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-21 18:46:28 +00:00
terencechain
546bb5ed53
Clean up misc warnings (#10900)
* Clean up misc warnings

* Gazelle

* Rm state assignments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-21 15:26:56 +00:00
Håvard Anda Estensen
b55ddb5a34
Use go:build lines and remove obsolete +build lines (#10704)
* Use go:build lines and remove obsolete +build lines

* Run gazelle

* Update crypto/bls/blst/stub.go

* Update crypto/bls/blst/stub.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-06-14 11:47:27 +00:00
terencechain
7fcadbe3ef
Add optimistic status to chainhead (#10842)
* Add optimistic status to chainhead

* Fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-09 17:37:52 +00:00
james-prysm
db687bf56d
Validator client builder support (#10749)
* Startinb builder service and interface

* Get header from builder

* Add get builder block

* Single validator registration

* Add mev-builder http cli flag

* Add method to verify registration signature

* Add builder registration

* Add submit validator registration

* suporting yaml

* fix yaml unmarshaling

* rolling back some changes from unmarshal from file

* adding yaml support

* adding register validator support

* added new validator requests into client/validator

* fixing gofmt

* updating flags and including gas limit, unit tests are still broken

* fixing bazel

* more name changes and fixing unit tests

* fixing unit tests and renaming functions

* fixing unit tests and renaming to match changes

* adding new test for yaml

* fixing bazel linter

* reverting change on validator service proto

* adding clarifying logs

* renaming function name to be more descriptive

* renaming variable

* rolling back some files that will be added from the builder-1 branch

* reverting more

* more reverting

* need placeholder

* need placeholder

* fixing unit test

* fixing unit test

* fixing unit test

* fixing unit test

* fixing more unit tests

* fixing more unit tests

* rolling back mockgen

* fixing bazel

* rolling back changes

* removing duplicate function

* fixing client mock

* removing unused type

* fixing missing brace

* fixing bad field name

* fixing bazel

* updating naming

* fixing bazel

* fixing unit test

* fixing bazel linting

* unhandled err

* fixing gofmt

* simplifying name based on feedback

* using corrected function

* moving default fee recipient and gaslimit to beaconconfig

* missing a few constant changes

* fixing bazel

* fixing more missed default renames

* fixing more constants in tests

* fixing bazel

* adding update proposer setting per epoch

* refactoring to reduce complexity

* adding unit test for proposer settings

* Update validator/client/validator.go

Co-authored-by: terencechain <terence@prysmaticlabs.com>

* trying out renaming based on feedback

* adjusting based on review comments

* making tests more appropriate

* fixing bazel

* updating flag description based on review feedback

* addressing review feedback

* switching to pushing at start of epoch for more time

* adding new unit test and properly throwing error

* switching keys in error to count

* fixing log variable

* resolving conflict

* resolving more conflicts

* adjusting error message

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-06-06 19:32:41 +00:00
terencechain
3e36eacd1e
Add registration DB methods (#10812) 2022-06-03 15:13:04 +00:00
terencechain
95c140b512
Validator client: add submit registration (#10785)
* Add client registration methods

* Mocken

* Run mockgen

* Fix bad tests

* Fix rest of the tests

* Tests

* Fix time tests

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-06-01 01:53:25 +00:00
terencechain
61cbe3709b
Ignore subset aggregates (#10674)
* Ignore subset aggregates

* Add test

* Update BUILD.bazel

* Update validate_sync_contribution_proof_test.go

* Update validate_sync_contribution_proof_test.go

* Don't utilize pooled objects. Use direct caches

* Handle mainnet/minimal better

* Revert att changes

* Check overlaps before set

* Feedbacks

* Fixed a copy bug

* Fixed the same copy bug in committee indices cache

* Use SafeCopyBytes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-25 05:40:06 +00:00
kasey
c66d9e9a11
Builder client (#10703)
* builder api client

* unexport error

* thanks, DeepSource!

* replace hexSlice w/ hexutil.Bytes

* use uint256 for BaseFeePerGas

* more confidence in correct endianness

* comment fix per Terence

* fix proto conversion for uint256

* couple more value checks in the http client tests

* TestMarshalBlindedBeaconBlockBodyBellatrix

* appease deepsource

* middleware to log requests

* big int round trip test

* very superficial test to make deepsource happy

* round trip test between proto payloads

* round trip starting from marshaled struct

* deepsource... for you, the moon

* remove unused receiver

* gofmt

* remove test destroying line added while debugging

* handle nil body in logging middleware

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-23 17:30:51 +00:00
Raul Jordan
16bbf5602f
Move Consensus Type Wrappers Into Consensus Types Package (#10598)
* builds

* move block to consensus-types

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-05-02 15:43:40 +00:00
Raul Jordan
84916672c6
Remove Eth2-Types Dependency in Prysm (#10578)
* replace eth2 types

* replace protos

* regen proto

* replace

* gaz

* deps

* amend

* regen proto

* mod

* gaz

* gaz

* ensure build

* ssz

* add dep

* no more eth2 types

* no more eth2

* remg

* all builds

* buidl

* tidy

* clean

* fmt

* val serv

* gaz

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-04-29 10:32:11 -04:00
Håvard Anda Estensen
d2f4a8cc7c
Replace ioutil with io and os (#10541)
* Replace ioutil with io and os

* Fix build errors
2022-04-18 20:42:07 +00:00
terence tsao
3ce26ae7e2
Add process payload header (#10529)
* Add process payload header

* Update block.go

* Remove unused receivers

* Use errors vars

* Add some comments

* Add some comments
2022-04-17 07:02:59 +00:00
terence tsao
1acedd5b01
Rename receipt root to receipts root (#10535) 2022-04-16 13:18:07 +00:00
Radosław Kapka
36e4f49af0
Bellatrix evaluators (#10506)
* defensive nil check

* separate ExecutionPayload/Header from codegen

* tell bazel about this new file

* Merge: support terminal difficulty override (#9769)

* Fix finding terminal block hash calculation

* Update mainnet_config.go

* Update beacon_block.pb.go

* Various fixes to pass all spec tests for Merge (#9777)

* Proper upgrade altair to merge state

* Use uint64 for ttd

* Correctly upgrade to merge state + object mapping fixes

* Use proper receive block path for initial syncing

* Disable contract lookback

* Disable deposit contract lookback

* Go fmt

* Merge: switch from go bindings to raw rpc calls (#9803)

* Disable genesis ETH1.0 chain header logging

* Update htrutils.go

* all gossip tests passing

* Remove gas validations

* Update penalty params for Merge

* Fix gossip and tx size limits for the merge part 1

* Remove extraneous p2p condition

* Add and use

* Add and use TBH_ACTIVATION_EPOCH

* Update WORKSPACE

* Update Kintsugi engine API (#9865)

* Kintsugi ssz (#9867)

* All spec tests pass

* Update spec test shas

* Update Kintsugi consensus implementations (#9872)

* Remove secp256k1

* Remove unused merge genesis state gen tool

* Manually override nil transaction field. M2 works

* Fix bad hex conversion

* Change Gossip message size and Chunk SIze from 1 MB t0 10MB (#9860)

* change gossip size and chunk size after merge

* change ssz to accomodate both changes

* gofmt config file

* add testcase for merge MsgId

* Update beacon-chain/p2p/message_id.go

Change MB to Mib in comment

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* change function name from altairMsgID to postAltairMsgID

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Sync with develop

* Merge branch 'develop' of github.com:prysmaticlabs/prysm into kintsugi

* Update state_trie.go

* Clean up conflicts

* Fix build

* Update config to devnet1

* Fix state merge

* Handle merge test case for update balance

* Fix build

* State pkg cleanup

* Fix a bug with loading mainnet state

* Fix transactions root

* Add v2 endpoint for merge blocks (#9802)

* Add V2 blocks endpoint for merge blocks

* Update beacon-chain/rpc/apimiddleware/structs.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* go mod

* fix transactions

* Terence's comments

* add missing file

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* Sync

* Go mod tidy

* change EP field names

* latest kintusgi execution api

* fix conflicts

* converting base fee to big endian format (#10018)

* ReverseByteOrder function does not mess the input

* sync with develop

* use merge gossip sizes

* correct gossip sizes this time

* visibility

* clean ups

* Sync with develop, fix payload nil check bug

* Speed up syncing, hide cosmetic errors

* Sync with develop

* Clean up after sync

* Update generate_keys.go

* sync with develop

* Update mainnet_config.go

* Clean ups

* Sync optimistically candidate blocks (#10193)

* Revert "Sync optimistically candidate blocks (#10193)"

This reverts commit f99a0419ef1efc4bda0be77f22165b8fe4377c0d.

* Sync optimistically candidate blocks (#10193)

* allow optimistic sync

* Fix merge transition block validation

* Update proposer.go

* Sync with develop

* delete deprecated client, update testnet flag

* Change optimistic logic (#10194)

* Logs and err handling

* Fix build

* Clean ups

* Add back get payload

* c

* Done

* Rm uncommented

* Optimistic sync: prysm validator rpcs (#10200)

* Logs to reproduce

* Use pointers

* Use pointers

* Use pointers

* Update json_marshal_unmarshal.go

* Fix marshal

* Update json_marshal_unmarshal.go

* Log

* string total diff

* str

* marshal un

* set string

* json

* gaz

* Comment out optimistic status

* remove kiln flag here (#10269)

* Sync with devleop

* Sync with develop

* clean ups

* refactor engine calls

* Update process_block.go

* Fix deadlock, uncomment duty opt sync

* Update proposer_execution_payload.go

* Sync with develop

* Rm post state check

* Bypass eth1 data checks

* Update proposer_execution_payload.go

* Return early if ttd is not reached

* Sync with devleop

* Update process_block.go

* Update receive_block.go

* Update bzl

* Revert "Update receive_block.go"

This reverts commit 5b4a87c512325d9f26de4db7f941dcd7303cd0d8.

* Fix run time

* add in all the fixes

* fix evaluator bugs

* latest fixes

* sum

* fix to be configurable

* Update go.mod

* Fix AltairCompatible to account for future state version

* Update proposer_execution_payload.go

* fix broken conditional checks

* fix all issues

* Handle pre state Altair with valid payload

* Handle pre state Altair with valid payload

* Log bellatrix fields

* Update log.go

* Revert "fix broken conditional checks"

This reverts commit e118db6c20172783a3bb573d2f78670f0207ec1e.

* LH multiclient working

* Friendly fee recipient log

* Remove extra SetOptimisticToValid

* fix race

* fix test

* Fix base fee per gas

* Fix notifypayload headroot

* tx fuzzer

* clean up with develop branch

* save progress

* 200tx/block

* add LH flags

* Sync with devleop

* cleanup

* cleanup

* hash

* fix build

* fix test

* fix go check

* fmt

* gosec

* Blocked stream

(cherry picked from commit f362af9862db680b6352692217ad5c08d44a1e86)

# Conflicts:
#	proto/prysm/v1alpha1/validator.pb.go

* remove duplicate param

* test

* revert some test changes

* Initial version of EE tx count

* evaluate all txs in epoch

* remove logs

* uncomment tests

* remove unwanted change

* parameterize ExpectedExecEngineTxsThreshold

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Zahoor Mohamed <zahoor@zahoor.in>
Co-authored-by: kasey <489222+kasey@users.noreply.github.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
Co-authored-by: Zahoor Mohamed <zahoor@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-04-11 13:45:22 +00:00
terence tsao
f44c99d92a
Add blinded beacon block protobufs and wrappers (#10473)
Co-authored-by: rkapka <rkapka@wp.pl>
2022-04-05 14:57:19 +02:00
terence tsao
7279349ae2
Cleanup helpers in beacon-chain/core/blocks/payload.go (#10435)
* less fragile check for execution payload

* gaz

* gaz

* include field

* clean up helpers for checking merge status

* Update beacon-chain/core/blocks/payload.go

Co-authored-by: Potuz <potuz@prysmaticlabs.com>

* Update payload.go

* Rename

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Potuz <potuz@prysmaticlabs.com>
2022-03-28 15:25:49 +00:00
Preston Van Loon
b697463da9
wrapper: Remove deprecated fork specific wrapper methods for signed beacon blocks (#10369)
* Remove deprecated wrappers, lots of refactoring

* Revert proto/prysm/v1alpha1/validator.proto

* fix tests

* fix test

* fix conversion in e2e

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-25 23:00:44 +00:00
kasey
49826ebe28
Checkpoint Sync 2/5 - API support for retrieving weak subjectivity data (#10384)
also refactor replayer code for better reuse:
- separately expose stategen's canonical block func
- CanonicalHistory in ws api, NewCanonicalBuilder
- refactor CanonicalHistory into a ReplayerBuilder

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-23 22:54:07 +00:00
james-prysm
df8da80db8
Implement Fee Recipient: CLI flags and Calling Beacon API (#10312)
* send proposer data to config

* wip implementation with file and url based config import

* improving logic on get validator index

* fix function

* optimizing function for map and address bug

* fixing log

* update cache if it doesn't exist

* updating flags

* initial unit test scaffold

* fixing validator to call rpc call, removed temporary dependency

* adding the API calls for the runner

* fixing broken build

* fixing deepsource

* fixing interface

* fixing fatal

* fixing more deepsource issues

* adding test placeholders

* updating proposer config to add validation

* changing how if statement throws error

* removing unneeded validation, validating in a different way

* wip improving tests

* more unit test work

* fixing unit test

* fixing unit tests and edge cases

* adding unit tests and adjusting how the config is created

* fixing bazel builds

* fixing proto generation

* fixing imports

* fixing unit tests

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* updating flags based on comments, fixing unit tests

* fixing bazel

* removing unneeded function

* fixing unit tests

* refactors and unit test fixes based on comments

* fixing bazel build

* refactor the cache out fo the fee recipient function

* adding usecase for multiple fee recipient

* refactor burn name

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* fixing bug with validator index based on code review

* edited flag descriptions to better communicate usage

* fixing manual reference to flag name

* fixing code review comments

* fixing linting

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing comments and renaming functions

* fixing linting

* Update cmd/validator/flags/flags.go

* Update cmd/validator/flags/flags.go

* Update cmd/validator/flags/flags.go

* improving comments

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-03-21 13:48:02 -05:00
kasey
0b1a777d62
ran gofmt @ v1.17, these are the resulting changes (#10362)
* ran gofmt @ v1.17, these are the resulting changes

* fix some flaky tests that are tripping up this PR

* gofmt

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
2022-03-14 15:58:13 -05:00
Preston Van Loon
c1197d7881
Add static analysis for unsafe uint casting (#10318)
* Add static analysis for unsafe uint casting

* Fix violations of uintcast

* go mod tidy

* Add exclusion to nogo for darwin build

* Add test for math.Int

* Move some things to const so they are assured not to exceed int64

* Self review

* lint

* fix tests

* fix test

* Add init check for non 64 bit OS

* Move new deps from WORKSPACE to deps.bzl

* fix bazel build for go analysis runs

* Update BUILD.bazel

Remove TODO

* add math.AddInt method

* Add new test casts

* Add case where builtin functions and declared functions are covered

* Fix new findings

* cleanup

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-03-11 09:34:30 +00:00
Raul Jordan
f6eed74500
PrepareBeaconProposer Protobuf Schema (#10332)
* prepare v1 protos

* add in protos

* prepare beacon proposer

* stubs

* builds

* gaz

* build

* rem

* ssz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-09 21:40:26 +00:00
kasey
17a43c1158
Retrieving state by slot to always apply canonical block at slot, when available (#10255)
* new stategen.StateReplayer/ReplayerBuilder to give more fine-grained
  control of replaying state+block history
* all rpc/api methods updated to use the new interface, return post-state

Co-authored-by: Kasey Kirkham <kasey@users.noreply.github.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-09 13:33:18 -06:00
Radosław Kapka
77b8b13eff
Bellatrix API support for block endpoints (#10324)
* refactor GetBlockV2

* Add bellatrix to GetBlockSSZV2

* Add bellatrix to ListBlockAttestations

* Add bellatrix to SubmitBlock

* gzl

* return error from SubmitBlock

* return nil

* Better code flow when getting blocks

* remove tautology

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-09 17:18:52 +00:00
Potuz
57a323f083
Forkchoice featureflag (#10299)
* Compiling main beacon-chain binary

* Add feature flag

* passing protoarray tests

* passing nodetree tests

* passing blockchain package tests

* passing rpc tests

* go fmt

* re-export forkchoice store from blockchain package

* remove duplicated import

* remove unused var

* add nodetree rpc method

* remove slot from IsOptimisticForRoot

* release lock in IsOptimistic

* change package name

* Revert "change package name"

This reverts commit 679112f9ef795922c631e7823dbdfb3746838f3c.

* rename package

* Update doc

* Fix span names

* Terence + Raul review

* remove go:build flags

* add errors dep

* spec tests

* fix call to IsOptimisticForRoot

* fix test

* Fix conflict

* change name of function

* remove ctx from store.head

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2022-03-09 03:05:51 +00:00
Preston Van Loon
78fe712e53
Validator: deduplicate fork logic for block proposals (#10297)
* Deduplicate block proposals

* fix tests, more dedup of tests

* Add godoc to BuildSignedBeaconBlock.

* Rename SignRequest_Object => SignRequestObject

* Fix error messages

* Add tests for new methods

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-03-07 19:18:54 +00:00
Nishant Das
339540274b
Integration of Vectorized Sha256 In Prysm (#10166)
* add changes

* fix for vectorize

* fix bug

* add new bench

* use new algorithms

* add latest updates

* save progress

* hack even more

* add more changes

* change library

* go mod

* fix deps

* fix dumb bug

* add flag and remove redundant code

* clean up better

* remove those ones

* clean up benches

* clean up benches

* cleanup

* gaz

* revert change

* potuz's review

* potuz's review

* potuz's review

* gaz

* potuz's review

* remove cyclical import

* revert ide changes

* potuz's review

* return
2022-02-28 21:56:12 +08:00
Raul Jordan
12ba8f3645
Renaming Random in ExecutionPayloads to PrevRandao (#10283)
* rename proto

* p header

* regen

* regen ssz

* fix randao

* random name changes

* bazel builds

* bt

* incorrect prev randao

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-26 03:47:16 +00:00
Preston Van Loon
7f41b69281
Wrapper: Update block interface and reorganize fork logic (#10267)
* Add ssz.HashRoot interface composition to BeaconBlock interface, move fork specific logic into it's own files

* Remove needless underscore
2022-02-20 20:53:05 +00:00
terence tsao
0232b5f8f5
Regen pbs (#10227)
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-02-16 10:06:41 -06:00
terence tsao
d51b52d432
Add pow block protobuf (#10205)
* Add pow_block protobuf

* Revert hash changes

This reverts commit 4afb280385a411b65e960033f4d74f0ba74bc686.

* Revert "Add pow_block protobuf"

This reverts commit 4afb280385a411b65e960033f4d74f0ba74bc686.

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-09 22:27:30 +00:00
Nishant Das
80aa4f483c
Make Opt Max Cover Default (#10109)
* make default

* radek's review

* clean up

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-02-02 16:26:48 +00:00
Raul Jordan
4322185d80
Use Execution Payload Defined in Engine V1 Namespace (#10170)
* payload use

* incl

* cloners

* regen pb

* gen ssz

* use proper payload

* broken imports

* build

* state

* builds

* proper engine

* builds

* fmt

* baz

* ssz

* baz

* fmt

* regen ssz

* receipts root

* regen pb

* ssz

* receipts root

* ci lint

* proto

* test pkg

* bazel
2022-02-02 03:30:35 +00:00
Michael Neuder
8b2b109939
Rename imported keymanager to local. (#10155)
* imported -> local

* reverting to the state of the PR at eb1e3c3d1

* use changes from develop

* del

* rem patch

* patch

* rename to local

* gazelle

* add back build

* imported rename

* gaz

* local

* merge fix + remove proto changes

* comment revert

* build

* gofmt and one new reference

* gofmt pt2

* Update validator/accounts/wallet_edit.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/rpc/accounts.go

* rename

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
2022-02-01 19:54:19 +00:00
james-prysm
aed6e13498
Web3Signer: CLI implementation (#10056)
* initial commit for cli integration of web3signer

* resolving conflicts and execution

* remove aggregation slot from proto

* rem aggregation slot

* define a sync message block root struct

* fix sync message name

* sync message block root struct

* amend where sync committee block root is used

* altered switch statement to return correct json request by type

* fixing fork data import, types, and unit tests

* reverting unwanted changes

* reverting more unwanted changes

* fixing deepsource issues

* fixing formatting

* more fixes for deepsource and code clean up

* only want to fetch once for fetch validating public keys

* adding more comments

* new unit tests for requests and fixing a mapper issue

* Update validator/client/validator.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* adjusting comment

* adjusting comment

* fixing import organization

* including more unit tests

* adding new cli edit

* adding in checks for wallet initialize

* adding web3signer flags to main.go

* some how resolved files did not save correctly

* adding in check to make sure web flag only works with types imported and derived

* Update validator/client/sync_committee.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/accounts/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/main.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/wallet/wallet.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* reverting changes that accidently got checked in

* reverting

* reverting

* continuing to revert unintenteded changes

* reverting

* removing more unneeded changes

* addressing review comment

* initial refactor

* adding in more clarifying comments

* fixing mock

* resolving desource issues

* addressing gosec scan for helper go file

* addressing gosec

* trying to fix bazel build

* removal of interface to fix build

* fixing maligned struct

* addressing deepsource

* fixing deepsource

* addressing efficiency of type checking

* fixing bazel test failure

* fixing go linter errors

* gaz

* web changes

* add w3signer

* new kind

* proper use

* align

* adding prysm validator flags to help flags list

* addressing root comment

* ci lint

* fixing standardapi tests

* fixing accounts_test after removal of keymanager from rpc server

* fixing more unit tests

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update cmd/validator/flags/flags.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing missed err checks

* fixing mock tests

* fixing gofmt

* unskipping minimal e2e test and removing related TODOs

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update testing/endtoend/components/validator.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* adding some error wrapers to clarify failure point

* fixing bazel build with new error checks

* taking preston's advice to make test fail faster to understand what's going on with the test

* checking if genesis validators root is not zero hash

* adding check for genesis validators root giving zero hash

* fixing missing dependency

* adding check for wallet

* log all

* fixing errors for http responses

* switching marshal to pretty print

* adding pretty sign request test

* fixing base url setting

* adding in check for web3signer and temporary wallet instead of having to open the wallet

* refactoring web3signer to not require wallet

* bazel build fix

* fixing gazelle build

* adding content type of request

* fixing more bazel

* removing unused code

* removing unused comments

* adding skip test back in

* addressing a validation and error message

* fix parse

* body

* fixing logic for datadir

* improving error handling

* show resp

* fix

* sign resp as str

* point of pointer remove

* sign resp

* unmarshal sig resp

* read body as str

* adding more verbose logging

* removing unused result

* fixing unit test

* reconfiguring files to properly nest code and mocks

* fix build issue

* using context when using client function calls

* fixing based on suggestion

* addressing comments

* gaz

* removing defined max timeout

* reverting json print pretty

* Update validator/accounts/wallet_edit.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* removing unneeded code restrictions

* should not introduce new code that may impact existing key manager types

* adjusting comments

* adding in json validation

* running go mod tidy

* some logging

* more logs

* fixing typo

* remove logs

* testing without byte trim

* fixing order or properties

* gaz

* tidy

* reverting some logs

* removing the confusing comments

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update validator/client/aggregate.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing pr comments

* editing bytes test

* Run gazelle update-repos

* run gazelle

* improving unit test coverage

* fixing text

* fixing a potential escaped error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-31 10:44:17 -06:00
Leo Lara
d9799e6b6c
#10036 Replace codename Merge with Bellatrix (2nd step) (#10116)
* Rename BeaconBlockBodyMerge to BeaconBlockBodyBellatrix

* Rename SignedBeaconBlockMerge to SignedBeaconBlockBellatrix

* Rename CopyBeaconBlockMerge to CopyBeaconBlockMerge

* Rename NewBeaconBlockMerge to NewBeaconBlockBellatrix

* Rename BeaconBlockMerge to BeaconBlockBellatrix

* Rename some comments and strings in pkg proto: Merge -> Bellatrix

* Rename PbMergeBlock to PbBellatrixBlock

* Many renames of merge -> bellatrix in proto package

* Rename some Merge -> Bellatrix in beacon chain package

* More names

* Fix formating in config/params/config.go

* Rename Merge -> Bellatrix in proto/prysm/storage

* Several renames and corrections Merge -> Bellatrix

Co-authored-by: Potuz <potuz@potuz.net>

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Potuz <potuz@potuz.net>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-26 07:24:47 +00:00
james-prysm
8da8855ad5
Web3Signer: Sign Method Implementation (#10084)
* breaking up changes from cli pr

* reverting some changes, adding in changes from remote-web3signer

* adding raul's change

* adding fork info to signing calls

* fixing imports

* gaz

* fixing gofmt

* removing unneeded comment

* Update validator/client/aggregate.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/client/sync_committee.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* addressing comments

* revert proto changes

* proto changes

* reserve

* reserve

* switching to passing slot from passing fork, using slot to find the fork

* removing unneeded check

* fixing missed unit test

* optional

* optional

* gaz

* improving some definitions with constants

* improving some definitions with constants

* rem opt

* rem

* gaz

* moving mocks to its own folder

* adding in bazel field to fix

* fixing type check error

* fixing build

* fixing strict imports

* fixing dependencies

* changing bazel build

* changing bazel build

* changing bazel build

* removing testing only dependency

* removing dependency on testing util package

* update bazel build

* Update checktags_test.go

* Update active_balance.go

* Update sync_committee_minimal.go

* Update sync_committee_mainnet.go

* Update active_balance_disabled.go

* Update committee.go

* Update committee_disabled.go

* Update sync_committee.pb.gw.go

* Update powchain.pb.gw.go

* Update proposer_indices.go

* Update proposer_indices_disabled.go

* Update sync_committee.go

* Update mainnet.go

* Update p2p_messages.pb.gw.go

* Update finalized_block_root_container.pb.gw.go

* Update beacon_block.pb.gw.go

* Update attestation.pb.gw.go

* Update secret_key_test.go

* Update beacon_state.pb.gw.go

* Update version.pb.gw.go

* Update sync_committee.pb.gw.go

* Update sync_committee_disabled.go

* Update mainnet_test.go

* Update minimal.go

* Update signature_test.go

* Update gocast.go

* Update cgo_symbolizer.go

* Update validator.pb.gw.go

* Update beacon_state.pb.gw.go

* Update signature.go

* Update public_key_test.go

* Update minimal_test.go

* Update checktags_test.go

* Update bls_benchmark_test.go

* Update public_key.go

* Update secret_key.go

* Update aliases.go

* Update init.go

* Update stub.go

* Update journald_linux.go

* Update attestation.pb.gw.go

* Update config_utils_develop.go

* Update stub.go

* Update stub.go

* Update beacon_block.pb.gw.go

* Update validator.pb.gw.go

* Update node.pb.gw.go

* Update config_utils_prod.go

* Update journald.go

* Update beacon_block.pb.gw.go

* Update beacon_chain.pb.gw.go

* Update beacon_state.pb.gw.go

* Update events.pb.gw.go

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* addressing comments from review

* updating length of comment

* Update validator/keymanager/remote-web3signer/keymanager.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Update stub.go

revert changes

* Update validator/client/aggregate_test.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* addressing final comments

* fixing gofmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2022-01-18 14:31:58 -06:00
Mohamed Zahoor
4ca04dcc4b
Add storage wrapper for BeaconState (#9963)
* adding proto wrapper for BeaconState

* remove BUILD.bazel file spelling mistake

* address terence review comments

* remove capitalization

Co-authored-by: Nishant Das <nishdas93@gmail.com>

* renamed as storage, use v1alpha1 ojects

* fix lint in the generated file

* store state as ssz bytes

* added a seperate version for managing storage

* goimports generated files to satisfy lint

* changed proto to oneof

Co-authored-by: Nishant Das <nishdas93@gmail.com>
2022-01-18 15:21:56 +00:00
terence tsao
33d1ae0792
Eth2api: GetBlockSSZ supports bellatrix block (#10077)
* add rpc and tests

* rename
2022-01-17 02:30:09 +00:00
Preston Van Loon
182bd615ac
refactoring: Deduplicate code (#10090)
* Deduplicate sync committee indices from state

* Deduplicate code in listblocks RPC endpoints, which only differ in response object

* Deduplicate test code in config/fieldparams

* Delete stale benchmark target. This is already included in go_default_test

* deduplicate test cases in TestIsSlashableValidator_OK and fix blst
BUILD file

* Deduplicate TestStore_IsFinalizedChildBlock

* Revert crypto/bls/blst/BUILD.bazel

* Deduplicate TestStore_SaveBlock_NoDuplicates

* Use a generic wrapper for beacon blocks to streamline test

* Deduplicate TestStore_BlocksCRUD

* Deduplicate TestStore_BlocksHandleZeroCase

* Deduplicate TestStore_BlocksBatchDelete

* deduplicate TestStore_BlocksHandleInvalidEndSlot

* Deduplicate TestStore_BlocksCRUD_NoCache

* Deduplicate common block test setup. Make TestStore_Blocks_FiltersCorrectly support multiple forks in test

* Deduplicate the rest of these tests

* lint

* Deprecation warning

* Add test for WrappedSignedBeaconBlock

* Test error path
2022-01-16 18:44:42 +00:00
Michael Neuder
9166f40a04
Eth1 connections (#10073)
* Adding Eth1 connections data message to service definition

* PoC implementation for ETH1 connections RPC

* naming revisions

* cleanup

* register powchain info fetcher

* regenerate protos

* gazelle Bazel

* fixing comments

Co-authored-by: Michael Neuder <michaelneuder@Michaels-MacBook-Pro.local>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-01-14 11:13:04 -05:00
Leo Lara
b1c2454658
#10036 Replace codename Merge with Bellatrix (1st step) (#10044)
* Rename BeaconStateMerge to BeaconStateBellatrix

* Rename version.Merge to version.Bellatrix

* Rename ComputeFieldRootsWithHasherMerge to ComputeFieldRootsWithHasherBellatrix

* Rename test names to Bellatrix

* Rename comments and strings to Bellatrix

* Fix formatting in a few files

* Revert wrong renaming in test name

* Revert renaming to Bellatrix in mainnet_config.go

* Revert renaming of db key without migration

* Regenerate from proto changes

* Rename new use of already renamed symbols

* gofmt and goimports after regenerating protofiles

* revert weird imports

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
2022-01-10 16:47:30 +00:00
terence tsao
b38b0186b8
Use signature length from fieldparams (#10019)
* Use bls sig length from fieldparams

* fmt

* fix tests

* fix tests

* fix tests

* Update tags_test.go

* fix tests

* Update BUILD.bazel
2021-12-15 15:14:30 -05:00
Raul Jordan
37ca409cc1
Add More Fields to Remote Signer Sign Request (#10004)
* add in new fields

* sign request added

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-09 21:53:54 +00:00
Radosław Kapka
5569a68452
Code cleanup (#9992)
* Value assigned to a variable is never read before being overwritten

* The result of append is not used anywhere

* Suspicious assignment of range-loop vars detected

* Unused method receiver detected

* Revert "Auxiliary commit to revert individual files from 54edcb445484a2e5d79612e19af8e949b8861253"

This reverts commit bbd1e1beabf7b0c5cfc4f514dcc820062ad6c063.

* Method modifies receiver

* Fix test

* Duplicate imports detected

* Incorrectly formatted error string

* Types of function parameters can be combined

* One more "Unused method receiver detected"

* Unused parameter detected in function
2021-12-07 17:52:39 +00:00
terence tsao
2bcda2e021
Add more merge related protobufs (#9986)
* Add merge protos

* Update proto/eth/v2/beacon_block.proto

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-12-07 08:08:14 +00:00
terence tsao
c9d5b4ba0e
Add merge beacon block wrappers (#9906) 2021-11-24 14:26:17 -08:00
Raul Jordan
4ae7513835
Import Keystores Standard API Implementation (#9924)
* begin

* rem deleted code

* delete keystores all tests

* surface errors to user

* add in changes

* del

* tests

* slice

* begin import process

* add import keystores logic

* unit tests for import

* tests for all import keystores keymanager issues

* change proto

* pbs

* renaming works

* use proper request

* pb

* comment

* gaz

* fix up cli cmd

* test

* add gw

* precond

* tests

* radek comments
2021-11-24 10:40:49 -05:00