Commit Graph

430 Commits

Author SHA1 Message Date
Nishant Das
e83c9d5862
Revert "Replace a Few IntFlags with Uint64Flags (#9959)" (#10163)
* Revert "Replace a Few IntFlags with Uint64Flags (#9959)"

This reverts commit 790bf03123.

* fix

* fix
2022-02-01 08:51:17 +00:00
Preston Van Loon
7df01feb58
Remove libfuzzer / afl based builds in favor of go 1.18 fuzz testing (#10065)
* Remove libfuzzer / afl based builds in favor of go 1.18 fuzz testing

* rm beacon-fuzz archive

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2022-01-20 04:51:02 +00:00
Radosław Kapka
5569a68452
Code cleanup (#9992)
* Value assigned to a variable is never read before being overwritten

* The result of append is not used anywhere

* Suspicious assignment of range-loop vars detected

* Unused method receiver detected

* Revert "Auxiliary commit to revert individual files from 54edcb445484a2e5d79612e19af8e949b8861253"

This reverts commit bbd1e1beabf7b0c5cfc4f514dcc820062ad6c063.

* Method modifies receiver

* Fix test

* Duplicate imports detected

* Incorrectly formatted error string

* Types of function parameters can be combined

* One more "Unused method receiver detected"

* Unused parameter detected in function
2021-12-07 17:52:39 +00:00
Raul Jordan
790bf03123
Replace a Few IntFlags with Uint64Flags (#9959)
* use uints instead of ints

* fix method

* fix

* fix

* builds

* deepsource

* deep source
2021-12-01 23:34:53 +00:00
Nishant Das
236a5c4167
Cleanup From Deepsource (#9961)
* ds cleanup

* fix

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-12-01 18:56:07 +00:00
terence tsao
c21e43e4c5
Refactor: move functions beacon-chain/core/time -> time/slots (#9719)
* Move necessary functions beacon-chain/core/time -> time/slots

* Fix fuzz

* Fix build

* Update slot_epoch.go
2021-10-01 15:17:57 -05:00
terence tsao
8219af46e4
Move slot epoch from core to time pkg (#9714)
* Move slot epoch from core to time pkg

* Fix fuzz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-30 19:00:14 +00:00
Raul Jordan
a9a4bb9163
Move Shared/Testutil into Testing (#9659)
* move testutil

* util pkg

* build

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 18:53:46 +00:00
Raul Jordan
29513c804c
Create Encoding Bytesutil (#9658)
* bytesutil

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-23 15:23:37 +00:00
Raul Jordan
f3d6dbcc1e
Move Shared/Params Into Config/Params (#9642)
* config params into pkg

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-21 19:59:25 +00:00
Raul Jordan
11a1f681e0
Move Shared Packages Into Async/ (#9620)
* async packages

* change pkg

* build

* working

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-18 17:26:11 +00:00
Raul Jordan
7dadc780b8
Move Shared Packages into Math/ and IO/ (#9622)
* amend

* building

* build

* userprompt

* imports

* build val

* gaz

* io file

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-17 21:55:24 +00:00
Raul Jordan
d2f74615ab
Move Miscellaneous Shared/ Packages Into Semantic Groups (#9624)
* remove shared network and ip util

* forkutil

* gaz

* build

* gaz

* nogo

* genrule

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-17 19:20:50 +00:00
Raul Jordan
61c24c3870
Move Fuzz/ Into Testing/ (#9617)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 17:32:51 +00:00
Raul Jordan
23e2d62c0c
Create Container Package for Shared/ Subpackages (#9607)
* ensure run

* amend

* fix initial sync test target

Co-authored-by: prestonvanloon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-16 12:05:58 -05:00
terence tsao
ee5d75732d
Add pkg crypto (#9603)
* Add pkg crypto

* Update go.yml

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-15 22:55:11 +00:00
terence tsao
31a78ab22a
Create config package for Shared/featureconfig (#9593)
* add config/features

* Gazelle

* Gazelle

* Fix build

* Go mod tidy

* active cache

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2021-09-15 01:18:39 +00:00
terence tsao
3e71997290
Create time Package for Shared/timeutil, mclock and slotutil (#9594)
* add time pkg

* Go fmt
2021-09-15 00:09:04 +00:00
Preston Van Loon
b586d3784b
Rename beacon-chain/core/state to beacon-chain/core/transition pkg (#9530)
* Move, refactor, fix

* @rkapka PR feedback: revert irrelevant changes

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-08 10:41:47 +00:00
Preston Van Loon
4beb352e6f
Move Slot and Epoch functions from helpers to core pkg (#9519)
* Move Slot and Epoch functions from helpers to core

* limited viz

* goimports

* fix fuzz build

* fix fuzz build

* fix

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-03 20:10:31 +00:00
Nishant Das
cc790ceb2e
Add RPC Changes to Sync For Block Methods (#9497)
* add changes

* radek's review

* kasey's review

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-09-03 16:00:56 +00:00
Nishant Das
114a14a4b6
Patch Initial Sync For Non Finalized Blocks (#9452)
* fix for now

* off by 1

* preston's review
2021-08-24 10:20:45 -05:00
Preston Van Loon
362212b5ed
Fix 'headSlot' log statement in init-sync (#9451)
* Fix 'headSlot' log statement in init-sync

* gofmt -w beacon-chain/sync/initial-sync/round_robin.go
2021-08-24 07:00:18 +00:00
Raul Jordan
9145310647
Eliminate Proto V2 Namespace (#9297)
* get rid of v2 in prysm codebase

* replace block2

* builds

* terence comments

* gazelle
2021-07-28 21:23:44 +00:00
terence tsao
8db0c9a0e6
Move proto/interfaces -> proto/prysm/v2 (#9270)
* Move proto/interfaces -> proto/prysm

* Update paths

* Fix block2

* Update blocks_fetcher_utils.go

* Update BUILD.bazel

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-23 20:10:15 +00:00
Raul Jordan
2d10bcf179
Move State Protos Into V2 Namespace (#9257)
* move state protos

* regen ssz

* edit v1 code

* fix imports

* building

* beacon chain builds

* validator and shared builds

* fuzz builds

* changes

* spectest builds

* tools build

* remove import cycle

* generate ssz

* pcli

* gaz

* kafka

* gaz
2021-07-23 08:07:40 -05:00
Raul Jordan
6dadb80cc4
Move v1alpha1 into Prysm API namespace (#9245) 2021-07-21 21:34:07 +00:00
terence tsao
1d3a9983cc
Move block interface next to generated pb (#9146)
* Move block interface next to pb

* Update fuzz build bazel

* Move interface to /proto/interface and wrapper next to generated pb

* Fix fuzz build bazel

* Add //proto/eth/v1alpha1/wrapper

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-07-06 15:34:05 +00:00
Preston Van Loon
d6bd619429
Bazel: minimal test build transitions (#9122)
* Add build transitions for minimal config based tests

* respect existing gotags

* clean up import

* Add some commentary

* gaz

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2021-06-30 19:22:09 +00:00
Raul Jordan
b23f63a064
Beacon State V2 Interface Definition With Semantic Version Paths (#9125)
* v2 state initialize and semantic paths

* ensure build

* gaz changes to ignored build files

* gaz
2021-06-30 15:06:19 +00:00
Radosław Kapka
12403d249f
[Feature] - API Middleware (#8926)
* HTTP proxy server for Eth2 APIs (#8904)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* Run several new endpoints through API middleware (#8922)

* Implement API HTTP proxy server

* cleanup + more comments

* gateway will no longer be dependent on beaconv1

* handle error during ErrorJson type assertion

* simplify handling of endpoint data

* fix mux v1 route

* use URL encoding for all requests

* comment fieldProcessor

* fix failing test

* change proxy port to not interfere with e2e

* gzl

* simplify conditional expression

* Move appending custom error header to grpcutils package

* add api-middleware-port flag

* fix documentation for processField

* modify e2e port

* change field processing error message

* better error message for field processing

* simplify base64ToHexProcessor

* fix json structs

* /eth/v1/beacon/states/{state_id}/validators

* /eth/v1/beacon/states/{state_id}/validators/{validator_id}

* /eth/v1/beacon/states/{state_id}/validator_balances

* /eth/v1/beacon/states/{state_id}/committees

* allow skipping base64-encoding for query params

* /eth/v1/beacon/pool/attestations

* replace break with continue

* Remove unused functions (#8924)

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd71e2c1e3a14be85abd29a59b57ae9b5, reversing
changes made to 2d0f8e012ecb006888ed8e826b45625a3edc2eeb.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Process SSZ-serialized beacon state through API middleware (#8925)

* update field names

* Process SSZ-serialized beacon state through API middleware

* revert changes to go.mod and go.sum

* Revert "Merge branch '__develop' into feature/api-middleware"

This reverts commit 7c739a8fd71e2c1e3a14be85abd29a59b57ae9b5, reversing
changes made to 2d0f8e012ecb006888ed8e826b45625a3edc2eeb.

* update ethereumapis

* update validator field name

* update deps.bzl

* update json tags (#8942)

* Run `/node/syncing` through API Middleware (#8944)

* add IsSyncing field to grpc response

* run /node/syncing through the middleware

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* Return HTTP status codes other than 200 and 500 from node and debug endpoints (#8937)

* error codes for node endpoints

* error codes for debug endpoints

* better comment about headers

* gzl

* review comments

* comment on return value

* update fakeChecker used for fuzz tests

* fix failing tests

* Allow to pass URL params literally, without encoding to base64 (#8938)

* Allow to pass URL params literally, without encoding to base64

* fix compile error

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

* unused import

* Return correct status codes from beacon endpoints (#8960)

* Various API Middleware fixes (#8963)

* Return correct status codes from `/states` endpoints

* better error messages in debug and node

* better error messages in state

* returning correct error codes from validator endpoints

* correct error codes for getting a block header

* gzl

* fix err variable name

* fix nil block comparison

* test fixes

* make status enum test better

* fix ineffectual assignment

* make PR unstuck

* return proper status codes

* return uppercase keys from /config/spec

* return lowercase validator status

* convert requested enum values to uppercase

* validator fixes

* Implement `/beacon/headers` endpoint (#8966)

* Refactor API Middleware into more manageable code  (#8984)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* API Middleware unit tests (#8998)

* move endpoint registration out of shared package

* divide main function into smaller components

* return early on error

* implement hooks

* implement custom handlers and add documentation

* fix test compile error

* restrict package visibility

* remove redundant error checking

* rename file

* api_middleware_processing

* endpoints

* gzl

* remove gazelle:ignore

* merge

* Implement SSZ version of `/blocks/{block_id}` (#8970)

* Implement SSZ version of `/blocks/{block_id}`

* add dependencies back

* fix indentation in deps.bzl

* parameterize ssz functions

* get block ssz

* update ethereumapis dependency

* gzl

* Do not reuse `Endpoint` structs between API calls (#9007)

* code refactor

* implement endpoint factory

* fix test

* fmt

* include pbs

* gaz

* test naming fixes

* remove unused code

* radek comments

* revert endpoint test

* bring back bytes test case

* move `signedBeaconBlock` to `migration` package

* change `fmt.Errorf` to `errors.Wrap`

* capitalize SSZ

* capitalize URL

* more review feedback

* rename `handleGetBlockSSZ` to `handleGetBeaconBlockSSZ`

* rename `IndexOutOfRangeError` to `ValidatorIndexOutOfRangeError`

* simplify parameter names

* test header

* more corrections

* properly allocate array capacity

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
2021-06-15 10:28:49 -05:00
Raul Jordan
5aac06f04e
Move EthereumAPIs Into Prysm (#8968)
* begin move

* use same import path

* imports

* regen protos

* regen

* no rename

* generate ssz

* gaz

* fmt

* edit build file

* imports

* modify

* remove generated files

* remove protos

* edit imports in prysm

* beacon chain all builds

* edit script

* add generated pbs

* add replace rules

* license for ethereumapis protos

* change visibility

* fmt

* update build files to gaz ignore

* use proper form

* edit imports

* wrap block

* revert scripts

* revert go mod
2021-06-02 18:49:52 -05:00
terence tsao
276d03553c
Block interface clean up (#8947) 2021-05-26 11:33:46 -07:00
Nishant Das
caf9bdbc6f
Use Block Interface Across Prysm (#8918)
* commit initial work

* checkpoint current work

* gaz

* checkpoint

* req/resp changes

* initial-sync

* finally works

* fix error

* fix bugs

* fix issue

* fix issues

* fix refs

* tests

* more text fixes

* more text fixes

* more text fixes

* fix tests

* fix tests

* tests

* finally fix builds

* finally

* comments

* fix fuzz

* share common library

* fix

* fix

* add in more defensive nil checks

* add in more defensive nil checks

* imports

* Apply suggestions from code review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Apply suggestions from code review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_interface.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_wrapper.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update shared/interfaces/block_interface.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* imports

* fix bad changes

* fix

* terence's review

* terence's review

* fmt

* Update beacon-chain/rpc/beacon/blocks.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>

* fix tests

* fix

* fix all tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
2021-05-26 16:19:54 +00:00
Nishant Das
35dc8c31fa
Add in Context For Req/Resp (#8895)
* Add in context

* use better function signatures

* add in chainservice

* fix rpc requests

* add in test

* fix tests

* add in comments

* Update beacon-chain/sync/rpc_chunked_response.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/sync/rpc_send_request.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* rename files

* fmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-05-17 19:25:59 +00:00
Ivan Martinez
2f10b1c7b1
Change gogoproto compiler to protoc-gen-go-cast (#8697)
* Remove gogoproto compiler

* Remove more gogoproto

* Improvements

* Fix gengo

* More scripts

* Gazelle, fix deps

* Fix version and errors

* Fix gocast for arrays

* Fix ethapis

* Fixes

* Fix compile errors

* fix go.mod

* //proto/... builds

* Update for protov2

* temp fix compilation to move on

* Change everything to emptypb.empty

* Add grpc to proto/slashings

* Fix almost all build failures

* Oher build problems

* FIX THIS FUCKING THING

* gaz literally every .bazel

* Final touches

* Final final touches

* Fix proto

* Begin moving proto.Marshal to native

* Fix site_data

* Fixes

* Fix duplicate gateway

* Fix gateway target

* Fix ethapis

* Fixes from review

* Update

* Fix

* Fix status test

* Fix fuzz

* Add isprotoslice to fun

* Change DeepEqual to DeepSSZEqual for proto arrays

* Fix build

* Fix gaz

* Update go

* Fixes

* Fixes

* Add case for nil validators after copy

* Fix cast

* Fix test

* Fix imports

* Go mod

* Only use extension where needed

* Fixes

* Split gateway from gengo

* gaz

* go mod

* Add back hydrated state

* fix hydrate

* Fix proto.clone

* Fies

* Revert "Split gateway from gengo"

This reverts commit 7298bb2054d446e427d9af97e13b8fabe8695085.

* Revert "gaz"

This reverts commit ca952565701a88727e22302d6c8d60ac48d97255.

* Merge all gateway into one target

* go mod

* Gaz

* Add generate v1_gateway files

* run pb again

* goimports

* gaz

* Fix comments

* Fix protos

* Fix PR

* Fix protos

* Update grpc-gateway and ethapis

* Update ethapis and gen-go-cast

* Go tidy

* Reorder

* Fix ethapis

* fix spec tests

* Fix script

* Remove unused import

* Fix fuzz

* Fix gomod

* Update version

* Error if the cloned result is nil

* Handle optional slots

* ADd more empty checks to clone

* Undo fuzz changes

* Fix build.bazel

* Gaz

* Redo fuzz changes

* Undo some eth1data changes

* Update go.mod

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Undo clone beacon state

* Remove gogo proto more and unused v1_gateway

* Add manual fix for nil vals

* Fix gaz

* tidy

* Tidy again

* Add detailed error

* Revert "Add detailed error"

This reverts commit 59bc053dcd59569a54c95b07739d5a379665ec5d.

* Undo varint changes

* Fix nil validators in deposit test

* Commit

* Undo

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-05-17 18:32:04 +00:00
Nishant Das
e6a1d5b1b9
Refactor RPC Topic Construction (#8892)
* check in changes

* goimports

* export it

* add new rpc type
2021-05-15 08:58:56 +08:00
kevlu93
eca67cec4c
Embed Config Pattern for Sync Services (#8636)
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2021-03-21 19:07:42 +00:00
terence tsao
ce725ceec3
Move state pkg to stateV0 pkg (#8620)
* Move state pkg to stateV0 pkg

* Build.bazel

* Remove unused RootsArrayHashTreeRoot

* Revert "Remove unused RootsArrayHashTreeRoot"

This reverts commit bf0bda30d1a8eb7a071f6e3ce9ee85041b45aca6.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-03-17 19:49:49 +00:00
Victor Farazdagi
fa2084330b
Fix race condition in TestService_Initialized (#8597) 2021-03-11 21:16:40 +00:00
Victor Farazdagi
fa92766095
Fixes flaky TestBlocksFetcher_nonSkippedSlotAfter test (#8586)
* Fixes flaky TestBlocksFetcher_nonSkippedSlotAfter test

* fix conditional
2021-03-10 20:03:26 +00:00
Raul Jordan
090fbbf18c
Add Goland Standard "cmd" Pattern for Beacon-Chain Binaries (#8540)
* beacon chain cmd pattern

* imports spacing

* more import fix

* edit build file

* e2e viz

* amend e2e
2021-03-02 13:36:03 -06:00
Victor Farazdagi
a069738c20
ETH2 Types: Slot (#8408)
* update shared/params

* update eth2-types deps

* update protobufs

* update shared/*

* fix testutil/state

* update beacon-chain/state

* update beacon-chain/db

* update tests

* fix test

* update beacon-chain/core

* update beacon-chain/blockchain

* update beacon-chain/cache

* beacon-chain/forkchoice

* update beacon-chain/operations

* update beacon-chain/p2p

* update beacon-chain/rpc

* update sync/initial-sync

* update deps

* update deps

* go fmt

* update beacon-chain/sync

* update endtoend/

* bazel build //beacon-chain - works w/o issues

* update slasher code

* udpate tools/

* update validator/

* update fastssz

* fix build

* fix test building

* update tests

* update ethereumapis deps

* fix tests

* update state/stategen

* fix build

* fix test

* add FarFutureSlot

* go imports

* Radek's suggestions

* Ivan's suggestions

* type conversions

* Nishant's suggestions

* add more tests to rpc_send_request

* fix test

* clean up

* fix conflicts

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
2021-02-16 07:45:34 +00:00
Radosław Kapka
0716519be9
Fix error formatting inside fmt.Errorf (#8439)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-12 23:44:46 +00:00
Radosław Kapka
47fdb3b99a
Revert "Rename NewService to New (#8337)" (#8440)
* Revert "Rename `NewService` to `New` (#8337)"

This reverts commit d121b19145.

# Conflicts:
#	beacon-chain/sync/initial-sync/round_robin_test.go

* fix name in test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-02-12 17:45:22 +00:00
Victor Farazdagi
a8e501b3cf
ETH2 Types: Epoch (#8373)
* update deps

* update deps

* update protos/*

* update deps

* reset protos

* update protos

* update shared/params/config

* update protos

* update /shared

* update shared/slotutil and shared/testutil

* update beacon-chain/core/helpers

* updates beacon-chain/state

* update beacon-chain/forkchoice

* update beacon-chain/blockchain

* update beacon-chain/cache

* update beacon-chain/core

* update beacon-chain/db

* update beacon-chain/node

* update beacon-chain/p2p

* update beacon-chain/rpc

* update beacon-chain/sync

* go mod tidy

* make sure that beacon-chain build suceeds

* go fmt

* update e2e tests

* update slasher

* remove redundant alias

* update validator

* gazelle

* fix build errors in unit tests

* go fmt

* update deps

* update fuzz/BUILD.bazel

* fix unit tests

* more unit test fixes

* fix blockchain UTs

* more unit test fixes
2021-02-09 10:05:22 +00:00
Radosław Kapka
86a9d4c6a4
Configurable testutil's BeaconState (#8407)
* Configurable testutil's BeaconState

* fix shared and fuzz tests

* return state copy

* use mainnet config values for default state

* handle error in block fuzz

* goimports

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-02-08 20:00:09 +00:00
Panagiotis Georgiadis
0a180dc662
Make a const instead of 3 'abc' occurences (#8406) 2021-02-08 17:08:29 +01:00
Victor Farazdagi
372dc47b64
Deprecate enableSyncBacktracking flag (#8371)
* Deprecate enableSyncBacktracking flag

* keep deprecated flags
2021-02-01 11:18:24 +00:00
terence tsao
d121b19145
Rename NewService to New (#8337)
* Hide beacon operation field if it's 0

* Rename NewSerivce to New

* Revert "Hide beacon operation field if it's 0"

This reverts commit 896fa11a0bb605212293d6fff02a8974c54710ab.

* Fix NewServiceRegistry

* Update slasher/detection/service.go

Co-authored-by: Radosław Kapka <rkapka@wp.pl>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-26 10:26:57 +00:00
terence tsao
d5ec248691
Rename getter functions to be idiomatic (#8320)
* Rename getter functions

* Rename new

* Radek's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-01-25 21:27:30 +00:00
terence tsao
9ff825a570
Update variable names that are same as imports (#8220)
* Fix names that are same as import name

* Review feedbacks
2021-01-07 10:42:03 -06:00
Radosław Kapka
9b3e1eb643
Implement GetHealth in the node API (#8217)
* Implement GetHealth in the node API

* repair fuzz mock

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-01-06 20:11:20 +00:00
Radosław Kapka
392e61fbee
Revert changes to sync service status check (#8215) 2021-01-06 11:19:53 +00:00
Radosław Kapka
7135a8542f
Check if initial sync service has been initialized (#8214) 2021-01-06 10:45:22 +00:00
Victor Farazdagi
e772e8c8c2
Init sync: minor style fixes (#8156)
* simplify FSM calls

* improve calculateHeadAndTargetEpochs API

* simplify

* simplify nil assignment
2020-12-18 14:31:15 +00:00
terence tsao
20dede7532
Move state summary cache to DB (#8101) 2020-12-16 08:56:21 -08:00
Victor Farazdagi
0f18867f08
Remove duplicate package imports (#8104)
* remove duplicate imports

* remove unused lookupLimit

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-14 22:22:55 +00:00
Steven Allen
2428880058
Update go-libp2p to 0.12.0 (#8015)
* Update go-libp2p to 0.12.0

go-libp2p 0.12.0 made some significant changes to the stream interfaces around
stream closing:

* Close now closes in both directions and frees the stream. However, unlike
FullClose did, it doesn't _wait_ for the remote peer to respond with an EOF.
* To close for writing, call CloseWrite (like one would on a TCP connection, etc.).

This patch:

* Replaces calls to FullClose with Close where appropriate.
* Replaces calls to Close with CloseWrite where appropriate.
* Removes redundant Close calls.
* Calls Reset to where appropriate to indicate that the request/response was
  aborted. Unlike Close, this will not flush and will not cause the remote peer
  to read an EOF. Instead, the remote peer will read an ErrReset error.
* Ensures we always either close or reset streams. Send wasn't closing the
  stream on some error paths.
* Now that stream closing is async, we explicitly wait for a response when
  "hanging up" on a peer (so we don't hang up before they receive our
  response/goodbye message).

* update bazel

* Gazelle

* revert unintentional bazel workspace change

* appease an overzealous linter

* update to latest

* Refactor encoder

* gazelle

* Gazelle

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-14 17:22:25 +00:00
Victor Farazdagi
b51aec6981
Init sync: minor fixes (#8060)
* fixes typo

* simplify code
2020-12-07 09:33:06 +00:00
Victor Farazdagi
be078d6a16
Update TestMain(): do not call os.Exit() explicitly (#8046)
* update workspace

* update testmain
2020-12-04 16:10:07 +00:00
Victor Farazdagi
afbfaedea4
Unskip fixed init-sync service test (#8042) 2020-12-04 11:57:46 +00:00
Victor Farazdagi
647b4cf108
Round robin: half open interval in syncToFinalizedEpoch (#8039) 2020-12-03 21:55:42 +00:00
Victor Farazdagi
7dd0c24fea
Skip bogus init-sync test (#8038) 2020-12-03 20:20:04 +00:00
Victor Farazdagi
3092f75ec2
Init sync: conditional syncing to finalized slot (#7999)
* extract sync methods

* add unit test

* gazelle

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-authored-by: Shay Zluf <thezluf@gmail.com>

* better set back step

Co-authored-by: Shay Zluf <thezluf@gmail.com>
2020-12-03 14:10:51 +00:00
Victor Farazdagi
5417e8cf31
Init-sync: enable peer scorer by default (#7974)
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-12-03 00:45:20 +00:00
Victor Farazdagi
c7f7a29d7e
remove redundant start slot (#7991)
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-12-02 20:47:39 +03:00
Potuz
9a1423d62d
Log with field error instead of err (#7998) 2020-12-01 05:38:42 -08:00
Victor Farazdagi
54a42ce4a8
more robust processing of invalid head slot (#7990) 2020-11-29 15:03:25 -06:00
Victor Farazdagi
f361450e8d
Update TestMain() to use os.Exit() (#7814)
* update TestMain

* fix sync/initial-sync test

* restore code in rate limiter

* fix rate_limiter tests
2020-11-13 18:28:14 -08:00
Raul Jordan
4290ba416c
Fix Prysm Runtime Data Races (#7770)
* handle state trie data races

* race fixes

* added proper locks

* fix gaz

* use thread-safe refs() function
2020-11-10 20:57:07 -06:00
Victor Farazdagi
09e3f0360e
Remove redundant calls to os.exit() in TestMain (#7761)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-10 14:56:47 +00:00
Victor Farazdagi
51796d77f6
Init-sync: backtracking to alternative forks (#7564)
* cherry-pick from previous commits

* use finalized checpoint instead of head slot

* add helpers

* add dedicated error for missing blocks

* add backtracking options to fetcher

* TestBlocksFetcher_alternativeSlotBefore regression test

* update peer block provider test

* update injected deps

* update round robing deps

* update fetcher

* patch status and stream handlers

* test template

* gazelle

* revert BestNonFinalized changes

* extract waitForBandwidth method

* extend test

* remove redundant code

* add flag

* update queue

* update thresholds

* upd inequality

* minor cleanup

* check for mode

* add backtracking queue test

* merge from master

* remove redundant test

* correct logging level

* fix tests

* remove test template

* Nishant's suggestion to parametrize magic number
2020-11-06 15:54:20 +00:00
Victor Farazdagi
6e21b7a623
Add one more assertion to fetcher test (#7733) 2020-11-05 19:48:49 +00:00
Victor Farazdagi
f6cbfd5e27
Init-sync fetcher: backtracking (#7704)
* fetcher: backtracking

* gazelle

* amend comments

* cleanup

* cleanup

* minor fixes

* simplify

* remove unrelevant test

* add TestBlocksFetcher_findAncestor

* more tests

* Address Nishant's feedback

* optimize backtracking

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-05 07:27:46 +00:00
Victor Farazdagi
c5a8363998
Remove redundant error return from filterPeers() (#7730) 2020-11-04 23:29:50 +00:00
Victor Farazdagi
7acd73e1fe
Refactor p2p requests: extract BeaconBlocksByRangeRequest and BeaconBlocksByRootRequest (#7718)
* extract SendBeaconBlocksByRangeRequest

* adds extra processing

* extract SendBeaconBlocksByRootRequest

* extract into rpc_send_request.go

* add tests

* more tests

* gazelle

* remove redundant import

* for consistency

* for consistency

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-04 20:09:19 +00:00
Victor Farazdagi
926d3b9b34
Init-sync: more tests + minor refactoring (#7692)
* cherry-pick commits

* re-arrange calls
2020-10-31 21:33:57 +00:00
Nishant Das
df762bbfee
cleaner logging (#7689) 2020-10-31 19:10:08 +08:00
Victor Farazdagi
4fc0a50569
Turn init-sync FSM logs to trace level (#7670)
* trace

* go fmt

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-29 04:55:07 +00:00
Preston Van Loon
fcbb168c76
Code health: review map usage (#7635)
* remove unused cache states map

* correct typo

* Remove unused array

* Add lock around deposits cache chainstart pubkeys

* Copy attestation before grabbing lock. This may reduce lock contention time as other callers wanting the lock do not need to wait as long for the lock to become available.

* Copy attestation before grabbing lock. This may reduce lock contention time as other callers wanting the lock do not need to wait as long for the lock to become available.

* Set capacity to 1 since it is known that the slice will be 1 after insertion

* require validatorSlashingPreconditionCheck caller to hold lock

* Add lock for voluntary exits pool HasBeenIncluded

* Require rate limiter retrieveCollector to hold lock

* Add lock requirement assertions in sync

* Remove unused struct

* remove ClearCachedStates API

* field initSyncState is unused

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-10-26 21:17:07 +00:00
Victor Farazdagi
e6d688f6d5
Init-sync: re-arrange tests (#7641)
* move tests

* formatting

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-26 08:54:06 +00:00
Victor Farazdagi
8a3b75e9e3
Update init-sync FSM (#7640)
* update fsm

* more tests
2020-10-26 05:26:35 +00:00
Radosław Kapka
7e2112b4ba
Fix panic issues on beacon node shutdown (#7625)
* allow service cleanup on node shutdown

* revert error to fatal

* remove unused struct field
2020-10-23 12:43:48 +00:00
Victor Farazdagi
0b53a89d00
More verbose output on init-sync (#7578)
* more verbose output on init-sync

* log slot

* no error stack

* remove redundant data

* less verbose
2020-10-20 14:59:38 +00:00
terence tsao
7076a1ec4a
More feature flag deletions (#7533)
* Delete disable state lock and init sync verbose flags

* Delete disable slashing broadcast

* Remove disable wait for sync, noise, eth1 cache, static subnet

* Remove enable broadcast recovery attemp and make it as default

* Remove disable head update on per attestation

* Revert disable att braodcast discovery attempt

* gazelle

* Fixed an anti pattern

* Add enableAttBroadcastDiscoveryAttempts back

* Add back WaitForSync

* Remove extra lines

* Use DisableDynamicCommitteeSubnets path per @prestonvanloon feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 23:28:49 +00:00
Victor Farazdagi
a005c77b3f
Update error usage patterns to go1.13+ (#7529)
* rely on errors.Is

* wrap errors
2020-10-14 17:39:52 +00:00
terence tsao
8f04c555fc
More feature flags removal (#7526)
* Remove disable domain cache

* Remove don't verify att sig flag

* Remove batch verify and attester copies

* Remove batch verify usage

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 16:18:40 +00:00
Nishant Das
022b6667e5
Use Custom SSZ for P2P Types (#7436)
* checkpoint progress

* add roundtrip tests

* change all

* remove error response

* clean up

* Update beacon-chain/sync/error_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* gaz

* fix tests

* fmt

* gaz

* change back

* fix again

* clean up

* deep source

* fix all tests

* add gaz

* fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-14 07:55:28 +00:00
Victor Farazdagi
b9844024b4
Update comments describing init-sync process (#7521)
* updates comments

* fetcher mode from config

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 01:44:46 +00:00
Victor Farazdagi
ac1a4a078c
Fix reported anti patterns (#7501)
* merge var decl

* single append

* replace bytes.Compare -> bytes.Equal

* remove redundant spritnf

* remove redundant spritnf

* trimprefix

* remove redundant nil check

* remove redundant return

* plain channel or unblock on context closing

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-12 16:12:00 +00:00
Nishant Das
43765b5cb0
Register Subscribers After Node Is Synced (#7468)
* wait for synced

* fix again

* add test

* fix all

* fixes deepsource reported issue

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-10 16:50:28 +08:00
Victor Farazdagi
d98a6dda8f
Verify interface compliance using typed nil instead of dereferencing and conversion (#7489)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-10 00:36:48 +00:00
Victor Farazdagi
f92244d497
Remove redundant err checking (#7488)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-09 23:39:23 +00:00
Victor Farazdagi
9ce64e2428
Refactor peer scorer into peerdata + scorers (#7452)
* updates comment

* manager -> service

* rename receiver

* refacgtor bad_responses

* refactor store

* update status service

* extends data service

* status service test

* refactor block provider scorer

* misc updates

* fix tests

* data -> peerdata

* gazelle

* peerdata/store test

* limit the visibility scope

* Nishant's suggestion

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-07 13:08:51 +00:00
Radosław Kapka
3e0b20529b
Miscellaneous code quality improvements (#7414)
* anti-patterns

* performance issues

* handle skipped defer

* gazelle fix

* misc bug risks

* make logging of proposer slashings more robust

* simplify calling span.End()

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-04 15:03:10 +00:00
Victor Farazdagi
d169b490fa
Fix ineffectual assignments (#7403)
* update rpc/beacon
* more fixes to beacon-chain/rpc
* update beacon-chain/sync
* Merge refs/heads/master into fix-ineffectual-assignments
* updates beacon-chain/p2p
* Merge branch 'fix-ineffectual-assignments' of github.com:prysmaticlabs/prysm into fix-ineffectual-assignments
* update beacon-chain/*
* fix imports
* update beacon-chain/blockchain
* more updates
* Merge refs/heads/master into fix-ineffectual-assignments
* Merge branch 'master' into fix-ineffectual-assignments
* Merge refs/heads/master into fix-ineffectual-assignments
* next round of updated
* Merge branch 'fix-ineffectual-assignments' of github.com:prysmaticlabs/prysm into fix-ineffectual-assignments
* wrap up remaining items
2020-10-01 18:53:36 +00:00
Victor Farazdagi
4aea039324
Remove unused constant in round robin (#7402)
* remove unused constant
2020-10-01 13:02:47 +00:00
terence tsao
68d0c09daf
Add, use and test VerifyWeakSubjectivityRoot (#7344)
* Add, use and test `VerifyWeakSubjectivityRoot`
* Comments
* Make bazel test happy
* Merge branch 'master' into verify-ws
* Add flag to help
* Merge branch 'verify-ws' of github.com:prysmaticlabs/prysm into verify-ws
* Merge refs/heads/master into verify-ws
* Merge refs/heads/master into verify-ws
* Merge refs/heads/master into verify-ws
* Merge branch 'master' of github.com:prysmaticlabs/prysm into verify-ws
* Update beacon-chain/blockchain/service.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Update beacon-chain/blockchain/weak_subjectivity_checks.go

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Merge branch 'verify-ws' of github.com:prysmaticlabs/prysm into verify-ws
* s/&&/|| for VerifyWeakSubjectivityRoot circuit breaker
* Merge refs/heads/master into verify-ws
* Merge refs/heads/master into verify-ws
2020-09-26 08:04:07 +00:00
Victor Farazdagi
5b814009fa
Improves test coverage of init-sync queue (#7320)
* basic invariants for all state transition handlers

* TestBlocksQueue_onCheckStaleEvent extended

* TestBlocksQueue_onScheduleEvent extended

* TestBlocksQueue_onDataReceivedEvent extended

* removes redundant ctx setting

* TestBlocksQueue_onReadyToSendEvent extended

* TestBlocksQueue_onProcessSkippedEvent extended

* test to highest slot update

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-24 21:39:46 +08:00