Commit Graph

313 Commits

Author SHA1 Message Date
Raul Jordan
4290ba416c
Fix Prysm Runtime Data Races (#7770)
* handle state trie data races

* race fixes

* added proper locks

* fix gaz

* use thread-safe refs() function
2020-11-10 20:57:07 -06:00
Victor Farazdagi
09e3f0360e
Remove redundant calls to os.exit() in TestMain (#7761)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-10 14:56:47 +00:00
Preston Van Loon
93c11e0e53
Update rules_go (#7202)
* Update rules_go

* go 1.15

* try with v0.24.2

* Update Mac OS X SDK

* gaz

* update SDK in toolchain config

* -I flag

* another -I flag

* Update rules_go, gazelle, bazel version

* regen, update rules_docker

* Revert "another -I flag"

This reverts commit 9255133d99bee1e94560a64316a4a7539a363935.

* Revert "-I flag"

This reverts commit 2954a41d76adc81ccb6281af243f4be1f79152e9.

* giving up

* Use OS X 10.12

* Use OS X 10.12

* Revert "Use OS X 10.12"

This reverts commit 4f60d5cb807b2ccccd484a85a6a7477a5fd13e28.

* Revert "Use OS X 10.12"

This reverts commit a79177fab7182ad792d1cee7fb61214ef8bd73ab.

* osx toolchain tweaks necessary to work with 10.15 mac sdk

* Update docker image, regen

* gaz

* test using custom image

* Revert "test using custom image"

This reverts commit 95b8666810a254f16061c99e2ea0ae4d020b2f2d.

* explicit go version

* Clean up docker image rules with new definitions. gazelle

* please the linter

* Update protobuf compiler to 3.13.0, run gazelle

* Update gazelle to fix empty build files. https://github.com/bazelbuild/bazel-gazelle/pull/926

* update skylib

* fix herumi fuzz build

* remove comment from tools/cross-toolchain/regenerate.sh

Co-authored-by: rkapka <rkapka@wp.pl>
Co-authored-by: SuburbanDad <gts.mobile@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-11-10 03:01:56 +00:00
terence tsao
be40e1a3b9
Update delete state(s) functions (#7754) 2020-11-09 15:37:36 -08:00
Nishant Das
5995d2394c
Pass By Value Instead Of Reference (#7710)
* change to value from reference

* fix up

* make it a pointer

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-11-09 10:08:08 +00:00
terence tsao
4dcae8707a
Startup routine to clean up dirty states in DB (#7729)
* Clean up dirty states in DB

* Add Passthrough

* Fix finalized check

* Add tests

* Remove unused test vars

* Update beacon-chain/db/kv/state.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>

* Use non-strict inequality

* Prevent overflow and test

* Use slotsPerArchivedPoint/3 to loosen criteria

* Update test

* Background

* Handle ctx cancellation

* Update beacon-chain/state/stategen/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/db/kv/state.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Handle error

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-11-06 21:05:44 +00:00
terence tsao
2da1ec8052
Remove expensive validators only method (#7717)
* Remove validators only method

* Gazelle
2020-11-03 12:47:21 -06:00
terence tsao
817c16a2f4
Use a short cut in migration step (#7686)
* Don't generate state if it's already in DB

* Use proper missing root

* Update logic for clarity
2020-10-31 19:01:05 +00:00
terence tsao
92b6e0b6af
Add and use HasStateInCache (#7691) 2020-10-31 11:38:01 -07:00
terence tsao
fb2dfec1f4
Use state by root to get finalized state (#7677) 2020-10-29 14:04:06 -05:00
Nishant Das
2e4dee5aeb
Stategen Bug Fixes (#7674)
* bug fixes

* fix

* terence's review

* Make comments more explicit

* Update variable names

* Update tests

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-10-29 16:14:57 +00:00
Raul Jordan
b1c047b9ee
Prevent Panics in Field Trie Helpers (#7613)
* add tests to prevent panics

* if > 0

* fix typ
2020-10-23 21:41:45 +00:00
Nishant Das
6eb022ffa1
Clean Up Block Retrieval Methods (#7593)
* fix and add tests

* return for 0

* add test case

* fix

* fix fail

* add comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-23 04:23:15 +00:00
terence tsao
840ffc84ac
Save state to DB during long non-finality (#7597)
* Starting saving state during hot

* Add a log

* Add helpers to turn on/off mode

* Add locks

* Add missing return

* Clean up

* Add logic to migration to handle db roots

* Add tests for on and off

* Add more tests

* Add test for migrate

* @prestonvanloon's feedback

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-23 00:35:30 +00:00
Raul Jordan
e54ac48f9d
Deep Source Security Issues (#7617)
* deep source issues

* fix broken build

* fix beacon chain build

* gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-22 23:31:52 +00:00
Raul Jordan
d2f7240255
Context Cancelation Handling for Hot States (#7596)
* attempt deadline for state info

* cancel check
2020-10-21 17:06:48 +00:00
Nishant Das
329fbff814
Use StateRoot as Key for Cache (#7540)
* checkpoint

* review

* comment

* fix bad bug

* fix up
2020-10-19 09:13:47 -07:00
Nishant Das
b502876e98
Fix Mismatched BlockRoots In State Replay (#7559)
* fix mismatched roots

* alternate approach

* preston's review

* comment
2020-10-18 09:39:27 -07:00
Ivan Martinez
daf0b51361
Eth2 API: Implement block endpoints (#7433)
* Start Beacon API outline

* Rename to v1

* Add impl

* Change to outline

* Add comments

* Remove unneeded items

* Fix linting

* tidy

* Fix visibility

* go.sum

* Fix deps

* Tidy

* Implement blocks API endpoints

* Add check for interface type and fix pointers

* Fix pointer name

* gaz

* Fix comments

* Fix imports

* Fix analysis

* Add more coverage

* Add coverage and fix errors

* Fix head test

* Fix test remove println

* Fix error text and cleanup

* Change tests to TDD

* Add tests for finalized

* Fix att test

* Fix analysis

* Fix go mo d

* Fix proto

* fix go mod

* Extend testing

* Fix tests

* Move migration to package and test block atts

* Fix migration

* Gaz

* Check for block canonical before returning

* Fix text

* Gaz

* Fix tests

* Fix tests

* Fix canonical

* Fix test again

* Fix tests

* Remove unneeded comment

* Plug in RPC service

* Fix err msg
2020-10-15 18:00:49 +00:00
terence tsao
7076a1ec4a
More feature flag deletions (#7533)
* Delete disable state lock and init sync verbose flags

* Delete disable slashing broadcast

* Remove disable wait for sync, noise, eth1 cache, static subnet

* Remove enable broadcast recovery attemp and make it as default

* Remove disable head update on per attestation

* Revert disable att braodcast discovery attempt

* gazelle

* Fixed an anti pattern

* Add enableAttBroadcastDiscoveryAttempts back

* Add back WaitForSync

* Remove extra lines

* Use DisableDynamicCommitteeSubnets path per @prestonvanloon feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 23:28:49 +00:00
terence tsao
8f04c555fc
More feature flags removal (#7526)
* Remove disable domain cache

* Remove don't verify att sig flag

* Remove batch verify and attester copies

* Remove batch verify usage

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-14 16:18:40 +00:00
Victor Farazdagi
a019a0db4c
Combines func params of the same type (#7500)
* combines func params

* update leftovers

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-12 15:43:19 +00:00
Preston Van Loon
7cc32c4dda
Various code inspection resolutions (#7438)
* remove unused code

* remove defer use in loop

* Remove unused methods and constants

* gofmt and gaz

* nilness check

* remove unused args

* Add TODO for refactoring subscribeWithBase to remove unused arg. It seems too involved to include in this sweeping PR. https://github.com/prysmaticlabs/prysm/issues/7437

* replace empty slice declaration

* Remove unnecessary type conversions

* remove redundant type declaration

* rename receivers to be consistent

* Remove bootnode query tool. It is now obsolete by discv5

* Remove relay node. It is no longer used or supported

* Revert "Remove relay node. It is no longer used or supported"

This reverts commit 4bd7717334dad85ef4766ed9bc4da711fb5fa810.

* Delete unused test directory

* Delete unsupported gcp startup script

* Delete old k8s script

* build fixes

* fix build

* go mod tidy

* revert slasher/db/kv/block_header.go

* fix build

* remove redundant nil check

* combine func args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-10-12 08:11:05 +00:00
Victor Farazdagi
4c09e59b3b
Remove duplicate imports (#7491)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-10 03:54:31 +00:00
Victor Farazdagi
f92244d497
Remove redundant err checking (#7488)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-09 23:39:23 +00:00
Preston Van Loon
d4e6ce6998
Fix deepsource findings (#7457)
* remove unused code

* Incomplete condition fix

* club append to single call

* anti-pattern fix 'should use for range instead of for { select {} }'

* use strings.ReplaceAll

* replace `len(parts[0]) == 0` with `parts[0] == ""`

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-08 08:36:47 +00:00
terence tsao
7ad2929f0f
Revamp proposer cache to fix lookahead bug (#7442)
* Add and test proposer indices cache

* Remove proposer indices usages from committee cache

* Adopt the new proposer indices cache

* Add comments on why genesis epoch is skipped

* Fix the failing tests

* Update beacon-chain/blockchain/process_block.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/core/helpers/committee.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/core/helpers/slot_epoch.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/core/helpers/slot_epoch.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/core/helpers/slot_epoch_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/core/helpers/validators.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>

* Address additional feedbacks from @prestonvanloon

* Add comments on why genesis epoch is skipped

* Refactor EndSlot to use StartSlot within

* Add proposer indices disabled

* Add libfuzz tags

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-10-07 17:25:08 +00:00
Radosław Kapka
3e0b20529b
Miscellaneous code quality improvements (#7414)
* anti-patterns

* performance issues

* handle skipped defer

* gazelle fix

* misc bug risks

* make logging of proposer slashings more robust

* simplify calling span.End()

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-04 15:03:10 +00:00
Preston Van Loon
2916d183e8
Pubsub: ensure better TopicIDs validation (#7418)
* Ensure better topicIDs validation, add tests

* go mod tidy

* fix tests

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-03 01:51:58 +00:00
Victor Farazdagi
d169b490fa
Fix ineffectual assignments (#7403)
* update rpc/beacon
* more fixes to beacon-chain/rpc
* update beacon-chain/sync
* Merge refs/heads/master into fix-ineffectual-assignments
* updates beacon-chain/p2p
* Merge branch 'fix-ineffectual-assignments' of github.com:prysmaticlabs/prysm into fix-ineffectual-assignments
* update beacon-chain/*
* fix imports
* update beacon-chain/blockchain
* more updates
* Merge refs/heads/master into fix-ineffectual-assignments
* Merge branch 'master' into fix-ineffectual-assignments
* Merge refs/heads/master into fix-ineffectual-assignments
* next round of updated
* Merge branch 'fix-ineffectual-assignments' of github.com:prysmaticlabs/prysm into fix-ineffectual-assignments
* wrap up remaining items
2020-10-01 18:53:36 +00:00
terence tsao
f9e062407f
Fix an edge case when loading replay start state by slot (#7313)
* Ensure start state is the parent of the first block
* Regression test
* Merge branch 'master' into state-by-slot-state-state
* Add back the line
* Merge branch 'state-by-slot-state-state' of github.com:prysmaticlabs/prysm into state-by-slot-state-state
* Add a test for lower slot block condition
2020-09-23 21:39:16 +00:00
Radosław Kapka
dca93ce641
Unnecessary Slice-to-Slice Conversion analyzer (#7321)
* analyzer with tests
* fix bazel file
* modify analyzer to fix build issues
* add analyzer to tool chain
* remove arrays from inspections
* fix redundant [:] operator
* Merge branch 'master' into use-slice-directly
* Merge branch 'master' into use-slice-directly
* fix another inspection
* add package-level comment
2020-09-23 16:14:34 +00:00
Preston Van Loon
cebb62997d
Add beacon state unmarshal fuzzer, afl support (#6625)
* Add AFL third_party libraries

* add beacon state fuzzing, add afl fuzz bundle

* rm fuzzing engine

* fix and lint

* Check for array out of bounds when calculating proposer delta

* failing test

* fix

* Checkpoint progress

* Add requirement that inclusion distance is not zero, add regression test

* No need for HTR since that is covered in process slots

* Removing some fuzzit logic, old fuzz tests

* Add ssz encoder test and fix

* Fuzzing checkpoint, adding fuzzing to the p2p layer

* ignore some libfuzzer files

* Full testing of p2p processing of blocks, with some mocked stuff

* use tmpdir and always process blocks

* use checkptr

* Update ethereumapis

* go mod tidy

* benchmarks for ferran's fast ssz hash tree root

* Update fastssz

* fmt

* gaz

* goimports

* Fix

* fix ethereumapis

* fix again

* kafka

* fix gen file

* fix compute signing root

* gofmt

* checkpoint progress

* progress

* checkpoint

* updates

* updates

* merge fix

* WIP

* merge

* fix build

* fix merge related issues

* cleanup

* revert unrelated

* lint

* lint

* lint

* manual tags for fuzz

* Commentary on upload script

* some import fixes, but not all

* fix //fuzz:fuzz_tests

* rm unused test

* update generated ssz

* Set // +build libfuzzer

* remove debug code

* A bit of refactoring ot explain why there is a committee_disabled file

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-09-14 11:42:08 -07:00
terence tsao
fea2cc9e2f
Cleanup stategen pkg (#7127)
* Clean up stategen
* Sync with master
* Fix tests
* Merge refs/heads/master into cleanup-newstate
* Merge refs/heads/master into cleanup-newstate
* Rename file
* Merge branch 'cleanup-newstate' of github.com:prysmaticlabs/prysm into cleanup-newstate
* gaz
* Forgot to add metrics page back
2020-08-27 22:29:59 +00:00
Victor Farazdagi
bd489e345a
Fixes docs.go -> doc.go file name (#7128)
* fixes doc file name
* Merge refs/heads/master into fix-doc-file-name
2020-08-27 20:28:54 +00:00
Preston Van Loon
b7175b3482
Update fastssz: Attempt 2 (#7115)
* Revert "Revert "Update fastssz" (#7100)"

This reverts commit b954db9704.
* Preston's patch
* Merge branch 'master' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* Update fssz, add regression test case
* more HTR with fssz
* fix some tests
* only one test left
* Make it so that HTR will work
* gofmt, imports
* gofmt, imports
* fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* fix
* Merge branch 'master' into revert-7100-revert-6760-update-fssz
* Merge refs/heads/master into revert-7100-revert-6760-update-fssz
* gaz
* Merge branch 'revert-7100-revert-6760-update-fssz' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* Merge refs/heads/master into revert-7100-revert-6760-update-fssz
* fix test
* Merge branch 'revert-7100-revert-6760-update-fssz' of github.com:prysmaticlabs/prysm into revert-7100-revert-6760-update-fssz
* Merge refs/heads/master into revert-7100-revert-6760-update-fssz
2020-08-27 18:13:32 +00:00
Victor Farazdagi
ba07ccb484
Apply testutils assertions: final cleanup (#7003)
* slasher/beaconclient tests
* slasher/db/kv tests
* Merge branch 'master' into apply-testutils-assertions-to-slasher
* fix build
* slasher/detection tests
* rest of the tests
* misc tests
* tools tests
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge branch 'master' into apply-testutils-assertions-misc
* agg tests
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge branch 'master' into apply-testutils-assertions-misc
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* updates aggregated_test
* beacon-chain/operations/attestations/kv/* tests updated
* beacon-chain/operations/attestations tests updated
* beacon-chain/operations/slashings tests updated
* Merge branch 'master' into apply-testutils-assertions-misc
* gazelle
* beacon-chain/core tests updated
* fixes test
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* beacon-chain/rpc tests updated
* beacon-chain/sync/initial-sync tests
* misc tests
* optimizes error message parsing in testutils
* Merge branch 'assertutils-optimize-processing' into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* endtoend tests
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* gazelle
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* beacon-chain/blockchain tests updated
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* beacon-chain/state/stategen tests updated
* beacon-chain all left-over tests are done
* Merge refs/heads/master into apply-testutils-assertions-misc
* validator tests updated
* slasher tests
* Merge branch 'master' into apply-testutils-assertions-misc
* gofmt
* gazelle
* Merge refs/heads/master into apply-testutils-assertions-misc
* shared upd
* end2end tests deps fixed
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* misc
* all tests are updated
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
2020-08-25 15:23:06 +00:00
Nishant Das
e556ac348a
Add Shared Field Metrics (#7081)
* add metrics
* Merge branch 'master' into sharedFieldMetrics
* Merge branch 'master' into sharedFieldMetrics
* Merge branch 'master' into sharedFieldMetrics
* Merge branch 'master' into sharedFieldMetrics
2020-08-24 23:03:23 +00:00
terence tsao
b954db9704
Revert "Update fastssz" (#7100)
* Revert "Update fastssz (#6760)"

This reverts commit 78a25f99c3.
* Merge refs/heads/master into revert-6760-update-fssz
2020-08-24 20:06:28 +00:00
Preston Van Loon
78a25f99c3
Update fastssz (#6760)
* Update fastssz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fmt
* gaz
* Merge refs/heads/master into update-fssz
* goimports
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Fix
* fix ethereumapis
* fix again
* kafka
* fix gen file
* fix compute signing root
* gofmt
* checkpoint progress
* progress
* checkpoint
* progress
* Fix build
* checkpoint
* helpers
* Another test fixed
* gaz
* another test fix
* gofmt
* some fixes
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fix one test
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fill empty checkpoint roots
* more padding
* more padding
* Fix //beacon-chain/rpc/debug:go_default_test
* fix //beacon-chain/core/state:go_default_test
* fix //beacon-chain/core/state:go_default_test
* fix some htr errors
* fix //slasher/rpc:go_default_test
* Progress on //beacon-chain/core/blocks:go_default_test
* Progress on //beacon-chain/core/blocks:go_default_test
* Progress on //beacon-chain/core/blocks:go_default_test
* fix //slasher/db/kv:go_default_test
* progress
* fix //beacon-chain/sync/initial-sync:go_raceon_test
* gofmt and gaz
* fix one more test, taking a break
* Fix //beacon-chain/core/blocks:go_default_test
* Complete beacon-chain/powchain
* Do most of beacon-chain/rpc/beacon/
* Do most of beacon-chain/blockchain
* fix //beacon-chain/operations/attestations/kv:go_default_test
* Fix //beacon-chain/cache/depositcache:go_default_test
* Fix //slasher/detection:go_default_test
* Progress
* fix //beacon-chain/rpc/validator:go_default_test
* gofmt
* fix //validator/client:go_default_test
* fix
* fix //beacon-chain/blockchain:go_raceoff_test
* fix //beacon-chain/rpc/beacon:go_default_test
* fix 1 of 4 shards in //beacon-chain/sync:go_default_test
* Fix //beacon-chain/sync:go_default_test and gofmt
* prevent panic
* fix //beacon-chain/state/stategen:go_default_test
* fix
* Merge branch 'master' of github.com:prysmaticlabs/prysm into update-fssz
* fix most tests
* Self review, go mod tidy, run regen scripts
* fix slasher
* Update ethereumapis
* disable spawn strategy override
* Merge refs/heads/master into update-fssz
* Merge refs/heads/master into update-fssz
* Remove extra line in imports
* Remove extra line in imports
* Gofmt
* PR feedback from @nisdas
2020-08-24 01:46:17 +00:00
terence tsao
ec1dd85c44
Fix identical expressions (#7069)
* Fix ident exps
* Merge refs/heads/master into fix-ident-exp
2020-08-20 22:34:52 +00:00
terence tsao
e93d31c60e
Use hot state cache for replay (#7063) 2020-08-20 08:53:10 -07:00
terence tsao
8bdf9db147
Add replay block count metric (#7060) 2020-08-20 06:20:26 -07:00
terence tsao
dbd1e8c247
Only cache check point state if it hasn't existed in other cache (#7037)
* Only cache epoch boundary state in the event of skipped epoch boundary slot

* checkpoint

* remove lods

* checkppoint again

* Cache: fill if empty

* Don't include the sync fixes

* Update sync utils

* Avoid extra process slots

Co-authored-by: nisdas <nishdas93@gmail.com>
2020-08-17 16:20:34 -05:00
Victor Farazdagi
74101612ce
Moves AssertLogsContains to assertions package (#6985)
* first working implementation
* assertions tests
* adds to requires
* merges assert and require tests into a single suite
* gazelle
* Merge branch 'merge-assert-require-tests' into assert-logs-contains-move-to-assertions
* gazelle
* updates references
* fixes build issue
* Merge branch 'master' into assert-logs-contains-move-to-assertions
* Merge refs/heads/master into assert-logs-contains-move-to-assertions
* Merge branch 'master' into assert-logs-contains-move-to-assertions
* fixes build issue
* Merge branch 'assert-logs-contains-move-to-assertions' of github.com:prysmaticlabs/prysm into assert-logs-contains-move-to-assertions
* Merge refs/heads/master into assert-logs-contains-move-to-assertions
* Merge refs/heads/master into assert-logs-contains-move-to-assertions
2020-08-13 16:22:25 +00:00
terence tsao
537fe3f721
Fix loading historical blocks to replay cold state (#6956)
* Replay: cold state only uses finalized blocks
* Replay: regression test
* Merge refs/heads/master into fix-finalized-replay
2020-08-10 23:35:28 +00:00
rkapka
1f21e196b6
Fix multiple anti-patterns from DeepSource analysis (#6951)
* Merge variable declaration and assignment
* Use result of type assertion to simplify cases
* Replace call to bytes.Compare with bytes.Equal
* Drop unnecessary use of the blank identifier
* Replace x.Sub(time.Now()) with time.Until(x)
* Function literal can be simplified
* Use a single append to concatenate two slices
* Replace time.Now().Sub(x) with time.Since(x)
* Omit comparison with boolean constant
* Omit redundant nil check on slices
* Nested if can be replaced with else-if
* Function call can be replaced with helper function
* Omit redundant control flow
* Use plain channel send or receive
* Simplify returning boolean expression
* Merge branch 'origin-master' into fix-antipatterns
* Merge branch 'master' into fix-antipatterns
2020-08-10 16:16:45 +00:00
rkapka
33105f1bbc
Simplify make call by omitting redundant arguments (#6949)
* removed redundant args
* Merge refs/heads/master into omit-redundant-make-args
2020-08-10 11:06:53 +00:00
Nishant Das
78ec783604
Add New Validator Map Handler (#6910)
* add new method
* Merge branch 'master' into addMapHandler
* comments
* Merge branch 'addMapHandler' of https://github.com/prysmaticlabs/geth-sharding into addMapHandler
* Update beacon-chain/state/types.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* Merge refs/heads/master into addMapHandler
* gofmt
2020-08-07 01:42:50 +00:00
rkapka
e492343ffd
Code quality issues - Remove unused code (#6864)
* remove unused code

* gofmt

* remove junk file

* bring back metrics-related functions

* bring back keymanager's code
2020-08-05 13:59:46 +02:00
Shay Zluf
0d6fb72496
Slasher protection race condition fix (#6871)
* Slasher lock on update
* gaz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into slasher_lock_on_update
* goimports
* line space
* ivan feedback
* Merge refs/heads/master into slasher_lock_on_update
2020-08-04 19:50:23 +00:00
terence tsao
ee7da0d451
ForceCheckpoint return early when zero hashes (#6852)
* Setter: return early with zero hashes
* Test: zero hashes don't panic
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge refs/heads/master into zero-hashes
* Merge refs/heads/master into zero-hashes
* Merge refs/heads/master into zero-hashes
2020-08-03 18:34:54 +00:00
Shay Zluf
6582014b80
Fix deposit data copy (#6847)
* fix deposit data copy

* goimports
2020-08-03 16:25:19 +03:00
Preston Van Loon
06ee5695fb
Save finalized checkpoint on beacon chain close (#6431)
* Save finalized checkpoint on save

* gofmt and goimport

* remove head.root arg

* test

* fixes

* remove comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-28 13:35:17 -05:00
Nishant Das
2cd359bde5
preset memory (#6722) 2020-07-25 11:54:54 -07:00
terence tsao
0b9212c685
Revert "Optimize AppendValidator for beacon state" (#6710)
* Revert "Optimize `AppendValidator` for beacon state (#6659)"

This reverts commit c41e382255.
* Test. H/t to @prestonvanloon
* Merge branch 'master' of github.com:prysmaticlabs/prysm into revert-6659-deposit
* Merge branch 'master' into revert-6659-deposit
2020-07-24 05:43:15 +00:00
terence tsao
cc773a1641
Rm unused HashTreeRootState (#6697)
* Remove hash tree root implementations

* Delete benchmark tests

* Update existing tests

* Gaz

* Remove outdated ssz cache fuzz test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-23 17:19:05 -05:00
Preston Van Loon
2c11fcb242
Stategen: always invalidate hot cache state in StateByRootInitialSync (#6698)
* Stategen: always invalidate hot cache state in StateByRootInitialSync
* Sanity test. @nisdas feedback
* Merge refs/heads/master into defer-invalidate
2020-07-23 04:25:50 +00:00
terence tsao
c41e382255
Optimize AppendValidator for beacon state (#6659) 2020-07-21 12:58:30 -07:00
Raul Jordan
bc16fa9f50
Accounts V2: Derived Keymanager, Wallet & Account Creation (#6624)
* initialize derived wallet
* derived wallet + account creation
* initialize wallet seed
* encrypted seed file creation
* generate next acct
* create seed from pass
* properly creating derived accounts
* fix up formatting
* prep for review
* start tests for derived create account
* add derived test
* linter
* gaz
* derived keymanager create account test complete
* Merge branch 'master' into derived-keymanager
* tests pass
* gaz
* fix list test
* Merge refs/heads/master into derived-keymanager
* ivan feedback
* skip mnemonic confirm
* Merge branch 'derived-keymanager' of github.com:prysmaticlabs/prysm into derived-keymanager
* comment
* tidy
* fmt
* organize
* test interface conformity
* Update validator/accounts/v2/iface/wallet.go
* ivan comments
* Merge branch 'derived-keymanager' of github.com:prysmaticlabs/prysm into derived-keymanager
* Merge refs/heads/master into derived-keymanager
* Merge branch 'master' of github.com:prysmaticlabs/prysm into derived-keymanager
* Fix
* Fix test
* Merge refs/heads/master into derived-keymanager
* fix errs
* imports
* Gaz
2020-07-21 02:05:23 +00:00
Preston Van Loon
c419e4ed8f
Improved cold state checkpoints: migrate database index and usage (#6461)
* Add database migrations, still need to update the API usage...
* gofmt goimports
* progress
* Merge branch 'master' of github.com:prysmaticlabs/prysm into index-migration
* use slot instead of index
* rename LastArchivedIndex to LastArchivedSlot
* rename LastArchivedIndexRoot to LastArchivedRoot
* remove unused HighestSlotStates method
* deprecate old key, include in migration
* deprecate old key, include in migration
* remove blocks index in migration
* rename bucket variable
* fix code to pass tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into index-migration
* gofmt, goimports
* fix
* Add state slot index
* progress
* lint
* fix build
* Merge branch 'master' of github.com:prysmaticlabs/prysm into index-migration
* kafka
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* remove SaveArchivedPointRoot, a few other big changes
* Merge branch 'index-migration' of github.com:prysmaticlabs/prysm into index-migration
* fix tests and lint
* lint again
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* block migration, some renaming
* gaz, gofmt
* add tests
* change index to uint bytes
* Merge branch 'index-migration' of github.com:prysmaticlabs/prysm into index-migration
* rm method notes
* stop if the bucket doesn't exist
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* @rauljordan pr feedback
* Simplify
* Merge refs/heads/master into index-migration
* Remove unused method, add roundtrip test
* gofmt
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge refs/heads/master into index-migration
* Merge branch 'master' of github.com:prysmaticlabs/prysm into index-migration
2020-07-18 18:05:04 +00:00
Victor Farazdagi
7ba500c417
Applies assertion funcs to state tests (#6623)
* applies assertion funcs to beacon-chain/state tests
* applies assertion funcs to beacon-chain/state/stategen tests
* applies assertion funcs to beacon-chain/state/stateutil tests
* gazelle
* Merge refs/heads/master into state-apply-testutils-assertions
* Merge refs/heads/master into state-apply-testutils-assertions
* Merge refs/heads/master into state-apply-testutils-assertions
2020-07-18 07:56:48 +00:00
terence tsao
62df4995e6
Clean up block chain pkg initial sync (#6562) 2020-07-12 20:44:06 -07:00
terence tsao
d54cefbe42
Part 1 of blockchain pkg clean up (#6537)
* Fix span names

* Clean up on block func

* Add updateFinalized helper

* Deprecate ReceiveBlockNoPubsub for ReceiveBlock

* Proposer to broadcast block

* Update migrate interface

* Replace ReceiveBlock for all

* Go fmt
2020-07-09 18:50:48 -05:00
Preston Van Loon
c2deab8948
State: use a constant for field count (#6513)
* State: use a constant for field count
* one more field count for alloc
* one more field count for alloc
* Merge branch 'master' into field-const-21
* Merge branch 'master' into field-const-21
* Merge branch 'master' into field-const-21
* Merge branch 'master' into field-const-21
2020-07-09 03:37:08 +00:00
Victor Farazdagi
af4dfd4c36
Tiny optimization in state refs lock/unlock (#6520)
* tiny optimization in state refs count
2020-07-08 12:53:36 +00:00
Nishant Das
074e3c9aa7
Use Custom Block Hashing Method (#6501)
* blockroot
* gaz
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
* Merge refs/heads/master into useCustomMethod
2020-07-08 03:53:16 +00:00
Preston Van Loon
90b8b76ae8
Remove unused EpochAttestationsRoot method (#6514)
* Remove unused EpochAttestationsRoot method
2020-07-07 23:39:11 +00:00
Ivan Martinez
d9fea7cb8c
[QSP] Deduplicate functions for audit best practices (#6476)
* Deduplicate DB functions and getters

* Rename function

* Reduce to prestons suggestions

* Fix bug

* Fix

* Start changing state db funcs

* Fix tests

* Change return

* Add checks

* Fix test

* Fix

* Nishant comment

* Fix

* Fix

* Fix

* Fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-07-07 11:29:25 -05:00
Nishant Das
b868dfa481
Save Archived State (#6503) 2020-07-07 08:04:55 -07:00
Nishant Das
64fa474434
Batch Verify Blocks (#6469)
* add everything so far
* checkpoint progress
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* fix
* checkpoint
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* checkpoint again
* checkpoint again
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* commenting
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* bls cleanup
* revert this back
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* revert core changes
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* add flag
* add test
* add one more test
* clean up
* comment
* lint
* terence's review
* Merge refs/heads/master into fastBLS
* Merge refs/heads/master into fastBLS
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* Merge refs/heads/master into fastBLS
* remove additional method
* Merge branch 'fastBLS' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
* fix
* Merge refs/heads/master into fastBLS
* copy
* Merge branch 'fastBLS' of https://github.com/prysmaticlabs/geth-sharding into fastBLS
2020-07-07 04:16:12 +00:00
Jim McDonald
7bb0ee78af
Update beacon state locks (#6326)
* Rationalise state locking
* Rationalise state locking
* Merge branch 'state-locks' of github.com:mcdee/prysm into state-locks
* Merge branch 'master' into state-locks
* Add feature flag
* Merge
* Merge branch 'master' into state-locks
* Merge branch 'master' into state-locks
* Update locks
* Merge branch 'master' into state-locks
* Gazelle
* Tidy-ups
* Merge branch 'master' into state-locks
* Remove commentary to a docs.go file for better presentation on godocs
* Add newBeaconStateLocks as a --dev flag
* gofmt
* Merge branch 'master' into state-locks
2020-07-06 20:52:53 +00:00
terence tsao
def7b602e3
Hot states use no DB (#6488)
* Add cache to service struct
* Update hot getters/setters to use cache
* Update migration
* Update other services to adapt
* Fix initial sync get state
* Update getter related tests
* Update hot related tests
* Update migrate related tests
* New awesome tests for migration
* Clean up rest of the tests
* Merge refs/heads/master into hot-state-no-db
* Fix block chain head tests
* Fix block chain processor tests
* Fixed RPC tests
* Update cold getter and test
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
* Fix sync tests
* Short cut if state is already in DB
* Remove uneeded saves
* Update beacon-chain/state/stategen/hot_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/getter_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/getter_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/service.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/setter_test.go

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Preston's feedback
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
* Return a copy of cache states
* Remove hot state caches check in StateByRoot
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
* Merge refs/heads/master into hot-state-no-db
* Raul's feedback
* Merge branch 'hot-state-no-db' of github.com:prysmaticlabs/prysm into hot-state-no-db
2020-07-06 17:22:12 +00:00
terence tsao
8bb5532377
Epoch boundary cache (#6486)
* Add epoch boundary state cache
* Clean up epoch boundary state cache
* Starting tests
* More tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into epoch-boundary-cache
* Merge refs/heads/master into epoch-boundary-cache
2020-07-03 17:29:30 +00:00
terence tsao
d62e989a67
Minor clean up on state gen package (#6480)
* Remove unused fields and function
* Rename splitInfo to finalizedInfo
* Merge branch 'master' into stategen-cleanup
* Fmt
* Merge branch 'stategen-cleanup' of github.com:prysmaticlabs/prysm into stategen-cleanup
* Merge refs/heads/master into stategen-cleanup
2020-07-03 06:33:58 +00:00
Raul Jordan
091525deb8
Revert Stream Duties (#6473)
* ensure gazelle
* fix up propose test
* build
* comment
2020-07-02 17:50:05 +00:00
terence tsao
43b1bcb84f
Use CopyValidator for ValidatorAtIndex (#6430)
* Use CopyValidator
* Merge refs/heads/master into validator-at-index
2020-06-26 23:21:11 +00:00
Raul Jordan
252f758baa
QSP-9 Prevent Casting to Int if Possible (#6349)
* no cast to int

* fix up significant casting issues

* more casting

* even more casting fixes

* more casts

* fix subnets

* back to ints

* final touches

* broken test fix

* add in blocks test fix

* unskip

* revert bytes fixes

* casting fixes

* Update beacon-chain/db/kv/state.go

* Update beacon-chain/db/kv/blocks.go

* fmt

* slash:

* fix val tests

* fix up conf

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
2020-06-26 11:07:00 -05:00
Raul Jordan
0b178b446a
QSP-64 Conditional for Rebuilding BlockRoots Trie in State Setters (#6390)
* rebuild trie for block roots
* Merge refs/heads/master into rebuild-trie-fix
* Merge refs/heads/master into rebuild-trie-fix
2020-06-25 02:03:18 +00:00
terence tsao
d0e2e0e979
Best practice feedback - part 1 (#6385)
* log error for metric
* reuse epoch e
* Better node comments
* Remove not needed breaks
* Use j over i
* Merge branch 'master' of github.com:prysmaticlabs/prysm into best-practice-pt1
* Descendent/Descendant
* Merge refs/heads/master into best-practice-pt1
* Update beacon-chain/forkchoice/protoarray/nodes.go
* Merge refs/heads/master into best-practice-pt1
* Merge refs/heads/master into best-practice-pt1
2020-06-24 21:09:47 +00:00
Nishant Das
24d09f083f
Release State Reference Copy (#6341)
* release flag
* add comment
* improve
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge branch 'master' into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into releaseStateRef
* remove old flags
* gaz
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
* Merge refs/heads/master into releaseStateRef
2020-06-24 00:24:59 +00:00
terence tsao
c3adde3b32
QSP10, QSP11 - Add proper locking (#6350)
* Proper locking based on audit feedbacks
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Preston's feedback
* Merge branch 'qsp10-11' of github.com:prysmaticlabs/prysm into qsp10-11
* Unlock in invalid index branch
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge branch 'master' into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
* Merge refs/heads/master into qsp10-11
2020-06-23 23:08:08 +00:00
Raul Jordan
e1310adb4c
QSP-Docs All Documentation Improvements from Quantstamp Audit Report (#6367)
* docs improvements so far
* all misc docs items
* finalized
* Merge branch 'master' into qsp-docs
* fix broken test
* Merge branch 'master' into qsp-docs
2020-06-23 21:11:20 +00:00
Preston Van Loon
f9038674d1
Refactor setupDB to return the state summary cache used by the database (#6369) 2020-06-23 13:40:55 -07:00
Victor Farazdagi
33b2636bf1
Enables new aggregation implementation (#6344)
* adds feature flag
* aggregations/attestation package
* better tests
* bazel visibility issues
* removes redundant code
* Merge branch 'master' into aggregation-attestations-package
* updates previous implementation
* Merge branch 'master' into aggregation-enable-new-aggregation
* resolve merge conflict
* resolve merge conflict (more)
* Merge branch 'master' into aggregation-enable-new-aggregation
* force max_cover aggregation on --dev
* Merge branch 'aggregation-enable-new-aggregation' of github.com:prysmaticlabs/prysm into aggregation-enable-new-aggregation
* Merge branch 'master' into aggregation-enable-new-aggregation
* Merge refs/heads/master into aggregation-enable-new-aggregation
* Merge refs/heads/master into aggregation-enable-new-aggregation
* Merge refs/heads/master into aggregation-enable-new-aggregation
* Merge refs/heads/master into aggregation-enable-new-aggregation
* Merge refs/heads/master into aggregation-enable-new-aggregation
2020-06-23 19:13:55 +00:00
terence tsao
c417b00675
Fix cold state replay to start at correct slot (#6361)
* Fix off by 1
* Regression tests
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-offset
* Go fmt
* Merge refs/heads/master into fix-offset
* Merge refs/heads/master into fix-offset
2020-06-23 16:04:42 +00:00
Victor Farazdagi
5c90038007
Adds aggregation/attestations package (#6343)
* adds feature flag

* aggregations/attestation package

* better tests

* bazel visibility issues

* removes redundant code

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-22 23:00:38 -05:00
terence tsao
c837dfb2f2
ActivationEligibilityEpoch nil case (#6347) 2020-06-22 14:41:21 -05:00
Nishant Das
3b2dc8346d
Release State Field Trie (#6340)
* remove flag and references
* fix flag
* Merge refs/heads/master into releaseStateFieldTrie
* Merge refs/heads/master into releaseStateFieldTrie
2020-06-22 15:25:55 +00:00
Preston Van Loon
d4545233cd
Fix race condition issues in beacon state (#6322)
* Fix race condition issues in beacon state
* Merge branch 'master' of github.com:prysmaticlabs/prysm into race-condition-fixes
* Checkout beacon-chain/state/stateutil/BUILD.bazel from master
* @nisdas PR feedback. defer unlock
2020-06-20 00:27:23 +00:00
Raul Jordan
7067c84c69
Stream Duties Client Implementation (#5867)
* include validator client stream

* Update validator/client/validator_attest.go

* gazelle

* rem extraneous logs

* fixing tests

* resolve most tests

* gaz

* add lock

* ivan feedback

* pass tests for update protect

* gaz

* duties gaz

* no need for canonical head slot

* fix ctx leak

* fmt

* add in feature flag

* add streaming subpackage

* add polling/streaming separation

* able to build

* fix duplicate package names

* fix polling

* imports

* confirm it works

* fixed up comment

* go lint comments

* gaz

* build

* Update validator/client/streaming/service_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* tidy

* fmt

* add stream duties to e2e

* add stream duties to e2e flags

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-06-18 13:30:05 -05:00
Paul Nelson
c1c3b75867
Move generic HashTreeRoot funcs to //shared (#6167)
* Functions that are to be moved to //shared/htrutils captured in separate file.

* Move files into new //shared/htutils package

hash_function.go renamed to hashers.go

* Refactor to reference moved methods in new package

- Added import for htrutils to state and stateutil packages
- Update references to imported funcs (append "htrutils.")
- Updated funcs in htrutils to be exported where necessary

* Add tests
2020-06-18 10:15:13 +08:00
Raul Jordan
2fb485598b
Resolve Nil Validator Panic (#6218)
* resolve proposer panic
* Merge refs/heads/master into panic-proposer-slashed
2020-06-11 17:38:03 +00:00
terence tsao
edeb359ded
Use SSZ HTR as baseline (#6207)
* Use ssz htr as baseline to compare
* Merge branch 'master' into fix-test
* One more. Fix for TestBlockBodyRoot_NilIsSameAsEmpty
* Merge branch 'fix-test' of github.com:prysmaticlabs/prysm into fix-test
2020-06-10 17:51:06 +00:00
Nishant Das
d1a1043ef1
Update to v0.12 (#5614) 2020-06-09 15:40:48 -07:00
Ivan Martinez
1022d9dba8
Add AttestationDataRoot to stateutil (#6119)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-06-04 13:59:41 -05:00
Raul Jordan
75aaa5e07b
fix bad root (#6096) 2020-06-03 11:26:07 -05:00
terence tsao
6e514b96fd
Remove unused code (#6065)
* Ran code cleanup from goland

* Typo

* Remove unused code

* Gazelle

* Delete comments
2020-05-31 15:47:15 -07:00
Preston Van Loon
49a0d3caf0
Refactor dependencies, make Prysm "go gettable" (#6053)
* Fix a few deps to work with go.mod, check in generated files

* Update Gossipsub to 1.1 (#5998)

* update libs

* add new validators

* add new deps

* new set of deps

* tls

* further fix gossip update

* get everything to build

* clean up

* gaz

* fix build

* fix all tests

* add deps to images

* imports

Co-authored-by: rauljordan <raul@prysmaticlabs.com>

* Beacon chain builds with go build

* fix bazel

* fix dep

* lint

* Add github action for testing go

* on PR for any branch

* fix libp2p test failure

* Fix TestProcessBlock_PassesProcessingConditions by updating the proposer index in test

* Revert "Fix TestProcessBlock_PassesProcessingConditions by updating the proposer index in test"

This reverts commit 43676894ab01f03fe90a9b8ee3ecfbc2ec1ec4e4.

* Compute and set proposer index instead of hard code

* Add back go mod/sum, fix deps

* go build ./...

* Temporarily skip two tests

* Fix kafka confluent patch

* Fix kafka confluent patch

* fix kafka build

* fix kafka

* Add info in DEPENDENCIES. Added a stub link for Why Bazel? until https://github.com/prysmaticlabs/documentation/issues/138

* Update fuzz ssz files as well

* Update fuzz ssz files as well

* getting closer

* rollback rules_go and gazelle

* fix gogo protobuf

* install librdkafka-dev as part of github actions

* Update kafka to a recent version where librkafkfa is not required for go modules

* clarify comment

* fix kafka build

* disable go tests

* comment

* Fix geth dependencies for end to end

* rename word

* lint

* fix docker

Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-05-31 14:44:34 +08:00
terence tsao
4c2ed410b0
Invert state ref copy flag (#6015)
* Revert "Release state-ref-copy flag (#6012)"

This reverts commit ebf7122d6f.

* Invert state ref copy flag
2020-05-27 14:30:54 -05:00
terence tsao
ebf7122d6f
Release state-ref-copy flag (#6012)
* Release state ref copy flag
* Merge refs/heads/master into invert-flag1
2020-05-27 18:08:12 +00:00
Nishant Das
6a9112bfae
remove byte mempool (#5951) 2020-05-21 20:39:06 -07:00
terence tsao
8eb7c7a0bc
Recover skipped archived point (#5950) 2020-05-21 18:15:16 -07:00
Shay Zluf
17169e5a2d
External slashing protection (#5895)
* slasher grpc client

* do not export

* slasher on a different package

* fix featureconfig

* change to rough time

* revert roughtime

* remove extra comma

* revert order change

* goimports

* fix comments and tests

* fix package name

* revert reorder

* comment for start

* service

* fix visibility

* external slasher validator protection implementation

* gaz

* fix comment

* add comments

* nishant feedback

* raul feedback

* preston feedback

* fix flags

* fix imports

* fix imports

* port 4002

* added tests

* fix log

* fix imports

* fix imports name

* raul feedback

* gaz

* terence comment

* change name

* runtime fixes

* add flag check

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-20 10:23:22 -05:00
terence tsao
678347e64b
Handle missing archived index (#5899)
* Make `archivedRoot` handle missing root case
* Regression tests
* Fixed existing tests
* Removed debug log
* Merge refs/heads/master into handle-missing-archived-index
* Merge refs/heads/master into handle-missing-archived-index
* More comments on the look back
* Merge branch 'handle-missing-archived-index' of github.com:prysmaticlabs/prysm into handle-missing-archived-index
* Merge refs/heads/master into handle-missing-archived-index
2020-05-18 19:46:06 +00:00
terence tsao
5515c20ffd
Add skip-regen-historical-states flag (#5871)
* Add feature flag

* Comments to clairfy a few things

* Use feature flag

* Update state gen resume

* More comments

* Remove from dv
2020-05-15 14:49:13 -05:00
Ivan Martinez
52c302307a
Remove out of date scripts and fix mockgen script (#5859) 2020-05-14 17:11:52 -05:00
terence tsao
2f27966b44
Fix state gen migration (#5858)
* Remove useless `currentSplitSlot == 0`
* Increase slots per archived point to 2048
* Don't delete last archived state
* Revert back to 256
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-migration-delete-latest-state
* Don't delete last archived state
* Regression test
* Revert
2020-05-14 20:03:14 +00:00
Ivan Martinez
7a8e85af9e
Add testing for EnableFieldTrie in state_trie.go (#5820)
* Add testing for EnableFieldTrie
* Merge branch 'master' into test-field-trie
* Merge branch 'master' into test-field-trie
* Merge refs/heads/master into test-field-trie
* goimports
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Merge refs/heads/master into test-field-trie
* Add tests for generic HTR
* Merge refs/heads/master into test-field-trie
2020-05-14 17:38:34 +00:00
Ivan Martinez
946a2caafd
Add fix for nil state in InitializeFromProto (#5817)
* Add nil check and tests
* Add unsafe test
* Update tools/genesis-state-gen/main.go
* Undo genesis state gen changes
* Merge branch 'proto-fuzz-fix' of github.com:prysmaticlabs/prysm into proto-fuzz-fix
* gaz
* Fix
2020-05-12 00:23:05 +00:00
terence tsao
34f28a31b1
No magic number for BlockBodyRoot (#5803)
* `GetValidatorPerformance` return `Unavailable` when syncing

* Use proper config values for BlockBodyRoot

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-11 11:45:31 +08:00
Preston Van Loon
b00bbfeec7
Release enableCustomBlockHTR to all (#5742)
* Release deprecatedEnableCustomBlockHTR to all

* Release deprecatedEnableCustomBlockHTR to all

* Fix builds

* Treat nil body as empty

* Use custom HTR in signing root

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-08 13:50:39 -07:00
terence tsao
7a27d89e68
Reduce memory usage for new-state-mgmt at init sync (#5778)
* Add `GetWithoutCopy`
* Add `StateByRootInitialSync`
* Use `StateByRootInitialSync` for initial syncing using new-state-mgmt
* Merge branch 'master' into new-state-init-sync-mem
* Skip if split slot is 0
* Merge branch 'new-state-init-sync-mem' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
* Make sure we invalidate cache
* Add test part 1
* Add tests part 2
* Update beacon-chain/cache/hot_state_cache.go

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
* Comment
* Merge branch 'new-state-init-sync-mem' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
* Merge branch 'master' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
* Dont need to run fork choice and save head during intial sync
* Invalidate cache at onBlockInitialSyncStateTransition
* Merge branch 'new-state-init-sync-mem' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
* Revert saveHeadNoDB changes
* Add back DeleteHotStateInCache
* Removed extra deletion
* Merge branch 'master' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
* Proper set splitslot for tests
* Merge branch 'master' into new-state-init-sync-mem
* Merge branch 'master' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
* Merge branch 'new-state-init-sync-mem' of github.com:prysmaticlabs/prysm into new-state-init-sync-mem
2020-05-08 19:02:48 +00:00
Nishant Das
6fed51d380
Don't Copy The Whole Validator Map each time (#5780)
* fix copying issues

* revert some changes
2020-05-08 18:14:18 +08:00
Ivan Martinez
9a1157465e
Change non-mutating validator accesses to ReadOnly (#5776)
* Change instances of ValidatorAtIndex to ReadOnly where possible

* Use ReadOnly for VerifyExit and Slashings

* Move length check to before lock

* Improve readonly tests

* undo process attester changes

* Fix test
2020-05-07 14:15:51 -04:00
terence tsao
ef4dead3fe
Fix cold state generation (#5770) 2020-05-06 13:25:37 -07:00
Victor Farazdagi
bbde2a6820
Assert no side effects occur on test (#5714)
* add reset funcs
* init -> TestMain for beacon config params
* fixes build file
* core/epoch spectests updated
* core/blocks spectests updated
* fix the rest of spectests
* SetupTestConfigCleanup
* revert copy() api
* rever all cases of copy()
* fixes spectests
* updates fork_test
* config_test update
* fixes spectest/config_test
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge branch 'master' into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge branch 'master' into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
* Merge refs/heads/master into assert-no-side-effects-occur-in-tests
2020-05-05 08:39:38 +00:00
Preston Van Loon
eb2abbdd8b
libfuzz based tests (#5095)
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* use opt for fuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* use ubuntu as the default build image
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Move fuzz tests
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* making progress on p2p fuzz
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* use a single stream repeatedly
* use a single stream repeatedly
* use many streams
* fixes
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Add basic test for ssz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge branch 'master' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Use a custom go_library to add libfuzzer
* delete old corpus, fix rpc_status_fuzz
* move fuzz.bzl
* minor fixes
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* add c-shared library (doesnt work)
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* fix build
* fixes
* Merge refs/heads/master into go-1.14-libfuzz
* lint
* more lint
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* more lint
* allow environment variable
* Merge refs/heads/master into go-1.14-libfuzz
* Add a fuzzit image with the beacon_states
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* lint
* readme
* lint
* fix lint again i think
* gaz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* breakup deps
* Merge branch 'go-1.14-libfuzz' of github.com:prysmaticlabs/prysm into go-1.14-libfuzz
* don't panic on failure
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* panic if no beacon states to read
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* update rules_foreign_cc
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* remove c_shared target, manually tag tests
* Add readme to fuzz package
* delete a few comments
* delete a few comments
* lint
* gaz
* gaz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
* Merge refs/heads/master into go-1.14-libfuzz
2020-05-05 07:22:26 +00:00
Preston Van Loon
eb27f35982
Use the custom HTR in a few more places (#5743) 2020-05-05 14:42:53 +08:00
Preston Van Loon
9be8a456eb
Use stateutil.BlockRoot everywhere (#5739)
* Sweeping replace of ssz.HashTreeRoot(block) to stateutil.BlockRoot, minor tweaks to critical path of proposeBlock
* imports
* Merge branch 'master' into better-block-htr
* Add --enable-custom-block-htr to e2e flags
* Merge branch 'better-block-htr' of github.com:prysmaticlabs/prysm into better-block-htr
2020-05-05 04:30:24 +00:00
Victor Farazdagi
140aff4398
fixes runtime panic in state/getters (#5728)
* fixes runtime panic in state/getter
* Merge branch 'master' into check-non-nil-in-state-getter
* return empty map
* Merge branch 'check-non-nil-in-state-getter' of github.com:prysmaticlabs/prysm into check-non-nil-in-state-getter
2020-05-04 10:15:33 +00:00
Victor Farazdagi
b5c4dc2a75
Refactor db teardown to testing.TB.Cleanup (#5725)
* init-sync updates
* slasher/db/kv tests
* beacon-chain/rpc/beacon tests
* update kv_test
* beacon-chain/rpc-validator tests updated
* slasher/db/kv - remove teardown method
* beacon-chain/sync tests updated
* beacon-chain/db/kv tests updated
* beacon-chain/blockchain tests updated
* beacon-chain/state/stategen tests updated
* beacon-chain/powchain updates
* updates rest of slasher tests
* validator/db tests
* rest of the tests
* minor comments update
* gazelle
* Merge refs/heads/master into teardowndb-to-cleanup
2020-05-04 01:14:34 +00:00
Nishant Das
2c3f974361
Custom Method For Block Signing Root (#5704)
* custom block signing root

* add comment and test

* Update beacon-chain/core/helpers/signing_root.go

* Update beacon-chain/core/helpers/signing_root.go

* fix changes
2020-05-01 03:37:37 -04:00
Nishant Das
7bea237397
Revert Validator Reference Copy (#5701)
* revert all changes
* revert test
* unreachable code
* Merge refs/heads/master into revertValidatorRefCopy
* Merge refs/heads/master into revertValidatorRefCopy
2020-05-01 02:48:46 +00:00
Raul Jordan
3416962fc2
All Beacon Node Package-Level Godocs (#5677)
* package level godocs fixed

* all beacon node godocs

* comment and gaz

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-29 12:40:33 -05:00
Victor Farazdagi
44b93d9df9
Fix tests depending on configuration flag (#5650)
* fixes segfault
* Merge branch 'master' into fix-nil-segfaults
* upd state
* Merge branch 'master' into fix-nil-segfaults
* updates shard=1
* Merge refs/heads/master into fix-nil-segfaults
* Merge refs/heads/master into fix-nil-segfaults
* Merge refs/heads/master into fix-nil-segfaults
* fixes tests
* Merge branch 'fix-nil-segfaults' of github.com:prysmaticlabs/prysm into fix-nil-segfaults
* reset config to nil
* add flag assertion to other beacon-chain/rpc files
* Merge refs/heads/master into fix-nil-segfaults
* fix the rest featureconfig.init calls
* Merge branch 'fix-nil-segfaults' of github.com:prysmaticlabs/prysm into fix-nil-segfaults
* remove config setting from init
* gazelle
* reset shard back to 4
2020-04-27 19:44:35 +00:00
Nishant Das
e9191e7d6c
Create Pcli (#5644)
* add new utility
* Merge branch 'master' into pCli
* Update tools/pcli/main.go

Co-Authored-By: Victor Farazdagi <simple.square@gmail.com>
* Update tools/pcli/main.go

Co-Authored-By: Victor Farazdagi <simple.square@gmail.com>
* Merge refs/heads/master into pCli
* readme and docker fix
* readme
* Merge refs/heads/master into pCli
* Merge refs/heads/master into pCli
* Merge refs/heads/master into pCli
* pending feedback
* builds
* Merge refs/heads/master into pCli
* Merge refs/heads/master into pCli
2020-04-27 16:47:28 +00:00
Nisen
7d7480527e
Missing defer before unlock (#5643)
* missing defer before unlock
2020-04-27 11:20:09 +00:00
Nishant Das
482a054ee1
Handle Nil State Object (#5575)
* handle panic
* Merge branch 'master' into handlePanic
* Merge refs/heads/master into handlePanic
* Merge refs/heads/master into handlePanic
2020-04-22 04:44:33 +00:00
terence tsao
480ddb7c3d
Continue if archival state is not in DB (#5565)
* Continue if archival state is not availble to generate
* Fixed test
* Merge branch 'master' into save-blocks-splitslot
* Merge refs/heads/master into save-blocks-splitslot
2020-04-21 18:18:55 +00:00
terence tsao
199c50be47
Migration should not fail process block routine (#5557)
* Continue rather than fail
* Merge branch 'master' into migrate-should-not-fail-block
* Merge refs/heads/master into migrate-should-not-fail-block
2020-04-21 14:44:43 +00:00
terence tsao
2a0711ab01
Regen and save historical states as default (#5544)
* Better error
* Use copy
* Conflict
* Fixed a bug on using pre state. Added a LRU cache for saving pre state.
* Removed interaction menu. Ran Gazelle
* Fixed delete state to continue than exit
* Gazelle
* Merge branch 'master' of github.com:prysmaticlabs/prysm into regen-default
* Merge refs/heads/master into regen-default
* Added a warning message
* Merge branch 'regen-default' of github.com:prysmaticlabs/prysm into regen-default
* Merge refs/heads/master into regen-default
2020-04-20 20:19:53 +00:00
Raul Jordan
37cba662f1
Run Gofmt on Repo (#5541)
* toggle flags
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* run gofmt
* Merge branch 'master' into lint-repo
2020-04-20 16:35:04 +00:00
Nishant Das
626b3e0c66
Add Reference Copying (#5245)
* add state changes
* add new changes
* add flag
* lint
* add flag
* change to correct bool
* fixing and consolidating trie
* lint
* Apply suggestions from code review
* Merge refs/heads/master into addBetterCopying
* Merge branch 'master' into addBetterCopying
* refCopy -> stateRefCopy
* Merge refs/heads/master into addBetterCopying
* tests whether unexpected mutation of validators within state is avoided
* Merge branch 'addBetterCopying' of github.com:prysmaticlabs/prysm into addBetterCopying
* remove unnecessary fields
* gazelle
* updates test
* avoid unexpected mutation in block roots on refcopy
* avoid unexpected mutation in state roots on refcopy
* Merge refs/heads/master into addBetterCopying
* Merge branch 'master' into addBetterCopying
* fix test
* randao tests
* simplify tests
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* test cur/prev attestations mutation
* Merge branch 'addBetterCopying' of github.com:prysmaticlabs/prysm into addBetterCopying
* gazelle
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* fixes tests
* minor naming update
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
* Merge refs/heads/master into addBetterCopying
2020-04-19 13:57:43 +00:00
terence tsao
c3217abb83
Save init sync cached blocks before regenerate state (#5501)
* Revert "Add WaitForSynced to beacon node for validator startup (#5366)"

This reverts commit 1224e75ba1.
* Merge branch 'master' of github.com:prysmaticlabs/prysm
* Revert "Add Separate Network Config (#5454)"

This reverts commit 28733f2c9e.
* Save blocks to db before generating state
* Revert "Revert "Add Separate Network Config (#5454)""

This reverts commit d43c2b7dbdac58e3d7b38c0c6b7b2f1fdbd99828.
* Revert "Revert "Add WaitForSynced to beacon node for validator startup (#5366)""

This reverts commit 927d8d9e06ab700e10c5848eb46def6b40f1ecfa.
2020-04-18 23:29:58 +00:00
terence tsao
62213ca602
Retrieve last ancestor state from DB. (Radical improvement from 5466) (#5475)
* Add `lastAncestorState` method
* Add new tests and fix existing test
* Check context deadline exceeds
* Merge refs/heads/master into last-ancestor-state
* Merge refs/heads/master into last-ancestor-state
2020-04-18 01:04:25 +00:00
terence tsao
50d67f52bb
Replay and generate state using proper parent root (#5466)
* Start replay using parent root
* Merge branch 'master' into fix-replay-bug
* Fixed TestStateByRoot_HotStateDB
* Merge branch 'fix-replay-bug' of github.com:prysmaticlabs/prysm into fix-replay-bug
* Fixed TestOnAtts
* Update beacon-chain/state/stategen/getter_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/state/stategen/getter_test.go
* Fixed TestLoadHoteStateByRoot_FromDBCanProcess
* Merge branch 'fix-replay-bug' of github.com:prysmaticlabs/prysm into fix-replay-bug
2020-04-17 02:06:23 +00:00
Victor Farazdagi
0ecd2a6dc2
updates UpdateRandaoMixesAtIndex signature (#5447)
* updates signature
* Merge branch 'master' into fix-update-randao-mixes-signature
2020-04-15 20:35:19 +00:00
terence tsao
cb045dd0e3
Align code base to v0.11 (#5127)
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* add in new patch and workspace
* update cloners
* Handle rewards overflow (#5122)

* Refactoring of initial sync (#5096)

* implements blocks queue

* refactors updateCounter method

* fixes deadlock on stop w/o start

* refactors updateSchedulerState

* more tests on schduler

* parseFetchResponse tests

* wraps up tests for blocks queue

* eod commit

* fixes data race in round robin

* revamps fetcher

* fixes race conditions + livelocks + deadlocks

* less verbose output

* fixes data race, by isolating critical sections

* minor refactoring: resolves blocking calls

* implements init-sync queue

* udpate fetch/send buffers in blocks fetcher

* blockState enum-like type alias

* refactors common code into releaseTicket()

* better gc

* linter

* minor fix to round robin

* moves original round robin into its own package

* adds enableInitSyncQueue flag

* fixes issue with init-sync service selection

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* initsyncv1 -> initsyncold

* adds span

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>

* Handle rewards overflow

* Revert "Refactoring of initial sync (#5096)"

This reverts commit 3ec2a0f9e0.

Co-authored-by: Victor Farazdagi <simple.square@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* updated block operations
* updated validator client
* Merge refs/heads/master into v0.10.1
* updated block operations test
* skip benchmark test
* updated transition test
* updated db kv tests
* updated ops tests
* updated ops tests
* updated slashing tests
* updated rpc tests
* updated state utils
* updated test utils and miscs
* Temp skips minimal spec tests
* Fixed proposer slashing test
* Gaz
* Skip 2 more minimal tests
* Skip 2 more minimal tests
* Update readme
* gaz
* Conflict
* Fix import and not use
* Update workspace for new spec test
* Fix workspace
* Merge refs/heads/master into v0.10.1
* Update workspace with new ethapi commit
* Unblock a few tests
* Merge refs/heads/master into v0.10.1
* fixed block op test
* gaz
* Merge refs/heads/master into v0.10.1
* Skip gen state test (test setup issue
* Updated hysteresis config
* Updated epoch processing for new hyteresis
* Updated tests
* regen proto beacon
* update state util for state root
* update state types
* update getter and setters
* update compute domain and get domain and tests
* update validators
* Add forkdata proto
* Updated compute domain api, moved it to helper pkg
* Merge refs/heads/master into v0.10.1
* Fixed all core tests
* Fixed all the sync tests
* Fixed all the rpc tests
* Merge refs/heads/master into v0.10.1
* Merge refs/heads/master into v0.10.1
* Fixed conflict
* Fixed conflict
* Conflict fix
* visibility
* Fixed validator tests
* Fixing test util
* Fixed rest of non spec tests
* Fixed a bug proposer index wasn't included
* gaz
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Updated eth1 data voting period to epoch based
* Fixed failed tests
* fix bug
* fix error
* Fixed more misc tests
* Add new SignedAggregateAndProof to pass spec test
* Update minimalConfig.PersistentCommitteePeriod
* allow to rebuild trie
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Skip e2e tests
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Align aggregator action with v0.11 (#5146)
* Remove Head Root from Beacon Block by Range Request (#5165)

* make proto changes
* remove head root
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* add back herumi's library
* Update ethapi in workspace, started fixing test. Hand off to Nishant
* fix build
* All tests passing
* Align finalized slot check with v0.11 (#5166)
* Merge branch 'master' into v0.11
* Add DoS resistance for v0.11 (#5158)
* Add Fork Digest Helper (#5173)
* Extend DoS prevention to rest of operation objects (#5174)

* Update mapping

* Add caches

* Update seen block in validation pipeline

* Update seen att in validation pipeline

* Update seen att in validation pipeline

* Fixed rest of tests

* Gazelle

* Better writes

* Lint

* Preston's feedback

* Switched to LRU cache and fixed tests

* Gazelle

* Fix test

* Update proposer slashing

* Update proposer slashing

* Fixed a block test

* Update exit

* Update atteser slashing

* Raul's feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add remote keymanager (#5133)

* Add remote keymanager

* Add generic signRoot() helper

* Add tests for remote keymanager

* NewRemote -> NewRemoteWallet

* signRoot -> signOject, to increase reuse

* Fix end-to-end compile error

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Add Snappy Framing to the Encoder (#5172)

* change to framing

* more fixes

* fix everything

* add stricter limits

* preston feedback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: rauljordan <raul@prysmaticlabs.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Move Subnet Functionality to its Own File (#5179)

* move subnets to their own file

* fix build fail

* build

* Update beacon-chain/p2p/discovery_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Sync with master
* Verify proposer signature in sync (#5206)
* Fix Signed Attestation In Sync (#5207)
* Add Eth2 Fork ENR Functionality (#5181)

* add fork entry enr

* add in fork

* add the required fork entry to node

* add and retrieve fork entry

* await state initialized

* utilize new structure

* more progress, utilizing a config map instead

* send the genesis validators root via the event feed

* struct method for discovery

* fix broken builds

* fixed up more tsts using state feed initializer

* fix up most tests

* only one more failing test

* almost done with tests

* p2p tests all pass

* config fix

* fix blockchain test

* gaz

* add in todo

* lint

* add compare func

* ensure fork ENR versions match between peers

* add in test for discovery

* test name

* tests complete

* tests done

* done

* comments

* fix all flakes

* addressed comments

* build using ssz gen

* marshal record

* use custom ssz

* deduplicate import

* fix build

* add enr proto

* p2p tests done

Co-authored-by: nisdas <nishdas93@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Verify aggregator signature in sync (#5208)
* Add Fork Digest For Gossip Topics (#5191)

* update for the day

* fix remaining failing test

* fix one more test

* change message

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

* implement fork digest'

* align digest to interface'

* passed all tests

* spawn in goroutine

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Fix Incorrect Attester Slashing Method (#5229)
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Remove keystore keymanager from validator (#5236)

* Remove keystore keymanager from validator

* Update dependency

* Update validator/flags/flags.go

* Update validator/flags/flags.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* fix broadcaster
* update metrics with fork digest for p2p (#5251)

* update metrics with fork digest for p2p

* update p2p metrics

* update metrics using att values

* wrapped up

* fix bug

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Fix incorrect domain type comments (#5250)

* Fix incorrect domain type comments
* resolve conflicts
* fix broken broadcast test
* fix tests
* include protocol suffix
* fix confs
* lint
* fix test
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* resolve broken slasher test'
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge branch 'master' into v0.11
* fix config override
* Remove deprecated parameters (#5249)
* Avoid div by zero in extreme balance case (#5273)

* Return effective balance increment instead of 1

* Update to new spec tests v0.11.1

* Revert "Regen historical states for `new-state-mgmt` compatibility (#5261)"

This reverts commit df9a534826.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Remove deprecated parameters (#5249)" (#5276)

This reverts commit 7d17c9ac3455ee15c67b3645485693309216bc97.
* Verify block proposer index before gossip  (#5274)

* Update pipeline

* Update tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Add in Proposer Index to Custom HTR (#5269)

* fix test

* Update beacon-chain/state/stateutil/blocks_test.go

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Resolve Flakey P2P Tests (#5285)

* double time for flakey test

* fix test flakeyness in p2p:

* flakey

* time tolerance

* greater tolerance
* Merge branch 'master' into v0.11
* release resources correctly (#5287)
* Merge refs/heads/master into v0.11
* Enable NOISE Handshake by Default v0.11 (#5272)

* noise handshakes by default

* fix build

* noisy noise everywhere

* deprecated noisy noise flag with more noise

* add secio as fallback

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: nisdas <nishdas93@gmail.com>
* Merge refs/heads/master into v0.11
* new ports
* fix broken build
* Make `new-state-mgmt` canonical  (#5289)

* Invert the flags
* Update checking messages
* Fixed all db tests
* Fixed rest of the block chain tests
* Fix chain race tests
* Fixed rpc tests
* Disable soudns better...
* Merge branch 'v0.11' into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Fix export
* Merge branch 'invert-new-state-mgmt' of github.com:prysmaticlabs/prysm into invert-new-state-mgmt
* Fix conflict tests
* Gazelle
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge refs/heads/v0.11 into invert-new-state-mgmt
* Merge branch 'master' into v0.11
* resolve flakeyness
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Detect Proposer Slashing Implementation (#5139)

* detect blocks

* detect blocks

* use stub

* use stub

* use stub

* todo

* fix test

* add tests and utils

* fix imports

* fix imports

* fix comment

* todo

* proposerIndex

* fix broken test

* formatting and simplified if

* Update slasher/detection/service.go

* Update slasher/detection/testing/utils.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fixed up final comments

* better naming

* Update slasher/detection/service.go

* Update slasher/detection/service.go

* Update slasher/detection/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* no more named args

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Add Metadata And Ping RPC methods (#5271)

* add new proto files

* add flag and helper

* add initializer

* imports

* add ping method

* add receive/send ping request

* add ping test

* refactor rpc methods and add ping test

* finish adding all tests

* fix up tests

* Apply suggestions from code review

* lint

* imports

* lint

* Update beacon-chain/p2p/service.go

* Update shared/cmd/flags.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into v0.11
* Updates for remote keymanager (#5260)
* Merge branch 'spec-v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Merge remote-tracking branch 'origin' into v0.11
* Update to slash by slot instead of epoch (#5297)

* change to slash by slot instead of epoch

* gaz

* fix test

* fix test

* fix infinite loop on error parse
* Sync with master
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Update proposer protection to v0.11 (#5292)

* Complete most of changes

* Fix other tests

* Test progress

* Tests

* Finish tests

* update pbs

* Fix mocked tests

* Gazelle

* pt 2

* Fix

* Fixes

* Fix tests wit hwrong copying
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Implement `SubscribeCommitteeSubnet` method (#5299)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append
* Merge refs/heads/master into v0.11
* resolve confs
* Validator subscribe subnet to next epoch (#5312)

* Alert to subscribe to next epoch

* Fixed tests

* Comments

* Fixed tests

* Update validator/client/validator.go

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)

This reverts commit 47e5a2cf96f5add151bf135a5352c2dad7922615.
* Aggregate on demand for v0.11 (#5302)

* Add client implementation

* Update workspace

* Update server

* Update service

* Gaz

* Mocks

* Fixed validator tests

* Add round tirp tests

* Fixed subnet test

* Wait 1/3 on validator side

* Lint

* Comment

* Update committee cache

* Comment

* Update RPC

* Fixed test

* Nishant's comment

* Gaz

* Refresh ENR is for epoch

* Needs to be append

* Fixed duplication

* Tests

* Skip e2e

* Update beacon-chain/rpc/validator/aggregator.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
* Refactor Dynamic Subscriptions (#5318)

* clean up

* comment

* metrics

* fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Fix listindexed attestations and detect historic attestations (#5321)

* fix list indexed attestations

* fix tests

* goimports

* names
* Add check for slot == 0 (#5322)
* Change attester protection to return default if DB is empty (#5323)

* Change how default values are set

* Remove unused imports

* Remove wasteful db call

* Fix db tests

* Fix db test
* Merge refs/heads/master into v0.11
* fix it (#5326)
* V0.11 run time fixes to use interop config (#5324)

* Started testing
* Bunch of fixes
* use-interop
* Sync with v0.11
* Conflict
* Uncomment wait for activation
* Move pending block queue from subscriber to validator pipeline
* Merge branch 'v0.11' into use-interop-config
* passing tests
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/v0.11 into use-interop-config
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Merge refs/heads/master into v0.11
* Nil Checks in Process Attestation v0.11 (#5331)

* Started testing

* Bunch of fixes

* use-interop

* Sync with v0.11

* Uncomment wait for activation

* Move pending block queue from subscriber to validator pipeline

* passing tests

* nil checks to prevent panics

* lint

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Validator batch subscribe subnets (#5332)

* Update both beacon node and validator

* Comments

* Tests

* Lint

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Validator smarter subscribe (#5334)
* Fix incorrect proposer index calculation (#5336)

* Use correct parent state

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* enhance error
* enhance error
* Update P2P Service to Handle Local Metadata (#5319)

* add metadata to ENR

* add new methods

* glue everything

* fix all tests and refs

* add tests

* add more tests

* Apply suggestions from code review

* fix method

* raul's review

* gaz

* fix test setup

* fix all tests

* better naming

* fix broken test

* validate nil

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Revert "Revert "Revert "Remove deprecated parameters (#5249)" (#5276)" (#5277)" (#5343)

This reverts commit e5aef1686e582fc2077767c42187c8527f3a742f.
* Wait for Genesis Event to Start P2P (#5303)

* use event feed for state initialized events

* add in handler for tests

* wait till genesis for p2p

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge refs/heads/master into v0.11
* Avoid duplicated aggregation request (#5346)

* Avoid duplicated aggregation request

* Test and lock

* Gaz
* Fix Validate For Metadata (#5348)

* return true

* shay's review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Multiple Proposer Slots Allowed Per Epoch for Validators (#5344)

* allow multiple proposer slots

* multi propose

* proposer indices to slots map

* remove deprecated comm assign

* Apply suggestions from code review

* resolve broken tests, add logic in validator client

* fix val tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Networking Fixes (#5349)

* close stream later

* add ping method

* add method

* lint
* More efficient aggregation on demand (#5354)
* Return Nil Error if Pre-Genesis in P2P Service Healthz Check (#5355)

* pregenesis healthz check:

* optimal

* right order

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* no comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Release DiscoveryV5 for Testnet Restart (#5357)

* release discv5

* fix build
* Fix Overflow in Status Check (#5361)

* fix overflow

* Apply suggestions from code review
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* fix after merge
* Merge refs/heads/master into v0.11
* Make Mainnet Config Default, No More Demo Config  (#5367)

* bye bye demo config

* gaz

* fix usage

* fix dep

* gaz

* Update default balance for sendDeposits

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
* Use FastSSZ Marshal/Unmarshal for DB Encodings in v0.11.1 (#5351)

* try

* use marshaler structure for db instead of proto

* white list types

* attempt

* revert

* testutil.NewBeaconState()

* Fully populate fields for round trip ssz marshal

* fix //beacon-chain/db/kv:go_default_test

* more passing tests

* another test target passed

* fixed stategen

* blockchain tests green

* passing sync

* more targets fixed

* more test fixes in rpc/validator

* most rpc val

* validators test fixes

* skip round robin old

* aggregate test

* whitelist done

* Update beacon-chain/rpc/validator/attester_test.go

* edit baz

* Fixed tests

* Fixed getblock test

* Add back init

* reduce test size

* fix broken build

* tests pass

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
* Reconnect slasher streams on beacon node shutdown (#5376)

* restart streams on beacon node shutdown

* fix comment

* remove export

* ivan feedback

* ivan feedback

* case insensitive

* Update slasher/beaconclient/receivers.go

* raul feedback

Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Merge branch 'master' into v0.11
* Merge refs/heads/master into v0.11
* Amend Faucet to Offer 32.5 ETH for v0.11 (#5378)

* deposit amount in faucet

* fix eth amount

* gas cost
* unskip exec transition test
* Revert "Enable NOISE Handshake by Default v0.11 (#5272)" (#5381)

This reverts commit a8d32d504a8f923cdf7fa9dfc2684f8804fbab92.
* Merge refs/heads/master into v0.11
* use string for deposit flag
* Update Bootnode to v0.11 (#5387)

* fix bootnode

* add changes

* gaz

* fix docker
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* build fix
* fix flaky test
* Merge refs/heads/master into v0.11
* Unskip E2E for V0.11 (#5386)

* Begin work on fixing e2e for v0.11

* Start bootnode work

* Begin implementing bootnode into e2e

* Fix E2E for v0.11

* Remove extra

* gaz

* Remove unused key gen code

* Remove trailing multiaddr code

* add skip for slashing

* Fix slashing e2e

* Fix docker image build
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into v0.11
* Merge refs/heads/master into v0.11
* Merge branch 'master' of github.com:prysmaticlabs/prysm into v0.11
* Update beacon-chain/p2p/broadcaster_test.go
* Merge refs/heads/master into v0.11
* Pass E2E Tests for v0.11 and Enable Attestation Subnets By Default (#5407)
* Update README.md

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/p2p/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/keystore/deposit_input.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update tools/faucet/server.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update proto/beacon/p2p/v1/BUILD.bazel

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/benchutil/pregen_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Update shared/bls/spectest/aggregate_verify_test.go
* Addressed feedback. All test passing
* Merge branch 'v0.11' of github.com:prysmaticlabs/prysm into v0.11
* Update beacon-chain/core/blocks/block_operations_fuzz_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update shared/testutil/helpers.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Update beacon-chain/core/helpers/signing_root.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>
* Resolve Misc v0.11 Items (Raul) (#5414)

* address all comments

* set faucet

* nishant feedback

* Update beacon-chain/p2p/service.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Revert keymanager changes (#5416)

* Revert "Updates for remote keymanager (#5260)"

This reverts commit bbcd895db50ce5e7c0ecb64210471cf56f63b373.

* Revert "Remove keystore keymanager from validator (#5236)"

This reverts commit 46008770c162e741251e13772fd7356b43a9af87.

* Revert "Update eth2 wallet keymanager (#4984)"

This reverts commit 7f7ef43f218598a671aaeb327342d7e5130fe8b1.

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
* Update BLS and limit visibility (#5415)

* remove duplicated BLS, add golang.org/x/mod

* Update BLS and restrict visibility

* fix build
* Fix eth1data test and fix order of ops (#5413)
* use multiaddr builder (#5419)
* Unskip benchutil and minor v0.11 fixes (#5417)

* Unskip benchutil tests

* Remove protos and gaz

* Fixes
* Networking Fixes (#5421)

* check

* fix test

* fix size

* fix test

* more fixes

* fix test again
* Update ethereum APIs with latest master
* Error handling for v0.11 tests (#5428)

* Proper err handling for tests

* Lint

* Fixed rest of the tests

* Gaz

* Fixed old master tests
* Sync with master
* Rm old aggregate_test.go
2020-04-14 20:27:03 +00:00
Victor Farazdagi
748d513c62
proper error checking and type assertions (#5424)
* proper error checking and type assertions
2020-04-14 16:41:09 +00:00
terence tsao
3b9a03d517
State service clean up - better cold state getter (#5393)
* Update load cold states by root and slot

* Update cold tests

* Add a way to recover state summary

* Short circuit slot 0 to return genesis state

* Short circuit if there's no block in DB

* Fixed test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-13 11:32:02 -05:00
Preston Van Loon
d5ddd012bc
Enforce error handling and checking type assertions (#5403)
* Enforce error handling and checking type assertions
* Reference issue #5404 in the TODO message
* doc description
* Merge branch 'master' into errcheck
* fix tests and address @nisdas feedbacK
* gaz
* fix docker image
2020-04-13 04:11:09 +00:00
terence tsao
e59721f264
State service clean up - better migration (#5391) 2020-04-11 13:54:19 -07:00
Preston Van Loon
8c9ae8e3bf
Use state validator index map (#5383)
* remove validator index tracking from DB

* Builds production code path

* tests build

* fix tests

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-04-11 14:47:22 -05:00
Preston Van Loon
973a0a89b5
Update state root cache to fuzz in parallel (#5364)
* Update state root cache to fuzz in parallel

* confirmed arrays passing

Co-authored-by: rauljordan <raul@prysmaticlabs.com>
2020-04-10 13:12:59 -05:00
terence tsao
a7bee79988
Use the config value (#5342) 2020-04-07 20:11:19 -05:00
Nishant Das
7e50c36725
Add Hasher To State Data Types (#5244)
* add hasher
* re-order stuff
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
* Merge refs/heads/master into addHasher
2020-03-31 18:57:19 +00:00
terence tsao
e22365c4a8
Uncomment out cold state tests (#5252)
* Fixed most of the tests

* All tests passing

* All tests passing

* Fix merge conflict

* Fixed error test

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-31 11:23:39 -05:00
terence tsao
7f7866ff2a
Micro optimizations on new-state-mgmt service for initial syncing (#5241)
* Starting a quick PoC

* Rate limit to one epoch worth of blocks in memory

* Proof of concept working

* Quick comment out

* Save previous finalized checkpoint

* Test

* Minor fixes

* More run time fixes

* Remove panic

* Feature flag

* Removed unused methods

* Fixed tests

* E2e test

* comment

* Compatible with current initial sync

* Starting

* New cache

* Cache getters and setters

* It should be part of state gen

* Need to use cache for DB

* Don't have to use finalized state

* Rm unused file

* some changes to memory mgmt when using mempool

* More run time fixes

* Can sync to head

* Feedback

* Revert "some changes to memory mgmt when using mempool"

This reverts commit f5b3e7ff4714fef9f0397007f519a45fa259ad24.

* Fixed sync tests

* Fixed existing tests

* Test for state summary getter

* Gaz

* Fix kafka passthrough

* Fixed inputs

* Gaz

* Fixed build

* Fixed visibility

* Trying without the ignore

* Didn't work..

* Fix kafka

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-03-30 17:10:45 -05:00
Nishant Das
63df1d0b8d
Add Merkleize With Customized Hasher (#5234)
* add buffer for merkleizer
* add comment
* Merge refs/heads/master into merkleize
* Merge refs/heads/master into merkleize
* Merge refs/heads/master into merkleize
* Merge refs/heads/master into merkleize
* Merge refs/heads/master into merkleize
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into merkleize
* Merge branch 'merkleize' of https://github.com/prysmaticlabs/geth-sharding into merkleize
* lint
* Merge refs/heads/master into merkleize
2020-03-29 06:13:24 +00:00