Commit Graph

605 Commits

Author SHA1 Message Date
Nishant Das
904898e405 Revert "Change BLS Library to Herumi (#3752)" (#4006)
This reverts commit 24583864b4.
2019-11-14 13:00:50 -05:00
Nishant Das
24583864b4 Change BLS Library to Herumi (#3752)
* change to herumi's bls

* change alias

* change to better

* add benchmark

* build

* change to bazel fork

* fix prefix

* make it work with library

* update to latest

* change again

* add import

* update to latest

* add sha commit

* new static lib with groups swapped

* using herumis new lib

* fix dep paths in c headers

* update again

* new changes

* fix commit

* fix serialization

* comment

* fix test

* fix to herumis latest version

* fix test

* fix benchmarks

* add new workspace

* change commit and remove init

* get test to pass

* remove parameter

* remove reverse byte order

* make gazelle happy

* set pure to off

* fix failing tests

* remove old ref

* use HashWithDomain functions

* update to latest version

* clean up

* gaz

* add back removed code

* switch off pure
2019-11-14 09:51:42 -06:00
Preston Van Loon
fc7c530696 Use a data table for common power of 2 roots (#3995)
* use a data table for common power of 2 roots

* revert beacon-chain/rpc/proposer/server.go
2019-11-13 14:03:42 -06:00
terence tsao
81c53c26fb Update run time to v0.9 (#3935)
* Update seed domains (#3872)

* Remove Transfers (#3870)

* Remove active index roots and compact committee roots (#3869)

* Update inclusion reward (#3886)

* Alter proposer selection logic (#3884)

* Fix early committee bias (#3888)

* Remove shards and committees (#3896)

* Epoch spec tests v0.9 (#3907)

* Block spec test v0.9 (#3905)

* rm'ed in protobuf

* build proto

* build proto

* build proto

* fix core package

* Gazelle

* Fixed all the tests

* Fixed static test

* Comment out spec test for now

* One more skip

* fix-roundRobinSync (#3862)

* Starting but need new seed function

* Revert initial sync

* Updated Proposer Slashing

* Fixed all tests

* Lint

* Update inclusion reward

* Fill randao mixes with eth1 data hash

* Test

* Fixing test part1

* All tests passing

* One last test

* Updated config

* Build proto

* Proper skip message

* Conflict and fmt

* Removed crosslinks and shards. Built

* Format and gazelle

* Fixed all the block package tests

* Fixed all the helper tests

* All epoch package tests pass

* All core package tests pass

* Fixed operation tests

* Started fixing rpc test

* RPC tests passed!

* Fixed all init sync tests

* All tests pass

* Fixed blockchain tests

* Lint

* Lint

* Preston's feedback

* Starting

* Remove container

* Fixed block spec tests

* All passing except for block_processing test

* Failing block processing test

* Starting

* Add AggregateAndProof

* All mainnet test passes

* Update deposit contract (#3906)

* Proto spec tests v0.9 (#3908)

* Starting

* Add AggregateAndProof

* Unskip block util tests (#3910)

* rm'ed in protobuf

* build proto

* build proto

* build proto

* fix core package

* Gazelle

* Fixed all the tests

* Fixed static test

* Comment out spec test for now

* One more skip

* fix-roundRobinSync (#3862)

* Starting but need new seed function

* Revert initial sync

* Updated Proposer Slashing

* Fixed all tests

* Lint

* Update inclusion reward

* Fill randao mixes with eth1 data hash

* Test

* Fixing test part1

* All tests passing

* One last test

* Updated config

* Build proto

* Proper skip message

* Conflict and fmt

* Removed crosslinks and shards. Built

* Format and gazelle

* Fixed all the block package tests

* Fixed all the helper tests

* All epoch package tests pass

* All core package tests pass

* Fixed operation tests

* Started fixing rpc test

* RPC tests passed!

* Fixed all init sync tests

* All tests pass

* Fixed blockchain tests

* Lint

* Lint

* Preston's feedback

* Starting

* Remove container

* Fixed block spec tests

* All passing except for block_processing test

* Failing block processing test

* Starting

* Add AggregateAndProof

* All mainnet test passes

* Unskip block util tests

* Slot processing spec test V0.9 (#3912)

* Starting

* Add AggregateAndProof

* Unskip slot processing mainnet test

* Unskip minimal spec test for finalization (#3920)

* Remove outdated interop tests (#3922)

* Rm outdated interop tests

* Rm test runner

* Gazelle

* Update validator to use proposer slot (#3919)

* Fix committee assignment (#3931)

* Replace shard with committee index (#3930)

* Conflict

* Clean up (#3933)

* Remove shard filter in db (#3936)

* Remove lightouse compatibility test (#3939)

* Update Committee Cache for v0.9 (#3948)

* Updated committee cache

* Removed shuffled indices cache

* Started testing run time

* Lint

* Fixed test

* Safeguard against nil head state

* address edge case

* add test

* Fixed TestRoundRobinSync by doubling the epochs

* Unskip TestProtoCompatability (#3958)

* Unskip TestProtoCompatability

* Update WORKSPACE

* Fix minimal config (#3959)

* fix minimal configs

* fix hardcoded value in test

* Simplify verify att time (#3961)

* update readme for deposit contract, regen bindings for vyper 0.1.0b12 (#3963)

* update readme for deposit contract, regen bindings

* medium

* Check nil base state (#3964)

* Copy Block When Receiving it From Sync (#3966)

* copy block

* clone for other service methods too

* Change logging of Bitfield  (#3956)

* change logging of bits

* preston's review

* Unskip Beacon Server Test (#3962)

* run test till the end

* fix up proto message types

* fmt

* resolve broken tests

* better error handling

* fixing new logic to use archived proposer info

* fix up logic

* clip using the max effective balance

* broken build fix with num arg mismatch

* amend archive

* archival logic changed

* rename test

* archive both proposer and attester seeds

* page size 100

* further experiments

* further experimentation, archivedProposerIndex seems wrong

* test passes

* rem log

* fix broken test

* fix test

* gaz

* fix imports

* ethapis
2019-11-11 17:03:43 -05:00
Nishant Das
cc18b2f4d3 Handle Runtime Error Type (#3954) 2019-11-09 05:53:20 -08:00
mkinney
e9bd530221 do not die if user does not have git installed (#3944) 2019-11-06 13:41:06 -05:00
Preston Van Loon
371f808aa4
bump fork version (#3926) 2019-11-05 01:05:42 -05:00
Nishant Das
6e90931837
Handle Missing Blocks (#3880)
* add fixes

* more changes

* simplify the solution

* add shuffle

* remove prom metric

* add to mock

* go doc

* use non default source

* add test

* terence's review

* all tests passing
2019-11-05 10:37:18 +08:00
Preston Van Loon
7d1633230d Feature gate all caches (#3894) 2019-11-03 17:20:08 -08:00
Nishant Das
db26c0d012 Hardcode Domain Version For Compute Domain (#3876)
* hardcode domain version to zeroed byte array

* remove todo

* gazelle
2019-11-03 18:37:01 -05:00
terence tsao
2b0acffe7f Set it to 3 (#3857) 2019-11-03 18:18:52 -05:00
Jim McDonald
485fc538c3 Utility to parallelise functions over arrays (#3813)
* Initial version of scatter

* Add mutex for scatter batch

* Provide mutex to worker processes

* Embed mputil

* Add notes on scatter

* Tidy-up

* Add mutex test

* Simplify scatter for users

* Tidy-ups/code coverage

* Gazelle update

* Add benchmark

* Mutex to RWMutex

* Add test against internal functions

* Bazel fixes

* Fix benchmark

* Benchmark values to constants

* Update shared/mputil/scatter.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Update shared/mputil/scatter.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Moved struct to top of file

* Add featureconfig for scatter

* Gate scatter behind feature flag

* Lint fixes

* fmt
2019-11-03 16:25:52 -05:00
Preston Van Loon
cc16a10a33
Remove --enable-finalized-block-root-index (#3887)
* Deprecated do not use

* remove finalized block indexing feature flag

* remove flag some more

* dont rollback a managed tx

* require blocks to be in a chain in tests
2019-11-02 14:57:35 -04:00
Nishant Das
ab92326dfb set bool correctly (#3913) 2019-11-01 09:48:24 -05:00
mkinney
da6c270d46 report git commit and date on local build (#3904)
* report git commit and date on local build

* switch to using time package
2019-11-01 13:47:30 +08:00
Preston Van Loon
86cd873e67
Add span and annotate errors for validating beacon attestations (#3861)
* Add span and annotate errors for validating beacon attestations

* fix test
2019-10-31 18:32:22 -04:00
Nishant Das
49e0ddf861 Cache Skipped Slots (#3879)
* use LRU cache on skip slots

* add a metric

* gaz

* more metrics

* temporary log

* added more logging

* fix

* only update, if its higher than the cache

* only update, if its higher than the cache

* increase the cache size, just in case

* split cache related stuff into its own file

* add feature flag

* clean up

* add new test

* make test proper

* godoc

* preston's review
2019-10-31 13:38:51 -04:00
mkinney
9ca95530fa allow passing no,prompt,force options to clear-db (#3871)
* allow passing no,prompt,force options to clear-db

* ensure the node test runs ok

* remove clear-db option and add force-clear-db; remove no longer needed file

* add clear-db option back; force-clear-db overrides

* add clear-db option back to usage

* revert to only using --clear-db
2019-10-31 17:55:32 +08:00
Preston Van Loon
5f79abd828 Safer panic handler (#3892)
* safer panic handler

* safer panic handler
2019-10-30 15:40:39 -07:00
Preston Van Loon
2b444ea954 Add feature flag to enable shuffled index cache (#3893)
* add feature flag to enable shuffled index cache

* gaz'
2019-10-30 10:26:05 +08:00
Preston Van Loon
ae89cce593 Deprecated do not use (#3882) 2019-10-30 01:28:41 +08:00
Preston Van Loon
749f4b776b
Add flag deprecation (#3881)
* add deprecated flags

* move flags to their own list

* gofmt

* more warning
2019-10-29 11:30:14 -04:00
Preston Van Loon
0481eb4872
Track finalized block root index and filter in blocks by range (#3873)
* checkpoint progress

* feature flag and test

* return true if feature flag is not on

* add lint msg

* rollback tx if failure

* Revert validator/BUILD.bazel
2019-10-29 11:14:17 -04:00
Nishant Das
f1627c0b67 Make No Genesis Delay the Default (#3878) 2019-10-29 07:31:10 -07:00
Nishant Das
1c4b4c8393 fix references to use ComputeDomain (#3854) 2019-10-25 08:05:53 -07:00
Raul Jordan
0a3825e79e Optimize Integer Square Root (#3785) 2019-10-24 06:32:04 -07:00
Preston Van Loon
acf11262de Add feature flag for state pruning (#3845)
* Add feature flag for state pruning

* gaz
2019-10-23 23:45:52 -07:00
Nishant Das
c9a7a9c709
Attestation is Decoupled From Proposal (#3800)
* fix

* add correct test

* add helper

* gaz

* add helper

* gaz
2019-10-22 07:15:32 +08:00
Jim McDonald
053fa5e616 Remove hard-coded config values (#3762)
* Remove hard-coded config values

* Partial reversion to use hard-coded fork version of 0

* Re-add fetching genesis fork version from config
2019-10-19 13:52:34 -07:00
terence tsao
a41ac6b498 Implement ProcessEpochPrecompute (#3798)
* Implemented ProcessSlashingsPrecompute

* Tests for ProcessSlashingsPrecompute

* Gaz

* Lint

* Feature flags

* Hook in ProcessEpochPrecompute

* Hook it to spec test

* Seperate test into it's own package to avoid circular dependency

* Lint

* Gazelle

* Preston's feedback

* Nishant's feedback
2019-10-18 16:38:47 +08:00
Preston Van Loon
fef6b95fed
bump fork version (#3795) 2019-10-17 08:50:13 -07:00
terence tsao
2d863a1e63
Methods to precompute process_epoch records (#3788) 2019-10-16 17:48:26 -07:00
terence tsao
9bf55e53e7 Add context and tracing to attestation pool (#3737)
* Test case for overlapping aggregation bits

* Add ctx and tracing for attestation pool and beyond

* No nil

* Use real ctx

* Use real ctx

* Fix test

* Fix test

* Fix test

* Fix test

* Fixed imports
2019-10-09 06:42:17 +08:00
Preston Van Loon
4e22f52ab3 Testnet restart and hotfixes (#3736)
* hotfix for round robin, hotfix for grpc recovery

* gofmt

* break

* wrong subtraction

* lint

* testnet fork version update

* ignore relay / DHT protocol not supported error
2019-10-08 07:59:08 +08:00
Nishant Das
093c32e229
update repo (#3717) 2019-10-07 15:50:58 +08:00
terence tsao
750bc83369
Clean up feature flag namings (#3715) 2019-10-07 14:11:49 +09:00
Ivan Martinez
14d9a83cda Add Block Generation Util to testutil package (#3674) (#3709) 2019-10-04 16:07:46 -07:00
terence tsao
66dcf2b80d Moved /shared/ test code in different package (#3714)
* Fixed bytesutil

* Fix featureflag

* Fix hashutil

* Fix interop

* Fix iputils

* Fix mathutils

* Fix messagehandler

* Fix pagination

* Fix params

* Fix sliceutil

* Fix merkletrie
2019-10-04 15:46:49 -07:00
Nishant Das
50b1d209ab
Resubscribe Headers from ETH 1 Chain (#3706)
* resubscribe headers

* Update beacon-chain/powchain/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
2019-10-04 10:48:14 +08:00
Preston Van Loon
a6f6bb12fa Add message events to spans for pubsub (#3703) 2019-10-03 09:33:16 -07:00
Preston Van Loon
4bee60826d
add flag to enable BLS pubkey cache (#3699) 2019-10-03 08:26:15 -07:00
Preston Van Loon
7a04ff6368 Add database API for creating backups (#3694)
* Save db backup

* Fix DB backup method

* Add backup db webhook

* gaz

* if err != nil

* more verbose filename

* Don't obliterate everything :)
2019-10-03 17:29:49 +08:00
Nishant Das
4daf62fc28 Fix Validator Activation (#3684) 2019-10-02 13:26:11 -07:00
Ivan Martinez
8bab55d88e Add Block Generation Util to testutil package (#3674)
* Create block generation util in testutil

* Gazelle

* Fix deps

* Fix imports

* Change tests to use config and fix integer division

* Remove logs

* Fix build

* Add test to ensure finalization occurs

* Add check for finalization

* Add comment for incrementing the state

* Fix test

* Fix tes

* Fix testutil us

* Fix tests

* Change var name

* Add regression test for bug with large validator count

* Fix bazel test
2019-10-02 08:15:40 -07:00
terence tsao
c632b96454 Validator logging improvements (#3661)
* Starting

* Update logging for service

* Update logging for assignment

* Update logging for attest

* Update logging for propose

* Update logging for balance update

* Final touchup

* Fixed test

* Fixed test

* Feedback

* Fix

* Fix all the tests
2019-10-02 12:18:01 +08:00
Jim McDonald
323ee8dfac SetupInitialDeposits() now returns deposit data roots (#3683) 2019-10-02 10:50:34 +08:00
Preston Van Loon
42a2d5c1ee log buf.String() instead of map[reflect.Type]error (#3681) 2019-10-02 08:51:11 +08:00
kilic
d5e02eaa43 Change BLS Pairing Engine (#3670)
* change bls pairing engine

* fix linter warnings

* curve order

* add back spec test

* use only one dep

* fix test

* remove toBytes

* gaz

* add it back

* fix tests

* imports

* imports

* gaz

* remove hash function

* change naming

* preston's comments

* gaz

* fix test failure

* change back

* revert test changes

* gaz
2019-10-02 08:13:59 +08:00
Raul Jordan
d9c0e65cef
Improve Beacon Node Logging UX (#3600)
* info logs beacon node improvements

* prom test fixes

* info logging changes

* wrapped up node info logging

* changed to debug level

* warn logs taken care of

* Terence suggestion

* warn spacing

* better logging in initial sync

* debug level standardized

* complete debug standardization

* participation at epoch end

* fix archive tests

* even more test fixes

* prom test

* ops test

* powtest

* rpc sync test

* rem part

* log formatting
2019-10-01 15:05:17 -05:00
Preston Van Loon
f78d6e66b3
only enable libp2p logs when trace level logging (#3680) 2019-10-01 08:38:21 -07:00
Raul Jordan
3d37a4e038
Optimize Domain Data RPC Request (#3671)
* gaz

* fix broken build

* fix broken test

* fix broken test
2019-10-01 09:36:36 -05:00
Jim McDonald
628da919a4 Use deterministic method to create test deposits (#3639)
* Use deterministic method to create test deposits

* More descriptive failure messages for tests
2019-10-01 08:56:26 +08:00
Alex
6a203dce81 remove roughtime servers hardcode (#3666) 2019-09-30 15:30:45 -05:00
Preston Van Loon
571efc11d1
add error spans, interceptrs (#3641) 2019-09-29 11:48:55 -07:00
terence tsao
d02e73c5fe
Feature flag for new caching scheme (#3619) 2019-09-27 13:14:22 -07:00
Raul Jordan
24df2d3e44
Skip BLS With a Feature Flag at Runtime (#3618)
* skip bls verification with a feature flag at runtime

* gazelle

* more bls mocks
2019-09-27 13:28:43 -05:00
Jim McDonald
e1e36e1424 Default genesisTime to now when generating a genesis state. (#3615)
* Default genesisTime to now when generating a genesis state.

* Use roughtime for genesis creation timestamp
2019-09-27 10:49:55 -05:00
Jim McDonald
b089cdd216 Allow overwriting of default bootstrap node (#3616)
* Allow overwriting of default bootstrap node

* Update shared/cmd/flags.go

Co-Authored-By: Nishant Das <nish1993@hotmail.com>

* Provide warning at more suitable time
2019-09-27 20:05:16 +08:00
Nishant Das
ddf6f7d4d9 add new bootnode and contract endpoint (#3612) 2019-09-26 21:05:38 -07:00
terence tsao
b8bd28cca2
Move SplitOffset to Sliceutil (#3606) 2019-09-26 16:25:55 -07:00
Nishant Das
9b008522b8 Refactor Validator Start Routine (#3594)
* make demo default

* make minimal config a flag

* lint

* initialize config at the start

* gaz

* make main method cleaner

* remove interop.go

* fix test

* lint

* gaz

* Update validator/accounts/interop.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fix docker build

* fix docker build
2019-09-26 13:23:25 -05:00
terence tsao
3ef681e649
Initial sync no verify block (#3586) 2019-09-25 20:32:00 -07:00
terence tsao
fb74dae835
Remove dead cache (#3584) 2019-09-25 18:23:15 -07:00
Preston Van Loon
ba4f45b180 BLS pubkey from bytes cache (#3583)
* BLS pubkey from bytes cache

* lint
2019-09-25 13:51:14 -05:00
terence tsao
6c892dc376 General indices helpers (#3575)
* Implemented Power of 2 helpers

* Test for power of 2 helpers

* Gazelle

* Fmt

* Implemented MerkleTree

* Test for MerkleTree

* Fixed tests

* Implemented ConcatGeneralizedIndices and GeneralizedIndexLength

* Tests for the above

* Benchmarked copy, it's faster

* Implemented rest of the indices helpers

* Tests for indices helpers

* Delete
2019-09-25 17:05:35 +08:00
terence tsao
2335b5eae7
Merkle tree implementation (#3572) 2019-09-24 17:37:45 -07:00
terence tsao
e64287773c
Merkle tree math helpers (#3571) 2019-09-24 09:58:18 -07:00
Nishant Das
0e329fc115 Attestation Server Fix (#3570) 2019-09-24 08:19:37 -07:00
Preston Van Loon
ad47817bcd Only write interop ssz states to disk with flag ON (#3566)
* only write SSZ states to disk with flag on

* lint

* also write blocks
2019-09-23 18:36:12 -05:00
Nishant Das
b5b10a8d35 Add Back Kademlia DHT to Prysm (#3557)
* serve nodes

* remove testing flag

* add back bootnode

* add dht

* add back dht

* gaz

* fix build

* bootnode works in runtime

* fix all references

* all tests pass

* remove feature flag

* separate out ports

* lint

* fix docker build

* use one error package
2019-09-23 10:24:16 -07:00
Preston Van Loon
cba44e5151 log a warning on unhealthy healthz (#3529) 2019-09-19 20:39:55 +05:30
Nishant Das
d9664d3b6b Fix Withdrawal Credentials (#3517) 2019-09-18 07:52:34 -07:00
Raul Jordan
7e7941b0af
bls endianness (#3495) 2019-09-16 16:39:45 -05:00
Raul Jordan
9683a83750 Properly Use Demo Config (#3494) 2019-09-16 12:55:30 -07:00
Nishant Das
bef58620fc Update Renovate (#3489)
* Update libp2p

* Update com_google_protobuf commit hash to 763c358

* Update graknlabs_bazel_distribution commit hash to 1ec7e2d

* Update dependency com_github_coreos_go_semver to v0.3.0

* Update dependency com_github_minio_sha256_simd to v0.1.1

* Update dependency com_github_prometheus_common to v0.7.0

* Update dependency com_github_prometheus_procfs to v0.0.5

* Update dependency com_google_cloud_go to v0.46.2

* Update dependency io_bazel_rules_docker to v0.10.1

* Update libp2p

* Update dependency com_github_beorn7_perks to v1

* Update dependency com_github_grpc_ecosystem_go_grpc_middleware to v1

* change back protobuf
2019-09-16 06:05:16 -07:00
Preston Van Loon
a2685245f2 demo-config flag (#3473)
* democonfig

* 3.2
2019-09-15 14:24:08 -05:00
Nishant Das
d36061d62f Fix Deterministic Key Generator (#3467) 2019-09-14 10:46:38 -04:00
terence tsao
8c7ef61238 Add back configure validator features (#3456) 2019-09-12 14:02:53 -05:00
Preston Van Loon
0a0d579822 run minimal tests (#3454) 2019-09-12 14:24:35 -04:00
terence tsao
91f824fe10
Clean up unused flags (#3449) 2019-09-12 11:48:34 -04:00
terence tsao
bee3aff6c5
Delete deprecated p2p (#3451) 2019-09-12 11:20:46 -04:00
Nishant Das
3e8a94516d
remove bootstrap node (#3447) 2019-09-12 12:46:16 +05:30
Nishant Das
273b917319
Allow Separate Ports for Different Transports (#3414)
* update workspace

* change to new version

* gaz

* set keys

* try more things

* finally fixed all tests

* fix bootnode

* Update beacon-chain/p2p/discovery.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* preston's and raul's review

* add http server

* add tool

* add image

* change comment

* add multiaddr comment

* lint

* cosmetic changes

* fix docker

* remove dep

* preston's requested changes

* new flags

* add support for separate tcp port

* fix refs

* change tcp port
2019-09-12 10:22:27 +05:30
terence tsao
798bbbdc82 Cold start for interop (#3437)
* coldstart flags for validator

* WIP beacon node flags

* wip beacon chain, flag fix in validator, arg fix in validator

* checkpoint

* Added interop service

* working on mock chainstart

* save the state lol

* fix tests

* Save genesis validators

* gaz

* fix validator help flags

* WaitForChainStart actually waits for genesis time

* cold start fixes

* cache

* change back

* allow for genesis state too

* remove logs

* increase mmap size

* dont process if head doesn't exist

* add 10ms tolerance

* enable libp2p debug at debug, fix pubsub

* works with checkpt

* initialize justified and finalized in db

* Removed preloadStatePath from blockchain service

* Clean up

* Write to disk for now post state

* revert b466dd536f8eadbdae2264a545a755370223d917

* Builds

* Only RPC test fails now

* use minimal config, no demo config

* clean up branch

* Lint

* resolve lint

* more lint fixes

* lint

* fix viz

* Fixing RPC test

* skip before epoch 2

* RPC time out

* Fixed ordering

* rename

* remove some dbg statements

* ensure index is correct

* fix some panics

* getting closer

* fix tests

* Fix private key

* Fixed RPC test

* Fixed beacon chain build for docker

* Add interop.go to validator go_image

* Fixed docker build

* handle errors

* skip test, skip disconnecting peers

* Fixed docker build

* tolerance for attestation processing

* revert copy

* clearer err message parse

* fix launching from dep contract
2019-09-11 13:38:35 -05:00
shayzluf
1edeb8ec4c Beaconblock over wire (#3436) 2019-09-10 10:24:14 -04:00
Preston Van Loon
3708a8f476 Add tool and script for interop testing (#3417)
* add tool and script for interop testing

* identity

* lint

* merge upstream, fix conflict, update script, add comment

* add comma separated support for --peer=

* remove NUM_VALIDATORS, comma fix

* WIP docker image

* use CI builder

* pr feedback

* whatever antoine says

* ignore git in docker

* jobs=auto

* disable remote cache

* try to cache the golang part

* try to cache the golang part

* nvm

* From Antoine with love

* fix
2019-09-09 17:31:19 -04:00
Ivan Martinez
5e939378d0 Update to spec v0.8.3 (#3355)
* Ignore latest messages in fork choice prior to latest justified

* Make sure Compact Committee Roots isn't changed by process_final_updates

* WIP add attestation bitfields length to match committee length

* Begin work on updating spec tests to 0.8.2

* WIP set up for new spec test structure

* Fix slashings

* Get mainnet tests mostly passing for attestations and attester slashings

* Fix process attestation test

* Undo change

* Complete spec tests for all operations
Still need sanity block tests

* Fix BLS sigs

* Reduce amount of reused code in core/blocks/spectests/

* Fix tests

* Update block sanity tests to 0.8.2

* Update epoch spec tests to 0.8.2

* Clean up all tests and fix shuffling/epoch tests

* WIP update bls tests to 0.8.2

* WIP update bls tests to 0.8.3

* Finish BLS spectest update to 0.8.3

* Fix shuffling spec tests

* Fix more tests

* Update proto ssz spec tests to 0.8.3

* Attempt to fix PrevEpochFFGDataMismatches test

* Goimports

* Fix documentation

* fix test

* Use custom general spec tests

* Reduce code footprint

* Remove unneeded minimal skip

* Fix for comments

* Fix for comments

* Fix test

* Small fixes

* Cleanup block spec tests a bit

* Undo change

* fix validator

* Fix validator tests

* Run gazelle

* Fix error output for epoch spec tests
2019-09-08 12:41:52 -07:00
terence tsao
14c59b2ff9 Remove deprecated services and --next (#3371)
* Save new validators in DB

* Use info

* Add total validator count

* Fixed tests

* Add new test

* Revert light client config

* Add state metrics back

* Gaz

* Mark old ones as deprecated

* Deprecate not --next services

* Fixed all operation tests

* Fixed node test

* All tests passing locally

* Add deprecated-p2p back, blocked by boostrap-query

* Revert message proto

* delete deprecated DB items

* delete all other instances of old db

* gaz

* cycle rem

* clear db
2019-09-05 11:04:06 -05:00
terence tsao
75bce9b7e1
Align metrics to interop (#3406) 2019-09-05 08:32:35 -07:00
Raul Jordan
75c0b01932
Genesis State Generator + Interop Docs (#3405)
* add main.go

* interop readme

* proper visibility

* standardize and abstract into simpler funcs

* formatting

* no os pkg

* add test

* no panics anywhere, properly and nicely handle errors

* proper comments

* fix broken test

* readme

* comment

* recommend ssz

* install

* tool now works

* README

* build

* readme

* 64 validators

* rem print
2019-09-04 13:47:44 -05:00
terence tsao
60c048a0ec
Remove lock from store struct (#3400) 2019-09-03 13:14:23 -07:00
Raul Jordan
8ce8717676
Fix Prysm Deposit Formatting (#3394)
* proofs with proper size

* getting to the root of the problem, no pun intended

* add regression test and fix proofs

* debugging the receipt root

* debug

* fixed spec tests

* fixed up proofs!

* tests all pass
2019-09-03 12:47:47 -05:00
Preston Van Loon
90b2a880c6 Add /p2p page (#3391)
* add /p2p page

* fix tests
2019-09-03 11:07:40 -05:00
terence tsao
b52f32d17c Clean up configs (#3389) 2019-09-02 17:13:33 -07:00
Raul Jordan
da630f349f
Add Test for Aggregating Large Amount of Attestations (#3358)
* test for verifying large amount of agg sigs

* agg sgi could not verify

* 128 fails

* confirmed works for 512

* comprehensive test for handle att

* commented test

* fix up test

* include the proper wait group

* concurrency managed to reproduce verification bug

* concurrent test passes

* revert config changes

* use new db in operations tests

* debugging for the special attestations

* resolve tests

* fmt
2019-09-02 13:49:37 -05:00
shayzluf
b32c19a004
Slasher db (#3270)
* first version of the watchtower api

* first commit

* remove watchtower

* working version

* fix < 0

* gaz

* Update slasher/db/db.go

* remove clear history

* moved constant to config

* gaz

* feedback changes

* compare uint64

* add constant config

* PruneSlasherStoragePeriod change
2019-09-02 18:36:29 +03:00
Nishant Das
1f0aad31d2
Add Hello Tracking (#3342)
* lint

* add requests

* add all new stuff

* comment

* preston's review

* change to send

* remove topic and add lock

* add test

* lint

* change num of peers

* preston's review

* Update beacon-chain/p2p/handshake.go
2019-08-29 22:02:52 +05:30
Raul Jordan
7beafa159d
Support Starting Validator Binary from Unencrypted Keys JSON (#3308)
* next compatible, tests pass

* terence feedback

* skip comment

* fixes

* misc fix

* on block

* parse from unencrypted keys json

* mod val client

* launching unencrypted workssss

* fix broken build

* fix up build

* rem prints

* resolve lint

* bls comment

* fix docker deps

* gaz
2019-08-26 16:07:09 -05:00
Nishant Das
919877f301 Ignore Messages From Local Peer (#3299)
* validate message coming into pipeline

* gaz

* add to deprecated p2p

* add new lib

* change lib
2019-08-24 14:41:24 -04:00
Preston Van Loon
06c97256bc p2p --next: Register p2p peer count metrics (#3301) 2019-08-24 10:07:03 -06:00
Preston Van Loon
b59b3ec09c
P2P implement message send (#3278)
* return a stream with send, for reading response

* gofmt

* added sender impl

* fix imports
2019-08-22 19:02:46 -04:00
Preston Van Loon
ce28feea45 Regular sync: pubsub subscriber for beacon blocks (#3220)
* add validation

* add block db check in validation

* merge

* in memory caching of seen blocks

* basic block processing

* Update BUILD.bazel

* use new receiveBlockNoPubsub

* fix build

* add TODO issue numbers

* add TODO issue numbers

* lint
2019-08-22 13:11:52 -05:00
Nishant Das
83083b9c65
Fix BLS Aggregation Method (#3269)
* lint

* update to new method

* fix all tests
2019-08-22 11:45:02 +05:30
terence tsao
b91639a32e
Deprecate old block chain service (#3268)
* seperate out block chain services

* Fix blockchain service config

* Gazelle

* Fixed tests
2019-08-21 19:14:24 -06:00
Preston Van Loon
01de412956
Minor runtime fixes for --next (#3265)
* some runtime fixes

* fixes

* fixes

* fixes

* fixes

* fixes
2019-08-21 16:58:38 -04:00
Nishant Das
79e57e8e8e Add Flag for SSZ Encoding (#3256)
* add flag and enum

* change help message

* linter

* add flag

* add comment

* one more comment

* fix panic

* preston's comments

* add panic
2019-08-21 12:33:48 -04:00
Preston Van Loon
acb20e269c Add flags to support new database, new sync (#3252) 2019-08-21 10:04:00 -06:00
Preston Van Loon
0b8cbd06b6
Add flag for testing new p2p (#3243)
* refactor a bit to select p2p

* lint

* fix build

* fix build

* fix build

* fix build

* fix build
2019-08-19 17:20:56 -04:00
Preston Van Loon
e88bbaf614
Block networking in sandbox test by default, fix roughtime panic (#3240)
* block networking in sandbox test by default, fix roughtime panic

* Update .bazelrc
2019-08-19 12:13:05 -04:00
Raul Jordan
5f2e0493eb
Fix Slice Union Helpers for Variadic Arguments (#3228)
* union changes

* slice util fixes
2019-08-18 22:42:50 -05:00
Preston Van Loon
81f868bd48
Regular Sync - First Pass (#3201)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* Update WORKSPACE

* Update WORKSPACE
2019-08-16 13:13:04 -04:00
Raul Jordan
65ee6eb3af
Refactor Slice Utils as Variadic Functions (#3206)
* variadic approach working

* transform to variadic

* comments

* all variadic funcs simplified and tests passing
2019-08-14 17:08:53 -05:00
Raul Jordan
d0bdbe5a33
Add Byte Slice Intersection Utils (#3204)
* remove unused code

* table drive tests for byte slice intersections

* include all table driven tests

* gazelle imports

* imports
2019-08-14 10:27:18 -05:00
Preston Van Loon
82efca9b6f
Move p2p to deprecated-p2p (#3191)
* move to deprecated-p2p

* fix lint

* lint?

* fix lint

* lint

* lint

* lint

* lint
2019-08-13 14:52:04 -04:00
skillful-alex
4e886a84f9 Added roughtime to IsSlotValid and fixed test TestIsValidBlock_InvalidSlot (#3186)
* add roughtime to IsSlotValid

* gazelle

* gofmt -s
2019-08-13 13:59:11 -04:00
Preston Van Loon
fa0ef76561
Lower jaegar BufferMaxCount (#3188)
* Lower BufferMaxCount

* revert tools/cluster-pk-manager/client/main.go
2019-08-12 22:50:47 -04:00
Preston Van Loon
0ab969a87d fix panic on invalid bls key (#3184) 2019-08-12 13:58:25 -07:00
Jean-André Santoni
e1dfe73525 Query a roughtime server to mitigate NTP attacks (#3151) 2019-08-09 07:05:08 -07:00
Preston Van Loon
ccc7d8d7b7
Update BLS with @protolambda's improvements (#3152)
* Add @protolambda's fork until https://github.com/phoreproject/bls/pull/11

* update workspace
2019-08-07 22:54:33 -04:00
Preston Van Loon
cb5c920502 Add quick bls benchmark (#3148) 2019-08-05 17:11:38 -07:00
terence tsao
bccd2f95cc
Finish error wrapping (#3135) 2019-08-04 15:45:03 -07:00
terence tsao
0beb919fc0
Switch over to sha256-simd (#3125) 2019-08-02 14:42:59 -07:00
Preston Van Loon
953c59a302 Wrap errors (#3123) 2019-08-01 19:27:38 -07:00
Preston Van Loon
63cf0f07a2 Disable caches, allow toggle via feature flag (#3107) 2019-07-29 20:38:05 -07:00
Nishant Das
f72f7677b3 Replace Deposit Hash with HashTreeRoot (#3102)
* change to hashTreeRoot

* remove function and run gaz

* fix panic
2019-07-29 08:43:24 -05:00
Raul Jordan
4ebe2fb5b5 Implement AttestationPool and ListAttestations RPC Functions (#3061) 2019-07-26 10:07:20 -07:00
terence tsao
c1eeeef853 Implement ListValidatorAssignments RPC function (#3067)
* Need to sync latest ethereum API, will do it in master

* Added pagination wrapper

* Implemented ListValidatorAssignments

* Finished tests

* Fixed tests

* Raul's feedback

* Fmt

* Pagination test

* Removed extra loggings
2019-07-25 15:45:31 -04:00
Ivan Martinez
e452b46873
Remove Optional Signature Verification for VoluntaryExit and BlockHeader (#3053)
* Remove verifySignatures from ProposerSlashings

* Remove flag from process transfers

* resolve all conflicts

* fix more references to old pbs

* Fix merge conflicts

* Remove verifySignature flag from ProcessBlockHeader

* fx spectest

* Fix test errors

* Fix tests

* Fix tests

* Goimports

* Fix test finally

* Move test helpers to testutil

* Goimports

* Fix imports

* Add tests for new helpers

* Run gazelle

* Fix tests
2019-07-25 13:53:46 -04:00
Preston Van Loon
59253afb96
compare hex/hash bytes instead of strings (#3072) 2019-07-25 10:41:15 -04:00
Preston Van Loon
94a73e847f
Log errors in regular sync (#3070)
* add a error level logging for regular sync

* add a error level logging for regular sync

* add a error level logging for regular sync
2019-07-24 22:29:40 -04:00
terence tsao
17576af752
Implement GetValidators gRPC server (#3054) 2019-07-23 19:36:35 -07:00
Raul Jordan
a2caba9956
Optimize Sparse Merkle Trie (#3056)
* calc tree from leaves simpler

* fast generate proof

* align api to be the same

* ensure tests pass

* err condition

* travis

* fix build

* zero hashes work
2019-07-23 14:17:39 -05:00
Preston Van Loon
be514076c1 Remove optional verify signatures argument when verifying deposits (#3052)
* remove optional verification of deposit signatures

* use minimal config for easier setup

* progress

* progress

* Fix a few test errors

* Fix more of tests

* fix imports, gazelle

* fix rpc package

* fix blocks package

* fixed state test

* fixed powchain tests

* add comments

* remove todo

* Update beacon-chain/rpc/validator_server_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
2019-07-23 22:19:14 +05:30
Raul Jordan
8cfbf0309d
Resolve Proto Lint Issues (#3044)
* skip proto lint

* golang ci lint no -t flag

* regenerate protos to match schema

* move to compatibility folder

* build file compatibility

* foo
2019-07-22 14:10:17 -05:00
terence tsao
1b5b8a57e0 Remove unused proto schemas (#3005)
* Update io_kubernetes_build commit hash to 1246899

* Update dependency build_bazel_rules_nodejs to v0.33.1

* Update dependency com_github_hashicorp_golang_lru to v0.5.1

* Update libp2p

* Update io_bazel_rules_k8s commit hash to e68d5d7

* Starting to remove old protos

* Bazel build proto passes

* Fixing pb version

* Cleaned up core package

* Fixing tests

* 6 tests failing

* Update proto bugs

* Fixed incorrect validator ordering proto

* Sync with master

* Update go-ssz commit

* Removed bad copies from v1alpha1 folder

* add json spec json to pb handler

* add nested proto example

* proto/testing test works

* fix refactoring build failures

* use merged ssz

* push latest changes

* used forked json encoding

* used forked json encoding

* fix warning

* fix build issues

* fix test and lint

* fix build

* lint
2019-07-22 10:03:57 -04:00
Preston Van Loon
dfc64121c6
Remove unused feature flags (#3002)
* remove a few feature flags that are no longer needed

* remove other unused flags

* forgot a few more
2019-07-19 21:27:35 -04:00
Preston Van Loon
e744d1a07e Spec freeze updates (#2312)
* Optimize Shuffled Indices Cache (#2728)

* Refactor Deposit Contract Test Setup (#2731)

* add new package

* fix all tests

* lint

* change hash function (#2732)

* Remove Deprecated Validator Protobuf (#2727)

* Remove deprecated validator protos

* Fix to comments

* Fix most of skipped tests (#2735)

* Cache Active Validator Indices, Count, and Balances (#2737)

* Optimize Base Reward Calculation (#2753)

* benchmark process epoch

* revert prof.out

* Add some optimizations

* beware where we use ActiveValidatorIndices...

* revert extra file

* gaz

* quick commit to get feedback

* revert extra file

* started fixing tests

* fixed broken TestProcessCrosslink_NoUpdate

* gaz

* cache randao seed

* fixed all the tests

* fmt and lint

* spacing

* Added todo

* lint

* revert binary file

* started regression test

* basic tests done

* using a fifo for active indices cache

* using a fifo for active count cache

* using a fifo for total balance cache

* using a fifo for active balance cache

* using a fifo for start shard cache

* using a fifo for seed cache

* gaz

* clean up

* fixing tests

* fixed all the core tests

* fixed all the tests!!!

* lint

* comment

* rm'ed commented code

* cache size to 1000 should be good enough

* optimized base reward

* revert binary file

* Added comments to calculate adjusted quotient outside

* removed deprecated configs (#2755)

* Optimize Process Eth1 Data Vote (#2754)

* Cleanup and Docs update (#2756)

* Add graffiti and update generate seed (#2759)

* Benchmark Process Block with Attestations (#2758)

* Tidying up Godoc for Core Package (#2762)

* Clean up Old RPC Endpoints (#2763)

* Update RPC end point for Proposer (#2767)

* add RequestBlock

* run mockgen

* implemented RequestBlock

* updated proto definitions

* updated tests

* updated validator attest tests

* done

* comment

* todo issue

* removed unused proto

* Update attesting indices v0.6 (#2449)

* sort participants slice

* add bitfield functions and tests

* added BitfieldBit test

* add AttestationParticipantsNew

* revert AttestationParticipants to its previous change

* add tests

* remove verifybitfieldnew

* fix tests and remove multiple tests

* remove duplicate test

* change magic number into ceildiv8

* Implement Justification and finalization Processing (#2448)

* Add convert to indexed (#2519)

* sort participants slice

* add bitfield functions and tests

* added BitfieldBit test

* add AttestationParticipantsNew

* revert AttestationParticipants to its previous change

* add tests

* remove verifybitfieldnew

* fix tests and remove multiple tests

* remove duplicate test

* start work

* convert attestation to indexed attestations

* fix test for convert index

* remove calling getter

* add more tests

* remove underscore

* changes name to signature (#2535)

* update registry updates func (#2521)

* update registry updates func

* added tests and moved to epoch processing

* fixed naming issues

* Update Committee Helpers to v0.6.0 (#2398)

* Update Committee Helper Part 2 (#2592)

* Implement Process Slashings for 0.6 (#2523)

* Update Proposer/Attester Slashings and Slashing Helpers (#2603)

* Implement Final Updates 0.6 (#2562)

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* implemented process_final_updates

* move to the top of the file

* add comment back

* gaz

* Test for process final updates

* fixed tests

* fixed all the tests

* Update Reward Helper v0.6 (#2470)

* added BaseReward

* added rewards helper

* added test for BaseReward

* extra space

* move exported function above

* update to new spec (#2614)

* Update Block Processing Voluntary Exits (#2609)

* Update processEth1Data for v0.6 (#2516)

* Clean up Helper Functions Part 1 (#2612)

* Finalize helper functions for 0.6 (#2632)

* Process Beacon Chain Transfers v0.6 (#2642)

* add transfers

* beacon transfer operations complete

* full cov

* transfer testing

* finished tests

* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/core/blocks/block_operations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Implement Process Crosslink From 0.6 (#2460)

* Process Block Eth1 Data v0.6 (#2645)

* Get attestation data slot v0.6 (#2593)

* attestation.go is ready, tests not

* ready for review

* fixing linter issues

* modified Crosslink and AttestationData proto fields to spec 2.0,marked deprecated fields

* gazelle

* fixed tests

* fixed error

* error msg

* Process Block Deposits v0.6 (#2647)

* imports fixes

* deposits tests pass

* wrapped up gazelle

* spacing

* Implement Crosslink Delta Rewards for 0.6 (#2517)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* merged master

* all tests passing

* start testing

* done

* add ProcessBlockHeader v0.6 (#2534)

* add ProcessBlockHeader

* function has all its dependancies in place

* arange the basic ok test

* gazzele and skip test update

* skip wrong sig test

* fmt imports and change requests

* goimports fmt

* map for struct fields to be location independent

* reorder protobuf fields

* added tests

* gazzle fix

* few change requests fixes

* revert changes in types.proto

* revert changes in types

* fix tests

* fix lint

* fmt imports

* fix gazelle

* fix var naming

* pb update

* var naming

* tarance change request fixes

* fix test

* Add Process Registry for Epoch Processing (#2668)

* update update-process-registry

* added back the old tests

* fmt

* gaz

* Follow up on process block header v0.6 (#2666)

* Putting Crosslink Delta Back (#2654)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* merged master

* all tests passing

* start testing

* done

* fixed tests

* addressed shay's feedback

* Implement Attestation Delta for v0.6 (#2646)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* implemented process_attestation_delta

* comments

* comments

* merged master

* all tests passing

* start testing

* done

* merged master

* fixed tests

* tests, more to come

* tests done

* lint

* spaces over tabs

* addressed shay's feedback

* merged master

* Implement process_rewards_and_penalties for 0.6 (#2665)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* implemented process_attestation_delta

* comments

* comments

* merged master

* all tests passing

* start testing

* done

* merged master

* fixed tests

* tests, more to come

* tests done

* lint

* spaces over tabs

* addressed shay's feedback

* starting but need to merge a few things...

* tests

* fmt

* Update Slot Processing and State Transition v0.6 (#2664)

* edit state transition to add slot processing logic, reorder logic

* fix build

* lint

* tests passing

* spacing

* tests pass

* imports

* passing tests

* Implement Process Epoch for 0.6 (#2675)

* can't find process j f functons

* implemented process_epoch

* tests done

* lint

* nishant's feedback

* stupid goland replace

* goimports

* Update CommitteeAssignment (#2693)

* cleaned up skipped tests for core processing (#2697)

* Process Block Attestations v0.6 (#2650)

* attestation.go is ready, tests not

* ready for review

* fixing linter issues

* modified Crosslink and AttestationData proto fields to spec 2.0,marked deprecated fields

* gazelle

* fixed tests

* fixed error

* add att processing:

* process atts

* error msg

* finish process attestations logic

* spacing

* ssz move

* inclusion delay failure passing

* more attestation tests

* more att tests passing

* more tests

* ffg data mismatching test

* ffg tests complete

* gofmt

* fix testing to match attestation updates

* ssz

* lint

* Fixed Skipped Tests for RPC Server (#2712)

* Remove Obsolete Deposit Proto Objects (#2673)

* remove deposit data

* remove deposit input

* fix references

* remove deposit helpers

* fix all refs

* gaz

* rgene proto

* fix all tests

* remove deposit data deprecated field

* fix remaining references

* fix all tests

* fix lint

* regen proto

* fix test

* Remove Deprecated Protobuf State Fields (#2713)

* Remove Deprecated Protobuf Crosslink/Slashing/Block Fields (#2714)

* Remove Deprecated Beacon Block Proto Fields (#2717)

* Remove Deprecated Attestation Proto Fields (#2723)

* Cache Shuffled Validator Indices (#2682)

* YAML shuffle tests for v0.6 (#2667)

* new shuffle tests

* added comment for exported function

* fix format and print

* added config files handling

* gazelle fix

* shuffle test debugging

* added shuffle list and benchmark

* hash function addition from nishant code

* gazelle fix

* remove unused function

* few minor changes

* add test to test protos optimization

* test a bigger list

* remove commented code

* small changes

* fix spec test and test indices to pass

* remove empty line

* abstraction of repeated code and comment arrangement

* terence change requests

* fix new test

* add small comment for better readability

* change from unshuflle to shuffle

* comment

* better comment

* fix all tests

* Remove Latest Block (#2721)

* lint

* remove latest block

* lint

* add proto

* fix build

* Fix Deposit Trie (#2686)

* remove deposit data

* remove deposit input

* fix references

* remove deposit helpers

* fix all refs

* gaz

* rgene proto

* fix all tests

* remove deposit data deprecated field

* fix remaining references

* fix all tests

* fix lint

* new tests with contract

* gaz

* more tests

* fixed bugs

* new test

* finally fixed it

* gaz

* fix test

* Remove Committee Cache (#2729)

* Benchmark Compute Committee (#2698)

* Fixed Skipped Attestation Tests (#2730)

* Optimize Shuffled Indices Cache (#2728)

* Refactor Deposit Contract Test Setup (#2731)

* add new package

* fix all tests

* lint

* change hash function (#2732)

* Remove Deprecated Validator Protobuf (#2727)

* Remove deprecated validator protos

* Fix to comments

* Fix most of skipped tests (#2735)

* Optimize Base Reward Calculation (#2753)

* benchmark process epoch

* revert prof.out

* Add some optimizations

* beware where we use ActiveValidatorIndices...

* revert extra file

* gaz

* quick commit to get feedback

* revert extra file

* started fixing tests

* fixed broken TestProcessCrosslink_NoUpdate

* gaz

* cache randao seed

* fixed all the tests

* fmt and lint

* spacing

* Added todo

* lint

* revert binary file

* started regression test

* basic tests done

* using a fifo for active indices cache

* using a fifo for active count cache

* using a fifo for total balance cache

* using a fifo for active balance cache

* using a fifo for start shard cache

* using a fifo for seed cache

* gaz

* clean up

* fixing tests

* fixed all the core tests

* fixed all the tests!!!

* lint

* comment

* rm'ed commented code

* cache size to 1000 should be good enough

* optimized base reward

* revert binary file

* Added comments to calculate adjusted quotient outside

* removed deprecated configs (#2755)

* Optimize Process Eth1 Data Vote (#2754)

* Cleanup and Docs update (#2756)

* Add graffiti and update generate seed (#2759)

* Benchmark Process Block with Attestations (#2758)

* Tidying up Godoc for Core Package (#2762)

* Clean up Old RPC Endpoints (#2763)

* Update RPC end point for Proposer (#2767)

* add RequestBlock

* run mockgen

* implemented RequestBlock

* updated proto definitions

* updated tests

* updated validator attest tests

* done

* comment

* todo issue

* removed unused proto

* Cache Active Validator Indices, Count, and Balances (#2737)

* Update Deposit Contract (#2648)

* lint

* add new contract

* change version

* remove log

* generating abi and binary files

* fix tests

* update to current version

* new changes

* add new hash function

* save hashed nodes

* add more things

* new method

* add update to trie

* new stuff

* gaz

* more stuff

* finally fixed build

* remove deposit data

* Revert "remove deposit data"

This reverts commit 9085409e91be0c94550af10290eaad72ad40a6de.

* more changes

* lint and gaz

* lint

* Update Shard Helpers for 0.6 (#2497)

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* Update Genesis State Function to v0.6 (#2465)

* add pseudocode

* make changes

* fix all tests

* fix tests

* lint

* regen protos and mocks

* regenerated protos

* started fixing core

* all core tests passing!

* removed shared/forkutils

* started fixing blockchain package

* lint

* updating rpc package

* add back deleted stuff

* add back deleted stuff that was deleted accidentally

* add back protos and mocks

* fix errors

* fix genesis issue

* fix genesis issue for slot ticker

* fix all genesis errors

* fix build files

* temp change for go-ssz

* fix test

* Revert "temp change for go-ssz"

This reverts commit 3411cb9d6d519cb521181486debc3b21603c8873.

* update to latest go-ssz

* unstaged changes

* Update Attester Server RPC Calls (#2773)

* Update config and function parameters to v0.7 (#2791)

* Minor Updates to 0.7 (#2795)

* Refactor Deposit Flow and Cleanup Tests (#2788)

* More WIP on cleaning deposit flow

* Fix tests

* Cleanup and imports

* run gazelle

* Move deposit to block_operations

* gazelle

* Update beacon-chain/core/blocks/block_operations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Fix docs

* Remove unneeded calculations

* Fix tests

* Fix tests finally (?)

* Optimize Committee Assignment RPC (#2787)

* Update BlockRoot to BlockHash (#2816)

* Fix Final Missing Items in Block Processing v0.6 (#2710)

* override config successfully

* passes processing

* add signing root helper

* blockchain tests pass

* tests blocked by signing root

* lint

* fix references

* fix protos

* proper use of signing root

* only few failing tests now

* fix final test

* tests passing

* lint and imports

* rem unused

* Update beacon-chain/core/blocks/block_operations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* lint

* Update beacon-chain/attestation/service_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/db/block_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* rename to hash tree root

* rename decode to unmarshal

* fix

* use latest ssz

* all tests passing

* lint

* fmt

* Add Config YAML for Spec Tests (#2818)

* Align Protobuf Type Names (#2825)

* gofmt

* Revert "Align Protobuf Type Names (#2825)" (#2827)

This reverts commit 882d067144c674bbf4eeee00404acaa0a9b1fd33.

* Update Domain Related Functions (#2832)

* Add Functions for Compressed and Uncompressed HashG2 With Domain (#2833)

* add tests

* gaz

* lint

* Revert "Add Functions for Compressed and Uncompressed HashG2 With Domain (#2833)" (#2835)

This reverts commit 7fb2ebf3f12f043d7bd12f43500dc2bd133df573.

* Add ConvertToPb to package testutil (#2838)

* Block Processing Bug Fixes (#2836)

* Update types PB with Size Tags (#2840)

* Epoch processing spec tests (#2814)

* Remove Deposit Index (#2851)

* Shuffle tests revisited (#2829)

* first commit

* remove old files, add log

* remove duplicate yaml testing code

* reduce visability

* nishant feedback changes

* skip TestFromYaml_Pass

* added tags to bazel build

* gazelle fix

* remove unused vars

* adda back config

* remove config handling

* remove unused var

* gazelle fix

* SSZ compatibility test for protobufs (#2839)

* update workspace spec sha

* remove yamls from branch

*  BLS spec tests (#2826) (#2856)

* bls spec tests

* add more bls tests

* use ioutil instead of bazel runfiles

* dont read bytes

* skip tests that overflow uint64

* manually fix input data

* add tests

* lint and gaz

* add all new changes

* some refactoring, cleanup, remove new API methods that only exist for tests

* gaz

* Remove yamls, skip test

* Slot processing spec test (#2813)

* eth1data rpc endpoint (#2733)

* eth1data rpc endpoint

* first version

* comment added

* gazelle fix

* new function to go once over the deposit array

* fix tests

* export DepositContainer

* terence feedback

* move structure decleration

* binary search

* fix block into Block

* preston feedback

* keep slice sorted to remove overhead in retrival

* merge changes

* feedback

* update to the latest go-ssz

* revert change

* chnages to fit new ssz

* revert merge reversion

* go fmt goimprts duplicate string

* exception for lint unused doesParentExist

* feedback changes

* latesteth1data to eth1data

* goimports and stop exposing Eth1Data

* revert unneeded change

* remove exposure of DepositContainer

* feedback and fixes

* fix workspace duplicate dependancy

* greatest number of deposits at current height

* add count votes function

* change method name

* revert back to latesteth1data

* latesteth1data

* preston feedback

* seperate function add tests fix bug

* stop exposing voteCountMap

* eth1data comment fix

* preston feedback

* fix tests

* new proto files

* workspace to default version of ssz

* new ssz

* chnage test size

* marshalled  marshaled

* Attesting Indices Fix (#2862)

* add change

* fix one test

* fix all tests

* add test

* clear cache

* removed old chaintest, simulated backend and state generator (#2863)

* Block Processing Sanity Spec Tests (#2817)

* update PrevEpoch

* add new changes

* shift to blocks package

* add more changes

* new changes

* updated pb with size tags

* add new changes

* fix errors

* uncomment code

* more changes

* add new changes

* rename and lint

* gaz

* more changes

* proccess slot SigningRoot instead of HashTreeRoot

* ensure yaml generated structs work

* block sanity all passing

* minimal and mainnet all pass

* remove commented code

* fix one test

* fix all tests

* fix again

* no state comparison

* matching spec

* change target viz

* comments gazelle

* clear caches before test cases

* latest attempts

* clean up test format

* remove debugging log, remove yaml

* unskip attestation

* remove skip, check post state, diff state diffs

* handle err

* add bug fixes

* fixed one more bug

* fixed churn limit bug

* change hashProto to HashTreeRoot

* all tests pass :)

* fix all tests

* gaz

* add regression tests

* fix test bug

* Mutation testing fixes for beacon-chain/core/helpers/attestation.go (#2868)

* mutation testing for attestation.go

* new line

* lint

* revert fmt.Errorf deletion

* gofmt

* Add some fixes for mutation testing on blocks.go (#2869)

* Fix sizes

* gaz

* Spec freeze release candidate spectests

* Align Protobuf Type Names  (#2872)

* Removes some deprecated fields from protobuf (#2877)

* search and replace checkpoints

* fix tests, except spec tests

* Update Configs for Freeze (#2876)

* update configs

* updated minimal configs

* almost there

* all tests passing except for spec tests

* better comment for MinGenesisTime

* done, ready for review

* rm seconds per day

* feedback

* Mutation testing fixes for beacon-chain/core/helpers/committee.go (#2870)

* Add some fixes for mutation testing on blocks.go

* working on mutation testing fo committee.go

* gofmt

* goimports

* update readme target

* update latest sha for spec tests

* fix build

* Update State Transition Function (#2867)

* Change Base Reward Factor (#2888)

* Update Freeze Spec Simplification Section - part 1 (#2893)

* finished changes to attesting_indices

* removed index_count <= 2**40 requirement

* lint

* reverted index_count <= 2**40 check

* added short cut len(a) > len(b)

* Update justification bits (#2894)

* updated all the helper pseudocodes (#2895)

* Make Constants Explicit and Minor Cleanups (#2898)

* Rename outdated configs, make constants explicitly delcared

* Remove activate_validator, not needed

* Remove GenesisSlot and GenesisEpoch

* Remove unused import

* Move Block Operation Length Checks to ProcessOperations (#2900)

* Move block operation length checks to ProcessOperations

* Write tests for each length check in ProcessOperations

* Remove unneeded test

* Move checks to a new function

* Move duplicate check back into ProcessOperations

* reorder proto fields (#2902)

* Slashing Penalty Calculation Change (#2889)

* lint

* change config val

* add max helper

* changes to slashing and process slashing, add a min function for integers

* gaz

* fix failing tests

* fix test

* fixed all tests

* Change Yaml tag

* lint

* remove gc hack

* fix test

* gaz

* preston's comments

* change failing field

* fix and regen proto

* lint

* Implement Compact Committee Root (#2897)

* add tags

* add function

* add new code

* add function

* add all new changes

* lint

* add tests

* fix tests

* fix more tests

* fix all outstanding tests

* gaz

* Update beacon-chain/core/helpers/committee.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* comment

* Remove deprecated fields from attestation data (#2892)

* fix broken tests

* remove comment

* fixes

* Update Deposit Contract (#2903)

* update to new contract

* fix references

* fix tests

* fix some more tests

* fix local deposit trie

* gaz

* shays review

* more changes

* update WORKSPACE to use 0.8 spec tests

* Perform Mutesting in Helpers Package (#2912)

* Perform mutesting on validator

* Mutesting in helpers package

* Mutested eth1data

* s/volundary/voluntary (#2914)

* Update BLS Domain (#2916)

* change from integer to byte slice

* add test

* fix func for bytes4

* Fix Spec tests (#2907)

* fix panics

* handle failed transitions

* remove log

* fix to protos

* new changes

* remove i

* change ssz commit

* new changes

* update epoch tests

* fix epoch testing

* fix shuffle tests

* fix test

* Perform Mutesting in Epoch and State Packages (#2913)

* done with updates (#2890)

* Add Max Size Tag for Protobuf Fields (#2908)

* No more space between ssz-size numbers

* Regen pb.go

* Fixed a few incorrect proto fields (#2926)

* Update Validator Workflow (#2906)

* Justification spec tests (#2896)

* update go-ssz

* Fix SSZ Compatibility Test (#2924)

* figuring out how to seqeeze in multiple fields in a tag for pb

* Added max tags and regenerated pb.go

* updated to new standard

* New bitfield types in proto (#2915)

* Cast bytes to correct bitfield, failing tests now though

* Add forked gogo/protobuf until https://github.com/gogo/protobuf/pull/582

* remove newline

* use proper override for gogo-protobuf

* fix a few tags

* forgot to include custody bits and Slashable not used

* Update yaml struct to use pb

* Update workspace to use latest ssz

* All tests fail

* Use the latest go-ssz commit

* All pass except for state (too long to taste)

* Update test.proto

* Added rest of the tests

* use 1 justification bits

* fix tag test, apply @rauljordan's suggestion

* add IsEmpty and use ssz struct

* delete unused file

* Update zero hash to sha256().digest() (#2932)

* update zero hash

* change zero hash to conform with spec

* goimports

* add test for zero hash

* Revert "Update zero hash to sha256().digest() (#2932)" (#2933)

This reverts commit b926ae0667b18aef3f7e0e8ec8a9b3e98b3d2ccc.

* Fix compress validator (#2936)

* fix compress validator

* update go-ssz

* build without the bytes test

* try minimal

* Block operations spec tests (#2828)

* update PrevEpoch

* debugging proposer slashing tests

* fmt

* add deposit tests

* Added skeleton for attestation minimal test

* remove bazel runfiles thing

* add deposits

* proposer slashing test is done

* comment

* complete test, some failing cases

* sig verify on

* refactor slightly to support mainnet and minimal

* included mainnet stuff

* Add block header tests

* volunary exit done

* transfer done

* new changes

* fix all tests

* update domain functions

* fmt

* fixed lint

* fixed all the tests

* fixed a transfer bug

* finished attester slashing tests and fixed a few bugs

* started fixing...

* cleaned up exit and proposr slashing tests

* attester slashing passing

* refactored deposit tests

* remove yamls, update ssz

* Added todo for invalid sig

* gazelle

* deposits test done!

* transfer tests done and pass!

* fix attesting indices bug

* temporarily disabled signature verification

* cleaned up most of the block ops, except for att

* update committee AttestingIndices

* oops, i dont know how or why i changed this file

* fixed all the rpc tests

* 6 more failing packages

* test max transfer in state package

* replace hashproto with treehash in package blockchain

* gazelle

* fix test

* fix test again

* fixed transition test, 2 more left

* expect an error in attestation tests

* Handle panic when no votes in aggregate attestation

* clear cache

* Add differ, add logging, tests pass yay

* remove todo, add tag

* fixed TestReceiveBlock_RemovesPendingDeposits

* TestAttestationMinimal/success_since_max_epochs_per_crosslink fails now...

* handle panics

* Transfer tests were disabled in https://github.com/ethereum/eth2.0-specs/pull/1238

* more fixes after merge, updating block_operations.yaml.go to match yaml

* figuring out how to seqeeze in multiple fields in a tag for pb

* Added max tags and regenerated pb.go

* updated to new standard

* New bitfield types in proto (#2915)

* Cast bytes to correct bitfield, failing tests now though

* Add forked gogo/protobuf until https://github.com/gogo/protobuf/pull/582

* remove newline

* fix references and test panic

* change to proto objects from custom types

* fix panics in tests

* use proper override for gogo-protobuf

* fix a few tags

* fix tests

* forgot to include custody bits and Slashable not used

* fix tests

* sort again

* Update yaml struct to use pb

* Update workspace to use latest ssz

* All tests fail

* Use the latest go-ssz commit

* All pass except for state (too long to taste)

* Update test.proto

* Added rest of the tests

* use 1 justification bits

* minor fixes

* wrong proto.Equal

* fix tag test, apply @rauljordan's suggestion

* add IsEmpty and use ssz struct

* inverted logic

* update zero hash

* change zero hash to conform with spec

* goimports

* add test for zero hash

* Revert "Update zero hash to sha256().digest() (#2932)"

This reverts commit b926ae0667b18aef3f7e0e8ec8a9b3e98b3d2ccc.

* update ssz, fix import, shard big test

* checkpoint

* fix compress validator

* update go-ssz

* missing import

* missing import

* tests now pass

* been a good day

* update test size

* fix lint

* imports and remove unused const

* update bazel jobs flag

* update bazel jobs flag

* satisfy deprecation warning

* Add ssz regression tests for investigation of test failures in PR #2828 (#2935)

* Adding regression tests for investigation

* add another example

* goimports

* add quick comment about test case 0

* Epoch Process Slashings Spec Tests (#2930)

* updated justification bits, tests passing OK

* regen pb.go, clarify bit operations

* justification and finalization tests; failing

* Add wrapper, so we call the correct method

* checkpoint

* Update tar ref

* TestSlashingsMinimal/small_penalty still failing

* Use bigint instead of () and float

* Revert a bad merge from workspace

* Fmt

* add note about https://github.com/ethereum/eth2.0-specs/issues/1284

* improve tests

* gaz

* Perform Mutesting In core/state Package (#2923)

* Perform mutesting on validator

* Mutesting in helpers package

* Mutested eth1data

* Perform mutesting in epoch and state packages

* Fix voluntary exits test

* Fix typo

* Fix comments

* Fix formatting

* Fix error message

* Handle missing errors

* Handle all errors

* Perform Mutesting In State Package

* Fix block roots size

* Remove comment

* Fix error

* add backend service

* Add ssz compatibility tests for signing root (#2931)

* Added tests for signing root

* imports

* fix lint on travis

* fix bes flag

* Final updates spec tests (#2901)

* set up tests

* need to reorder pbs

* figuring out how to seqeeze in multiple fields in a tag for pb

* Added max tags and regenerated pb.go

* updated to new standard

* New bitfield types in proto (#2915)

* Cast bytes to correct bitfield, failing tests now though

* Add forked gogo/protobuf until https://github.com/gogo/protobuf/pull/582

* remove newline

* use proper override for gogo-protobuf

* fix a few tags

* forgot to include custody bits and Slashable not used

* playing with tags idea, can revert this commit later

* fixes after merge

* reset caches before test

* all epoch tests pass

* gazelle

* Genesis trigger (#2905)

* genesis change

* integrate changes

* bodyroot

* remove unused code

* HasChainStarted

* added isValidGenesisState to ProcessLog

* state fix

* fix gazelle

* uint64 timestamp

* SetupInitialDeposits adds proof

* remove unneeded parts of test

* deposithash

* merkleproof from spec utils

* Revert "merkleproof from spec utils"

This reverts commit 1b0a124352e7b62e3c3220fb0d64e295b474b430.

* fix test failures

* chain started and hashtree root in tests

* simple eth2genesistime

* eth2 genesis time

* fix zero time

* add comment

* remove eth1data and feedback

* fix build issues

* main changes: add fields and methods to track active validator
count

* gaz

* fix test

* fix more tests

* improve test utils

* shift spec method to state package, improve test setup

* fixed log processing tests

* remove log

* gaz

* fix invalid metric

* use better tag names, not latest

* Remove Block Signing Root (#2945)

* replace with hash tree root

* Revert "replace with hash tree root"

This reverts commit 77d8f16a160e42f3c3d598df66c30a66657de1bf.

* replace with signing root instead

* remove one more ref

* Create Test Runner for Genesis State Spec Tests (#2940)

* genesis change

* integrate changes

* bodyroot

* remove unused code

* HasChainStarted

* added isValidGenesisState to ProcessLog

* state fix

* fix gazelle

* uint64 timestamp

* SetupInitialDeposits adds proof

* remove unneeded parts of test

* deposithash

* merkleproof from spec utils

* Revert "merkleproof from spec utils"

This reverts commit 1b0a124352e7b62e3c3220fb0d64e295b474b430.

* fix test failures

* chain started and hashtree root in tests

* simple eth2genesistime

* eth2 genesis time

* fix zero time

* add comment

* remove eth1data and feedback

* fix build issues

* main changes: add fields and methods to track active validator
count

* gaz

* fix test

* fix more tests

* improve test utils

* Start genesis spec tests

* shift spec method to state package, improve test setup

* Add Genesis validity spec test

* Bazel

* fixed log processing tests

* remove log

* gaz

* fix invalid metric

* use json tags

* fix up latest changes

* Fix most of test errors

* Attempts to see whats wrong with genesis validity

* Fix merge

* skip minimal

* fix state test

* new commit

* fix nishant comment

* gaz

* Static check on branch spec-v0.6 (#2946)

* Ran staticcheck and fixed the important complains

* commit

* commit

* Create Test Runner for Genesis State Spec Tests (#2940)

* genesis change

* integrate changes

* bodyroot

* remove unused code

* HasChainStarted

* added isValidGenesisState to ProcessLog

* state fix

* fix gazelle

* uint64 timestamp

* SetupInitialDeposits adds proof

* remove unneeded parts of test

* deposithash

* merkleproof from spec utils

* Revert "merkleproof from spec utils"

This reverts commit 1b0a124352e7b62e3c3220fb0d64e295b474b430.

* fix test failures

* chain started and hashtree root in tests

* simple eth2genesistime

* eth2 genesis time

* fix zero time

* add comment

* remove eth1data and feedback

* fix build issues

* main changes: add fields and methods to track active validator
count

* gaz

* fix test

* fix more tests

* improve test utils

* Start genesis spec tests

* shift spec method to state package, improve test setup

* Add Genesis validity spec test

* Bazel

* fixed log processing tests

* remove log

* gaz

* fix invalid metric

* use json tags

* fix up latest changes

* Fix most of test errors

* Attempts to see whats wrong with genesis validity

* Fix merge

* skip minimal

* fix state test

* new commit

* fix nishant comment

* gaz

* Add Back Eth1Data After Bad Merge (#2953)

* eth1data rpc endpoint

* first version

* comment added

* gazelle fix

* new function to go once over the deposit array

* fix tests

* export DepositContainer

* terence feedback

* move structure decleration

* binary search

* fix block into Block

* preston feedback

* keep slice sorted to remove overhead in retrival

* merge changes

* feedback

* update to the latest go-ssz

* revert change

* chnages to fit new ssz

* revert merge reversion

* go fmt goimprts duplicate string

* exception for lint unused doesParentExist

* feedback changes

* latesteth1data to eth1data

* goimports and stop exposing Eth1Data

* revert unneeded change

* remove exposure of DepositContainer

* feedback and fixes

* fix workspace duplicate dependancy

* greatest number of deposits at current height

* add count votes function

* change method name

* revert back to latesteth1data

* latesteth1data

* preston feedback

* seperate function add tests fix bug

* stop exposing voteCountMap

* eth1data comment fix

* preston feedback

* fix tests

* new proto files

* workspace to default version of ssz

* new ssz

* chnage test size

* marshalled  marshaled

* everything passing again

* add skip reason

* cleanup deposit contract slightly

* remove unused chainstart param (#2957)

* fix breakages from #2957 (#2958)

* fix breakages from #2957

* oops

* Fix deposit input data (#2956)

* fix deposit input data

* fix deposit input data

* gaz and build fix

* Add Tests for Genesis Deposits Caching (#2952)

* remove old method and replace with an improved one

* add new files

* gaz

* add test

* added all these tests

* gaz

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fix merkle proof error

* fix config

* Minor fixes for runtime (#2960)

* Minor fixes for runtime

* use comments

* goimports

* revert beacon-chain/core/state/state.go and fix comment for lint

* fix test too

* Minor runtime fixes  (#2961)

* Add support for bundling binaries and fix ARM64 builds (#2970)

* Add support for bundling binaries and fix ARM64 builds

* Fix exports

* ignore manual targets wrt vis check

* fix graknlabs

* update spec tests (#2979)

* hotfix until https://github.com/graknlabs/bazel-distribution/pull/169

* Overflow slashing calculation fix (#2977)

* Runtime Fixes (#2736)

* first batch of fixes

* add log

* more fixes

* another bug fixed

* update deposit contract and other fixes

* remove logs

* new changes

* fixes

* fix build

* remove config

* more fixes

* add more changes

* add back todo

* make compute state root work

* remove commented out and fix condition

* fix commented code

* fix config

* gaz

* remove flag

* remove init

* new fixes

* fix test

* one more fix

* fix all tests

* change back config

* fix one more bug

* remove logging bool

* Only build test targets when running bazel test //...

* Align prysm to spec v0.8.1 (#2978)

* Bazel problem

* Update zero hash representation to be clear (cosmetic)

* Update minor cosmetic fixes

* Fixed lookahead off by 1

* Update randao.go

* update ssz

* test failures fixed

* test fixes

* fix up workspace

* lint

* fixed errs

* Updated pubkey loggings (#2983)

* Fix proposer assignment (#2984)

* add jvm limits

* add jvm limits

* Removed logging from state transition functions (#2980)

* Match spec on proposer index division (#2985)

* Match spec on proposer index division

* gaz

* fixes

* Fix Default Eth1Data (#2982)

* fix bug

* Update beacon-chain/core/state/transition.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* more fixes

* reg test

* Set ejection balance to 1.6 (#2987)

* improve block processing log (#2990)

* Fix HistoricalRootsLimit (#2989)

* fix historical lenght

* fix genesis state initialization and test

* fix genesis state initialization and test

* fix genesis state initialization and test

* fix genesis state initialization and test

* hack config until https://github.com/prysmaticlabs/prysm/issues/2993

* More Runtime Fixes (#2986)

* local changes

* add val sig

* attester fix

* one more fix

* fixed all tests

* rem validator issue

* fix finality issue (#2994)

* Fix validator prev balance calculation (#2992)

* attester fix

* one more fix

* fixed all tests

* Fix validator prev balances calculation

* go fmt

* 48 bytes

* Fix Querier to handle Deposit Logs Race (#2999)

* fix sync issue

* fix build

* add regression test

* add var for magic number

* Fix eth1data and deposits (#2996)

* Work in progress, eth1data works, deposits are included at the appropriate time, and activation happens at the correct time

* revert blockInfo being public

* git tests to build, not yet pass though

* add tests

* some commentary

* fix comment

* goimports

* fmt and remove unused method

* Update rules go (#2975)

* update rules_go

* Fix some cross compile builds stuff

* add missing deps

* update to 0.19.1

* Update Protobufs to Match Ethereum APIs (#2998)

* add beacon block and attestation files

* add all types

* include all new proto type definitions

* add add all proto definitions

* fix all comments to say 48 bytes

* include latest changes

* readd common

* no swag

* add build file

* deps issue

* right package names

* address feedback, maintain parity between upstream ethereumapis

* delete pb

* bad gens

* Update "Testing Prysm" readme section (#3000)

Resolves invalid url link to golangci-lint.

* Update badge to version 0.8.1

* elaborate on test skip

* revert shared/p2p/options.go

* make travis happy with goimports

* Update beacon-chain/core/blocks/block.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
2019-07-19 19:16:10 -05:00
Preston Van Loon
f12fdfda0f Complain about improperly sized tests (#2873) 2019-06-30 09:00:22 -07:00
Dan
9ac950f480 Added the 'enable-upnp' flag to the list of command line args (#2860)
* Added the 'enable-upnp' flag to the list of supported command line arguments.

If the user specifies this feature flag (adds --enable-upnp as an argument) - the Beacon-chain and Validator services, when started, will initialize libp2p with the UPNP options.

* Added the new arg to usage.go due to test failure

* Update shared/p2p/service.go

Changed the logging according to Preston's recommendation.

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Code review changes:

1. File formatting.
2. Command line arg more detailed description.
2019-06-26 12:59:37 -07:00
Preston Van Loon
9bd6147027
BLS spec tests (#2826)
* bls spec tests

* add more bls tests

* use ioutil instead of bazel runfiles

* dont read bytes

* skip tests that overflow uint64

* manually fix input data

* add tests

* lint and gaz

* add all new changes

* some refactoring, cleanup, remove new API methods that only exist for tests

* gaz

* Remove yamls, skip test
2019-06-25 12:57:47 -04:00
Preston Van Loon
cebefde335
Update config.go to fix BLS issue (#2821)
* Update config.go

* fix test
2019-06-18 18:02:01 -04:00
Hsien-Tang Kao
3a167e54b5 Properly log and handle HTTP server error (#2685) 2019-06-12 19:58:49 -04:00
Nishant Das
663490ee1f Update Phore Dependency to the latest version (#2792)
* add new changes

* fix typos
2019-06-12 11:06:39 -05:00
Preston Van Loon
d9ee55013d
Use generalized examples in test (#2786) 2019-06-10 14:29:15 -04:00
renovate[bot]
9793de59a6 Update libp2p (#2783)
* Update libp2p

* Update libp2p, remove unused WORKSPACE go_repositories, fix test
2019-06-10 12:17:42 -04:00
Preston Van Loon
fbac09c1f6
Add go maxprocs metric (#2765) 2019-06-07 15:43:22 -04:00
Preston Van Loon
55bedd0745 Move go-ssz to external repo under MIT license (#2722) 2019-05-29 18:04:25 -07:00
Antoine Toulme
932e68571b expose p2p private key for static peering (#2719)
* expose p2p private key for static peering

* Review revisions

* Use testutil.TempDir()

* Use testing.T to report fatal errors
2019-05-29 15:43:23 -04:00
Antoine Toulme
fcc54317a3 Fix static peering (#2725)
* Fix static peering

* Fix import and use a slice to assemble peers to watch for

* Add a check for zero-length

* Fix duplicate import
2019-05-29 15:30:26 -04:00
Preston Van Loon
9ce5de3d95 Add flag to whitelist certain connections (#2716) 2019-05-28 05:54:49 -07:00
Dan
b040ac909e Persistent logs (#2660)
* added file log feature

* moved the logic to one central location (shared/logutil/logutil.go), removed new line chars from file logs

* removed a resdundant temp file refrences that went into beachon-chain/BUILD.bazel by mistake

* Update shared/cmd/flags.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Update shared/cmd/flags.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* manually added loguitl dep to the go image target

* Manaully added the logutil dep to the go image target

* Update shared/logutil/logutil.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/main.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/main.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* syntax and styling changes required by code reviewers

* Update beacon-chain/main.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/main.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* changed the return type of 'ConfigurePersistentLogging' from bool, error to error based on recommendation from code review

* ran goimports in beacon-chain/main.go after tests have failed

* Update beacon-chain/main.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/main.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update beacon-chain/main.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update shared/logutil/logutil.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/main.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Update validator/main.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Changes requested by code reviewer

* Added a mandatory comment (linter required)  to the 'Fire' event

* Changed the beacon-chain and validator to support only same format stdout and file logging due to complications in the outputs when using different formats.

* Had to run gazelle --fix due to check failure
2019-05-22 09:22:11 -04:00
Raul Jordan
50063912a8 Remove Expensive Participation Rate Prometheus Gauge (#2636)
* rem expensive prom gauge

* rem prom
2019-05-17 19:57:13 -04:00
Preston Van Loon
bdf4590b86 Fix config value, yaml fixes (#2634)
* minor fixes

* go back to 8
2019-05-17 19:27:13 -04:00
Preston Van Loon
208c5dfea6 Revert "Disable libp2p TLS security protocols for now (#2622)" (#2633) 2019-05-17 18:59:01 -04:00
Nishant Das
25ce3a3676 Add Excess Deposit Flag to allow Validator Balances more than 32 ETH (#2625)
* add flags and code

* adding tests

* gaz

* Update shared/featureconfig/config.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
2019-05-17 17:10:34 -04:00
terence tsao
3349fb4cba increase slots per epoch to 16 (#2627) 2019-05-17 14:41:49 -04:00
Preston Van Loon
15cac0c0b1
Disable libp2p TLS security protocols for now (#2622)
* Disable security protocols for now

* Enabling security for test only. See https://github.com/libp2p/go-libp2p-swarm/issues/124

* Fix spacing
2019-05-17 13:29:25 -04:00
terence tsao
b9fe8b172c Filter Canonical Attestation by Default (#2626)
* exclusive of finalized block

* filter canonical attestations by default
2019-05-17 13:17:29 -04:00
Preston Van Loon
d20c3d6cf7 Add better, incremental reputation (#2618)
* add better, incremental reputation

* remove space

* Lint
2019-05-17 22:04:38 +08:00
Preston Van Loon
4837629091 Add timestamp metadata to p2p messages (#2611) 2019-05-15 21:28:02 -07:00
Nishant Das
64ce41f9fc Add Check for Goroutines Count (#2608)
* changes

* revert ide

* goimports

* Update shared/cmd/flags.go
2019-05-15 10:38:27 -04:00
Preston Van Loon
fd4c7ffc07
Add new log formats and set fluentd as default for cluster (#2594)
* Add new log formats and set fluentd as default for cluster

* fix image build
2019-05-13 20:43:04 -04:00
Antoine Toulme
c9ce8b5246 Allow discovery to be removed, and add peers explicitly to peer store (#2557)
* Allow discovery to be removed, and add peers explicitly to peer store

* Changes after code review

* Update shared/cmd/flags.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
2019-05-11 18:02:58 -04:00
Preston Van Loon
9f7f7d6cff
Tracing improvements (#2570)
* some improvements

* fix

* gazelle

* disable lostcancel
2019-05-11 17:43:55 -04:00
Nishant Das
678ffa607e Fix Bitfield in Attestations (#2565)
* fix bitfield

* test

* fix reference

* fix tests

* remove test

* fix test

* add new helper

* add test

* fix tests

* fix test

* gaz

* add continue
2019-05-11 16:49:09 -04:00
Preston Van Loon
fc1fbf8017
Use a prysm specific DHT protocol (#2558)
* use a prysm specific DHT

* gazelle

* space
2019-05-10 11:56:30 -04:00
Preston Van Loon
9a82845c3c Fix lint issues (#2554)
* fix broadcast debug message

* feedback

* imports

* lint
2019-05-10 11:59:30 +08:00
Preston Van Loon
65f4c78750
Only marshal broadcast debug message when actually logging debug (#2553)
* fix broadcast debug message

* feedback
2019-05-09 22:57:47 -04:00
terence tsao
13e9bb5020 Filter Canonical Attester for RPC (#2551)
* exclusive of finalized block

* add filter to only include canonical attestation

* comments

* grammer

* gaz

* typo

* fixed existing tests

* added test for IsAttCanonical

* add nil blocks test
2019-05-09 18:53:19 -05:00
Preston Van Loon
991ee7e81b "Super sync" and naive p2p reputation (#2550)
* checkpoint on super sync with reputation

* ensure handling only expected peers msg

* exclusive of finalized block

* skip block saved already

* clean up struct

* remove 2 more fields

* _

* everything builds, but doesnt test yet

* lint

* fix p2p tests

* space

* space

* space

* fmt

* fmt
2019-05-09 16:02:24 -05:00
Preston Van Loon
e5a00fdca9
Add GCP test configuration and p2p-host-ip flag (#2510)
* Add GCP startup script

* add flag for external IP

* specify that it must be for linux

* /deploy/create

* gofmt
2019-05-06 13:33:19 -04:00
Raul Jordan
679493c786
issue warn log on disconnecting peer instead of error (#2491) 2019-05-05 11:22:04 -05:00
terence tsao
7f4eafb7b4
3.175 (#2482) 2019-05-03 16:57:37 -07:00
Raul Jordan
f876df42d7
ValidatorStatus Estimating Activation RPC Server (#2469)
* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename
2019-05-02 14:14:25 -05:00
Zak Cole
8b510e7fad typo (#2464)
negotiation was spelled wrong and it confused me for a second.
2019-05-01 16:49:59 -07:00
terence tsao
669ad3ee71
raise ejection bal to 3.15 (#2451) 2019-04-30 10:17:23 -07:00
terence tsao
63998b92aa Add and Use Truncate Hash Library (#2417)
* add truncate lib

* use trunc

* spaces
2019-04-29 13:11:51 -05:00
terence tsao
4a90f82bcc Fix Validator Ejection for Testnet (#2438)
* set ejection threshold and add gauge

* spaces

* gaz
2019-04-29 09:39:31 -05:00
Raul Jordan
63633a3bee Enable Historical State Pruning By Default (#2433)
* disable historical state pruning

* imports

* lint
2019-04-29 10:53:03 +08:00
terence tsao
85daf6e807
update to 3.2ETH (#2415) 2019-04-28 14:39:05 -07:00
Preston Van Loon
80d47468e4 Add a temporary commenting for issue #2408 (#2409) 2019-04-27 21:56:28 -05:00
Preston Van Loon
5167fadbd7 Serve pprof http (#2406)
* serve pprof http

* add comment for linter
2019-04-27 13:41:46 -07:00
Preston Van Loon
769fcb50ab
use Network, not Peerstore (#2404) 2019-04-27 16:05:19 -04:00
Preston Van Loon
210edfc940 P2P handshake handling (#2306)
* proto: illegal wireType 7 :(

* set addr to todo for now so somethign gets sent

* push latest progress

* Add feedback from @raulk. Stream never connects

* working handshake handler

* add exclusions for relay/bootstrap node

* fix tests, still need to add new ones

* remove race, fails coverage

* Add test for negotiation

* gazelle

* regen pb

* Update shared/p2p/handshake_handler.go

Co-Authored-By: prestonvanloon <preston@prysmaticlabs.com>
2019-04-27 14:08:27 -05:00
Raul Jordan
81c8b130c6
Only Sync With the Peer With the Highest Observed Slot (#2280)
* only accept the highest finalized slot from peers

* use best peer for state requests

* peer id

* rem old tests

* req peers tests pass

* warn level support for peer not having the same protocol

* pretty

* best peer

* comments

* req peers done

* tests passing

* fully functional

* enforce receiving from the best peer

* comments

* lint fixes
2019-04-26 10:18:43 -05:00
frederickalcantara
a170c69653 upgrading linter from gometalinter to golangci-lint (#2100)
* upgrading linter from gometalinter to golangci-lint

* fixed golangci-lint linting

* removed linting before_script command

* removed disable-all command

* Fixed golang config file

* fixed golang config file v2

* removed gosec issue rule

* formatting

* fixed travis build to run golangci-lint

* Add install golangci-lint command

* fixing golangci-lint script

* removed https://

* Added golangci-lint cmd script

* added go get for local lint install

* created a before_script

* add install before script

* Added get script

* added go mod download

* removed go mod downloads

* changed

* removed before script

* Added before script go get lint

* added exit zero to see what went wrong

* removed golang run script

* removed before script

* change lint command

* verbose output

* removed verbose

* change linter enable and disable configuration

* Update .golangci.yml

Removed gotype as a linter

* Update .golangci.yml

Added typecheck linter

* Update .golangci.yml

Added fixed lint version

* Update .golangci.yml

Added gotype

* Update .golangci.yml

Added typecheck

* removed env:lint

* Added env lint

* fixing lint upgrade

* Changing travis configuration

* FIxed spelling errors

* disabled typecheck

* Enabled typecheck

* remove binary

* Deleting lib binary

* adding more linters

* fixed constants

* fix spelling

* fixed all lint issues

* Revert "Changing travis configuration"

This reverts commit 334afe9d05e96261b01f275aa3ada20e7f36aac4.

* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into update-linter

* Changed from Infof to Info

* Fixing commits

* fixing commits with linter config

* added install

* Fixing

* fix log statement
2019-04-26 14:24:01 +08:00
Nishant Das
89600e8978
Remove Duplicated Functions in the Slice Package (#2389)
* remove duplicated funcs

* lint

* remove duplicated tests
2019-04-26 12:31:27 +08:00
Raul Jordan
706aa9b82f
Update Jaeger Export Tracing (#2382)
* added exporter

* fixed jaeger export

* lint

* imports
2019-04-25 10:59:18 -05:00
Preston Van Loon
8abd2e51bd
Update flags.go (#2376) 2019-04-25 07:59:12 -04:00
terence tsao
b81eeec374 Optimize block fetch with attestation target fetch (#2373)
* 1st attemp to optimize blk fetch w/ att tgt fetch

* test test test test
2019-04-24 12:21:00 -05:00
shayzluf
6c1740eefc Add Caching to Tree Hashing Algorithm (#1929)
* added todo to hash file in ssz

* params and copy of block cache

* start hash cache

* Hash cache implementation

* fixed some comments

* fixed promatheus duplicate counter name

* removed TODO

* change to use special expiration cache

* function name fixes junk object generator

* naming changes

* gazzle fix

* added pruning last read data test

* fixed gometallinter errors

* fix benchmarks and no int64 not serializable

* move struct from test

* add feature flag

* fix merge issues

* add featureflag to beacon and validator

* featureflag init for tests

* added feature flag to all ssz dependent tests

* remove setter func

* replace k8s tweaked expiration cache to https://github.com/karlseguin/ccache

* remove else

* change request by preston

* added init featureflags to genesis_test

* Update shared/ssz/hash_cache.go

add dot

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Update shared/ssz/hash_cache.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Update shared/ssz/hash_cache.go

remove extra space

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Update shared/params/config.go

add dot

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Update shared/featureconfig/config.go

remove dot

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Update shared/featureconfig/config.go

remove dot

Co-Authored-By: shayzluf <thezluf@gmail.com>

* remove powchain from prometheus hash cache name

* fixes fron change requests

* fix change requests

* remove faulty merge test

* gazelle fix

* fix fmt.sprintf

* remove debug binary

* fix gazelle
2019-04-24 13:39:02 +08:00
Raul Jordan
64e7db6241
Resolve Contract Address Automatically via HTTP (#2361)
* automatically resolve contract address

* Update beacon-chain/node/node.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* testnet contract endpoint

* fmt

* use config in msg
2019-04-23 16:33:47 -05:00
Raul Jordan
44bc1c1bc6
Suggest to ClearDB if Contract Mismatches Cluster (#2360)
* clear db by default

* revert changes

* comment

* run with clear db
2019-04-23 15:52:52 -05:00
Raul Jordan
e4bbf54f9f
Hardcoded Bootstrap + Relay Nodes (#2359)
* default bootstrap node and local network flag

* default relay node value

* Update shared/p2p/service.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* use no discovery
2019-04-23 15:30:59 -05:00
Raul Jordan
e36714805a
Enable Ancestor Cache By Default (#2358)
* remove ancestor cache flag

* ancestor cache on by default
2019-04-23 13:15:58 -05:00
Nishant Das
4ed86566c1 Disable RPC while Syncing (#2353)
* handle nil block

* blocking rpc till synced

* remove unused field

* remove log

* fix tests
2019-04-23 10:57:10 -05:00
Preston Van Loon
19c24ecbdf
remove unused method (#2337) 2019-04-22 13:11:44 -04:00
Preston Van Loon
d00350bf75 disable pubsub signatures and verification (#2339) 2019-04-22 09:44:36 -07:00
Raul Jordan
51d4965388 Enable Committee Cache By Default (#2323) 2019-04-22 06:37:42 -07:00
Preston Van Loon
a5da9f2dd6
Add feature flag to toggle gossip sub in p2p (#2322)
* add feature flag to enable gossip sub in p2p

* invert the enable/disable logic

* add the flag in k8s and fix tests

* gazellle

* return empty config if nil
2019-04-21 13:31:23 -04:00