Commit Graph

1247 Commits

Author SHA1 Message Date
Nishant Das
7eba8da9d2 Save Network Keys in Data Directory (#3488)
* change marshalling

* add networkkeys

* gaz

* fix test

* add new function

* resolve comments, rename to datadir
2019-09-16 17:09:16 -05:00
Raul Jordan
49a529388b
Resolve Miscellaneous Prysm TODOs (#3465)
* resolve

* resolve

* return

* remove deprecated protos

* rem deprecated pbs

* resolve cache

* resolve md TODO

* node server

* resolve config todo

* resolve even more

* broken build
2019-09-16 15:45:03 -05:00
Raul Jordan
9683a83750 Properly Use Demo Config (#3494) 2019-09-16 12:55:30 -07:00
Nishant Das
c9f48373cb allow rpc requests (#3490) 2019-09-16 12:54:46 -05:00
terence tsao
bf07cfcdab Clean up operation service (#3468)
* Cleaned up operation service

* Fixed all the tests

* Fixed node.go

* Review feedback

* Todo
2019-09-16 12:05:30 -05:00
terence tsao
c5b4cf7f7d Fixed attesting indices set (#3469)
* Fixed attesting indices set

* Typo

* Regression test

* Comment

* Validator count for tests
2019-09-15 13:54:55 -07:00
Preston Van Loon
a2685245f2 demo-config flag (#3473)
* democonfig

* 3.2
2019-09-15 14:24:08 -05:00
Nishant Das
d597410d9b MultiLocking in Operations Service (#3470)
* add lock

* add cache

* modify lock retrieval
2019-09-15 13:51:37 -05:00
terence tsao
9d62e542e5
Clean up validator server (#3466) 2019-09-14 13:31:38 -07:00
Raul Jordan
8887ccdd51 Add Consensus Regression Tests (#3464)
* Start working on adding consensus regression tests

* resolve issues

* table driven tests

* always check equality

* revert

* better naming

* better naming of tests

* build

* resolved

* remove unused deps

* fail on failure
2019-09-13 15:46:35 -07:00
Raul Jordan
1e086b63e8 Add Fork Choice Package Docs (#3463) 2019-09-13 13:03:14 -04:00
Preston Van Loon
0a0d579822 run minimal tests (#3454) 2019-09-12 14:24:35 -04:00
terence tsao
91f824fe10
Clean up unused flags (#3449) 2019-09-12 11:48:34 -04:00
Nishant Das
b04b542e64
fix panic (#3448) 2019-09-12 14:18:41 +05:30
Nishant Das
3e8a94516d
remove bootstrap node (#3447) 2019-09-12 12:46:16 +05:30
Nishant Das
273b917319
Allow Separate Ports for Different Transports (#3414)
* update workspace

* change to new version

* gaz

* set keys

* try more things

* finally fixed all tests

* fix bootnode

* Update beacon-chain/p2p/discovery.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* preston's and raul's review

* add http server

* add tool

* add image

* change comment

* add multiaddr comment

* lint

* cosmetic changes

* fix docker

* remove dep

* preston's requested changes

* new flags

* add support for separate tcp port

* fix refs

* change tcp port
2019-09-12 10:22:27 +05:30
Raul Jordan
e0e3dada7b
Abstract Usage of Deposit Cache Into Interface (#3443)
* deposit cache refactor begin

* use interface for has chain  started

* use deposit fetcher interface instead

* use moar interfaces

* comment

* gaz

* fix breaking build

* lint

* implement chainstart fetcher

* allow start to work

* fix broken tests
2019-09-11 23:30:04 -05:00
Preston Van Loon
14ed36a41e
Concensus issue on state transition (#3444)
* Add failing test

* remove glob

* remove extra deps

* Use min config for test

* Set for unslashedAttestingIndices

* add comment

* add minimal and manual tag
2019-09-11 19:15:20 -07:00
Preston Van Loon
ccece73483
Use the raw bytes, not the libp2p protobuf container for sepc256k1 private keys (#3445)
* use the raw bytes, not the libp2p protobuf container for sepc256k1 private keys

* fix tests
2019-09-11 17:04:35 -07:00
terence tsao
798bbbdc82 Cold start for interop (#3437)
* coldstart flags for validator

* WIP beacon node flags

* wip beacon chain, flag fix in validator, arg fix in validator

* checkpoint

* Added interop service

* working on mock chainstart

* save the state lol

* fix tests

* Save genesis validators

* gaz

* fix validator help flags

* WaitForChainStart actually waits for genesis time

* cold start fixes

* cache

* change back

* allow for genesis state too

* remove logs

* increase mmap size

* dont process if head doesn't exist

* add 10ms tolerance

* enable libp2p debug at debug, fix pubsub

* works with checkpt

* initialize justified and finalized in db

* Removed preloadStatePath from blockchain service

* Clean up

* Write to disk for now post state

* revert b466dd536f8eadbdae2264a545a755370223d917

* Builds

* Only RPC test fails now

* use minimal config, no demo config

* clean up branch

* Lint

* resolve lint

* more lint fixes

* lint

* fix viz

* Fixing RPC test

* skip before epoch 2

* RPC time out

* Fixed ordering

* rename

* remove some dbg statements

* ensure index is correct

* fix some panics

* getting closer

* fix tests

* Fix private key

* Fixed RPC test

* Fixed beacon chain build for docker

* Add interop.go to validator go_image

* Fixed docker build

* handle errors

* skip test, skip disconnecting peers

* Fixed docker build

* tolerance for attestation processing

* revert copy

* clearer err message parse

* fix launching from dep contract
2019-09-11 13:38:35 -05:00
Nishant Das
b4975f2b9d
Read P2P Peer Key Properly (#3442)
* fix conflict

* fix conflict

* gaz

* fix test

* gaz
2019-09-11 20:28:23 +05:30
shayzluf
1edeb8ec4c Beaconblock over wire (#3436) 2019-09-10 10:24:14 -04:00
Preston Van Loon
3708a8f476 Add tool and script for interop testing (#3417)
* add tool and script for interop testing

* identity

* lint

* merge upstream, fix conflict, update script, add comment

* add comma separated support for --peer=

* remove NUM_VALIDATORS, comma fix

* WIP docker image

* use CI builder

* pr feedback

* whatever antoine says

* ignore git in docker

* jobs=auto

* disable remote cache

* try to cache the golang part

* try to cache the golang part

* nvm

* From Antoine with love

* fix
2019-09-09 17:31:19 -04:00
Raul Jordan
af07c13730
[Interop] Improve RPC Codebase + Start Beacon Chain With Mock ETH1 Values (#3407)
* add main.go

* interop readme

* proper visibility

* standardize and abstract into simpler funcs

* formatting

* no os pkg

* add test

* no panics anywhere, properly and nicely handle errors

* proper comments

* fix broken test

* readme

* comment

* recommend ssz

* install

* tool now works

* README

* build

* readme

* 64 validators

* rem print

* register the no powchain flag

* work on mock eth1 start

* common interface

* getting closer with the interface defs

* only two uses of powchain

* remove powchain dependency

* remove powchain dependency

* common powchain interface

* proper comment in case of flag

* proper args into rpc services

* rename fields

* pass in mock flag into RPC

* conforms to iface

* use client instead of block fetcher iface

* broken tests

* block fetcher

* finalized

* resolved broken build

* fix build

* comment

* fix tests

* tests pass

* resolved confs

* took them out

* rename into smaller interfaces

* resolve some confs

* ensure tests pass

* properly utilize mock instead of localized mock

* res lint

* lint

* finish test for mock eth1data

* run gazelle

* include flag again

* fix broken build

* disable powchain

* dont dial eth1 nodes

* reenable pow

* use smaller interfaces, standardize naming

* abstract mock into its own package

* faulty mock lint

* fix stutter in lint

* rpc tests all passing

* use mocks for operations

* no more mocks in the entire rpc package

* no  mock

* viz

* testonly
2019-09-09 17:13:50 -04:00
Preston Van Loon
8d234014a4
Fix broadcast ssz (#3423)
* add two types of encoding/decoding ssz

* fix tests

* lint

* lint
2019-09-08 19:34:52 -07:00
Preston Van Loon
4dad28d1f6
Accept a filepath for bootnode ENR address (#3422)
* accept a filepath for bootnode ENR address

* fix
2019-09-08 19:05:28 -07:00
Ivan Martinez
5e939378d0 Update to spec v0.8.3 (#3355)
* Ignore latest messages in fork choice prior to latest justified

* Make sure Compact Committee Roots isn't changed by process_final_updates

* WIP add attestation bitfields length to match committee length

* Begin work on updating spec tests to 0.8.2

* WIP set up for new spec test structure

* Fix slashings

* Get mainnet tests mostly passing for attestations and attester slashings

* Fix process attestation test

* Undo change

* Complete spec tests for all operations
Still need sanity block tests

* Fix BLS sigs

* Reduce amount of reused code in core/blocks/spectests/

* Fix tests

* Update block sanity tests to 0.8.2

* Update epoch spec tests to 0.8.2

* Clean up all tests and fix shuffling/epoch tests

* WIP update bls tests to 0.8.2

* WIP update bls tests to 0.8.3

* Finish BLS spectest update to 0.8.3

* Fix shuffling spec tests

* Fix more tests

* Update proto ssz spec tests to 0.8.3

* Attempt to fix PrevEpochFFGDataMismatches test

* Goimports

* Fix documentation

* fix test

* Use custom general spec tests

* Reduce code footprint

* Remove unneeded minimal skip

* Fix for comments

* Fix for comments

* Fix test

* Small fixes

* Cleanup block spec tests a bit

* Undo change

* fix validator

* Fix validator tests

* Run gazelle

* Fix error output for epoch spec tests
2019-09-08 12:41:52 -07:00
Raul Jordan
adc27a0bc2 Update WORKSPACE With Latest SSZ (#3416) 2019-09-08 12:58:22 -04:00
terence tsao
a3c3a72e72
Chang flag name to interopXXX (#3418) 2019-09-08 09:29:23 -04:00
terence tsao
4235980511 Clean up post --next (#3411)
* Delete old code

* RPC mock testing

* Fixed BUILD

* Conflict

* Lint

* More lint
2019-09-06 22:39:14 -04:00
Nishant Das
fb20fc7881 change to wrap (#3413) 2019-09-06 14:01:59 -07:00
Nishant Das
171e5007c5
Update Discv5 to the Latest Version (#3392)
* update workspace

* change to new version

* gaz

* set keys

* try more things

* finally fixed all tests

* fix bootnode

* Update beacon-chain/p2p/discovery.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* preston's and raul's review

* add http server

* add tool

* add image

* change comment

* add multiaddr comment

* lint

* cosmetic changes

* fix docker

* remove dep

* preston's requested changes
2019-09-07 00:50:20 +05:30
terence tsao
56a395a297
Add /heads page (#3410) 2019-09-05 20:04:25 -07:00
terence tsao
14c59b2ff9 Remove deprecated services and --next (#3371)
* Save new validators in DB

* Use info

* Add total validator count

* Fixed tests

* Add new test

* Revert light client config

* Add state metrics back

* Gaz

* Mark old ones as deprecated

* Deprecate not --next services

* Fixed all operation tests

* Fixed node test

* All tests passing locally

* Add deprecated-p2p back, blocked by boostrap-query

* Revert message proto

* delete deprecated DB items

* delete all other instances of old db

* gaz

* cycle rem

* clear db
2019-09-05 11:04:06 -05:00
terence tsao
75bce9b7e1
Align metrics to interop (#3406) 2019-09-05 08:32:35 -07:00
terence tsao
c383b6a30c Load ssz formatted genesis state (#3408)
* Preload ssz genesis state

* Log
2019-09-04 17:32:38 -05:00
Raul Jordan
75c0b01932
Genesis State Generator + Interop Docs (#3405)
* add main.go

* interop readme

* proper visibility

* standardize and abstract into simpler funcs

* formatting

* no os pkg

* add test

* no panics anywhere, properly and nicely handle errors

* proper comments

* fix broken test

* readme

* comment

* recommend ssz

* install

* tool now works

* README

* build

* readme

* 64 validators

* rem print
2019-09-04 13:47:44 -05:00
Preston Van Loon
b0e6d7215c
Tracing: Add additional attributes (#3404)
* Add some attributes for tracing

* gaz
2019-09-03 20:03:09 -07:00
Preston Van Loon
b6e0d700ec
PubSub: Check messages received from self, do not double process (#3403)
* Check messages received from self, add them to the store

* tests

* fmt
2019-09-03 17:22:15 -07:00
Preston Van Loon
0a61c379a5
Rename / move logic about updating validator indices (#3402)
* rename, move

* pr feedback
2019-09-03 13:57:08 -07:00
terence tsao
6614816061
Log the slot of the block w/o parent (#3401) 2019-09-03 13:43:59 -07:00
terence tsao
60c048a0ec
Remove lock from store struct (#3400) 2019-09-03 13:14:23 -07:00
Preston Van Loon
5ec629af71 remove unnecessary lock (#3399) 2019-09-03 12:53:18 -07:00
Preston Van Loon
399f704bf5
Initial Sync: report healthy before chain started (#3388)
* Return error while syncing

* chainStarted
2019-09-03 11:25:20 -07:00
Preston Van Loon
8f342cc5bb
fix some parent context usage, add tracing to p2p handlers (#3395) 2019-09-03 11:06:35 -07:00
Preston Van Loon
90b2a880c6 Add /p2p page (#3391)
* add /p2p page

* fix tests
2019-09-03 11:07:40 -05:00
Preston Van Loon
d23ba8e69d
Temporarily ban peer if it fails to connect (#3390)
* temporarily ban peer if it fails to connect

* hotfix for handshake
2019-09-02 18:10:58 -07:00
Preston Van Loon
b1a102fd1d Return error while syncing (#3386) 2019-09-02 14:36:14 -05:00
Raul Jordan
da630f349f
Add Test for Aggregating Large Amount of Attestations (#3358)
* test for verifying large amount of agg sigs

* agg sgi could not verify

* 128 fails

* confirmed works for 512

* comprehensive test for handle att

* commented test

* fix up test

* include the proper wait group

* concurrency managed to reproduce verification bug

* concurrent test passes

* revert config changes

* use new db in operations tests

* debugging for the special attestations

* resolve tests

* fmt
2019-09-02 13:49:37 -05:00
Nishant Das
34a163b110 fix logging (#3384) 2019-09-02 06:29:59 -07:00
Nishant Das
876e0ea84d Fix Discv5 in Runtime (#3373)
* fix bug

* remove logs

* fix test

* add locks

* add ttl

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* change to ccache
2019-09-01 15:29:58 -07:00
terence tsao
25dbc5ea85
Add back state metrics (#3369) 2019-09-01 08:37:38 -07:00
terence tsao
146b611dc8
Use rough time for checking attestation is not from future epoch (#3370) 2019-08-31 13:18:31 -07:00
terence tsao
ca2a55874c
Save new validator index in DB (#3367) 2019-08-30 21:43:18 -07:00
Preston Van Loon
8e2dcb81ae
use roughtime (#3366) 2019-08-30 15:51:15 -07:00
terence tsao
f131585041 Initialize chain info w/o clear db (#3365)
* Initialize chain info upon restart

* Test
2019-08-30 15:24:37 -07:00
terence tsao
9a6410ec15
Lock when process attestation (#3364) 2019-08-30 14:07:42 -07:00
terence tsao
314bc513af
Fixed validator pubkey -> index getter (#3361) 2019-08-30 13:50:21 -07:00
Preston Van Loon
95c528f0bc First pass: single peer initial sync (#3363)
* lint

* add requests

* add all new stuff

* comment

* preston's review

* initial commit

* reorder sync so it isn't required to wait until start

* checkpoint

* fix

* improved handler API

* Set up prechain start values

* improved handler API

* ooops

* successful peer handshakes

* successful peer handshakes

* successful peer handshakes

* checkpoint

* chpkt

* handle init after chain start

* emit state initialized feed if existing db state

* merge error

* Done

* Test

* Fixed test

* emit state initialized

* force fork choice update

* wait for genesis time

* sync to current slot

* Use saved head in DB

* gaz

* fix tests

* lint

* lint

* lint

* lint

* Revert "Use saved head in DB"

This reverts commit c5f3404fdf333c8aac20bce8c349b1978494616b.

* remove db

* lint

* remove unused interfaces from composite

* resolve comments
2019-08-30 15:15:40 -05:00
terence tsao
205fe1baa5
DB: finalized and justified checkpoints can't return nil (#3362) 2019-08-30 10:03:55 -07:00
terence tsao
c425bf2c31 Refactor fork choice start up (#3360)
* Done

* Test

* Fixed test

* emit state initialized

* Fixed existing tests

* Lint

* Lint
2019-08-30 08:58:02 -05:00
terence tsao
f0332e1131 Save genesis state in DB (#3359)
* Done

* Test

* Fixed test

* emit state initialized
2019-08-29 15:32:35 -07:00
Nishant Das
1f0aad31d2
Add Hello Tracking (#3342)
* lint

* add requests

* add all new stuff

* comment

* preston's review

* change to send

* remove topic and add lock

* add test

* lint

* change num of peers

* preston's review

* Update beacon-chain/p2p/handshake.go
2019-08-29 22:02:52 +05:30
terence tsao
f49469a820 Fix chain info's pre chain start return values (#3353)
* Set up prechain start values

* ooops
2019-08-29 10:34:26 -05:00
terence tsao
d8fd7e502a Fix GetChainHead for RPC (#3352)
* Fix ChainHeadQuery

* Fixed test
2019-08-29 10:17:21 -05:00
terence tsao
206222c5bc Return cloned state (#3351) 2019-08-29 09:36:43 -05:00
Raul Jordan
816aac82d5 clone read access to head state and block (#3350) 2019-08-28 13:14:00 -07:00
terence tsao
5d7c33a8dc Check if slot is greater before process slots (#3349) 2019-08-28 14:24:33 -05:00
terence tsao
d84ae95309
Moved delay att inclusion to fork choice service (#3345) 2019-08-28 10:26:07 -07:00
Raul Jordan
e8f030977a
wait for chainstart (#3343) 2019-08-28 12:07:58 -05:00
terence tsao
cbb66dab50
Fix finalized block filtering in sync (#3334) 2019-08-28 08:29:45 -07:00
Preston Van Loon
2ee4f00b81 Add sync/p2p metric for number of messages received by topic (#3341)
* Add going msg metric

* fmt

* rename
2019-08-28 10:14:22 -05:00
skillful-alex
7bb5ac0dde do not panic if dv5Listener is not inited (#3339) 2019-08-28 16:29:34 +05:30
terence tsao
9f2c2f0197 Minor runtime fixes (#3335) 2019-08-27 22:19:47 -05:00
terence tsao
323bbe10ed
Add checkpoint to state caching (#3333) 2019-08-27 15:01:27 -07:00
Nishant Das
ca0f61bf24 Change Ordering of Gossipsub Registration (#3330)
* fix ordering

* Add comment
2019-08-27 10:23:22 -05:00
Nishant Das
701c70ae3b add better logging (#3329) 2019-08-27 06:27:04 -07:00
terence tsao
f188609137 Implement GetHead for RPC (#3326)
* next compatible, tests pass

* terence feedback

* skip comment

* fixes

* misc fix

* on block

* Update RPC service to use chain info

* OOps

* All tests pass, run time with 8 validators work!

* Remove saving genesis validator

* Revert gensis count

* Move redundant headstate

* Comments

* Implemented GetChainHead

* Test works

* Moved mock package

* Fixed visibility for BUILD file

* Conflict
2019-08-26 15:59:17 -05:00
terence tsao
64d0826469
Update RPC service to use chain info (#3309) 2019-08-26 13:06:16 -07:00
terence tsao
a1020585fd Fix aggregation with new DB (#3323)
* Check legacy when aggregate

* Typo
2019-08-26 14:30:39 -05:00
Andrei Ivasko
f99e2bd7c9 Benchmark active indices (#3153)
* new branch off master

* bloomfilter + benchmarks done

* cfilter benchmarks

* comments added

* goimports and gofmt added

* linter issues

* bazel run //:gazelle -- fix

* workspace definitions fixed

* fixed tree_test.go

* Update workspace

* final commit

* gazelle

* updated workspace

* workspace

* reverted workspace changes

* workspace newline

* applied git checkout origin/master WORKSPACE
2019-08-26 11:30:28 -05:00
terence tsao
0b5b3865ef Update validators db during epoch boundary (#3307) 2019-08-26 11:02:17 -05:00
Nishant Das
5828278807 Fix Bolt Fatal Crash (#3320)
* add fix and reg test

* nogo

* nogo
2019-08-26 09:00:40 -05:00
Raul Jordan
9ad00ffafb
Use New Attestation Receiver Method in RPC (#3287)
* next compatible, tests pass

* terence feedback

* skip comment

* fixes

* misc fix

* on block

* resolved err
2019-08-25 15:45:55 -05:00
terence tsao
6bcb68f862
Fix save the correct head (#3306) 2019-08-24 18:39:40 -07:00
terence tsao
045badc5f3
Fix competing attestation check (#3305) 2019-08-24 16:56:40 -07:00
Nishant Das
919877f301 Ignore Messages From Local Peer (#3299)
* validate message coming into pipeline

* gaz

* add to deprecated p2p

* add new lib

* change lib
2019-08-24 14:41:24 -04:00
terence tsao
122166b317
Fix transition logging (#3303) 2019-08-24 11:51:00 -06:00
terence tsao
8870bcea64
Fix attestation pool clean up for new db (#3304) 2019-08-24 11:36:31 -06:00
Preston Van Loon
06c97256bc p2p --next: Register p2p peer count metrics (#3301) 2019-08-24 10:07:03 -06:00
Nishant Das
9d15196bed Runtime Fixes (#3300) 2019-08-24 07:26:25 -06:00
Nishant Das
111f225177 Remove IsAttCanonical From Operations Service (#3298) 2019-08-24 06:50:43 -06:00
terence tsao
a31057de83
Fixed a few more init beacon node bugs (#3297) 2019-08-23 22:02:34 -06:00
terence tsao
5294caf5e8
Save validators upon chainstart (#3295) 2019-08-23 19:59:09 -06:00
Preston Van Loon
a852d610e2 Add panic handler (#3296) 2019-08-23 19:15:02 -06:00
terence tsao
3b422cb9c6
fixed conflict att log (#3294) 2019-08-23 17:23:19 -06:00
Preston Van Loon
b04bfb87a8
only attempt discv5 listener when no-discovery is not present (#3293) 2019-08-23 17:59:59 -04:00
Preston Van Loon
0353cc533e p2p error logging (#3292) 2019-08-23 15:46:54 -06:00
Preston Van Loon
0c0ec97343
fixes (#3291) 2019-08-23 17:34:03 -04:00
terence tsao
4484558d87 Part 11 of update fork choice - tracing and spans (#3285)
* Add tracing in forkchoice service

* Gazelle
2019-08-23 15:04:06 -05:00
Preston Van Loon
ce65b11801 Beacon attestation pubsub subscriber (#3289)
* beacon attestation subscriber

* register beacon attestation handler

* fix tests
2019-08-23 14:46:04 -05:00
Raul Jordan
2e8a06d6d4
Use New Blockchain Service in RPC Package (#3286)
* new chain service usage via interface

* put in the new chain service in propose blk

* deprecate with new service for canonical block roots

* remove old chain serv absolutely in validator server

* full legacy code compatible in beacon server

* fully compliant

* full deprecation at service level

* no more mock chain serv

* fix beacon server tests

* add changes to prop server

* broken build

* --next compatible

* conditional register of chain service

* proper conversion

* nil deref
2019-08-23 13:53:07 -05:00
terence tsao
02ca2290e1
Added metrics for monitoring processed objects and competing chain (#3283) 2019-08-23 12:18:39 -06:00
terence tsao
15f052c48d Update sync to use chain info for head and finalized check point (#3288)
* Starting

* Fixed all the tests
2019-08-23 12:48:40 -05:00
Nishant Das
74df2aa0c3 Add Recent Blocks RPC Request Handler (#3281)
* add new rpc handler

* gaz

* add it back

* remove ok

* preston's comments
2019-08-23 13:10:25 -04:00
Nishant Das
22f4807e0b Implement GoodBye RPC Handler (#3282)
* add handler

* gaz and addition to main rpc method

* remove todo

* preston's comments

* gaz
2019-08-23 12:53:38 -04:00
Raul Jordan
7f475bee00
no cache tests (#3284) 2019-08-23 09:56:48 -05:00
Nishant Das
ebb0e398d3 Deposit Cache Fix (#3280)
* fix cache

* fix spacing
2019-08-22 22:49:03 -04:00
Raul Jordan
f342224410
Full RPC Package Compliance With New DB Interface (#3275)
* deprecate db

* fix build

* begin integrating new db

* gaz

* use more of the new db

* newest implementation uses head state

* remove more deprecated items

* setup validators in state helper

* fix up some tests with the new db

* resolve broken build

* gaz

* begin ensuring tests pass

* optional idx

* list validator balances passing

* default page size passing

* only two failing

* fixed most tests, found edge case

* allow nil return and add proper tests

* pass tests

* fix head block root problem

* working with the new db

* every ethereumapis method now compliant with both dbs

* pass in db into server

* proposer server all compliant

* validator service fully compliant

* fix broken build, tests pass

* spacing

* compute state root and propose block tests passing with new db

* complete proposer server tests revamp

* validator tests halfway through passing with new db

* more validator server tests

* more than halfway there

* so so close

* all validators tests done

* attester server tests fixing

* use new api

* attester server complete

* complete
2019-08-22 20:39:06 -05:00
terence tsao
c47598514c Part 9 of update fork choice - HeadBlock and HeadState getters (#3279)
* Headblock and headstate getters

* Moved mutex around
2019-08-22 20:13:56 -05:00
terence tsao
0d64f7b80e Part 6 of update fork choice - implement new ReceiveAttestation (#3246)
* Implemented new fork choice service and helpers

* Added rest of the tests

* Lint

* Add back helpers test

* Reformatted to doc, helpers and metrics.go

* include new getter for block

* create block filters from indices

* give every block index a unique bucket

* construct block indices by bucket mmap

* almost done save for the block filters

* include block filters, need a few more small touches for fetching the proper indices by bucket

* full functionality to filter by parent root

* tests pass when using the same logic as attestations

* todo

* proper todo formatting

* first minimum slot range filter

* slot range filters pass

* more filter criteria passing

* tests passing

* add todos

* all block tests pass and work

* rem fmt

* range retrieval test

* fixed test conditions

* instantiate the other buckets

* simplify bucket lookups

* deprecate non map code

* revamp to remove old index prefixes

* create indices from data

* create indices from data

* fetch block roots by slot range

* better abstractions

* simpler abstractions

* roots rename

* comment

* preston feedback

* Fixed existing tests

* allow blocks without parent root

* Cleaned up a few things

* Removed todo

* Lint

* Cleaned up a few things

* A few functions don't need to be exported

* Gaz

* Fixed visibility

* Review feedback

* Review feedback part1

* Raul's feedback, refactored OnBlock and OnAttestation to its own file

* Fixed grammar

* Lint

* Implemented ReceiveAttestation

* Use time.Time

* Implemented ReceiveAttestation

* All tests pass

* Lint

* Oooops

* Typo
2019-08-22 20:00:55 -05:00
Preston Van Loon
b59b3ec09c
P2P implement message send (#3278)
* return a stream with send, for reading response

* gofmt

* added sender impl

* fix imports
2019-08-22 19:02:46 -04:00
Raul Jordan
8f01b76366
Integrate DB Refactor Into Ethereum APIs Beacon Chain Server (#3245)
* deprecate db

* fix build

* begin integrating new db

* gaz

* use more of the new db

* newest implementation uses head state

* remove more deprecated items

* setup validators in state helper

* fix up some tests with the new db

* resolve broken build

* gaz

* begin ensuring tests pass

* optional idx

* list validator balances passing

* default page size passing

* only two failing

* fixed most tests, found edge case

* allow nil return and add proper tests

* pass tests

* fix head block root problem

* working with the new db

* every ethereumapis method now compliant with both dbs

* pass in db into server
2019-08-22 15:28:53 -05:00
Preston Van Loon
ce28feea45 Regular sync: pubsub subscriber for beacon blocks (#3220)
* add validation

* add block db check in validation

* merge

* in memory caching of seen blocks

* basic block processing

* Update BUILD.bazel

* use new receiveBlockNoPubsub

* fix build

* add TODO issue numbers

* add TODO issue numbers

* lint
2019-08-22 13:11:52 -05:00
terence tsao
2e352cf5ff Add getter for GenesisTime (#3274)
* Add getter for gensis time

* Lint

* Lint
2019-08-22 11:41:05 -05:00
Nishant Das
e0d3e78746 Add Support for Static Peering (#3272)
* add test and support for static peering

* gaz

* remove delay

* add log

* handle all peers
2019-08-22 10:23:16 -05:00
Raul Jordan
bb542d2032
Basic Block/LatestVote Caching in DB Refactor (#3249)
* add block caching layer

* runlock

* lockinggg

* latest votes map

* validator latest vote deletion method added to interface

* seguin cache working

* cache size

* impl interface

* initialize caches at struct layer
2019-08-22 10:04:13 -05:00
Nishant Das
36c9a5665d
Implement Proposer Slashing Handler (#3273)
* validate proposer slashing

* add commetn

* add handler

* add

* remove

* gaz
2019-08-22 20:01:56 +05:30
Nishant Das
83083b9c65
Fix BLS Aggregation Method (#3269)
* lint

* update to new method

* fix all tests
2019-08-22 11:45:02 +05:30
Nishant Das
c09a6b87c3
Implement Attester Slashing Handler in Sync (#3260)
* add validation

* add test

* add new changes

* fix lint

* Update beacon-chain/sync/validate_attetser_slashing_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* terence's comments

* change key
2019-08-22 11:04:25 +05:30
terence tsao
b91639a32e
Deprecate old block chain service (#3268)
* seperate out block chain services

* Fix blockchain service config

* Gazelle

* Fixed tests
2019-08-21 19:14:24 -06:00
Preston Van Loon
4b17711702 fix startup contract check (#3267) 2019-08-21 17:48:29 -06:00
Raul Jordan
de82956088
Include Voluntary Exits Definitions in DB Refactor (#3266)
* new interface methods

* support proposer slashings

* add in the new buckets

* all crud for propoer slashings

* attester slashings complete

* all slashings crud done

* right comment

* deposit contract tests pass

* delete out of scope methods

* conform old beacon DB

* comment

* include interface implementations

* deprecations

* pass lint
2019-08-21 16:32:44 -05:00
Raul Jordan
3ca4d6fd91
Add Deposit Contract Methods to DB Refactor (#3264)
* new interface methods

* support proposer slashings

* add in the new buckets

* all crud for propoer slashings

* attester slashings complete

* all slashings crud done

* right comment

* deposit contract tests pass

* delete out of scope methods

* conform old beacon DB

* comment

* deprecations

* pass lint

* Update deposit_contract.go
2019-08-21 16:11:50 -05:00
Preston Van Loon
01de412956
Minor runtime fixes for --next (#3265)
* some runtime fixes

* fixes

* fixes

* fixes

* fixes

* fixes
2019-08-21 16:58:38 -04:00
Raul Jordan
8fef74ab25
Block Slashings CRUD Methods DB Refactor (#3261)
* new interface methods

* support proposer slashings

* add in the new buckets

* all crud for propoer slashings

* attester slashings complete

* all slashings crud done

* right comment

* delete out of scope methods

* conform old beacon DB
2019-08-21 15:21:04 -05:00
terence tsao
bfbff885fe Part 7 of update fork choice - chain info access (#3263)
* Implemented new fork choice service and helpers

* Added rest of the tests

* Lint

* Add back helpers test

* Add benchmark tests

* Add yaml driven framework tests

* Reformatted to doc, helpers and metrics.go

* include new getter for block

* create block filters from indices

* give every block index a unique bucket

* construct block indices by bucket mmap

* almost done save for the block filters

* include block filters, need a few more small touches for fetching the proper indices by bucket

* full functionality to filter by parent root

* tests pass when using the same logic as attestations

* todo

* proper todo formatting

* first minimum slot range filter

* slot range filters pass

* more filter criteria passing

* tests passing

* add todos

* all block tests pass and work

* rem fmt

* range retrieval test

* fixed test conditions

* Implemented new receive block methods

* Comments

* Remove mark evil block

* instantiate the other buckets

* simplify bucket lookups

* deprecate non map code

* revamp to remove old index prefixes

* create indices from data

* create indices from data

* fetch block roots by slot range

* better abstractions

* simpler abstractions

* roots rename

* comment

* preston feedback

* Fixed existing tests

* allow blocks without parent root

* Cleaned up a few things

* Removed todo

* Lint

* Cleaned up a few things

* A few functions don't need to be exported

* Gaz

* Fixed visibility

* Review feedback

* Review feedback part1

* Raul's feedback, refactored OnBlock and OnAttestation to its own file

* Fixed grammar

* Lint

* Renamed to receive_block.go

* Use time.Time

* Preston's feedback, removed OnTick and Store.time

* Dont have to cast it to kv

* add block caching layer

* runlock

* lockinggg

* Fixed

* Avoid 2 fetches of the same data

* latest votes map

* Gaz

* Test passes

* Lint

* Fixed db set up

* Fixed all the tests

* Gazelle

* Added tests

* Remove todo

* remove kv

* Last clean up

* Last clean up

Last clean up

* Lint

* Preston's feedback

* Starting

* Gazelle
2019-08-21 14:50:27 -05:00
terence tsao
b440891aea Part 5 of update fork choice - implement new ReceiveBlock (#3242)
* Implemented new fork choice service and helpers

* Added rest of the tests

* Lint

* Add back helpers test

* Add benchmark tests

* Add yaml driven framework tests

* Reformatted to doc, helpers and metrics.go

* include new getter for block

* create block filters from indices

* give every block index a unique bucket

* construct block indices by bucket mmap

* almost done save for the block filters

* include block filters, need a few more small touches for fetching the proper indices by bucket

* full functionality to filter by parent root

* tests pass when using the same logic as attestations

* todo

* proper todo formatting

* first minimum slot range filter

* slot range filters pass

* more filter criteria passing

* tests passing

* add todos

* all block tests pass and work

* rem fmt

* range retrieval test

* fixed test conditions

* Implemented new receive block methods

* Comments

* Remove mark evil block

* instantiate the other buckets

* simplify bucket lookups

* deprecate non map code

* revamp to remove old index prefixes

* create indices from data

* create indices from data

* fetch block roots by slot range

* better abstractions

* simpler abstractions

* roots rename

* comment

* preston feedback

* Fixed existing tests

* allow blocks without parent root

* Cleaned up a few things

* Removed todo

* Lint

* Cleaned up a few things

* A few functions don't need to be exported

* Gaz

* Fixed visibility

* Review feedback

* Review feedback part1

* Raul's feedback, refactored OnBlock and OnAttestation to its own file

* Fixed grammar

* Lint

* Renamed to receive_block.go

* Use time.Time

* Preston's feedback, removed OnTick and Store.time

* Dont have to cast it to kv

* add block caching layer

* runlock

* lockinggg

* Fixed

* Avoid 2 fetches of the same data

* latest votes map

* Gaz

* Test passes

* Lint

* Fixed db set up

* Fixed all the tests

* Gazelle

* Added tests

* Remove todo

* remove kv

* Last clean up

* Last clean up

Last clean up

* Lint

* Preston's feedback
2019-08-21 13:40:00 -04:00
Nishant Das
79e57e8e8e Add Flag for SSZ Encoding (#3256)
* add flag and enum

* change help message

* linter

* add flag

* add comment

* one more comment

* fix panic

* preston's comments

* add panic
2019-08-21 12:33:48 -04:00
Preston Van Loon
acb20e269c Add flags to support new database, new sync (#3252) 2019-08-21 10:04:00 -06:00
Nishant Das
3cb32c3792
Implement Discv5 in Prysm (#3211)
* add discovery

* gaz

* add build options

* add udpPort

* add more changes

* refactor private key

* added discovery loop

* add ttl

* add ttl

* use ip type instead of string

* tests pass

* gaz and new test file

* add test

* add more tests

* add one more test

* adding multiAddr tests

* adding new protocol , listener

* fix keys

* more fixes

* more changes dialing peers works now

* gaz

* add more changes

* add more changes

* gaz

* add new test helpers

* new test

* fixed all tests

* gaz

* reduce sleep

* lint

* new changes

* change formats

* fix all this stuff

* remove discv5 protocol

* remove protocol

* remove port condition,too restrictive

* preston's feedback

* preston's feedback

* close all peers

* gaz

* remove unused func

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* remove build options

* refactor tests
2019-08-21 11:38:30 +05:30
terence tsao
8fc3c55199
Forkchoice get head tiebreaker (#3253) 2019-08-20 21:26:04 -06:00
Preston Van Loon
e146bc35c0
revert this again (#3254) 2019-08-20 23:10:00 -04:00
terence tsao
6195a0bfa1
Fix GetHeadFromYaml is flakey (#3251)
* Forgot clear cache needs to be within the loop

* space
2019-08-20 19:57:35 -06:00
terence tsao
e330fa5733
Part 3 of fork choice update - yaml tests (#3213) 2019-08-20 16:20:54 -06:00
terence tsao
1c4b7329f2
Part 2 of fork choice update - benchmark tests (#3212) 2019-08-20 16:13:20 -06:00
Raul Jordan
121a277726 add better godocs (#3250) 2019-08-20 18:01:55 -04:00
Raul Jordan
900b550864
add all proper spans to methods (#3248) 2019-08-20 14:35:48 -05:00
Preston Van Loon
3f0d1c1d41 Reg sync beacon blocks (#3218)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* basic test with a hardcoded fork choice

* updated beacon config to use genesis fork version

* checkpoint on hello handler

* create a testing db method that can be used with the new database interface

* lint

* lint

* PR feedback

* checkpoint

* passing tests

* comments, errors

* comments, errors

* remove swarm

* Add basic sanity test, naive implementation

* add test with real data
2019-08-20 14:06:49 -05:00
terence tsao
01bbc552cd
Part 1 of fork choice update - fork choice as a service (#3209) 2019-08-20 12:26:43 -06:00
Raul Jordan
8f967d26d7
Allow Nil Return from DB Methods (#3247)
* allow nil return and add proper tests

* pass tests
2019-08-20 12:24:29 -05:00
Raul Jordan
a7d336a7d0
Misc Blocks DB Improvements (#3244)
* allow string constructions

* fix all other cases with formatting
2019-08-20 10:09:10 -05:00
Raul Jordan
1c8ac6658e
Implement Block DB Methods (#3221)
* include new getter for block

* create block filters from indices

* give every block index a unique bucket

* construct block indices by bucket mmap

* almost done save for the block filters

* include block filters, need a few more small touches for fetching the proper indices by bucket

* full functionality to filter by parent root

* tests pass when using the same logic as attestations

* todo

* proper todo formatting

* first minimum slot range filter

* slot range filters pass

* more filter criteria passing

* tests passing

* add todos

* all block tests pass and work

* rem fmt

* range retrieval test

* fixed test conditions

* instantiate the other buckets

* simplify bucket lookups

* deprecate non map code

* revamp to remove old index prefixes

* create indices from data

* create indices from data

* fetch block roots by slot range

* better abstractions

* simpler abstractions

* roots rename

* comment

* preston feedback

* allow blocks without parent root
2019-08-19 19:34:53 -05:00
Preston Van Loon
0b8cbd06b6
Add flag for testing new p2p (#3243)
* refactor a bit to select p2p

* lint

* fix build

* fix build

* fix build

* fix build

* fix build
2019-08-19 17:20:56 -04:00
terence tsao
6ac0d12f5b Part 4 of update fork choice - mark old functions deprecated (#3215)
* Mark these soon to be deprecated functions as "deprecated"

* Deprecate all

* Marked fork choice reorg test as deprecated

* Marked fork choice reorg test as deprecated

* Gaz
2019-08-19 11:34:25 -04:00
Preston Van Loon
a26ef9b44f
Regular sync: pubsub subscriber for voluntary exits (#3227)
* voluntary exit validator & handler skeleton

* pass through && register

* voluntary exits

* voluntary exits

* voluntary exits

* gaz

* lint
2019-08-18 11:33:58 -04:00
Preston Van Loon
b8e550b1e9
Add p2p broadcast implementation (#3226)
* add broadcaster impl

* change API so broadcast returns an error

* change API so broadcast returns an error

* add test for message not mapped

* lint msg

* lint msg
2019-08-18 00:32:39 -04:00
Preston Van Loon
68210eb733
remove the fork version changes since this would be required change for deposits and needs to be more carefully thought about (#3224) 2019-08-17 11:12:05 -04:00
Preston Van Loon
78bf39aff7
sync RPC: Hello handler (#3216)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* basic test with a hardcoded fork choice

* updated beacon config to use genesis fork version

* checkpoint on hello handler

* create a testing db method that can be used with the new database interface

* lint

* lint

* PR feedback

* checkpoint

* passing tests

* comments, errors

* comments, errors

* remove swarm

* Update WORKSPACE

* Update WORKSPACE

* merge

* lint

* lint

* touch

* touch

* imports
2019-08-16 16:03:11 -04:00
Preston Van Loon
81f868bd48
Regular Sync - First Pass (#3201)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* Update WORKSPACE

* Update WORKSPACE
2019-08-16 13:13:04 -04:00
Raul Jordan
6ec9d7e6e2
Utilize Indices for Key Lookup and Filtering Attestations in DB (#3202)
* begin indices approach

* use shard bucket

* continue the indices approach

* eliminate the filter checkers in favor of the single loop of root lookups

* elim extraneous println statement

* continue the indices approach

* intersection for multiple filter types works, but is complex, verbose, and nearly unreadable

* remove unused code

* table drive tests for byte slice intersections

* include all table driven tests

* gazelle imports

* better abstractions

* better comments

* variadic approach working

* transform to variadic

* comments

* comments

* separate bucket for indices for faster range scans

* attestation key as hash tree root of data and different indices buckets

* test pass

* default behavior without filter

* appropriate filter criterion errors if criterion does not apply to type

* better abstractions and prune keys on deletion

* better naming

* fix build

* fix build

* rem extraneous code
2019-08-15 19:57:43 -05:00
Preston Van Loon
eb192049b8
Create a testing db method that can be used with the new database interface (#3217)
* create a testing db method that can be used with the new database interface

* lint

* lint

* PR feedback

* Update setup_db.go
2019-08-15 17:41:51 -04:00
Preston Van Loon
c0627e29a8 Add varint prefixing to ssz network encoder (#3210)
* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* Update varint.go
2019-08-14 16:18:32 -05:00
terence tsao
11ac9585ad
Deprecate the old, and add new DB setup util for tests (#3208) 2019-08-14 11:48:28 -07:00
Raul Jordan
d0bdbe5a33
Add Byte Slice Intersection Utils (#3204)
* remove unused code

* table drive tests for byte slice intersections

* include all table driven tests

* gazelle imports

* imports
2019-08-14 10:27:18 -05:00
terence tsao
072bb4be27 Add gossipsub parameter test (#3200)
* added test for parameters

* Fixed test
2019-08-14 08:26:03 -04:00
Preston Van Loon
5b7182cf18
ssz network encoder (with snappy compression) (#3198)
* move to deprecated-p2p

* fix lint

* Add boilerplate p2p

* lint?

* fix imports

* fix lint

* lint

* lint

* lint

* lint

* comment

* skeleton

* checkpoint

* add a new message that should work with ssz

* add ssz fix and test snappy encoder

* clarify todo

* fix viz

* move, no need to be in subpackage

* testing pb

* end nl

* use merged ssz
2019-08-13 21:37:45 -04:00
Nishant Das
1eb29a2394 Clean Up In Memory Deposits in DB (#3065)
* lint

* clean up deposits in db

* fix all references

* fixed tests

* lint

* bring it into a separate package

* fix lint

* move test

* fix ref

* fix test

* fix test

* fix test
2019-08-13 19:13:47 -04:00
terence tsao
27319a8990 Implement State DB Methods (#3193)
* Added state implementation

* Gaze

* Fixed test

* Fixed build file

* Fixed all tests

* Merged with master

* Added comments to save and get from roots

* Make it explicit signing root

* s/./,

* s/marshalled/marshaled
2019-08-13 18:33:31 -04:00
Preston Van Loon
d2186726a3 New p2p package (#3196) 2019-08-13 14:12:00 -07:00
Preston Van Loon
82efca9b6f
Move p2p to deprecated-p2p (#3191)
* move to deprecated-p2p

* fix lint

* lint?

* fix lint

* lint

* lint

* lint

* lint
2019-08-13 14:52:04 -04:00
Preston Van Loon
e31792f999
deprecate node p2p config (#3192) 2019-08-13 14:24:09 -04:00
skillful-alex
4e886a84f9 Added roughtime to IsSlotValid and fixed test TestIsValidBlock_InvalidSlot (#3186)
* add roughtime to IsSlotValid

* gazelle

* gofmt -s
2019-08-13 13:59:11 -04:00
Raul Jordan
655f5830f4 Implement Blocks DB Methods (#3195) 2019-08-13 09:49:27 -07:00
Preston Van Loon
856dde497b
Move sync to deprecated- prefix (#3190)
* Move sync to deprecated_ prefix

* do not use underscore

* fix
2019-08-13 12:35:34 -04:00
Raul Jordan
8d8849feed
Implement Attestations DB Methods (#3183)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused

* add schema

* implementation simplicity

* has validator latest vote func impl

* retrieve validator latest vote

* has idx

* implement missing validator methods

* missing validator methods and test helpers

* validator index crud tests

* validator tests

* save attestation implementation

* attestation basic methods

* batch  save

* all buckets

* refactor with ok bool

* retrieval by root working

* todo for has attestations

* all tests passing, fmt, imports

* generate key use helper

* most att methods complete

* crud tests passing

* closer and closer to filtering all atts

* default no filter

* filter criteria functioning

* simplified conditional

* filter criteria func

* filter criteria

* filter criteria for atts there

* query filter map strategy

* internal filter api complete

* comments

* complete the passing of all other tests using criteria met

* imports

* fix broken build:

* breaking arg

* import sort groups

* keygen outside tx

* address feedback
2019-08-13 11:04:33 -05:00
Raul Jordan
551ed1d335
Internal Filter Criteria Builder API (#3185)
* query filter map strategy

* internal filter api complete

* comments

* terence feedback
2019-08-12 20:55:28 -05:00
Raul Jordan
6bd8ae8f67
Implement Validator DB Methods (#3172)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused

* add schema

* implementation simplicity

* has validator latest vote func impl

* retrieve validator latest vote

* has idx

* implement missing validator methods

* missing validator methods and test helpers

* validator index crud tests

* validator tests

* all buckets

* refactor with ok bool

* all tests passing, fmt, imports
2019-08-12 14:33:07 -05:00
terence tsao
715b9cd5ba Save head block root for new DB refactor (#3182)
* Save head block root instead of save head state

* Revert state
2019-08-12 12:13:30 -04:00
Nishant Das
22df351e89 Request Missing Logs (#3173)
* add request for missing logs

* fix formatting

* add metric

* add test

* fix test
2019-08-12 06:59:57 -04:00
Preston Van Loon
4c62b0410f
Filter deposits by index and block number (#3171)
* filter deposits by index and block number

* fix test and inverted logic

* nishant feedback

* -1

* Update beacon-chain/db/deposits_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul feedback: explicit variables

* revert everything :(

* a better fix

* fix +1

* fix test
2019-08-11 15:54:39 -04:00
Preston Van Loon
a938274d57
Return errors in log processing (#3170)
* Return errors in log processing

* error formatting
2019-08-11 10:39:22 -04:00
Raul Jordan
dce9c41094 Define Interface Stubs for New DB Interface (#3164)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused
2019-08-10 20:50:10 -04:00
Ivan Martinez
bb2d79be85 Aggregate attestations before adding into the DB (#3149)
* Implement Attestation Aggregation before inserting into the DB

* Nearly complete test for aggregating signatures

* Finish tests for aggregating signatures

* gazelle

* Rename tests

* add lock and advance state

* only advance if necessary

* Fix most tests

* Fix more of DB keys and changing keys to data hashes

* Fix a lot of tests and inconsistencies

* fix lock

* gaz

* undo local changes

* fix ref

* fix ref

* Fix some tests

* clear cache

* fix sync for attestations

* finally working across multiple nodes

* gen proto

* lint

* properly wrap error
2019-08-10 16:13:04 -04:00
Preston Van Loon
830a0a4bca
Fix mid-epoch assignment requests (#3168)
* fix mid-epoch assignments

* add quick test comment
2019-08-10 15:49:58 -04:00
terence tsao
d153abd992
Remove attestation announcement (#3165) 2019-08-09 16:31:44 -07:00
Raul Jordan
3d63bca127 Define New DB Interface (#3163)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method
2019-08-09 15:17:18 -04:00
Preston Van Loon
d860dbbb60
Fix deposits at genesis eth1data (#3161)
* Exclude additional deposits from genesis block

* fix tests

* add test to cover this scenario
2019-08-08 23:26:18 -04:00
terence tsao
32c426ed1b Replaced block and state roots construction to SlotsPerHistoricalRoot (#3160) 2019-08-08 13:12:35 -05:00
terence tsao
b3e29399aa
Reest array size to SlotsPerHistoricalRoot (#3158) 2019-08-08 08:03:24 -07:00
Raul Jordan
4e041c852b
utilize newest ssz and fix build (#3155) 2019-08-07 20:24:04 -05:00
Nishant Das
9ec54ae432 Optimize Verification Of Signatures in Attestations (#3146)
* add few changes

* add process attestation no verify

* gaz

* add reg test

* revert config

* add new method

* fix test

* preston's review

* preston's review

* space
2019-08-05 10:35:47 -04:00
terence tsao
bccd2f95cc
Finish error wrapping (#3135) 2019-08-04 15:45:03 -07:00
Preston Van Loon
d97b691f7d
stub insertions when flag off (#3131) 2019-08-03 16:28:04 -04:00
terence tsao
d59800210a
Skip empty criteria for public key instead of fail (#3130) 2019-08-03 10:23:10 -07:00
terence tsao
7e819990f6
Reject attestation older than finalized epoch (#3124)
* Reject Att older than current finalized epoch

* Fixed receive att test

* Don't use slot for old fork choice target map, use epoch

* Fixed one last conflict
2019-08-03 10:07:57 -07:00
terence tsao
d6b311ab84
Fix ListBlock RPC bugs (#3126) 2019-08-03 09:22:13 -07:00
Nishant Das
ea09a918d8 Fix Sync Service Status (#3128)
* fix bug

* remove log
2019-08-03 10:37:52 -04:00
Preston Van Loon
953c59a302 Wrap errors (#3123) 2019-08-01 19:27:38 -07:00
terence tsao
7eca7ba43b Fixed spelling of genesis (#3120) 2019-08-01 16:55:50 -04:00
Preston Van Loon
f019e54ebb
Remove metrics with high cardinality (#3121)
* remove metrics with high cardinality

* imports
2019-08-01 12:23:52 -04:00
Nishant Das
474fd20123 Fixes when Fetching Pending Deposits (#3119)
* fix bug and add reg test

* fix test

* add better test
2019-08-01 11:12:54 -04:00
Nishant Das
b504d3beb8 Attestation Fixes (#3113)
* change to hashTreeRoot

* remove function and run gaz

* fix panic

* remove cache and add fix

* Revert "remove cache and add fix"

This reverts commit 735986a2db6ba49a0b47611c773e9565f9a82061.

* add back fix

* comment out

* refactor and reg test

* some more fixes

* fix tests

* todo

* Revert config changes

* fix test

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/rpc/attester_server.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* preston's review
2019-07-31 14:58:03 -04:00
terence tsao
da551f688d Removed extra state root check (#3114) 2019-07-31 12:57:29 -04:00
Nishant Das
57d60d681a Eth1Data VoteCount Fix (#3105)
* change to hashTreeRoot

* remove function and run gaz

* fix panic

* remove cache and add fix

* Revert "remove cache and add fix"

This reverts commit 735986a2db6ba49a0b47611c773e9565f9a82061.

* add back fix

* comment out

* refactor and reg test

* Update beacon-chain/rpc/proposer_server_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Terence PR feedback
2019-07-31 11:00:31 -04:00
terence tsao
f70d94675b
Fix assignment to advance state up to requested slot (#3112) 2019-07-30 20:47:34 -07:00
Preston Van Loon
3c3e4a2cb5
Add tests for db deposits (#3111)
* add tests for deposits

* fix imports

* gaz

* travis lint
2019-07-30 12:30:07 -04:00
Preston Van Loon
5f4cdd6095
Need to ignore in progress cache while disabled (#3109) 2019-07-30 01:07:18 -04:00
Preston Van Loon
63cf0f07a2 Disable caches, allow toggle via feature flag (#3107) 2019-07-29 20:38:05 -07:00
Nishant Das
f72f7677b3 Replace Deposit Hash with HashTreeRoot (#3102)
* change to hashTreeRoot

* remove function and run gaz

* fix panic
2019-07-29 08:43:24 -05:00
terence tsao
3fe0933936 Fixed TestRetrieveAttestations_OK test (#3087) 2019-07-27 17:13:00 -05:00
terence tsao
ad82e84503
Updated pseudocode for get_indexed_attestation (#3086) 2019-07-27 14:47:32 -07:00
Nishant Das
fc907261e9
Fix State Root Verification in Sync (#3085)
* remove caches

* use already imported block package
2019-07-28 02:32:13 +05:30
terence tsao
96c32c3865 Implement ListBlocks RPC function (#3084)
* Implemented rpc list blocks

* Tests for list blocks

* lint

* lint
2019-07-27 15:26:28 -05:00
Nishant Das
8cbd1097d7 Fix Signature Verification for Attestations (#3080)
* fix bug and test

* terence's review
2019-07-26 23:18:44 -05:00
terence tsao
956b07f5c1 Implement state transition no verify (#3048)
* Implemented state transition w/o sig verification

* ExecuteStateTransitionNoValidateStateRoot -> ExecuteStateTransitionNoVerify

* Fixed all the tests

* Extra spaces

* Gazelle

* Conflict

* Added start shard cache back

* typos
2019-07-27 00:24:42 +05:30
Raul Jordan
4ebe2fb5b5 Implement AttestationPool and ListAttestations RPC Functions (#3061) 2019-07-26 10:07:20 -07:00
Raul Jordan
40fca7bb2c
Update SSZ Dependency With Tree Hash Cache Enabled (#3076)
* utilize new SSZ

* workspace
2019-07-25 23:49:49 -05:00
Ivan Martinez
ed78f1f406 Require Signature Verification in Randao, Attestations and AttesterSlashings (#3075) 2019-07-25 14:42:13 -07:00
terence tsao
b80d9f4f7f
Implement GetValidatorParticipation RPC function (#3069) 2019-07-25 14:01:18 -07:00
terence tsao
c1eeeef853 Implement ListValidatorAssignments RPC function (#3067)
* Need to sync latest ethereum API, will do it in master

* Added pagination wrapper

* Implemented ListValidatorAssignments

* Finished tests

* Fixed tests

* Raul's feedback

* Fmt

* Pagination test

* Removed extra loggings
2019-07-25 15:45:31 -04:00
Nishant Das
88b715d8f6 Fix Key in Eth1Data Cache (#3074) 2019-07-25 11:26:19 -07:00
Ivan Martinez
e452b46873
Remove Optional Signature Verification for VoluntaryExit and BlockHeader (#3053)
* Remove verifySignatures from ProposerSlashings

* Remove flag from process transfers

* resolve all conflicts

* fix more references to old pbs

* Fix merge conflicts

* Remove verifySignature flag from ProcessBlockHeader

* fx spectest

* Fix test errors

* Fix tests

* Fix tests

* Goimports

* Fix test finally

* Move test helpers to testutil

* Goimports

* Fix imports

* Add tests for new helpers

* Run gazelle

* Fix tests
2019-07-25 13:53:46 -04:00
Preston Van Loon
94a73e847f
Log errors in regular sync (#3070)
* add a error level logging for regular sync

* add a error level logging for regular sync

* add a error level logging for regular sync
2019-07-24 22:29:40 -04:00
Preston Van Loon
7d47be84ed
Template based protobuf parameters for ssz configurations (#3062)
* WIP on build time configuration changes

* add ssz_minimal tests

* split up spec tests into mainnet and minimal, skip any minimal test that are failing without --define ssz=minimal

* lint

* add commentary to ssz_proto_library
2019-07-24 22:03:05 -04:00
Preston Van Loon
f58afa62af
fix docker image builds (#3068) 2019-07-24 15:19:44 -04:00
blacktemplar
9f2543267e Ignore deposits with invalid signatures instead of throwing error (#3011)
* errors in signatures of deposits lead to ignoring the deposit instead of error (by spec)

* add appropriate error log for invalid signature in process deposit

* Small comment improvement

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* improved log

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* adds three unit tests for skipping deposits with invalid and uncompressed signatures and one for successfully performing a process deposit with a valid signature (inclusive signature checking)

* fix panics, still test failures

* adapts tests to use testutil

* add forgotten dependencies

* reordering imports according to goimports
2019-07-24 14:55:40 -04:00
Nishant Das
590aaaf370 Fix Batched Block Response (#3012) 2019-07-24 07:53:38 -07:00
terence tsao
17576af752
Implement GetValidators gRPC server (#3054) 2019-07-23 19:36:35 -07:00
Justin Page
9d0e9fa77d Improve sorted indices check in blocks package (#3060) 2019-07-23 18:17:47 -07:00
terence tsao
41e55a6902
Sync with latest eth api definitions (#3059) 2019-07-23 14:33:25 -07:00
Raul Jordan
930e992e85
Include Stubs for All Beacon Chain Server RPC Methods (#3058)
* add most rpc method stubs

* include all stubs

* use unimplemented error code
2019-07-23 15:29:13 -05:00
Raul Jordan
a2caba9956
Optimize Sparse Merkle Trie (#3056)
* calc tree from leaves simpler

* fast generate proof

* align api to be the same

* ensure tests pass

* err condition

* travis

* fix build

* zero hashes work
2019-07-23 14:17:39 -05:00
Preston Van Loon
be514076c1 Remove optional verify signatures argument when verifying deposits (#3052)
* remove optional verification of deposit signatures

* use minimal config for easier setup

* progress

* progress

* Fix a few test errors

* Fix more of tests

* fix imports, gazelle

* fix rpc package

* fix blocks package

* fixed state test

* fixed powchain tests

* add comments

* remove todo

* Update beacon-chain/rpc/validator_server_test.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
2019-07-23 22:19:14 +05:30
Raul Jordan
5374350a1c
Standardize Flags at Top Level and Remove Deprecated Utils (#3046)
* move shuffling to core

* remove old utils

* move flags to top level

* package lvl comment removal

* fix up references to flags

* revert node.go

* revert p2p_config.go

* revert main.go

* revert validator node.go

* revert validator main.go

* add flags pkg

* viz

* goimports
2019-07-23 08:58:20 -05:00
terence tsao
d42fab070d Implement ListValidatorBalances gRPC server (#3050)
* Implemented ListValidatorBalances in beacon chain server

* Fmt

* Tests

* Spacings

* goimports

* Apply suggestions from code review, thanks Raul!

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* Update beacon-chain/rpc/beacon_chain_server.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
2019-07-22 21:49:47 -05:00
Raul Jordan
b06876d698
Implement Node gRPC Server (#3049)
* add node server

* stub implementations

* node server impl

* gaz

* only missing genesis info now

* fmt imports

* all tests pass

* fmt

* revert change

* punctuation

* use internal err code

* view permission

* using real reflection

* spacing

* lint
2019-07-22 21:19:55 -05:00
Preston Van Loon
6a930ba175 Remove optional verifyTree argument (#3047)
* remove optional verifyTree argument

* remove fmt

* do not provide a default eth1data, return an error instead

* add a test for this new logic

* gaz
2019-07-22 16:47:11 -05:00
Ivan Martinez
1d71398b7c Remove VerifySignatures Flag From ProposerSlashings and Transfers (#3009)
* Remove verifySignatures from ProposerSlashings

* Remove flag from process transfers

* resolve all conflicts

* fix more references to old pbs
2019-07-22 14:25:32 -05:00
terence tsao
1b5b8a57e0 Remove unused proto schemas (#3005)
* Update io_kubernetes_build commit hash to 1246899

* Update dependency build_bazel_rules_nodejs to v0.33.1

* Update dependency com_github_hashicorp_golang_lru to v0.5.1

* Update libp2p

* Update io_bazel_rules_k8s commit hash to e68d5d7

* Starting to remove old protos

* Bazel build proto passes

* Fixing pb version

* Cleaned up core package

* Fixing tests

* 6 tests failing

* Update proto bugs

* Fixed incorrect validator ordering proto

* Sync with master

* Update go-ssz commit

* Removed bad copies from v1alpha1 folder

* add json spec json to pb handler

* add nested proto example

* proto/testing test works

* fix refactoring build failures

* use merged ssz

* push latest changes

* used forked json encoding

* used forked json encoding

* fix warning

* fix build issues

* fix test and lint

* fix build

* lint
2019-07-22 10:03:57 -04:00
Preston Van Loon
db5549f143
update some k8s deps (#3008)
* update some k8s deps

* merge

* fix build sizes
2019-07-21 15:34:36 -04:00
Ivan Martinez
acf1ebff2d
Implement Proposer Signature (#2973)
* Implement Proposer Signatures

* Remove logging flag

* Create verifySignature and verifySigningRoot

* Clean up logs

* Fix

* Remove log

* Fix test
2019-07-21 11:29:35 -04:00
Preston Van Loon
4a4316eb95
Add rule to push docker images (#3006)
* touch readme

* whatever

* Add docker_push to actually push the images

* revert readme

* revert pb.go
2019-07-20 20:40:03 -04:00
Preston Van Loon
cc696d90e3 Docker: use container bundles to upload multiple image tags (#3003)
* use docker tag from environment, if exists

* use container bundles to upload multiple image tags
2019-07-19 21:55:09 -07:00
Preston Van Loon
dfc64121c6
Remove unused feature flags (#3002)
* remove a few feature flags that are no longer needed

* remove other unused flags

* forgot a few more
2019-07-19 21:27:35 -04:00
Preston Van Loon
e744d1a07e Spec freeze updates (#2312)
* Optimize Shuffled Indices Cache (#2728)

* Refactor Deposit Contract Test Setup (#2731)

* add new package

* fix all tests

* lint

* change hash function (#2732)

* Remove Deprecated Validator Protobuf (#2727)

* Remove deprecated validator protos

* Fix to comments

* Fix most of skipped tests (#2735)

* Cache Active Validator Indices, Count, and Balances (#2737)

* Optimize Base Reward Calculation (#2753)

* benchmark process epoch

* revert prof.out

* Add some optimizations

* beware where we use ActiveValidatorIndices...

* revert extra file

* gaz

* quick commit to get feedback

* revert extra file

* started fixing tests

* fixed broken TestProcessCrosslink_NoUpdate

* gaz

* cache randao seed

* fixed all the tests

* fmt and lint

* spacing

* Added todo

* lint

* revert binary file

* started regression test

* basic tests done

* using a fifo for active indices cache

* using a fifo for active count cache

* using a fifo for total balance cache

* using a fifo for active balance cache

* using a fifo for start shard cache

* using a fifo for seed cache

* gaz

* clean up

* fixing tests

* fixed all the core tests

* fixed all the tests!!!

* lint

* comment

* rm'ed commented code

* cache size to 1000 should be good enough

* optimized base reward

* revert binary file

* Added comments to calculate adjusted quotient outside

* removed deprecated configs (#2755)

* Optimize Process Eth1 Data Vote (#2754)

* Cleanup and Docs update (#2756)

* Add graffiti and update generate seed (#2759)

* Benchmark Process Block with Attestations (#2758)

* Tidying up Godoc for Core Package (#2762)

* Clean up Old RPC Endpoints (#2763)

* Update RPC end point for Proposer (#2767)

* add RequestBlock

* run mockgen

* implemented RequestBlock

* updated proto definitions

* updated tests

* updated validator attest tests

* done

* comment

* todo issue

* removed unused proto

* Update attesting indices v0.6 (#2449)

* sort participants slice

* add bitfield functions and tests

* added BitfieldBit test

* add AttestationParticipantsNew

* revert AttestationParticipants to its previous change

* add tests

* remove verifybitfieldnew

* fix tests and remove multiple tests

* remove duplicate test

* change magic number into ceildiv8

* Implement Justification and finalization Processing (#2448)

* Add convert to indexed (#2519)

* sort participants slice

* add bitfield functions and tests

* added BitfieldBit test

* add AttestationParticipantsNew

* revert AttestationParticipants to its previous change

* add tests

* remove verifybitfieldnew

* fix tests and remove multiple tests

* remove duplicate test

* start work

* convert attestation to indexed attestations

* fix test for convert index

* remove calling getter

* add more tests

* remove underscore

* changes name to signature (#2535)

* update registry updates func (#2521)

* update registry updates func

* added tests and moved to epoch processing

* fixed naming issues

* Update Committee Helpers to v0.6.0 (#2398)

* Update Committee Helper Part 2 (#2592)

* Implement Process Slashings for 0.6 (#2523)

* Update Proposer/Attester Slashings and Slashing Helpers (#2603)

* Implement Final Updates 0.6 (#2562)

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* implemented process_final_updates

* move to the top of the file

* add comment back

* gaz

* Test for process final updates

* fixed tests

* fixed all the tests

* Update Reward Helper v0.6 (#2470)

* added BaseReward

* added rewards helper

* added test for BaseReward

* extra space

* move exported function above

* update to new spec (#2614)

* Update Block Processing Voluntary Exits (#2609)

* Update processEth1Data for v0.6 (#2516)

* Clean up Helper Functions Part 1 (#2612)

* Finalize helper functions for 0.6 (#2632)

* Process Beacon Chain Transfers v0.6 (#2642)

* add transfers

* beacon transfer operations complete

* full cov

* transfer testing

* finished tests

* Update beacon-chain/core/blocks/block_operations_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/core/blocks/block_operations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Implement Process Crosslink From 0.6 (#2460)

* Process Block Eth1 Data v0.6 (#2645)

* Get attestation data slot v0.6 (#2593)

* attestation.go is ready, tests not

* ready for review

* fixing linter issues

* modified Crosslink and AttestationData proto fields to spec 2.0,marked deprecated fields

* gazelle

* fixed tests

* fixed error

* error msg

* Process Block Deposits v0.6 (#2647)

* imports fixes

* deposits tests pass

* wrapped up gazelle

* spacing

* Implement Crosslink Delta Rewards for 0.6 (#2517)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* merged master

* all tests passing

* start testing

* done

* add ProcessBlockHeader v0.6 (#2534)

* add ProcessBlockHeader

* function has all its dependancies in place

* arange the basic ok test

* gazzele and skip test update

* skip wrong sig test

* fmt imports and change requests

* goimports fmt

* map for struct fields to be location independent

* reorder protobuf fields

* added tests

* gazzle fix

* few change requests fixes

* revert changes in types.proto

* revert changes in types

* fix tests

* fix lint

* fmt imports

* fix gazelle

* fix var naming

* pb update

* var naming

* tarance change request fixes

* fix test

* Add Process Registry for Epoch Processing (#2668)

* update update-process-registry

* added back the old tests

* fmt

* gaz

* Follow up on process block header v0.6 (#2666)

* Putting Crosslink Delta Back (#2654)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* merged master

* all tests passing

* start testing

* done

* fixed tests

* addressed shay's feedback

* Implement Attestation Delta for v0.6 (#2646)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* implemented process_attestation_delta

* comments

* comments

* merged master

* all tests passing

* start testing

* done

* merged master

* fixed tests

* tests, more to come

* tests done

* lint

* spaces over tabs

* addressed shay's feedback

* merged master

* Implement process_rewards_and_penalties for 0.6 (#2665)

* update process crosslink and update existing tests

* added a test case to cover no crosslink changes

* more test

* preston's feedback

* spellings

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* Starting, I need get_epoch_start_shard

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* need to use changes from latest crosslinks

* added BaseReward and totalActiveBalance

* added test for base reward

* implemented process_attestation_delta

* comments

* comments

* merged master

* all tests passing

* start testing

* done

* merged master

* fixed tests

* tests, more to come

* tests done

* lint

* spaces over tabs

* addressed shay's feedback

* starting but need to merge a few things...

* tests

* fmt

* Update Slot Processing and State Transition v0.6 (#2664)

* edit state transition to add slot processing logic, reorder logic

* fix build

* lint

* tests passing

* spacing

* tests pass

* imports

* passing tests

* Implement Process Epoch for 0.6 (#2675)

* can't find process j f functons

* implemented process_epoch

* tests done

* lint

* nishant's feedback

* stupid goland replace

* goimports

* Update CommitteeAssignment (#2693)

* cleaned up skipped tests for core processing (#2697)

* Process Block Attestations v0.6 (#2650)

* attestation.go is ready, tests not

* ready for review

* fixing linter issues

* modified Crosslink and AttestationData proto fields to spec 2.0,marked deprecated fields

* gazelle

* fixed tests

* fixed error

* add att processing:

* process atts

* error msg

* finish process attestations logic

* spacing

* ssz move

* inclusion delay failure passing

* more attestation tests

* more att tests passing

* more tests

* ffg data mismatching test

* ffg tests complete

* gofmt

* fix testing to match attestation updates

* ssz

* lint

* Fixed Skipped Tests for RPC Server (#2712)

* Remove Obsolete Deposit Proto Objects (#2673)

* remove deposit data

* remove deposit input

* fix references

* remove deposit helpers

* fix all refs

* gaz

* rgene proto

* fix all tests

* remove deposit data deprecated field

* fix remaining references

* fix all tests

* fix lint

* regen proto

* fix test

* Remove Deprecated Protobuf State Fields (#2713)

* Remove Deprecated Protobuf Crosslink/Slashing/Block Fields (#2714)

* Remove Deprecated Beacon Block Proto Fields (#2717)

* Remove Deprecated Attestation Proto Fields (#2723)

* Cache Shuffled Validator Indices (#2682)

* YAML shuffle tests for v0.6 (#2667)

* new shuffle tests

* added comment for exported function

* fix format and print

* added config files handling

* gazelle fix

* shuffle test debugging

* added shuffle list and benchmark

* hash function addition from nishant code

* gazelle fix

* remove unused function

* few minor changes

* add test to test protos optimization

* test a bigger list

* remove commented code

* small changes

* fix spec test and test indices to pass

* remove empty line

* abstraction of repeated code and comment arrangement

* terence change requests

* fix new test

* add small comment for better readability

* change from unshuflle to shuffle

* comment

* better comment

* fix all tests

* Remove Latest Block (#2721)

* lint

* remove latest block

* lint

* add proto

* fix build

* Fix Deposit Trie (#2686)

* remove deposit data

* remove deposit input

* fix references

* remove deposit helpers

* fix all refs

* gaz

* rgene proto

* fix all tests

* remove deposit data deprecated field

* fix remaining references

* fix all tests

* fix lint

* new tests with contract

* gaz

* more tests

* fixed bugs

* new test

* finally fixed it

* gaz

* fix test

* Remove Committee Cache (#2729)

* Benchmark Compute Committee (#2698)

* Fixed Skipped Attestation Tests (#2730)

* Optimize Shuffled Indices Cache (#2728)

* Refactor Deposit Contract Test Setup (#2731)

* add new package

* fix all tests

* lint

* change hash function (#2732)

* Remove Deprecated Validator Protobuf (#2727)

* Remove deprecated validator protos

* Fix to comments

* Fix most of skipped tests (#2735)

* Optimize Base Reward Calculation (#2753)

* benchmark process epoch

* revert prof.out

* Add some optimizations

* beware where we use ActiveValidatorIndices...

* revert extra file

* gaz

* quick commit to get feedback

* revert extra file

* started fixing tests

* fixed broken TestProcessCrosslink_NoUpdate

* gaz

* cache randao seed

* fixed all the tests

* fmt and lint

* spacing

* Added todo

* lint

* revert binary file

* started regression test

* basic tests done

* using a fifo for active indices cache

* using a fifo for active count cache

* using a fifo for total balance cache

* using a fifo for active balance cache

* using a fifo for start shard cache

* using a fifo for seed cache

* gaz

* clean up

* fixing tests

* fixed all the core tests

* fixed all the tests!!!

* lint

* comment

* rm'ed commented code

* cache size to 1000 should be good enough

* optimized base reward

* revert binary file

* Added comments to calculate adjusted quotient outside

* removed deprecated configs (#2755)

* Optimize Process Eth1 Data Vote (#2754)

* Cleanup and Docs update (#2756)

* Add graffiti and update generate seed (#2759)

* Benchmark Process Block with Attestations (#2758)

* Tidying up Godoc for Core Package (#2762)

* Clean up Old RPC Endpoints (#2763)

* Update RPC end point for Proposer (#2767)

* add RequestBlock

* run mockgen

* implemented RequestBlock

* updated proto definitions

* updated tests

* updated validator attest tests

* done

* comment

* todo issue

* removed unused proto

* Cache Active Validator Indices, Count, and Balances (#2737)

* Update Deposit Contract (#2648)

* lint

* add new contract

* change version

* remove log

* generating abi and binary files

* fix tests

* update to current version

* new changes

* add new hash function

* save hashed nodes

* add more things

* new method

* add update to trie

* new stuff

* gaz

* more stuff

* finally fixed build

* remove deposit data

* Revert "remove deposit data"

This reverts commit 9085409e91be0c94550af10290eaad72ad40a6de.

* more changes

* lint and gaz

* lint

* Update Shard Helpers for 0.6 (#2497)

* ValidatorStatus Estimating Activation RPC Server (#2469)

* fix spacing

* working on position in queue

* fmt

* spacing

* feedback

* tests

* rename

* Only Perform Initial Sync With a Single Peer (#2471)

* fix spacing

* use send instead of broadcast in initial sync

* Fix Estimation of Deposit Inclusion Slot in ValidatorActivationStatus (#2472)

* fix spacing

* fix time estimates

* correct slot estimation

* naming

* Update beacon-chain/rpc/validator_server.go

Co-Authored-By: rauljordan <raul@prysmaticlabs.com>

* SSZ web api for decoding input data (#2473)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* standardize slot numbers (#2475)

* Add CORS for ssz api (#2476)

* first pass ssz server for decoding deposit input data

* fix decoding

* revert viz change on helper

* add image target

* use /api prefix, add deployment for cluster

* fix lint

* needed CORS

* Allow Client to Retrieve Multiple Validator Statuses (#2474)

* multiple validator statuses

* gazelle

* context

* fixing bugs

* remove old way of checking

* fix logging

* make activation queue more accurate

* fix rpc test

* add test

* fix remaining tests

* lint

* comment

* review comments

* Update Prysm README (#2477)

* README updated

* readme updates

* no err throw (#2479)

* Fix Status Nil Pointer Error (#2480)

* no err throw

* nil errors

* 3.175 (#2482)

* Better Error Message if Failing to Exit Initial Sync (#2483)

* no err throw

* nil errors

* better error on init sync

* Only Log Active Balances (#2485)

* only log active balance

* dont need ()

* change logging (#2487)

* fix chainstart waiting on rpc server (#2488)

* shift ticker to after activation (#2489)

* Add drain script (#2418)

* Add drain script

* Fix script to drain contracts from newest to oldest

* Add README

* remove comments

* Only after block 400k, look up by deposit event

* issue warn log on disconnecting peer instead of error (#2491)

* Display Only Active Validator Data (#2490)

* Fix Validator Status Field in RPC Server (#2492)

* fix status of key

* status test fix

* fmt

* Estimate the Time Till Follow Distance Is Completed (#2486)

* use estimation instead

* fix test

* fixing another test

* fix tests and preston's comments

* remove unused var

* fix condition

* Revert "fix condition"

This reverts commit dee0e3112c01f68f30a2e50cd4eb35f29f672e1d.

* dont return error

* add production config for testnet release (#2493)

* Lookup Validator Index in State in Status Check (#2494)

* state lookup

* refactor duplicate code

* refactor with mapping

* fix broken tests

* finish refactor

* merged master

* updated EpochCommitteeCount and fixed tests

* implemented ShardDelta

* test for ShardDelta

* implemented EpochStartShard

* added epoch out of bound test

* test for accurate start shard

* lint

* Update Genesis State Function to v0.6 (#2465)

* add pseudocode

* make changes

* fix all tests

* fix tests

* lint

* regen protos and mocks

* regenerated protos

* started fixing core

* all core tests passing!

* removed shared/forkutils

* started fixing blockchain package

* lint

* updating rpc package

* add back deleted stuff

* add back deleted stuff that was deleted accidentally

* add back protos and mocks

* fix errors

* fix genesis issue

* fix genesis issue for slot ticker

* fix all genesis errors

* fix build files

* temp change for go-ssz

* fix test

* Revert "temp change for go-ssz"

This reverts commit 3411cb9d6d519cb521181486debc3b21603c8873.

* update to latest go-ssz

* unstaged changes

* Update Attester Server RPC Calls (#2773)

* Update config and function parameters to v0.7 (#2791)

* Minor Updates to 0.7 (#2795)

* Refactor Deposit Flow and Cleanup Tests (#2788)

* More WIP on cleaning deposit flow

* Fix tests

* Cleanup and imports

* run gazelle

* Move deposit to block_operations

* gazelle

* Update beacon-chain/core/blocks/block_operations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Fix docs

* Remove unneeded calculations

* Fix tests

* Fix tests finally (?)

* Optimize Committee Assignment RPC (#2787)

* Update BlockRoot to BlockHash (#2816)

* Fix Final Missing Items in Block Processing v0.6 (#2710)

* override config successfully

* passes processing

* add signing root helper

* blockchain tests pass

* tests blocked by signing root

* lint

* fix references

* fix protos

* proper use of signing root

* only few failing tests now

* fix final test

* tests passing

* lint and imports

* rem unused

* Update beacon-chain/core/blocks/block_operations.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* lint

* Update beacon-chain/attestation/service_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/db/block_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* rename to hash tree root

* rename decode to unmarshal

* fix

* use latest ssz

* all tests passing

* lint

* fmt

* Add Config YAML for Spec Tests (#2818)

* Align Protobuf Type Names (#2825)

* gofmt

* Revert "Align Protobuf Type Names (#2825)" (#2827)

This reverts commit 882d067144c674bbf4eeee00404acaa0a9b1fd33.

* Update Domain Related Functions (#2832)

* Add Functions for Compressed and Uncompressed HashG2 With Domain (#2833)

* add tests

* gaz

* lint

* Revert "Add Functions for Compressed and Uncompressed HashG2 With Domain (#2833)" (#2835)

This reverts commit 7fb2ebf3f12f043d7bd12f43500dc2bd133df573.

* Add ConvertToPb to package testutil (#2838)

* Block Processing Bug Fixes (#2836)

* Update types PB with Size Tags (#2840)

* Epoch processing spec tests (#2814)

* Remove Deposit Index (#2851)

* Shuffle tests revisited (#2829)

* first commit

* remove old files, add log

* remove duplicate yaml testing code

* reduce visability

* nishant feedback changes

* skip TestFromYaml_Pass

* added tags to bazel build

* gazelle fix

* remove unused vars

* adda back config

* remove config handling

* remove unused var

* gazelle fix

* SSZ compatibility test for protobufs (#2839)

* update workspace spec sha

* remove yamls from branch

*  BLS spec tests (#2826) (#2856)

* bls spec tests

* add more bls tests

* use ioutil instead of bazel runfiles

* dont read bytes

* skip tests that overflow uint64

* manually fix input data

* add tests

* lint and gaz

* add all new changes

* some refactoring, cleanup, remove new API methods that only exist for tests

* gaz

* Remove yamls, skip test

* Slot processing spec test (#2813)

* eth1data rpc endpoint (#2733)

* eth1data rpc endpoint

* first version

* comment added

* gazelle fix

* new function to go once over the deposit array

* fix tests

* export DepositContainer

* terence feedback

* move structure decleration

* binary search

* fix block into Block

* preston feedback

* keep slice sorted to remove overhead in retrival

* merge changes

* feedback

* update to the latest go-ssz

* revert change

* chnages to fit new ssz

* revert merge reversion

* go fmt goimprts duplicate string

* exception for lint unused doesParentExist

* feedback changes

* latesteth1data to eth1data

* goimports and stop exposing Eth1Data

* revert unneeded change

* remove exposure of DepositContainer

* feedback and fixes

* fix workspace duplicate dependancy

* greatest number of deposits at current height

* add count votes function

* change method name

* revert back to latesteth1data

* latesteth1data

* preston feedback

* seperate function add tests fix bug

* stop exposing voteCountMap

* eth1data comment fix

* preston feedback

* fix tests

* new proto files

* workspace to default version of ssz

* new ssz

* chnage test size

* marshalled  marshaled

* Attesting Indices Fix (#2862)

* add change

* fix one test

* fix all tests

* add test

* clear cache

* removed old chaintest, simulated backend and state generator (#2863)

* Block Processing Sanity Spec Tests (#2817)

* update PrevEpoch

* add new changes

* shift to blocks package

* add more changes

* new changes

* updated pb with size tags

* add new changes

* fix errors

* uncomment code

* more changes

* add new changes

* rename and lint

* gaz

* more changes

* proccess slot SigningRoot instead of HashTreeRoot

* ensure yaml generated structs work

* block sanity all passing

* minimal and mainnet all pass

* remove commented code

* fix one test

* fix all tests

* fix again

* no state comparison

* matching spec

* change target viz

* comments gazelle

* clear caches before test cases

* latest attempts

* clean up test format

* remove debugging log, remove yaml

* unskip attestation

* remove skip, check post state, diff state diffs

* handle err

* add bug fixes

* fixed one more bug

* fixed churn limit bug

* change hashProto to HashTreeRoot

* all tests pass :)

* fix all tests

* gaz

* add regression tests

* fix test bug

* Mutation testing fixes for beacon-chain/core/helpers/attestation.go (#2868)

* mutation testing for attestation.go

* new line

* lint

* revert fmt.Errorf deletion

* gofmt

* Add some fixes for mutation testing on blocks.go (#2869)

* Fix sizes

* gaz

* Spec freeze release candidate spectests

* Align Protobuf Type Names  (#2872)

* Removes some deprecated fields from protobuf (#2877)

* search and replace checkpoints

* fix tests, except spec tests

* Update Configs for Freeze (#2876)

* update configs

* updated minimal configs

* almost there

* all tests passing except for spec tests

* better comment for MinGenesisTime

* done, ready for review

* rm seconds per day

* feedback

* Mutation testing fixes for beacon-chain/core/helpers/committee.go (#2870)

* Add some fixes for mutation testing on blocks.go

* working on mutation testing fo committee.go

* gofmt

* goimports

* update readme target

* update latest sha for spec tests

* fix build

* Update State Transition Function (#2867)

* Change Base Reward Factor (#2888)

* Update Freeze Spec Simplification Section - part 1 (#2893)

* finished changes to attesting_indices

* removed index_count <= 2**40 requirement

* lint

* reverted index_count <= 2**40 check

* added short cut len(a) > len(b)

* Update justification bits (#2894)

* updated all the helper pseudocodes (#2895)

* Make Constants Explicit and Minor Cleanups (#2898)

* Rename outdated configs, make constants explicitly delcared

* Remove activate_validator, not needed

* Remove GenesisSlot and GenesisEpoch

* Remove unused import

* Move Block Operation Length Checks to ProcessOperations (#2900)

* Move block operation length checks to ProcessOperations

* Write tests for each length check in ProcessOperations

* Remove unneeded test

* Move checks to a new function

* Move duplicate check back into ProcessOperations

* reorder proto fields (#2902)

* Slashing Penalty Calculation Change (#2889)

* lint

* change config val

* add max helper

* changes to slashing and process slashing, add a min function for integers

* gaz

* fix failing tests

* fix test

* fixed all tests

* Change Yaml tag

* lint

* remove gc hack

* fix test

* gaz

* preston's comments

* change failing field

* fix and regen proto

* lint

* Implement Compact Committee Root (#2897)

* add tags

* add function

* add new code

* add function

* add all new changes

* lint

* add tests

* fix tests

* fix more tests

* fix all outstanding tests

* gaz

* Update beacon-chain/core/helpers/committee.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* comment

* Remove deprecated fields from attestation data (#2892)

* fix broken tests

* remove comment

* fixes

* Update Deposit Contract (#2903)

* update to new contract

* fix references

* fix tests

* fix some more tests

* fix local deposit trie

* gaz

* shays review

* more changes

* update WORKSPACE to use 0.8 spec tests

* Perform Mutesting in Helpers Package (#2912)

* Perform mutesting on validator

* Mutesting in helpers package

* Mutested eth1data

* s/volundary/voluntary (#2914)

* Update BLS Domain (#2916)

* change from integer to byte slice

* add test

* fix func for bytes4

* Fix Spec tests (#2907)

* fix panics

* handle failed transitions

* remove log

* fix to protos

* new changes

* remove i

* change ssz commit

* new changes

* update epoch tests

* fix epoch testing

* fix shuffle tests

* fix test

* Perform Mutesting in Epoch and State Packages (#2913)

* done with updates (#2890)

* Add Max Size Tag for Protobuf Fields (#2908)

* No more space between ssz-size numbers

* Regen pb.go

* Fixed a few incorrect proto fields (#2926)

* Update Validator Workflow (#2906)

* Justification spec tests (#2896)

* update go-ssz

* Fix SSZ Compatibility Test (#2924)

* figuring out how to seqeeze in multiple fields in a tag for pb

* Added max tags and regenerated pb.go

* updated to new standard

* New bitfield types in proto (#2915)

* Cast bytes to correct bitfield, failing tests now though

* Add forked gogo/protobuf until https://github.com/gogo/protobuf/pull/582

* remove newline

* use proper override for gogo-protobuf

* fix a few tags

* forgot to include custody bits and Slashable not used

* Update yaml struct to use pb

* Update workspace to use latest ssz

* All tests fail

* Use the latest go-ssz commit

* All pass except for state (too long to taste)

* Update test.proto

* Added rest of the tests

* use 1 justification bits

* fix tag test, apply @rauljordan's suggestion

* add IsEmpty and use ssz struct

* delete unused file

* Update zero hash to sha256().digest() (#2932)

* update zero hash

* change zero hash to conform with spec

* goimports

* add test for zero hash

* Revert "Update zero hash to sha256().digest() (#2932)" (#2933)

This reverts commit b926ae0667b18aef3f7e0e8ec8a9b3e98b3d2ccc.

* Fix compress validator (#2936)

* fix compress validator

* update go-ssz

* build without the bytes test

* try minimal

* Block operations spec tests (#2828)

* update PrevEpoch

* debugging proposer slashing tests

* fmt

* add deposit tests

* Added skeleton for attestation minimal test

* remove bazel runfiles thing

* add deposits

* proposer slashing test is done

* comment

* complete test, some failing cases

* sig verify on

* refactor slightly to support mainnet and minimal

* included mainnet stuff

* Add block header tests

* volunary exit done

* transfer done

* new changes

* fix all tests

* update domain functions

* fmt

* fixed lint

* fixed all the tests

* fixed a transfer bug

* finished attester slashing tests and fixed a few bugs

* started fixing...

* cleaned up exit and proposr slashing tests

* attester slashing passing

* refactored deposit tests

* remove yamls, update ssz

* Added todo for invalid sig

* gazelle

* deposits test done!

* transfer tests done and pass!

* fix attesting indices bug

* temporarily disabled signature verification

* cleaned up most of the block ops, except for att

* update committee AttestingIndices

* oops, i dont know how or why i changed this file

* fixed all the rpc tests

* 6 more failing packages

* test max transfer in state package

* replace hashproto with treehash in package blockchain

* gazelle

* fix test

* fix test again

* fixed transition test, 2 more left

* expect an error in attestation tests

* Handle panic when no votes in aggregate attestation

* clear cache

* Add differ, add logging, tests pass yay

* remove todo, add tag

* fixed TestReceiveBlock_RemovesPendingDeposits

* TestAttestationMinimal/success_since_max_epochs_per_crosslink fails now...

* handle panics

* Transfer tests were disabled in https://github.com/ethereum/eth2.0-specs/pull/1238

* more fixes after merge, updating block_operations.yaml.go to match yaml

* figuring out how to seqeeze in multiple fields in a tag for pb

* Added max tags and regenerated pb.go

* updated to new standard

* New bitfield types in proto (#2915)

* Cast bytes to correct bitfield, failing tests now though

* Add forked gogo/protobuf until https://github.com/gogo/protobuf/pull/582

* remove newline

* fix references and test panic

* change to proto objects from custom types

* fix panics in tests

* use proper override for gogo-protobuf

* fix a few tags

* fix tests

* forgot to include custody bits and Slashable not used

* fix tests

* sort again

* Update yaml struct to use pb

* Update workspace to use latest ssz

* All tests fail

* Use the latest go-ssz commit

* All pass except for state (too long to taste)

* Update test.proto

* Added rest of the tests

* use 1 justification bits

* minor fixes

* wrong proto.Equal

* fix tag test, apply @rauljordan's suggestion

* add IsEmpty and use ssz struct

* inverted logic

* update zero hash

* change zero hash to conform with spec

* goimports

* add test for zero hash

* Revert "Update zero hash to sha256().digest() (#2932)"

This reverts commit b926ae0667b18aef3f7e0e8ec8a9b3e98b3d2ccc.

* update ssz, fix import, shard big test

* checkpoint

* fix compress validator

* update go-ssz

* missing import

* missing import

* tests now pass

* been a good day

* update test size

* fix lint

* imports and remove unused const

* update bazel jobs flag

* update bazel jobs flag

* satisfy deprecation warning

* Add ssz regression tests for investigation of test failures in PR #2828 (#2935)

* Adding regression tests for investigation

* add another example

* goimports

* add quick comment about test case 0

* Epoch Process Slashings Spec Tests (#2930)

* updated justification bits, tests passing OK

* regen pb.go, clarify bit operations

* justification and finalization tests; failing

* Add wrapper, so we call the correct method

* checkpoint

* Update tar ref

* TestSlashingsMinimal/small_penalty still failing

* Use bigint instead of () and float

* Revert a bad merge from workspace

* Fmt

* add note about https://github.com/ethereum/eth2.0-specs/issues/1284

* improve tests

* gaz

* Perform Mutesting In core/state Package (#2923)

* Perform mutesting on validator

* Mutesting in helpers package

* Mutested eth1data

* Perform mutesting in epoch and state packages

* Fix voluntary exits test

* Fix typo

* Fix comments

* Fix formatting

* Fix error message

* Handle missing errors

* Handle all errors

* Perform Mutesting In State Package

* Fix block roots size

* Remove comment

* Fix error

* add backend service

* Add ssz compatibility tests for signing root (#2931)

* Added tests for signing root

* imports

* fix lint on travis

* fix bes flag

* Final updates spec tests (#2901)

* set up tests

* need to reorder pbs

* figuring out how to seqeeze in multiple fields in a tag for pb

* Added max tags and regenerated pb.go

* updated to new standard

* New bitfield types in proto (#2915)

* Cast bytes to correct bitfield, failing tests now though

* Add forked gogo/protobuf until https://github.com/gogo/protobuf/pull/582

* remove newline

* use proper override for gogo-protobuf

* fix a few tags

* forgot to include custody bits and Slashable not used

* playing with tags idea, can revert this commit later

* fixes after merge

* reset caches before test

* all epoch tests pass

* gazelle

* Genesis trigger (#2905)

* genesis change

* integrate changes

* bodyroot

* remove unused code

* HasChainStarted

* added isValidGenesisState to ProcessLog

* state fix

* fix gazelle

* uint64 timestamp

* SetupInitialDeposits adds proof

* remove unneeded parts of test

* deposithash

* merkleproof from spec utils

* Revert "merkleproof from spec utils"

This reverts commit 1b0a124352e7b62e3c3220fb0d64e295b474b430.

* fix test failures

* chain started and hashtree root in tests

* simple eth2genesistime

* eth2 genesis time

* fix zero time

* add comment

* remove eth1data and feedback

* fix build issues

* main changes: add fields and methods to track active validator
count

* gaz

* fix test

* fix more tests

* improve test utils

* shift spec method to state package, improve test setup

* fixed log processing tests

* remove log

* gaz

* fix invalid metric

* use better tag names, not latest

* Remove Block Signing Root (#2945)

* replace with hash tree root

* Revert "replace with hash tree root"

This reverts commit 77d8f16a160e42f3c3d598df66c30a66657de1bf.

* replace with signing root instead

* remove one more ref

* Create Test Runner for Genesis State Spec Tests (#2940)

* genesis change

* integrate changes

* bodyroot

* remove unused code

* HasChainStarted

* added isValidGenesisState to ProcessLog

* state fix

* fix gazelle

* uint64 timestamp

* SetupInitialDeposits adds proof

* remove unneeded parts of test

* deposithash

* merkleproof from spec utils

* Revert "merkleproof from spec utils"

This reverts commit 1b0a124352e7b62e3c3220fb0d64e295b474b430.

* fix test failures

* chain started and hashtree root in tests

* simple eth2genesistime

* eth2 genesis time

* fix zero time

* add comment

* remove eth1data and feedback

* fix build issues

* main changes: add fields and methods to track active validator
count

* gaz

* fix test

* fix more tests

* improve test utils

* Start genesis spec tests

* shift spec method to state package, improve test setup

* Add Genesis validity spec test

* Bazel

* fixed log processing tests

* remove log

* gaz

* fix invalid metric

* use json tags

* fix up latest changes

* Fix most of test errors

* Attempts to see whats wrong with genesis validity

* Fix merge

* skip minimal

* fix state test

* new commit

* fix nishant comment

* gaz

* Static check on branch spec-v0.6 (#2946)

* Ran staticcheck and fixed the important complains

* commit

* commit

* Create Test Runner for Genesis State Spec Tests (#2940)

* genesis change

* integrate changes

* bodyroot

* remove unused code

* HasChainStarted

* added isValidGenesisState to ProcessLog

* state fix

* fix gazelle

* uint64 timestamp

* SetupInitialDeposits adds proof

* remove unneeded parts of test

* deposithash

* merkleproof from spec utils

* Revert "merkleproof from spec utils"

This reverts commit 1b0a124352e7b62e3c3220fb0d64e295b474b430.

* fix test failures

* chain started and hashtree root in tests

* simple eth2genesistime

* eth2 genesis time

* fix zero time

* add comment

* remove eth1data and feedback

* fix build issues

* main changes: add fields and methods to track active validator
count

* gaz

* fix test

* fix more tests

* improve test utils

* Start genesis spec tests

* shift spec method to state package, improve test setup

* Add Genesis validity spec test

* Bazel

* fixed log processing tests

* remove log

* gaz

* fix invalid metric

* use json tags

* fix up latest changes

* Fix most of test errors

* Attempts to see whats wrong with genesis validity

* Fix merge

* skip minimal

* fix state test

* new commit

* fix nishant comment

* gaz

* Add Back Eth1Data After Bad Merge (#2953)

* eth1data rpc endpoint

* first version

* comment added

* gazelle fix

* new function to go once over the deposit array

* fix tests

* export DepositContainer

* terence feedback

* move structure decleration

* binary search

* fix block into Block

* preston feedback

* keep slice sorted to remove overhead in retrival

* merge changes

* feedback

* update to the latest go-ssz

* revert change

* chnages to fit new ssz

* revert merge reversion

* go fmt goimprts duplicate string

* exception for lint unused doesParentExist

* feedback changes

* latesteth1data to eth1data

* goimports and stop exposing Eth1Data

* revert unneeded change

* remove exposure of DepositContainer

* feedback and fixes

* fix workspace duplicate dependancy

* greatest number of deposits at current height

* add count votes function

* change method name

* revert back to latesteth1data

* latesteth1data

* preston feedback

* seperate function add tests fix bug

* stop exposing voteCountMap

* eth1data comment fix

* preston feedback

* fix tests

* new proto files

* workspace to default version of ssz

* new ssz

* chnage test size

* marshalled  marshaled

* everything passing again

* add skip reason

* cleanup deposit contract slightly

* remove unused chainstart param (#2957)

* fix breakages from #2957 (#2958)

* fix breakages from #2957

* oops

* Fix deposit input data (#2956)

* fix deposit input data

* fix deposit input data

* gaz and build fix

* Add Tests for Genesis Deposits Caching (#2952)

* remove old method and replace with an improved one

* add new files

* gaz

* add test

* added all these tests

* gaz

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* fix merkle proof error

* fix config

* Minor fixes for runtime (#2960)

* Minor fixes for runtime

* use comments

* goimports

* revert beacon-chain/core/state/state.go and fix comment for lint

* fix test too

* Minor runtime fixes  (#2961)

* Add support for bundling binaries and fix ARM64 builds (#2970)

* Add support for bundling binaries and fix ARM64 builds

* Fix exports

* ignore manual targets wrt vis check

* fix graknlabs

* update spec tests (#2979)

* hotfix until https://github.com/graknlabs/bazel-distribution/pull/169

* Overflow slashing calculation fix (#2977)

* Runtime Fixes (#2736)

* first batch of fixes

* add log

* more fixes

* another bug fixed

* update deposit contract and other fixes

* remove logs

* new changes

* fixes

* fix build

* remove config

* more fixes

* add more changes

* add back todo

* make compute state root work

* remove commented out and fix condition

* fix commented code

* fix config

* gaz

* remove flag

* remove init

* new fixes

* fix test

* one more fix

* fix all tests

* change back config

* fix one more bug

* remove logging bool

* Only build test targets when running bazel test //...

* Align prysm to spec v0.8.1 (#2978)

* Bazel problem

* Update zero hash representation to be clear (cosmetic)

* Update minor cosmetic fixes

* Fixed lookahead off by 1

* Update randao.go

* update ssz

* test failures fixed

* test fixes

* fix up workspace

* lint

* fixed errs

* Updated pubkey loggings (#2983)

* Fix proposer assignment (#2984)

* add jvm limits

* add jvm limits

* Removed logging from state transition functions (#2980)

* Match spec on proposer index division (#2985)

* Match spec on proposer index division

* gaz

* fixes

* Fix Default Eth1Data (#2982)

* fix bug

* Update beacon-chain/core/state/transition.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* more fixes

* reg test

* Set ejection balance to 1.6 (#2987)

* improve block processing log (#2990)

* Fix HistoricalRootsLimit (#2989)

* fix historical lenght

* fix genesis state initialization and test

* fix genesis state initialization and test

* fix genesis state initialization and test

* fix genesis state initialization and test

* hack config until https://github.com/prysmaticlabs/prysm/issues/2993

* More Runtime Fixes (#2986)

* local changes

* add val sig

* attester fix

* one more fix

* fixed all tests

* rem validator issue

* fix finality issue (#2994)

* Fix validator prev balance calculation (#2992)

* attester fix

* one more fix

* fixed all tests

* Fix validator prev balances calculation

* go fmt

* 48 bytes

* Fix Querier to handle Deposit Logs Race (#2999)

* fix sync issue

* fix build

* add regression test

* add var for magic number

* Fix eth1data and deposits (#2996)

* Work in progress, eth1data works, deposits are included at the appropriate time, and activation happens at the correct time

* revert blockInfo being public

* git tests to build, not yet pass though

* add tests

* some commentary

* fix comment

* goimports

* fmt and remove unused method

* Update rules go (#2975)

* update rules_go

* Fix some cross compile builds stuff

* add missing deps

* update to 0.19.1

* Update Protobufs to Match Ethereum APIs (#2998)

* add beacon block and attestation files

* add all types

* include all new proto type definitions

* add add all proto definitions

* fix all comments to say 48 bytes

* include latest changes

* readd common

* no swag

* add build file

* deps issue

* right package names

* address feedback, maintain parity between upstream ethereumapis

* delete pb

* bad gens

* Update "Testing Prysm" readme section (#3000)

Resolves invalid url link to golangci-lint.

* Update badge to version 0.8.1

* elaborate on test skip

* revert shared/p2p/options.go

* make travis happy with goimports

* Update beacon-chain/core/blocks/block.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
2019-07-19 19:16:10 -05:00
skillful-alex
d8e24af4c3 best peer assignment fix (#2976) 2019-07-16 11:31:51 -04:00
Preston Van Loon
f12fdfda0f Complain about improperly sized tests (#2873) 2019-06-30 09:00:22 -07:00
Dan
9ac950f480 Added the 'enable-upnp' flag to the list of command line args (#2860)
* Added the 'enable-upnp' flag to the list of supported command line arguments.

If the user specifies this feature flag (adds --enable-upnp as an argument) - the Beacon-chain and Validator services, when started, will initialize libp2p with the UPNP options.

* Added the new arg to usage.go due to test failure

* Update shared/p2p/service.go

Changed the logging according to Preston's recommendation.

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* Code review changes:

1. File formatting.
2. Command line arg more detailed description.
2019-06-26 12:59:37 -07:00
Preston Van Loon
9bd6147027
BLS spec tests (#2826)
* bls spec tests

* add more bls tests

* use ioutil instead of bazel runfiles

* dont read bytes

* skip tests that overflow uint64

* manually fix input data

* add tests

* lint and gaz

* add all new changes

* some refactoring, cleanup, remove new API methods that only exist for tests

* gaz

* Remove yamls, skip test
2019-06-25 12:57:47 -04:00
Preston Van Loon
cebefde335
Update config.go to fix BLS issue (#2821)
* Update config.go

* fix test
2019-06-18 18:02:01 -04:00
Preston Van Loon
c30bc3dd97
Revert "Add BoltDB Internal Stats (#2768)" (#2799)
This reverts commit c66186b54a.
2019-06-16 15:50:23 -04:00
Preston Van Loon
dd131561bf automaxprocs (#2770) 2019-06-13 07:53:42 -07:00
Nishant Das
c66186b54a Add BoltDB Internal Stats (#2768)
* lint

* add metrics

* rename file

* add review changes

* remove dumb pattern

* fix naming

* add all the buckets data

* change to seconds

* Update beacon-chain/db/db_metrics.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* preston's comments
2019-06-12 19:07:36 -04:00
Nishant Das
663490ee1f Update Phore Dependency to the latest version (#2792)
* add new changes

* fix typos
2019-06-12 11:06:39 -05:00
Nishant Das
d48e0925d0 Add gRPC Gateway Folder (#2793)
* add gateway proto files

* lint
2019-06-12 10:11:04 -05:00
gzuhlwang
78c3166ef2 fix typo (#2764) 2019-06-07 07:58:27 -04:00
Preston Van Loon
9e98e914a1 Add a gRPC gateway (#2604) 2019-06-02 08:33:44 -07:00
Dan
71b5d5beec Adding the BlockTreeBySlots function for getting the Block tree filte… (#2720) 2019-06-01 22:41:17 -07:00
Preston Van Loon
55bedd0745 Move go-ssz to external repo under MIT license (#2722) 2019-05-29 18:04:25 -07:00