Commit Graph

902 Commits

Author SHA1 Message Date
Preston Van Loon
a26ef9b44f
Regular sync: pubsub subscriber for voluntary exits (#3227)
* voluntary exit validator & handler skeleton

* pass through && register

* voluntary exits

* voluntary exits

* voluntary exits

* gaz

* lint
2019-08-18 11:33:58 -04:00
Preston Van Loon
b8e550b1e9
Add p2p broadcast implementation (#3226)
* add broadcaster impl

* change API so broadcast returns an error

* change API so broadcast returns an error

* add test for message not mapped

* lint msg

* lint msg
2019-08-18 00:32:39 -04:00
Preston Van Loon
68210eb733
remove the fork version changes since this would be required change for deposits and needs to be more carefully thought about (#3224) 2019-08-17 11:12:05 -04:00
Preston Van Loon
78bf39aff7
sync RPC: Hello handler (#3216)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* basic test with a hardcoded fork choice

* updated beacon config to use genesis fork version

* checkpoint on hello handler

* create a testing db method that can be used with the new database interface

* lint

* lint

* PR feedback

* checkpoint

* passing tests

* comments, errors

* comments, errors

* remove swarm

* Update WORKSPACE

* Update WORKSPACE

* merge

* lint

* lint

* touch

* touch

* imports
2019-08-16 16:03:11 -04:00
Preston Van Loon
81f868bd48
Regular Sync - First Pass (#3201)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* Update WORKSPACE

* Update WORKSPACE
2019-08-16 13:13:04 -04:00
Raul Jordan
6ec9d7e6e2
Utilize Indices for Key Lookup and Filtering Attestations in DB (#3202)
* begin indices approach

* use shard bucket

* continue the indices approach

* eliminate the filter checkers in favor of the single loop of root lookups

* elim extraneous println statement

* continue the indices approach

* intersection for multiple filter types works, but is complex, verbose, and nearly unreadable

* remove unused code

* table drive tests for byte slice intersections

* include all table driven tests

* gazelle imports

* better abstractions

* better comments

* variadic approach working

* transform to variadic

* comments

* comments

* separate bucket for indices for faster range scans

* attestation key as hash tree root of data and different indices buckets

* test pass

* default behavior without filter

* appropriate filter criterion errors if criterion does not apply to type

* better abstractions and prune keys on deletion

* better naming

* fix build

* fix build

* rem extraneous code
2019-08-15 19:57:43 -05:00
Preston Van Loon
eb192049b8
Create a testing db method that can be used with the new database interface (#3217)
* create a testing db method that can be used with the new database interface

* lint

* lint

* PR feedback

* Update setup_db.go
2019-08-15 17:41:51 -04:00
Preston Van Loon
c0627e29a8 Add varint prefixing to ssz network encoder (#3210)
* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* Update varint.go
2019-08-14 16:18:32 -05:00
terence tsao
11ac9585ad
Deprecate the old, and add new DB setup util for tests (#3208) 2019-08-14 11:48:28 -07:00
Raul Jordan
d0bdbe5a33
Add Byte Slice Intersection Utils (#3204)
* remove unused code

* table drive tests for byte slice intersections

* include all table driven tests

* gazelle imports

* imports
2019-08-14 10:27:18 -05:00
terence tsao
072bb4be27 Add gossipsub parameter test (#3200)
* added test for parameters

* Fixed test
2019-08-14 08:26:03 -04:00
Preston Van Loon
5b7182cf18
ssz network encoder (with snappy compression) (#3198)
* move to deprecated-p2p

* fix lint

* Add boilerplate p2p

* lint?

* fix imports

* fix lint

* lint

* lint

* lint

* lint

* comment

* skeleton

* checkpoint

* add a new message that should work with ssz

* add ssz fix and test snappy encoder

* clarify todo

* fix viz

* move, no need to be in subpackage

* testing pb

* end nl

* use merged ssz
2019-08-13 21:37:45 -04:00
Nishant Das
1eb29a2394 Clean Up In Memory Deposits in DB (#3065)
* lint

* clean up deposits in db

* fix all references

* fixed tests

* lint

* bring it into a separate package

* fix lint

* move test

* fix ref

* fix test

* fix test

* fix test
2019-08-13 19:13:47 -04:00
terence tsao
27319a8990 Implement State DB Methods (#3193)
* Added state implementation

* Gaze

* Fixed test

* Fixed build file

* Fixed all tests

* Merged with master

* Added comments to save and get from roots

* Make it explicit signing root

* s/./,

* s/marshalled/marshaled
2019-08-13 18:33:31 -04:00
Preston Van Loon
d2186726a3 New p2p package (#3196) 2019-08-13 14:12:00 -07:00
Preston Van Loon
82efca9b6f
Move p2p to deprecated-p2p (#3191)
* move to deprecated-p2p

* fix lint

* lint?

* fix lint

* lint

* lint

* lint

* lint
2019-08-13 14:52:04 -04:00
Preston Van Loon
e31792f999
deprecate node p2p config (#3192) 2019-08-13 14:24:09 -04:00
skillful-alex
4e886a84f9 Added roughtime to IsSlotValid and fixed test TestIsValidBlock_InvalidSlot (#3186)
* add roughtime to IsSlotValid

* gazelle

* gofmt -s
2019-08-13 13:59:11 -04:00
Raul Jordan
655f5830f4 Implement Blocks DB Methods (#3195) 2019-08-13 09:49:27 -07:00
Preston Van Loon
856dde497b
Move sync to deprecated- prefix (#3190)
* Move sync to deprecated_ prefix

* do not use underscore

* fix
2019-08-13 12:35:34 -04:00
Raul Jordan
8d8849feed
Implement Attestations DB Methods (#3183)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused

* add schema

* implementation simplicity

* has validator latest vote func impl

* retrieve validator latest vote

* has idx

* implement missing validator methods

* missing validator methods and test helpers

* validator index crud tests

* validator tests

* save attestation implementation

* attestation basic methods

* batch  save

* all buckets

* refactor with ok bool

* retrieval by root working

* todo for has attestations

* all tests passing, fmt, imports

* generate key use helper

* most att methods complete

* crud tests passing

* closer and closer to filtering all atts

* default no filter

* filter criteria functioning

* simplified conditional

* filter criteria func

* filter criteria

* filter criteria for atts there

* query filter map strategy

* internal filter api complete

* comments

* complete the passing of all other tests using criteria met

* imports

* fix broken build:

* breaking arg

* import sort groups

* keygen outside tx

* address feedback
2019-08-13 11:04:33 -05:00
Raul Jordan
551ed1d335
Internal Filter Criteria Builder API (#3185)
* query filter map strategy

* internal filter api complete

* comments

* terence feedback
2019-08-12 20:55:28 -05:00
Raul Jordan
6bd8ae8f67
Implement Validator DB Methods (#3172)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused

* add schema

* implementation simplicity

* has validator latest vote func impl

* retrieve validator latest vote

* has idx

* implement missing validator methods

* missing validator methods and test helpers

* validator index crud tests

* validator tests

* all buckets

* refactor with ok bool

* all tests passing, fmt, imports
2019-08-12 14:33:07 -05:00
terence tsao
715b9cd5ba Save head block root for new DB refactor (#3182)
* Save head block root instead of save head state

* Revert state
2019-08-12 12:13:30 -04:00
Nishant Das
22df351e89 Request Missing Logs (#3173)
* add request for missing logs

* fix formatting

* add metric

* add test

* fix test
2019-08-12 06:59:57 -04:00
Preston Van Loon
4c62b0410f
Filter deposits by index and block number (#3171)
* filter deposits by index and block number

* fix test and inverted logic

* nishant feedback

* -1

* Update beacon-chain/db/deposits_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul feedback: explicit variables

* revert everything :(

* a better fix

* fix +1

* fix test
2019-08-11 15:54:39 -04:00
Preston Van Loon
a938274d57
Return errors in log processing (#3170)
* Return errors in log processing

* error formatting
2019-08-11 10:39:22 -04:00
Raul Jordan
dce9c41094 Define Interface Stubs for New DB Interface (#3164)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused
2019-08-10 20:50:10 -04:00
Ivan Martinez
bb2d79be85 Aggregate attestations before adding into the DB (#3149)
* Implement Attestation Aggregation before inserting into the DB

* Nearly complete test for aggregating signatures

* Finish tests for aggregating signatures

* gazelle

* Rename tests

* add lock and advance state

* only advance if necessary

* Fix most tests

* Fix more of DB keys and changing keys to data hashes

* Fix a lot of tests and inconsistencies

* fix lock

* gaz

* undo local changes

* fix ref

* fix ref

* Fix some tests

* clear cache

* fix sync for attestations

* finally working across multiple nodes

* gen proto

* lint

* properly wrap error
2019-08-10 16:13:04 -04:00
Preston Van Loon
830a0a4bca
Fix mid-epoch assignment requests (#3168)
* fix mid-epoch assignments

* add quick test comment
2019-08-10 15:49:58 -04:00
terence tsao
d153abd992
Remove attestation announcement (#3165) 2019-08-09 16:31:44 -07:00
Raul Jordan
3d63bca127 Define New DB Interface (#3163)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method
2019-08-09 15:17:18 -04:00
Preston Van Loon
d860dbbb60
Fix deposits at genesis eth1data (#3161)
* Exclude additional deposits from genesis block

* fix tests

* add test to cover this scenario
2019-08-08 23:26:18 -04:00
terence tsao
32c426ed1b Replaced block and state roots construction to SlotsPerHistoricalRoot (#3160) 2019-08-08 13:12:35 -05:00
terence tsao
b3e29399aa
Reest array size to SlotsPerHistoricalRoot (#3158) 2019-08-08 08:03:24 -07:00
Raul Jordan
4e041c852b
utilize newest ssz and fix build (#3155) 2019-08-07 20:24:04 -05:00
Nishant Das
9ec54ae432 Optimize Verification Of Signatures in Attestations (#3146)
* add few changes

* add process attestation no verify

* gaz

* add reg test

* revert config

* add new method

* fix test

* preston's review

* preston's review

* space
2019-08-05 10:35:47 -04:00
terence tsao
bccd2f95cc
Finish error wrapping (#3135) 2019-08-04 15:45:03 -07:00
Preston Van Loon
d97b691f7d
stub insertions when flag off (#3131) 2019-08-03 16:28:04 -04:00
terence tsao
d59800210a
Skip empty criteria for public key instead of fail (#3130) 2019-08-03 10:23:10 -07:00
terence tsao
7e819990f6
Reject attestation older than finalized epoch (#3124)
* Reject Att older than current finalized epoch

* Fixed receive att test

* Don't use slot for old fork choice target map, use epoch

* Fixed one last conflict
2019-08-03 10:07:57 -07:00
terence tsao
d6b311ab84
Fix ListBlock RPC bugs (#3126) 2019-08-03 09:22:13 -07:00
Nishant Das
ea09a918d8 Fix Sync Service Status (#3128)
* fix bug

* remove log
2019-08-03 10:37:52 -04:00
Preston Van Loon
953c59a302 Wrap errors (#3123) 2019-08-01 19:27:38 -07:00
terence tsao
7eca7ba43b Fixed spelling of genesis (#3120) 2019-08-01 16:55:50 -04:00
Preston Van Loon
f019e54ebb
Remove metrics with high cardinality (#3121)
* remove metrics with high cardinality

* imports
2019-08-01 12:23:52 -04:00
Nishant Das
474fd20123 Fixes when Fetching Pending Deposits (#3119)
* fix bug and add reg test

* fix test

* add better test
2019-08-01 11:12:54 -04:00
Nishant Das
b504d3beb8 Attestation Fixes (#3113)
* change to hashTreeRoot

* remove function and run gaz

* fix panic

* remove cache and add fix

* Revert "remove cache and add fix"

This reverts commit 735986a2db6ba49a0b47611c773e9565f9a82061.

* add back fix

* comment out

* refactor and reg test

* some more fixes

* fix tests

* todo

* Revert config changes

* fix test

* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/rpc/attester_server.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* preston's review
2019-07-31 14:58:03 -04:00
terence tsao
da551f688d Removed extra state root check (#3114) 2019-07-31 12:57:29 -04:00
Nishant Das
57d60d681a Eth1Data VoteCount Fix (#3105)
* change to hashTreeRoot

* remove function and run gaz

* fix panic

* remove cache and add fix

* Revert "remove cache and add fix"

This reverts commit 735986a2db6ba49a0b47611c773e9565f9a82061.

* add back fix

* comment out

* refactor and reg test

* Update beacon-chain/rpc/proposer_server_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Terence PR feedback
2019-07-31 11:00:31 -04:00