Commit Graph

2323 Commits

Author SHA1 Message Date
Preston Van Loon
b8e550b1e9
Add p2p broadcast implementation (#3226)
* add broadcaster impl

* change API so broadcast returns an error

* change API so broadcast returns an error

* add test for message not mapped

* lint msg

* lint msg
2019-08-18 00:32:39 -04:00
Preston Van Loon
68210eb733
remove the fork version changes since this would be required change for deposits and needs to be more carefully thought about (#3224) 2019-08-17 11:12:05 -04:00
Preston Van Loon
78bf39aff7
sync RPC: Hello handler (#3216)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* basic test with a hardcoded fork choice

* updated beacon config to use genesis fork version

* checkpoint on hello handler

* create a testing db method that can be used with the new database interface

* lint

* lint

* PR feedback

* checkpoint

* passing tests

* comments, errors

* comments, errors

* remove swarm

* Update WORKSPACE

* Update WORKSPACE

* merge

* lint

* lint

* touch

* touch

* imports
2019-08-16 16:03:11 -04:00
Preston Van Loon
81f868bd48
Regular Sync - First Pass (#3201)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* Update WORKSPACE

* Update WORKSPACE
2019-08-16 13:13:04 -04:00
Raul Jordan
6ec9d7e6e2
Utilize Indices for Key Lookup and Filtering Attestations in DB (#3202)
* begin indices approach

* use shard bucket

* continue the indices approach

* eliminate the filter checkers in favor of the single loop of root lookups

* elim extraneous println statement

* continue the indices approach

* intersection for multiple filter types works, but is complex, verbose, and nearly unreadable

* remove unused code

* table drive tests for byte slice intersections

* include all table driven tests

* gazelle imports

* better abstractions

* better comments

* variadic approach working

* transform to variadic

* comments

* comments

* separate bucket for indices for faster range scans

* attestation key as hash tree root of data and different indices buckets

* test pass

* default behavior without filter

* appropriate filter criterion errors if criterion does not apply to type

* better abstractions and prune keys on deletion

* better naming

* fix build

* fix build

* rem extraneous code
2019-08-15 19:57:43 -05:00
Preston Van Loon
eb192049b8
Create a testing db method that can be used with the new database interface (#3217)
* create a testing db method that can be used with the new database interface

* lint

* lint

* PR feedback

* Update setup_db.go
2019-08-15 17:41:51 -04:00
Raul Jordan
65ee6eb3af
Refactor Slice Utils as Variadic Functions (#3206)
* variadic approach working

* transform to variadic

* comments

* all variadic funcs simplified and tests passing
2019-08-14 17:08:53 -05:00
Preston Van Loon
c0627e29a8 Add varint prefixing to ssz network encoder (#3210)
* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* Update varint.go
2019-08-14 16:18:32 -05:00
shayzluf
df65a8d118 watchtower api (#3134) 2019-08-14 12:16:17 -07:00
terence tsao
11ac9585ad
Deprecate the old, and add new DB setup util for tests (#3208) 2019-08-14 11:48:28 -07:00
Raul Jordan
d0bdbe5a33
Add Byte Slice Intersection Utils (#3204)
* remove unused code

* table drive tests for byte slice intersections

* include all table driven tests

* gazelle imports

* imports
2019-08-14 10:27:18 -05:00
terence tsao
072bb4be27 Add gossipsub parameter test (#3200)
* added test for parameters

* Fixed test
2019-08-14 08:26:03 -04:00
Preston Van Loon
5b7182cf18
ssz network encoder (with snappy compression) (#3198)
* move to deprecated-p2p

* fix lint

* Add boilerplate p2p

* lint?

* fix imports

* fix lint

* lint

* lint

* lint

* lint

* comment

* skeleton

* checkpoint

* add a new message that should work with ssz

* add ssz fix and test snappy encoder

* clarify todo

* fix viz

* move, no need to be in subpackage

* testing pb

* end nl

* use merged ssz
2019-08-13 21:37:45 -04:00
Nishant Das
1eb29a2394 Clean Up In Memory Deposits in DB (#3065)
* lint

* clean up deposits in db

* fix all references

* fixed tests

* lint

* bring it into a separate package

* fix lint

* move test

* fix ref

* fix test

* fix test

* fix test
2019-08-13 19:13:47 -04:00
Preston Van Loon
8ea586a3e6
add new p2p messages for RPC (#3199) 2019-08-13 18:48:25 -04:00
terence tsao
27319a8990 Implement State DB Methods (#3193)
* Added state implementation

* Gaze

* Fixed test

* Fixed build file

* Fixed all tests

* Merged with master

* Added comments to save and get from roots

* Make it explicit signing root

* s/./,

* s/marshalled/marshaled
2019-08-13 18:33:31 -04:00
Preston Van Loon
d2186726a3 New p2p package (#3196) 2019-08-13 14:12:00 -07:00
Preston Van Loon
82efca9b6f
Move p2p to deprecated-p2p (#3191)
* move to deprecated-p2p

* fix lint

* lint?

* fix lint

* lint

* lint

* lint

* lint
2019-08-13 14:52:04 -04:00
Preston Van Loon
e31792f999
deprecate node p2p config (#3192) 2019-08-13 14:24:09 -04:00
skillful-alex
4e886a84f9 Added roughtime to IsSlotValid and fixed test TestIsValidBlock_InvalidSlot (#3186)
* add roughtime to IsSlotValid

* gazelle

* gofmt -s
2019-08-13 13:59:11 -04:00
terence tsao
a3ac250ac1 Mark deprecated protobuf p2p msgs (#3194)
* Mark deprecated msgs

* Deprecate all

* Build
2019-08-13 13:33:04 -04:00
Raul Jordan
655f5830f4 Implement Blocks DB Methods (#3195) 2019-08-13 09:49:27 -07:00
Preston Van Loon
856dde497b
Move sync to deprecated- prefix (#3190)
* Move sync to deprecated_ prefix

* do not use underscore

* fix
2019-08-13 12:35:34 -04:00
Raul Jordan
8d8849feed
Implement Attestations DB Methods (#3183)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused

* add schema

* implementation simplicity

* has validator latest vote func impl

* retrieve validator latest vote

* has idx

* implement missing validator methods

* missing validator methods and test helpers

* validator index crud tests

* validator tests

* save attestation implementation

* attestation basic methods

* batch  save

* all buckets

* refactor with ok bool

* retrieval by root working

* todo for has attestations

* all tests passing, fmt, imports

* generate key use helper

* most att methods complete

* crud tests passing

* closer and closer to filtering all atts

* default no filter

* filter criteria functioning

* simplified conditional

* filter criteria func

* filter criteria

* filter criteria for atts there

* query filter map strategy

* internal filter api complete

* comments

* complete the passing of all other tests using criteria met

* imports

* fix broken build:

* breaking arg

* import sort groups

* keygen outside tx

* address feedback
2019-08-13 11:04:33 -05:00
Preston Van Loon
fa0ef76561
Lower jaegar BufferMaxCount (#3188)
* Lower BufferMaxCount

* revert tools/cluster-pk-manager/client/main.go
2019-08-12 22:50:47 -04:00
Raul Jordan
551ed1d335
Internal Filter Criteria Builder API (#3185)
* query filter map strategy

* internal filter api complete

* comments

* terence feedback
2019-08-12 20:55:28 -05:00
Preston Van Loon
0ab969a87d fix panic on invalid bls key (#3184) 2019-08-12 13:58:25 -07:00
Raul Jordan
6bd8ae8f67
Implement Validator DB Methods (#3172)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused

* add schema

* implementation simplicity

* has validator latest vote func impl

* retrieve validator latest vote

* has idx

* implement missing validator methods

* missing validator methods and test helpers

* validator index crud tests

* validator tests

* all buckets

* refactor with ok bool

* all tests passing, fmt, imports
2019-08-12 14:33:07 -05:00
terence tsao
715b9cd5ba Save head block root for new DB refactor (#3182)
* Save head block root instead of save head state

* Revert state
2019-08-12 12:13:30 -04:00
terence tsao
212f8d6c3f Mega renovate updates (#3181)
* Update io_bazel_rules_k8s commit hash to b815470

* Update io_kubernetes_build commit hash to 9f4571a

* Update dependency com_github_google_go_cmp to v0.3.1

* Update libp2p
2019-08-12 10:33:39 -04:00
Nishant Das
22df351e89 Request Missing Logs (#3173)
* add request for missing logs

* fix formatting

* add metric

* add test

* fix test
2019-08-12 06:59:57 -04:00
renovate[bot]
06950907c8 Update dependency com_google_cloud_go to v0.44.0 (#3178) 2019-08-11 21:55:48 -04:00
Preston Van Loon
4c62b0410f
Filter deposits by index and block number (#3171)
* filter deposits by index and block number

* fix test and inverted logic

* nishant feedback

* -1

* Update beacon-chain/db/deposits_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul feedback: explicit variables

* revert everything :(

* a better fix

* fix +1

* fix test
2019-08-11 15:54:39 -04:00
Preston Van Loon
a938274d57
Return errors in log processing (#3170)
* Return errors in log processing

* error formatting
2019-08-11 10:39:22 -04:00
Raul Jordan
dce9c41094 Define Interface Stubs for New DB Interface (#3164)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused
2019-08-10 20:50:10 -04:00
Ivan Martinez
bb2d79be85 Aggregate attestations before adding into the DB (#3149)
* Implement Attestation Aggregation before inserting into the DB

* Nearly complete test for aggregating signatures

* Finish tests for aggregating signatures

* gazelle

* Rename tests

* add lock and advance state

* only advance if necessary

* Fix most tests

* Fix more of DB keys and changing keys to data hashes

* Fix a lot of tests and inconsistencies

* fix lock

* gaz

* undo local changes

* fix ref

* fix ref

* Fix some tests

* clear cache

* fix sync for attestations

* finally working across multiple nodes

* gen proto

* lint

* properly wrap error
2019-08-10 16:13:04 -04:00
Preston Van Loon
830a0a4bca
Fix mid-epoch assignment requests (#3168)
* fix mid-epoch assignments

* add quick test comment
2019-08-10 15:49:58 -04:00
terence tsao
d153abd992
Remove attestation announcement (#3165) 2019-08-09 16:31:44 -07:00
Raul Jordan
3d63bca127 Define New DB Interface (#3163)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method
2019-08-09 15:17:18 -04:00
Jean-André Santoni
e1dfe73525 Query a roughtime server to mitigate NTP attacks (#3151) 2019-08-09 07:05:08 -07:00
Preston Van Loon
d860dbbb60
Fix deposits at genesis eth1data (#3161)
* Exclude additional deposits from genesis block

* fix tests

* add test to cover this scenario
2019-08-08 23:26:18 -04:00
terence tsao
32c426ed1b Replaced block and state roots construction to SlotsPerHistoricalRoot (#3160) 2019-08-08 13:12:35 -05:00
terence tsao
b3e29399aa
Reest array size to SlotsPerHistoricalRoot (#3158) 2019-08-08 08:03:24 -07:00
terence tsao
2ec8a46cb2
forgot to wrap these two errors (#3156) 2019-08-08 06:55:25 -07:00
Preston Van Loon
ccc7d8d7b7
Update BLS with @protolambda's improvements (#3152)
* Add @protolambda's fork until https://github.com/phoreproject/bls/pull/11

* update workspace
2019-08-07 22:54:33 -04:00
Raul Jordan
4e041c852b
utilize newest ssz and fix build (#3155) 2019-08-07 20:24:04 -05:00
Preston Van Loon
cb5c920502 Add quick bls benchmark (#3148) 2019-08-05 17:11:38 -07:00
Nishant Das
9ec54ae432 Optimize Verification Of Signatures in Attestations (#3146)
* add few changes

* add process attestation no verify

* gaz

* add reg test

* revert config

* add new method

* fix test

* preston's review

* preston's review

* space
2019-08-05 10:35:47 -04:00
terence tsao
dec694916b Renovate updates in batch (#3145)
* Update io_bazel_rules_k8s commit hash to 5648b17

* Update dependency build_bazel_rules_nodejs to v0.35.0

* Update libp2p

* Update dependency com_github_urfave_cli to v1
2019-08-05 09:33:40 -04:00
renovate[bot]
64f7569894 Update graknlabs_bazel_distribution commit hash to 8dc6490 (#3138) 2019-08-05 00:37:19 -04:00