Commit Graph

2538 Commits

Author SHA1 Message Date
Raul Jordan
121a277726 add better godocs (#3250) 2019-08-20 18:01:55 -04:00
Raul Jordan
900b550864
add all proper spans to methods (#3248) 2019-08-20 14:35:48 -05:00
Preston Van Loon
3f0d1c1d41 Reg sync beacon blocks (#3218)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* basic test with a hardcoded fork choice

* updated beacon config to use genesis fork version

* checkpoint on hello handler

* create a testing db method that can be used with the new database interface

* lint

* lint

* PR feedback

* checkpoint

* passing tests

* comments, errors

* comments, errors

* remove swarm

* Add basic sanity test, naive implementation

* add test with real data
2019-08-20 14:06:49 -05:00
terence tsao
01bbc552cd
Part 1 of fork choice update - fork choice as a service (#3209) 2019-08-20 12:26:43 -06:00
Raul Jordan
8f967d26d7
Allow Nil Return from DB Methods (#3247)
* allow nil return and add proper tests

* pass tests
2019-08-20 12:24:29 -05:00
Raul Jordan
a7d336a7d0
Misc Blocks DB Improvements (#3244)
* allow string constructions

* fix all other cases with formatting
2019-08-20 10:09:10 -05:00
Raul Jordan
1c8ac6658e
Implement Block DB Methods (#3221)
* include new getter for block

* create block filters from indices

* give every block index a unique bucket

* construct block indices by bucket mmap

* almost done save for the block filters

* include block filters, need a few more small touches for fetching the proper indices by bucket

* full functionality to filter by parent root

* tests pass when using the same logic as attestations

* todo

* proper todo formatting

* first minimum slot range filter

* slot range filters pass

* more filter criteria passing

* tests passing

* add todos

* all block tests pass and work

* rem fmt

* range retrieval test

* fixed test conditions

* instantiate the other buckets

* simplify bucket lookups

* deprecate non map code

* revamp to remove old index prefixes

* create indices from data

* create indices from data

* fetch block roots by slot range

* better abstractions

* simpler abstractions

* roots rename

* comment

* preston feedback

* allow blocks without parent root
2019-08-19 19:34:53 -05:00
Preston Van Loon
0b8cbd06b6
Add flag for testing new p2p (#3243)
* refactor a bit to select p2p

* lint

* fix build

* fix build

* fix build

* fix build

* fix build
2019-08-19 17:20:56 -04:00
Nishant Das
b7b62e24ad new changes (#3241) 2019-08-19 12:31:15 -04:00
Preston Van Loon
e88bbaf614
Block networking in sandbox test by default, fix roughtime panic (#3240)
* block networking in sandbox test by default, fix roughtime panic

* Update .bazelrc
2019-08-19 12:13:05 -04:00
terence tsao
6ac0d12f5b Part 4 of update fork choice - mark old functions deprecated (#3215)
* Mark these soon to be deprecated functions as "deprecated"

* Deprecate all

* Marked fork choice reorg test as deprecated

* Marked fork choice reorg test as deprecated

* Gaz
2019-08-19 11:34:25 -04:00
terence tsao
4c1ff2a897 Mega renovate updates (#3239)
* Update dependency build_bazel_rules_nodejs to v0.35.0

* Update io_bazel_rules_k8s commit hash to b815470

* Update dependency build_bazel_rules_nodejs to v0.36.0

* Update dependency com_github_googleapis_gnostic to v0.3.1

* Update dependency com_github_mattn_go_isatty to v0.0.9

* Update dependency com_google_cloud_go to v0.44.3

* Update dependency io_bazel_rules_go to v0.19.3

* Update libp2p

* Update io_bazel_rules_k8s commit hash to b799dd0
2019-08-19 09:57:19 -04:00
Raul Jordan
5f2e0493eb
Fix Slice Union Helpers for Variadic Arguments (#3228)
* union changes

* slice util fixes
2019-08-18 22:42:50 -05:00
Nishant Das
16c5d96e6a
Change BootNode to use Discv5 instead of Kademlia (#3203)
* add new test

* specify ecdsa keygen

* skip test

* fix ref

* comment again

* fix test and clean up

* gaz

* change to another format

* Apply suggestions from code review

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* fix docker build

* add close
2019-08-19 01:24:20 +05:30
Preston Van Loon
a26ef9b44f
Regular sync: pubsub subscriber for voluntary exits (#3227)
* voluntary exit validator & handler skeleton

* pass through && register

* voluntary exits

* voluntary exits

* voluntary exits

* gaz

* lint
2019-08-18 11:33:58 -04:00
Preston Van Loon
b8e550b1e9
Add p2p broadcast implementation (#3226)
* add broadcaster impl

* change API so broadcast returns an error

* change API so broadcast returns an error

* add test for message not mapped

* lint msg

* lint msg
2019-08-18 00:32:39 -04:00
Preston Van Loon
68210eb733
remove the fork version changes since this would be required change for deposits and needs to be more carefully thought about (#3224) 2019-08-17 11:12:05 -04:00
Preston Van Loon
78bf39aff7
sync RPC: Hello handler (#3216)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* basic test with a hardcoded fork choice

* updated beacon config to use genesis fork version

* checkpoint on hello handler

* create a testing db method that can be used with the new database interface

* lint

* lint

* PR feedback

* checkpoint

* passing tests

* comments, errors

* comments, errors

* remove swarm

* Update WORKSPACE

* Update WORKSPACE

* merge

* lint

* lint

* touch

* touch

* imports
2019-08-16 16:03:11 -04:00
Preston Van Loon
81f868bd48
Regular Sync - First Pass (#3201)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* Update WORKSPACE

* Update WORKSPACE
2019-08-16 13:13:04 -04:00
Raul Jordan
6ec9d7e6e2
Utilize Indices for Key Lookup and Filtering Attestations in DB (#3202)
* begin indices approach

* use shard bucket

* continue the indices approach

* eliminate the filter checkers in favor of the single loop of root lookups

* elim extraneous println statement

* continue the indices approach

* intersection for multiple filter types works, but is complex, verbose, and nearly unreadable

* remove unused code

* table drive tests for byte slice intersections

* include all table driven tests

* gazelle imports

* better abstractions

* better comments

* variadic approach working

* transform to variadic

* comments

* comments

* separate bucket for indices for faster range scans

* attestation key as hash tree root of data and different indices buckets

* test pass

* default behavior without filter

* appropriate filter criterion errors if criterion does not apply to type

* better abstractions and prune keys on deletion

* better naming

* fix build

* fix build

* rem extraneous code
2019-08-15 19:57:43 -05:00
Preston Van Loon
eb192049b8
Create a testing db method that can be used with the new database interface (#3217)
* create a testing db method that can be used with the new database interface

* lint

* lint

* PR feedback

* Update setup_db.go
2019-08-15 17:41:51 -04:00
Raul Jordan
65ee6eb3af
Refactor Slice Utils as Variadic Functions (#3206)
* variadic approach working

* transform to variadic

* comments

* all variadic funcs simplified and tests passing
2019-08-14 17:08:53 -05:00
Preston Van Loon
c0627e29a8 Add varint prefixing to ssz network encoder (#3210)
* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* Update varint.go
2019-08-14 16:18:32 -05:00
shayzluf
df65a8d118 watchtower api (#3134) 2019-08-14 12:16:17 -07:00
terence tsao
11ac9585ad
Deprecate the old, and add new DB setup util for tests (#3208) 2019-08-14 11:48:28 -07:00
Raul Jordan
d0bdbe5a33
Add Byte Slice Intersection Utils (#3204)
* remove unused code

* table drive tests for byte slice intersections

* include all table driven tests

* gazelle imports

* imports
2019-08-14 10:27:18 -05:00
terence tsao
072bb4be27 Add gossipsub parameter test (#3200)
* added test for parameters

* Fixed test
2019-08-14 08:26:03 -04:00
Preston Van Loon
5b7182cf18
ssz network encoder (with snappy compression) (#3198)
* move to deprecated-p2p

* fix lint

* Add boilerplate p2p

* lint?

* fix imports

* fix lint

* lint

* lint

* lint

* lint

* comment

* skeleton

* checkpoint

* add a new message that should work with ssz

* add ssz fix and test snappy encoder

* clarify todo

* fix viz

* move, no need to be in subpackage

* testing pb

* end nl

* use merged ssz
2019-08-13 21:37:45 -04:00
Nishant Das
1eb29a2394 Clean Up In Memory Deposits in DB (#3065)
* lint

* clean up deposits in db

* fix all references

* fixed tests

* lint

* bring it into a separate package

* fix lint

* move test

* fix ref

* fix test

* fix test

* fix test
2019-08-13 19:13:47 -04:00
Preston Van Loon
8ea586a3e6
add new p2p messages for RPC (#3199) 2019-08-13 18:48:25 -04:00
terence tsao
27319a8990 Implement State DB Methods (#3193)
* Added state implementation

* Gaze

* Fixed test

* Fixed build file

* Fixed all tests

* Merged with master

* Added comments to save and get from roots

* Make it explicit signing root

* s/./,

* s/marshalled/marshaled
2019-08-13 18:33:31 -04:00
Preston Van Loon
d2186726a3 New p2p package (#3196) 2019-08-13 14:12:00 -07:00
Preston Van Loon
82efca9b6f
Move p2p to deprecated-p2p (#3191)
* move to deprecated-p2p

* fix lint

* lint?

* fix lint

* lint

* lint

* lint

* lint
2019-08-13 14:52:04 -04:00
Preston Van Loon
e31792f999
deprecate node p2p config (#3192) 2019-08-13 14:24:09 -04:00
skillful-alex
4e886a84f9 Added roughtime to IsSlotValid and fixed test TestIsValidBlock_InvalidSlot (#3186)
* add roughtime to IsSlotValid

* gazelle

* gofmt -s
2019-08-13 13:59:11 -04:00
terence tsao
a3ac250ac1 Mark deprecated protobuf p2p msgs (#3194)
* Mark deprecated msgs

* Deprecate all

* Build
2019-08-13 13:33:04 -04:00
Raul Jordan
655f5830f4 Implement Blocks DB Methods (#3195) 2019-08-13 09:49:27 -07:00
Preston Van Loon
856dde497b
Move sync to deprecated- prefix (#3190)
* Move sync to deprecated_ prefix

* do not use underscore

* fix
2019-08-13 12:35:34 -04:00
Raul Jordan
8d8849feed
Implement Attestations DB Methods (#3183)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused

* add schema

* implementation simplicity

* has validator latest vote func impl

* retrieve validator latest vote

* has idx

* implement missing validator methods

* missing validator methods and test helpers

* validator index crud tests

* validator tests

* save attestation implementation

* attestation basic methods

* batch  save

* all buckets

* refactor with ok bool

* retrieval by root working

* todo for has attestations

* all tests passing, fmt, imports

* generate key use helper

* most att methods complete

* crud tests passing

* closer and closer to filtering all atts

* default no filter

* filter criteria functioning

* simplified conditional

* filter criteria func

* filter criteria

* filter criteria for atts there

* query filter map strategy

* internal filter api complete

* comments

* complete the passing of all other tests using criteria met

* imports

* fix broken build:

* breaking arg

* import sort groups

* keygen outside tx

* address feedback
2019-08-13 11:04:33 -05:00
Preston Van Loon
fa0ef76561
Lower jaegar BufferMaxCount (#3188)
* Lower BufferMaxCount

* revert tools/cluster-pk-manager/client/main.go
2019-08-12 22:50:47 -04:00
Raul Jordan
551ed1d335
Internal Filter Criteria Builder API (#3185)
* query filter map strategy

* internal filter api complete

* comments

* terence feedback
2019-08-12 20:55:28 -05:00
Preston Van Loon
0ab969a87d fix panic on invalid bls key (#3184) 2019-08-12 13:58:25 -07:00
Raul Jordan
6bd8ae8f67
Implement Validator DB Methods (#3172)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused

* add schema

* implementation simplicity

* has validator latest vote func impl

* retrieve validator latest vote

* has idx

* implement missing validator methods

* missing validator methods and test helpers

* validator index crud tests

* validator tests

* all buckets

* refactor with ok bool

* all tests passing, fmt, imports
2019-08-12 14:33:07 -05:00
terence tsao
715b9cd5ba Save head block root for new DB refactor (#3182)
* Save head block root instead of save head state

* Revert state
2019-08-12 12:13:30 -04:00
terence tsao
212f8d6c3f Mega renovate updates (#3181)
* Update io_bazel_rules_k8s commit hash to b815470

* Update io_kubernetes_build commit hash to 9f4571a

* Update dependency com_github_google_go_cmp to v0.3.1

* Update libp2p
2019-08-12 10:33:39 -04:00
Nishant Das
22df351e89 Request Missing Logs (#3173)
* add request for missing logs

* fix formatting

* add metric

* add test

* fix test
2019-08-12 06:59:57 -04:00
renovate[bot]
06950907c8 Update dependency com_google_cloud_go to v0.44.0 (#3178) 2019-08-11 21:55:48 -04:00
Preston Van Loon
4c62b0410f
Filter deposits by index and block number (#3171)
* filter deposits by index and block number

* fix test and inverted logic

* nishant feedback

* -1

* Update beacon-chain/db/deposits_test.go

Co-Authored-By: Raul Jordan <raul@prysmaticlabs.com>

* raul feedback: explicit variables

* revert everything :(

* a better fix

* fix +1

* fix test
2019-08-11 15:54:39 -04:00
Preston Van Loon
a938274d57
Return errors in log processing (#3170)
* Return errors in log processing

* error formatting
2019-08-11 10:39:22 -04:00
Raul Jordan
dce9c41094 Define Interface Stubs for New DB Interface (#3164)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused
2019-08-10 20:50:10 -04:00