Commit Graph

1030 Commits

Author SHA1 Message Date
terence tsao
5d7c33a8dc Check if slot is greater before process slots (#3349) 2019-08-28 14:24:33 -05:00
terence tsao
d84ae95309
Moved delay att inclusion to fork choice service (#3345) 2019-08-28 10:26:07 -07:00
Raul Jordan
e8f030977a
wait for chainstart (#3343) 2019-08-28 12:07:58 -05:00
terence tsao
cbb66dab50
Fix finalized block filtering in sync (#3334) 2019-08-28 08:29:45 -07:00
Preston Van Loon
2ee4f00b81 Add sync/p2p metric for number of messages received by topic (#3341)
* Add going msg metric

* fmt

* rename
2019-08-28 10:14:22 -05:00
skillful-alex
7bb5ac0dde do not panic if dv5Listener is not inited (#3339) 2019-08-28 16:29:34 +05:30
terence tsao
9f2c2f0197 Minor runtime fixes (#3335) 2019-08-27 22:19:47 -05:00
terence tsao
323bbe10ed
Add checkpoint to state caching (#3333) 2019-08-27 15:01:27 -07:00
Nishant Das
ca0f61bf24 Change Ordering of Gossipsub Registration (#3330)
* fix ordering

* Add comment
2019-08-27 10:23:22 -05:00
Nishant Das
701c70ae3b add better logging (#3329) 2019-08-27 06:27:04 -07:00
terence tsao
f188609137 Implement GetHead for RPC (#3326)
* next compatible, tests pass

* terence feedback

* skip comment

* fixes

* misc fix

* on block

* Update RPC service to use chain info

* OOps

* All tests pass, run time with 8 validators work!

* Remove saving genesis validator

* Revert gensis count

* Move redundant headstate

* Comments

* Implemented GetChainHead

* Test works

* Moved mock package

* Fixed visibility for BUILD file

* Conflict
2019-08-26 15:59:17 -05:00
terence tsao
64d0826469
Update RPC service to use chain info (#3309) 2019-08-26 13:06:16 -07:00
terence tsao
a1020585fd Fix aggregation with new DB (#3323)
* Check legacy when aggregate

* Typo
2019-08-26 14:30:39 -05:00
Andrei Ivasko
f99e2bd7c9 Benchmark active indices (#3153)
* new branch off master

* bloomfilter + benchmarks done

* cfilter benchmarks

* comments added

* goimports and gofmt added

* linter issues

* bazel run //:gazelle -- fix

* workspace definitions fixed

* fixed tree_test.go

* Update workspace

* final commit

* gazelle

* updated workspace

* workspace

* reverted workspace changes

* workspace newline

* applied git checkout origin/master WORKSPACE
2019-08-26 11:30:28 -05:00
terence tsao
0b5b3865ef Update validators db during epoch boundary (#3307) 2019-08-26 11:02:17 -05:00
Nishant Das
5828278807 Fix Bolt Fatal Crash (#3320)
* add fix and reg test

* nogo

* nogo
2019-08-26 09:00:40 -05:00
Raul Jordan
9ad00ffafb
Use New Attestation Receiver Method in RPC (#3287)
* next compatible, tests pass

* terence feedback

* skip comment

* fixes

* misc fix

* on block

* resolved err
2019-08-25 15:45:55 -05:00
terence tsao
6bcb68f862
Fix save the correct head (#3306) 2019-08-24 18:39:40 -07:00
terence tsao
045badc5f3
Fix competing attestation check (#3305) 2019-08-24 16:56:40 -07:00
Nishant Das
919877f301 Ignore Messages From Local Peer (#3299)
* validate message coming into pipeline

* gaz

* add to deprecated p2p

* add new lib

* change lib
2019-08-24 14:41:24 -04:00
terence tsao
122166b317
Fix transition logging (#3303) 2019-08-24 11:51:00 -06:00
terence tsao
8870bcea64
Fix attestation pool clean up for new db (#3304) 2019-08-24 11:36:31 -06:00
Preston Van Loon
06c97256bc p2p --next: Register p2p peer count metrics (#3301) 2019-08-24 10:07:03 -06:00
Nishant Das
9d15196bed Runtime Fixes (#3300) 2019-08-24 07:26:25 -06:00
Nishant Das
111f225177 Remove IsAttCanonical From Operations Service (#3298) 2019-08-24 06:50:43 -06:00
terence tsao
a31057de83
Fixed a few more init beacon node bugs (#3297) 2019-08-23 22:02:34 -06:00
terence tsao
5294caf5e8
Save validators upon chainstart (#3295) 2019-08-23 19:59:09 -06:00
Preston Van Loon
a852d610e2 Add panic handler (#3296) 2019-08-23 19:15:02 -06:00
terence tsao
3b422cb9c6
fixed conflict att log (#3294) 2019-08-23 17:23:19 -06:00
Preston Van Loon
b04bfb87a8
only attempt discv5 listener when no-discovery is not present (#3293) 2019-08-23 17:59:59 -04:00
Preston Van Loon
0353cc533e p2p error logging (#3292) 2019-08-23 15:46:54 -06:00
Preston Van Loon
0c0ec97343
fixes (#3291) 2019-08-23 17:34:03 -04:00
terence tsao
4484558d87 Part 11 of update fork choice - tracing and spans (#3285)
* Add tracing in forkchoice service

* Gazelle
2019-08-23 15:04:06 -05:00
Preston Van Loon
ce65b11801 Beacon attestation pubsub subscriber (#3289)
* beacon attestation subscriber

* register beacon attestation handler

* fix tests
2019-08-23 14:46:04 -05:00
Raul Jordan
2e8a06d6d4
Use New Blockchain Service in RPC Package (#3286)
* new chain service usage via interface

* put in the new chain service in propose blk

* deprecate with new service for canonical block roots

* remove old chain serv absolutely in validator server

* full legacy code compatible in beacon server

* fully compliant

* full deprecation at service level

* no more mock chain serv

* fix beacon server tests

* add changes to prop server

* broken build

* --next compatible

* conditional register of chain service

* proper conversion

* nil deref
2019-08-23 13:53:07 -05:00
terence tsao
02ca2290e1
Added metrics for monitoring processed objects and competing chain (#3283) 2019-08-23 12:18:39 -06:00
terence tsao
15f052c48d Update sync to use chain info for head and finalized check point (#3288)
* Starting

* Fixed all the tests
2019-08-23 12:48:40 -05:00
Nishant Das
74df2aa0c3 Add Recent Blocks RPC Request Handler (#3281)
* add new rpc handler

* gaz

* add it back

* remove ok

* preston's comments
2019-08-23 13:10:25 -04:00
Nishant Das
22f4807e0b Implement GoodBye RPC Handler (#3282)
* add handler

* gaz and addition to main rpc method

* remove todo

* preston's comments

* gaz
2019-08-23 12:53:38 -04:00
Raul Jordan
7f475bee00
no cache tests (#3284) 2019-08-23 09:56:48 -05:00
Nishant Das
ebb0e398d3 Deposit Cache Fix (#3280)
* fix cache

* fix spacing
2019-08-22 22:49:03 -04:00
Raul Jordan
f342224410
Full RPC Package Compliance With New DB Interface (#3275)
* deprecate db

* fix build

* begin integrating new db

* gaz

* use more of the new db

* newest implementation uses head state

* remove more deprecated items

* setup validators in state helper

* fix up some tests with the new db

* resolve broken build

* gaz

* begin ensuring tests pass

* optional idx

* list validator balances passing

* default page size passing

* only two failing

* fixed most tests, found edge case

* allow nil return and add proper tests

* pass tests

* fix head block root problem

* working with the new db

* every ethereumapis method now compliant with both dbs

* pass in db into server

* proposer server all compliant

* validator service fully compliant

* fix broken build, tests pass

* spacing

* compute state root and propose block tests passing with new db

* complete proposer server tests revamp

* validator tests halfway through passing with new db

* more validator server tests

* more than halfway there

* so so close

* all validators tests done

* attester server tests fixing

* use new api

* attester server complete

* complete
2019-08-22 20:39:06 -05:00
terence tsao
c47598514c Part 9 of update fork choice - HeadBlock and HeadState getters (#3279)
* Headblock and headstate getters

* Moved mutex around
2019-08-22 20:13:56 -05:00
terence tsao
0d64f7b80e Part 6 of update fork choice - implement new ReceiveAttestation (#3246)
* Implemented new fork choice service and helpers

* Added rest of the tests

* Lint

* Add back helpers test

* Reformatted to doc, helpers and metrics.go

* include new getter for block

* create block filters from indices

* give every block index a unique bucket

* construct block indices by bucket mmap

* almost done save for the block filters

* include block filters, need a few more small touches for fetching the proper indices by bucket

* full functionality to filter by parent root

* tests pass when using the same logic as attestations

* todo

* proper todo formatting

* first minimum slot range filter

* slot range filters pass

* more filter criteria passing

* tests passing

* add todos

* all block tests pass and work

* rem fmt

* range retrieval test

* fixed test conditions

* instantiate the other buckets

* simplify bucket lookups

* deprecate non map code

* revamp to remove old index prefixes

* create indices from data

* create indices from data

* fetch block roots by slot range

* better abstractions

* simpler abstractions

* roots rename

* comment

* preston feedback

* Fixed existing tests

* allow blocks without parent root

* Cleaned up a few things

* Removed todo

* Lint

* Cleaned up a few things

* A few functions don't need to be exported

* Gaz

* Fixed visibility

* Review feedback

* Review feedback part1

* Raul's feedback, refactored OnBlock and OnAttestation to its own file

* Fixed grammar

* Lint

* Implemented ReceiveAttestation

* Use time.Time

* Implemented ReceiveAttestation

* All tests pass

* Lint

* Oooops

* Typo
2019-08-22 20:00:55 -05:00
Preston Van Loon
b59b3ec09c
P2P implement message send (#3278)
* return a stream with send, for reading response

* gofmt

* added sender impl

* fix imports
2019-08-22 19:02:46 -04:00
Raul Jordan
8f01b76366
Integrate DB Refactor Into Ethereum APIs Beacon Chain Server (#3245)
* deprecate db

* fix build

* begin integrating new db

* gaz

* use more of the new db

* newest implementation uses head state

* remove more deprecated items

* setup validators in state helper

* fix up some tests with the new db

* resolve broken build

* gaz

* begin ensuring tests pass

* optional idx

* list validator balances passing

* default page size passing

* only two failing

* fixed most tests, found edge case

* allow nil return and add proper tests

* pass tests

* fix head block root problem

* working with the new db

* every ethereumapis method now compliant with both dbs

* pass in db into server
2019-08-22 15:28:53 -05:00
Preston Van Loon
ce28feea45 Regular sync: pubsub subscriber for beacon blocks (#3220)
* add validation

* add block db check in validation

* merge

* in memory caching of seen blocks

* basic block processing

* Update BUILD.bazel

* use new receiveBlockNoPubsub

* fix build

* add TODO issue numbers

* add TODO issue numbers

* lint
2019-08-22 13:11:52 -05:00
terence tsao
2e352cf5ff Add getter for GenesisTime (#3274)
* Add getter for gensis time

* Lint

* Lint
2019-08-22 11:41:05 -05:00
Nishant Das
e0d3e78746 Add Support for Static Peering (#3272)
* add test and support for static peering

* gaz

* remove delay

* add log

* handle all peers
2019-08-22 10:23:16 -05:00
Raul Jordan
bb542d2032
Basic Block/LatestVote Caching in DB Refactor (#3249)
* add block caching layer

* runlock

* lockinggg

* latest votes map

* validator latest vote deletion method added to interface

* seguin cache working

* cache size

* impl interface

* initialize caches at struct layer
2019-08-22 10:04:13 -05:00
Nishant Das
36c9a5665d
Implement Proposer Slashing Handler (#3273)
* validate proposer slashing

* add commetn

* add handler

* add

* remove

* gaz
2019-08-22 20:01:56 +05:30
Nishant Das
83083b9c65
Fix BLS Aggregation Method (#3269)
* lint

* update to new method

* fix all tests
2019-08-22 11:45:02 +05:30
Nishant Das
c09a6b87c3
Implement Attester Slashing Handler in Sync (#3260)
* add validation

* add test

* add new changes

* fix lint

* Update beacon-chain/sync/validate_attetser_slashing_test.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* terence's comments

* change key
2019-08-22 11:04:25 +05:30
terence tsao
b91639a32e
Deprecate old block chain service (#3268)
* seperate out block chain services

* Fix blockchain service config

* Gazelle

* Fixed tests
2019-08-21 19:14:24 -06:00
Preston Van Loon
4b17711702 fix startup contract check (#3267) 2019-08-21 17:48:29 -06:00
Raul Jordan
de82956088
Include Voluntary Exits Definitions in DB Refactor (#3266)
* new interface methods

* support proposer slashings

* add in the new buckets

* all crud for propoer slashings

* attester slashings complete

* all slashings crud done

* right comment

* deposit contract tests pass

* delete out of scope methods

* conform old beacon DB

* comment

* include interface implementations

* deprecations

* pass lint
2019-08-21 16:32:44 -05:00
Raul Jordan
3ca4d6fd91
Add Deposit Contract Methods to DB Refactor (#3264)
* new interface methods

* support proposer slashings

* add in the new buckets

* all crud for propoer slashings

* attester slashings complete

* all slashings crud done

* right comment

* deposit contract tests pass

* delete out of scope methods

* conform old beacon DB

* comment

* deprecations

* pass lint

* Update deposit_contract.go
2019-08-21 16:11:50 -05:00
Preston Van Loon
01de412956
Minor runtime fixes for --next (#3265)
* some runtime fixes

* fixes

* fixes

* fixes

* fixes

* fixes
2019-08-21 16:58:38 -04:00
Raul Jordan
8fef74ab25
Block Slashings CRUD Methods DB Refactor (#3261)
* new interface methods

* support proposer slashings

* add in the new buckets

* all crud for propoer slashings

* attester slashings complete

* all slashings crud done

* right comment

* delete out of scope methods

* conform old beacon DB
2019-08-21 15:21:04 -05:00
terence tsao
bfbff885fe Part 7 of update fork choice - chain info access (#3263)
* Implemented new fork choice service and helpers

* Added rest of the tests

* Lint

* Add back helpers test

* Add benchmark tests

* Add yaml driven framework tests

* Reformatted to doc, helpers and metrics.go

* include new getter for block

* create block filters from indices

* give every block index a unique bucket

* construct block indices by bucket mmap

* almost done save for the block filters

* include block filters, need a few more small touches for fetching the proper indices by bucket

* full functionality to filter by parent root

* tests pass when using the same logic as attestations

* todo

* proper todo formatting

* first minimum slot range filter

* slot range filters pass

* more filter criteria passing

* tests passing

* add todos

* all block tests pass and work

* rem fmt

* range retrieval test

* fixed test conditions

* Implemented new receive block methods

* Comments

* Remove mark evil block

* instantiate the other buckets

* simplify bucket lookups

* deprecate non map code

* revamp to remove old index prefixes

* create indices from data

* create indices from data

* fetch block roots by slot range

* better abstractions

* simpler abstractions

* roots rename

* comment

* preston feedback

* Fixed existing tests

* allow blocks without parent root

* Cleaned up a few things

* Removed todo

* Lint

* Cleaned up a few things

* A few functions don't need to be exported

* Gaz

* Fixed visibility

* Review feedback

* Review feedback part1

* Raul's feedback, refactored OnBlock and OnAttestation to its own file

* Fixed grammar

* Lint

* Renamed to receive_block.go

* Use time.Time

* Preston's feedback, removed OnTick and Store.time

* Dont have to cast it to kv

* add block caching layer

* runlock

* lockinggg

* Fixed

* Avoid 2 fetches of the same data

* latest votes map

* Gaz

* Test passes

* Lint

* Fixed db set up

* Fixed all the tests

* Gazelle

* Added tests

* Remove todo

* remove kv

* Last clean up

* Last clean up

Last clean up

* Lint

* Preston's feedback

* Starting

* Gazelle
2019-08-21 14:50:27 -05:00
terence tsao
b440891aea Part 5 of update fork choice - implement new ReceiveBlock (#3242)
* Implemented new fork choice service and helpers

* Added rest of the tests

* Lint

* Add back helpers test

* Add benchmark tests

* Add yaml driven framework tests

* Reformatted to doc, helpers and metrics.go

* include new getter for block

* create block filters from indices

* give every block index a unique bucket

* construct block indices by bucket mmap

* almost done save for the block filters

* include block filters, need a few more small touches for fetching the proper indices by bucket

* full functionality to filter by parent root

* tests pass when using the same logic as attestations

* todo

* proper todo formatting

* first minimum slot range filter

* slot range filters pass

* more filter criteria passing

* tests passing

* add todos

* all block tests pass and work

* rem fmt

* range retrieval test

* fixed test conditions

* Implemented new receive block methods

* Comments

* Remove mark evil block

* instantiate the other buckets

* simplify bucket lookups

* deprecate non map code

* revamp to remove old index prefixes

* create indices from data

* create indices from data

* fetch block roots by slot range

* better abstractions

* simpler abstractions

* roots rename

* comment

* preston feedback

* Fixed existing tests

* allow blocks without parent root

* Cleaned up a few things

* Removed todo

* Lint

* Cleaned up a few things

* A few functions don't need to be exported

* Gaz

* Fixed visibility

* Review feedback

* Review feedback part1

* Raul's feedback, refactored OnBlock and OnAttestation to its own file

* Fixed grammar

* Lint

* Renamed to receive_block.go

* Use time.Time

* Preston's feedback, removed OnTick and Store.time

* Dont have to cast it to kv

* add block caching layer

* runlock

* lockinggg

* Fixed

* Avoid 2 fetches of the same data

* latest votes map

* Gaz

* Test passes

* Lint

* Fixed db set up

* Fixed all the tests

* Gazelle

* Added tests

* Remove todo

* remove kv

* Last clean up

* Last clean up

Last clean up

* Lint

* Preston's feedback
2019-08-21 13:40:00 -04:00
Nishant Das
79e57e8e8e Add Flag for SSZ Encoding (#3256)
* add flag and enum

* change help message

* linter

* add flag

* add comment

* one more comment

* fix panic

* preston's comments

* add panic
2019-08-21 12:33:48 -04:00
Preston Van Loon
acb20e269c Add flags to support new database, new sync (#3252) 2019-08-21 10:04:00 -06:00
Nishant Das
3cb32c3792
Implement Discv5 in Prysm (#3211)
* add discovery

* gaz

* add build options

* add udpPort

* add more changes

* refactor private key

* added discovery loop

* add ttl

* add ttl

* use ip type instead of string

* tests pass

* gaz and new test file

* add test

* add more tests

* add one more test

* adding multiAddr tests

* adding new protocol , listener

* fix keys

* more fixes

* more changes dialing peers works now

* gaz

* add more changes

* add more changes

* gaz

* add new test helpers

* new test

* fixed all tests

* gaz

* reduce sleep

* lint

* new changes

* change formats

* fix all this stuff

* remove discv5 protocol

* remove protocol

* remove port condition,too restrictive

* preston's feedback

* preston's feedback

* close all peers

* gaz

* remove unused func

* Update beacon-chain/p2p/service.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>

* remove build options

* refactor tests
2019-08-21 11:38:30 +05:30
terence tsao
8fc3c55199
Forkchoice get head tiebreaker (#3253) 2019-08-20 21:26:04 -06:00
Preston Van Loon
e146bc35c0
revert this again (#3254) 2019-08-20 23:10:00 -04:00
terence tsao
6195a0bfa1
Fix GetHeadFromYaml is flakey (#3251)
* Forgot clear cache needs to be within the loop

* space
2019-08-20 19:57:35 -06:00
terence tsao
e330fa5733
Part 3 of fork choice update - yaml tests (#3213) 2019-08-20 16:20:54 -06:00
terence tsao
1c4b7329f2
Part 2 of fork choice update - benchmark tests (#3212) 2019-08-20 16:13:20 -06:00
Raul Jordan
121a277726 add better godocs (#3250) 2019-08-20 18:01:55 -04:00
Raul Jordan
900b550864
add all proper spans to methods (#3248) 2019-08-20 14:35:48 -05:00
Preston Van Loon
3f0d1c1d41 Reg sync beacon blocks (#3218)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* basic test with a hardcoded fork choice

* updated beacon config to use genesis fork version

* checkpoint on hello handler

* create a testing db method that can be used with the new database interface

* lint

* lint

* PR feedback

* checkpoint

* passing tests

* comments, errors

* comments, errors

* remove swarm

* Add basic sanity test, naive implementation

* add test with real data
2019-08-20 14:06:49 -05:00
terence tsao
01bbc552cd
Part 1 of fork choice update - fork choice as a service (#3209) 2019-08-20 12:26:43 -06:00
Raul Jordan
8f967d26d7
Allow Nil Return from DB Methods (#3247)
* allow nil return and add proper tests

* pass tests
2019-08-20 12:24:29 -05:00
Raul Jordan
a7d336a7d0
Misc Blocks DB Improvements (#3244)
* allow string constructions

* fix all other cases with formatting
2019-08-20 10:09:10 -05:00
Raul Jordan
1c8ac6658e
Implement Block DB Methods (#3221)
* include new getter for block

* create block filters from indices

* give every block index a unique bucket

* construct block indices by bucket mmap

* almost done save for the block filters

* include block filters, need a few more small touches for fetching the proper indices by bucket

* full functionality to filter by parent root

* tests pass when using the same logic as attestations

* todo

* proper todo formatting

* first minimum slot range filter

* slot range filters pass

* more filter criteria passing

* tests passing

* add todos

* all block tests pass and work

* rem fmt

* range retrieval test

* fixed test conditions

* instantiate the other buckets

* simplify bucket lookups

* deprecate non map code

* revamp to remove old index prefixes

* create indices from data

* create indices from data

* fetch block roots by slot range

* better abstractions

* simpler abstractions

* roots rename

* comment

* preston feedback

* allow blocks without parent root
2019-08-19 19:34:53 -05:00
Preston Van Loon
0b8cbd06b6
Add flag for testing new p2p (#3243)
* refactor a bit to select p2p

* lint

* fix build

* fix build

* fix build

* fix build

* fix build
2019-08-19 17:20:56 -04:00
terence tsao
6ac0d12f5b Part 4 of update fork choice - mark old functions deprecated (#3215)
* Mark these soon to be deprecated functions as "deprecated"

* Deprecate all

* Marked fork choice reorg test as deprecated

* Marked fork choice reorg test as deprecated

* Gaz
2019-08-19 11:34:25 -04:00
Preston Van Loon
a26ef9b44f
Regular sync: pubsub subscriber for voluntary exits (#3227)
* voluntary exit validator & handler skeleton

* pass through && register

* voluntary exits

* voluntary exits

* voluntary exits

* gaz

* lint
2019-08-18 11:33:58 -04:00
Preston Van Loon
b8e550b1e9
Add p2p broadcast implementation (#3226)
* add broadcaster impl

* change API so broadcast returns an error

* change API so broadcast returns an error

* add test for message not mapped

* lint msg

* lint msg
2019-08-18 00:32:39 -04:00
Preston Van Loon
68210eb733
remove the fork version changes since this would be required change for deposits and needs to be more carefully thought about (#3224) 2019-08-17 11:12:05 -04:00
Preston Van Loon
78bf39aff7
sync RPC: Hello handler (#3216)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* basic test with a hardcoded fork choice

* updated beacon config to use genesis fork version

* checkpoint on hello handler

* create a testing db method that can be used with the new database interface

* lint

* lint

* PR feedback

* checkpoint

* passing tests

* comments, errors

* comments, errors

* remove swarm

* Update WORKSPACE

* Update WORKSPACE

* merge

* lint

* lint

* touch

* touch

* imports
2019-08-16 16:03:11 -04:00
Preston Van Loon
81f868bd48
Regular Sync - First Pass (#3201)
* checkpoint

* checkpoint

* varint prefix for ssz

* move the encoding API around a little bit to support reader writer

* add a simple test for the happy path subscribe

* move wait timeout to testutil

* Add inverted topic mapping

* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* make anon methods more clear

* clean up log fields

* move topic mapping, reformat TODOs, get ready for brutal team review

* lint

* lint

* lint

* Update beacon-chain/p2p/gossip_topic_mappings.go

Co-Authored-By: Nishant Das <nishdas93@gmail.com>

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* PR feedback

* Update WORKSPACE

* Update WORKSPACE
2019-08-16 13:13:04 -04:00
Raul Jordan
6ec9d7e6e2
Utilize Indices for Key Lookup and Filtering Attestations in DB (#3202)
* begin indices approach

* use shard bucket

* continue the indices approach

* eliminate the filter checkers in favor of the single loop of root lookups

* elim extraneous println statement

* continue the indices approach

* intersection for multiple filter types works, but is complex, verbose, and nearly unreadable

* remove unused code

* table drive tests for byte slice intersections

* include all table driven tests

* gazelle imports

* better abstractions

* better comments

* variadic approach working

* transform to variadic

* comments

* comments

* separate bucket for indices for faster range scans

* attestation key as hash tree root of data and different indices buckets

* test pass

* default behavior without filter

* appropriate filter criterion errors if criterion does not apply to type

* better abstractions and prune keys on deletion

* better naming

* fix build

* fix build

* rem extraneous code
2019-08-15 19:57:43 -05:00
Preston Van Loon
eb192049b8
Create a testing db method that can be used with the new database interface (#3217)
* create a testing db method that can be used with the new database interface

* lint

* lint

* PR feedback

* Update setup_db.go
2019-08-15 17:41:51 -04:00
Preston Van Loon
c0627e29a8 Add varint prefixing to ssz network encoder (#3210)
* Add varint prefixing to ssz network encoder

* fix spacing

* fix comments

* fix comments

* Update varint.go
2019-08-14 16:18:32 -05:00
terence tsao
11ac9585ad
Deprecate the old, and add new DB setup util for tests (#3208) 2019-08-14 11:48:28 -07:00
Raul Jordan
d0bdbe5a33
Add Byte Slice Intersection Utils (#3204)
* remove unused code

* table drive tests for byte slice intersections

* include all table driven tests

* gazelle imports

* imports
2019-08-14 10:27:18 -05:00
terence tsao
072bb4be27 Add gossipsub parameter test (#3200)
* added test for parameters

* Fixed test
2019-08-14 08:26:03 -04:00
Preston Van Loon
5b7182cf18
ssz network encoder (with snappy compression) (#3198)
* move to deprecated-p2p

* fix lint

* Add boilerplate p2p

* lint?

* fix imports

* fix lint

* lint

* lint

* lint

* lint

* comment

* skeleton

* checkpoint

* add a new message that should work with ssz

* add ssz fix and test snappy encoder

* clarify todo

* fix viz

* move, no need to be in subpackage

* testing pb

* end nl

* use merged ssz
2019-08-13 21:37:45 -04:00
Nishant Das
1eb29a2394 Clean Up In Memory Deposits in DB (#3065)
* lint

* clean up deposits in db

* fix all references

* fixed tests

* lint

* bring it into a separate package

* fix lint

* move test

* fix ref

* fix test

* fix test

* fix test
2019-08-13 19:13:47 -04:00
terence tsao
27319a8990 Implement State DB Methods (#3193)
* Added state implementation

* Gaze

* Fixed test

* Fixed build file

* Fixed all tests

* Merged with master

* Added comments to save and get from roots

* Make it explicit signing root

* s/./,

* s/marshalled/marshaled
2019-08-13 18:33:31 -04:00
Preston Van Loon
d2186726a3 New p2p package (#3196) 2019-08-13 14:12:00 -07:00
Preston Van Loon
82efca9b6f
Move p2p to deprecated-p2p (#3191)
* move to deprecated-p2p

* fix lint

* lint?

* fix lint

* lint

* lint

* lint

* lint
2019-08-13 14:52:04 -04:00
Preston Van Loon
e31792f999
deprecate node p2p config (#3192) 2019-08-13 14:24:09 -04:00
skillful-alex
4e886a84f9 Added roughtime to IsSlotValid and fixed test TestIsValidBlock_InvalidSlot (#3186)
* add roughtime to IsSlotValid

* gazelle

* gofmt -s
2019-08-13 13:59:11 -04:00
Raul Jordan
655f5830f4 Implement Blocks DB Methods (#3195) 2019-08-13 09:49:27 -07:00
Preston Van Loon
856dde497b
Move sync to deprecated- prefix (#3190)
* Move sync to deprecated_ prefix

* do not use underscore

* fix
2019-08-13 12:35:34 -04:00
Raul Jordan
8d8849feed
Implement Attestations DB Methods (#3183)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused

* add schema

* implementation simplicity

* has validator latest vote func impl

* retrieve validator latest vote

* has idx

* implement missing validator methods

* missing validator methods and test helpers

* validator index crud tests

* validator tests

* save attestation implementation

* attestation basic methods

* batch  save

* all buckets

* refactor with ok bool

* retrieval by root working

* todo for has attestations

* all tests passing, fmt, imports

* generate key use helper

* most att methods complete

* crud tests passing

* closer and closer to filtering all atts

* default no filter

* filter criteria functioning

* simplified conditional

* filter criteria func

* filter criteria

* filter criteria for atts there

* query filter map strategy

* internal filter api complete

* comments

* complete the passing of all other tests using criteria met

* imports

* fix broken build:

* breaking arg

* import sort groups

* keygen outside tx

* address feedback
2019-08-13 11:04:33 -05:00
Raul Jordan
551ed1d335
Internal Filter Criteria Builder API (#3185)
* query filter map strategy

* internal filter api complete

* comments

* terence feedback
2019-08-12 20:55:28 -05:00