Commit Graph

23 Commits

Author SHA1 Message Date
Preston Van Loon
b87d0abc6c Fix finalized block root index (#3950)
* progress on fixing #3947

* upsert after, but only if not already exists

* return an error in some case

* format

* handle case where there has not been any finality yet

* it works

* remove todo

* use variable with comments

* regression test

* revert off by one issue, fix tests

* rename receiver

* @nisdas pr feedback
2019-11-08 11:00:46 +08:00
Preston Van Loon
cc16a10a33
Remove --enable-finalized-block-root-index (#3887)
* Deprecated do not use

* remove finalized block indexing feature flag

* remove flag some more

* dont rollback a managed tx

* require blocks to be in a chain in tests
2019-11-02 14:57:35 -04:00
Preston Van Loon
0481eb4872
Track finalized block root index and filter in blocks by range (#3873)
* checkpoint progress

* feature flag and test

* return true if feature flag is not on

* add lint msg

* rollback tx if failure

* Revert validator/BUILD.bazel
2019-10-29 11:14:17 -04:00
Preston Van Loon
2d98902eed
Require state to exist when updating finalized checkpoint (#3843)
* require state to exist before saving finalized checkpoint

* Add comment

* gaz

* fix test

* fix test
2019-10-23 22:50:33 -07:00
terence tsao
9a0fb5dca1
Delete DB states older than finalized checkpoint (#3824) 2019-10-23 08:30:21 -07:00
Preston Van Loon
6bf14dedcd
Better aggregated attestations pool (#3761)
* WIP of aggregated signatures in DB

* new lines at end

* taking a nap on the plane now

* fix tests

* remove duplication of attestations. so much for that airplane nap lol

* benchmark before flight lands

* gaz

* manual gaz

* fully contained checks

* quick improvement before landing

* new bitlist with fixes

* doesn't need real signatures

* it works, mostly

* print shard too

* some refactoring

* Revert "some refactoring"

This reverts commit 377ce7fbfb02321c20522da2eb27208a20a67669.

* Revert "Revert "some refactoring""

This reverts commit b46a458898acd214da44fc20dfae920d2725b6e8.

These changes are ok, just need to update the expected values

* fix tests

* lint

* lint

* upstream changes

* fix tests

* what

* resolve TODOs

* gofmt

* revert unrelated pb

* remove debug statement
2019-10-16 23:46:07 -07:00
Preston Van Loon
7a04ff6368 Add database API for creating backups (#3694)
* Save db backup

* Fix DB backup method

* Add backup db webhook

* gaz

* if err != nil

* more verbose filename

* Don't obliterate everything :)
2019-10-03 17:29:49 +08:00
Nishant Das
8ece8fb44b Expose DB Metrics (#3663)
* add in bolt metrics

* unregister in db teardown

* unregister in Close()

* fix clear db case

* fix test error

* gaz

* remove unregister

* remove gaz
2019-09-30 10:24:47 -05:00
Raul Jordan
36e3a9f82a
Implement Archival DB Methods (#3521)
* generate archive proto

* archived committee info

* archive methods added to iface definition

* impl

* update iface

* proto comments

* implement first method

* committee info

* save committee info

* participation checked in

* fully implemented

* tests

* test defs

* db impls done
2019-09-18 13:41:47 -05:00
Raul Jordan
b919429801
Archived Data Definitions & DB API (#3510)
* generate archive proto

* archived committee info

* archive methods added to iface definition

* impl

* update iface

* proto comments
2019-09-18 12:05:24 -05:00
terence tsao
c425bf2c31 Refactor fork choice start up (#3360)
* Done

* Test

* Fixed test

* emit state initialized

* Fixed existing tests

* Lint

* Lint
2019-08-30 08:58:02 -05:00
Raul Jordan
bb542d2032
Basic Block/LatestVote Caching in DB Refactor (#3249)
* add block caching layer

* runlock

* lockinggg

* latest votes map

* validator latest vote deletion method added to interface

* seguin cache working

* cache size

* impl interface

* initialize caches at struct layer
2019-08-22 10:04:13 -05:00
Raul Jordan
de82956088
Include Voluntary Exits Definitions in DB Refactor (#3266)
* new interface methods

* support proposer slashings

* add in the new buckets

* all crud for propoer slashings

* attester slashings complete

* all slashings crud done

* right comment

* deposit contract tests pass

* delete out of scope methods

* conform old beacon DB

* comment

* include interface implementations

* deprecations

* pass lint
2019-08-21 16:32:44 -05:00
Raul Jordan
3ca4d6fd91
Add Deposit Contract Methods to DB Refactor (#3264)
* new interface methods

* support proposer slashings

* add in the new buckets

* all crud for propoer slashings

* attester slashings complete

* all slashings crud done

* right comment

* deposit contract tests pass

* delete out of scope methods

* conform old beacon DB

* comment

* deprecations

* pass lint

* Update deposit_contract.go
2019-08-21 16:11:50 -05:00
Raul Jordan
8fef74ab25
Block Slashings CRUD Methods DB Refactor (#3261)
* new interface methods

* support proposer slashings

* add in the new buckets

* all crud for propoer slashings

* attester slashings complete

* all slashings crud done

* right comment

* delete out of scope methods

* conform old beacon DB
2019-08-21 15:21:04 -05:00
terence tsao
1c4b7329f2
Part 2 of fork choice update - benchmark tests (#3212) 2019-08-20 16:13:20 -06:00
Raul Jordan
900b550864
add all proper spans to methods (#3248) 2019-08-20 14:35:48 -05:00
Raul Jordan
6ec9d7e6e2
Utilize Indices for Key Lookup and Filtering Attestations in DB (#3202)
* begin indices approach

* use shard bucket

* continue the indices approach

* eliminate the filter checkers in favor of the single loop of root lookups

* elim extraneous println statement

* continue the indices approach

* intersection for multiple filter types works, but is complex, verbose, and nearly unreadable

* remove unused code

* table drive tests for byte slice intersections

* include all table driven tests

* gazelle imports

* better abstractions

* better comments

* variadic approach working

* transform to variadic

* comments

* comments

* separate bucket for indices for faster range scans

* attestation key as hash tree root of data and different indices buckets

* test pass

* default behavior without filter

* appropriate filter criterion errors if criterion does not apply to type

* better abstractions and prune keys on deletion

* better naming

* fix build

* fix build

* rem extraneous code
2019-08-15 19:57:43 -05:00
terence tsao
27319a8990 Implement State DB Methods (#3193)
* Added state implementation

* Gaze

* Fixed test

* Fixed build file

* Fixed all tests

* Merged with master

* Added comments to save and get from roots

* Make it explicit signing root

* s/./,

* s/marshalled/marshaled
2019-08-13 18:33:31 -04:00
Raul Jordan
655f5830f4 Implement Blocks DB Methods (#3195) 2019-08-13 09:49:27 -07:00
Raul Jordan
8d8849feed
Implement Attestations DB Methods (#3183)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused

* add schema

* implementation simplicity

* has validator latest vote func impl

* retrieve validator latest vote

* has idx

* implement missing validator methods

* missing validator methods and test helpers

* validator index crud tests

* validator tests

* save attestation implementation

* attestation basic methods

* batch  save

* all buckets

* refactor with ok bool

* retrieval by root working

* todo for has attestations

* all tests passing, fmt, imports

* generate key use helper

* most att methods complete

* crud tests passing

* closer and closer to filtering all atts

* default no filter

* filter criteria functioning

* simplified conditional

* filter criteria func

* filter criteria

* filter criteria for atts there

* query filter map strategy

* internal filter api complete

* comments

* complete the passing of all other tests using criteria met

* imports

* fix broken build:

* breaking arg

* import sort groups

* keygen outside tx

* address feedback
2019-08-13 11:04:33 -05:00
Raul Jordan
6bd8ae8f67
Implement Validator DB Methods (#3172)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused

* add schema

* implementation simplicity

* has validator latest vote func impl

* retrieve validator latest vote

* has idx

* implement missing validator methods

* missing validator methods and test helpers

* validator index crud tests

* validator tests

* all buckets

* refactor with ok bool

* all tests passing, fmt, imports
2019-08-12 14:33:07 -05:00
Raul Jordan
dce9c41094 Define Interface Stubs for New DB Interface (#3164)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused
2019-08-10 20:50:10 -04:00