Commit Graph

16 Commits

Author SHA1 Message Date
terence tsao
9a0fb5dca1
Delete DB states older than finalized checkpoint (#3824) 2019-10-23 08:30:21 -07:00
Raul Jordan
1a94ef12b9
Ensure Blocks Are Not Duplicated When Saved to DB (#3739)
* dedup

* tests pass when using the fallback
2019-10-08 13:47:48 +09:00
Raul Jordan
8d215feb25
Large Prysm Performance Improvements (#3622)
* skip bls verification with a feature flag at runtime

* gazelle

* more bls mocks

* block roots efficiency

* db block roots now does not show up on the flame graphs

* save validator latest votes batch

* batch save att

* misc improvements to pprof

* import

* include validator index cache

* error if no filter criteria

* resolved comments

* build fix

* lint

* remove delay global

* attestation and block test fixes

* preston suggestions

* fix db tests

* fix missing broken tests

* tests pass
2019-09-30 15:45:53 -05:00
Raul Jordan
4bd2730c5e
Batch Deletions for Blocks and Attestations (#3496)
* batch deletions for blocks and attestations

* test for atts delete

* test for blocks delete

* better naming of args in iface methods

* modify a bit

* convert to batch

* blocks batch delete

* batch fixes

* suspecting deadlock

* blocks batch delete tests pass

* more complex test
2019-09-27 11:11:10 -05:00
terence tsao
f0332e1131 Save genesis state in DB (#3359)
* Done

* Test

* Fixed test

* emit state initialized
2019-08-29 15:32:35 -07:00
Raul Jordan
8f01b76366
Integrate DB Refactor Into Ethereum APIs Beacon Chain Server (#3245)
* deprecate db

* fix build

* begin integrating new db

* gaz

* use more of the new db

* newest implementation uses head state

* remove more deprecated items

* setup validators in state helper

* fix up some tests with the new db

* resolve broken build

* gaz

* begin ensuring tests pass

* optional idx

* list validator balances passing

* default page size passing

* only two failing

* fixed most tests, found edge case

* allow nil return and add proper tests

* pass tests

* fix head block root problem

* working with the new db

* every ethereumapis method now compliant with both dbs

* pass in db into server
2019-08-22 15:28:53 -05:00
Raul Jordan
bb542d2032
Basic Block/LatestVote Caching in DB Refactor (#3249)
* add block caching layer

* runlock

* lockinggg

* latest votes map

* validator latest vote deletion method added to interface

* seguin cache working

* cache size

* impl interface

* initialize caches at struct layer
2019-08-22 10:04:13 -05:00
terence tsao
1c4b7329f2
Part 2 of fork choice update - benchmark tests (#3212) 2019-08-20 16:13:20 -06:00
Raul Jordan
900b550864
add all proper spans to methods (#3248) 2019-08-20 14:35:48 -05:00
Raul Jordan
8f967d26d7
Allow Nil Return from DB Methods (#3247)
* allow nil return and add proper tests

* pass tests
2019-08-20 12:24:29 -05:00
Raul Jordan
a7d336a7d0
Misc Blocks DB Improvements (#3244)
* allow string constructions

* fix all other cases with formatting
2019-08-20 10:09:10 -05:00
Raul Jordan
1c8ac6658e
Implement Block DB Methods (#3221)
* include new getter for block

* create block filters from indices

* give every block index a unique bucket

* construct block indices by bucket mmap

* almost done save for the block filters

* include block filters, need a few more small touches for fetching the proper indices by bucket

* full functionality to filter by parent root

* tests pass when using the same logic as attestations

* todo

* proper todo formatting

* first minimum slot range filter

* slot range filters pass

* more filter criteria passing

* tests passing

* add todos

* all block tests pass and work

* rem fmt

* range retrieval test

* fixed test conditions

* instantiate the other buckets

* simplify bucket lookups

* deprecate non map code

* revamp to remove old index prefixes

* create indices from data

* create indices from data

* fetch block roots by slot range

* better abstractions

* simpler abstractions

* roots rename

* comment

* preston feedback

* allow blocks without parent root
2019-08-19 19:34:53 -05:00
Raul Jordan
6ec9d7e6e2
Utilize Indices for Key Lookup and Filtering Attestations in DB (#3202)
* begin indices approach

* use shard bucket

* continue the indices approach

* eliminate the filter checkers in favor of the single loop of root lookups

* elim extraneous println statement

* continue the indices approach

* intersection for multiple filter types works, but is complex, verbose, and nearly unreadable

* remove unused code

* table drive tests for byte slice intersections

* include all table driven tests

* gazelle imports

* better abstractions

* better comments

* variadic approach working

* transform to variadic

* comments

* comments

* separate bucket for indices for faster range scans

* attestation key as hash tree root of data and different indices buckets

* test pass

* default behavior without filter

* appropriate filter criterion errors if criterion does not apply to type

* better abstractions and prune keys on deletion

* better naming

* fix build

* fix build

* rem extraneous code
2019-08-15 19:57:43 -05:00
Raul Jordan
655f5830f4 Implement Blocks DB Methods (#3195) 2019-08-13 09:49:27 -07:00
terence tsao
715b9cd5ba Save head block root for new DB refactor (#3182)
* Save head block root instead of save head state

* Revert state
2019-08-12 12:13:30 -04:00
Raul Jordan
dce9c41094 Define Interface Stubs for New DB Interface (#3164)
* begin db interface

* define the database interface

* interface definition simplifications

* include latest message proto

* modify pbs

* rem kv folder

* add filter interface

* lint

* ctx package is great

* interface getting better

* ctx everywhere...it's everywhere!

* block roots method

* new kv store initialization

* comments

* gaz

* implement interface

* refactor for proper naming conventions

* add todos

* proper comments

* rem unused
2019-08-10 20:50:10 -04:00