Commit Graph

3511 Commits

Author SHA1 Message Date
terence tsao
48e984f526
Add HighestSlotBlocksBelow getter for db (#5195)
* Add HighestSlotBlockAt

* Start testing

* Apply fixes

* Typo

* Test

* Rename for clarity

* Use length

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-25 08:41:48 -05:00
Preston Van Loon
fbee94a0e9
Remove deprecated aggregator (#5200) 2020-03-25 06:14:21 -07:00
Preston Van Loon
9740245ca5
Add enable-state-field-trie for e2e (#5198)
* Add enable-state-field-trie for e2e
* Merge refs/heads/master into e2e-enable-state-field-trie
* Merge refs/heads/master into e2e-enable-state-field-trie
* fix all this
* Update shared/sliceutil/slice.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* terence's review
* comment
* Merge branch 'e2e-enable-state-field-trie' of https://github.com/prysmaticlabs/geth-sharding into e2e-enable-state-field-trie
2020-03-25 06:54:56 +00:00
Preston Van Loon
48d4a8655a
Add ipv6 multiaddr support (#5199)
* Add ipv6 multiaddr support
* allow ipv6 for discv5
2020-03-25 04:03:51 +00:00
terence tsao
e15d92df06
Apply fixes to block slots methods in DB (#5194)
* Apply fixes
* Typo
* Merge refs/heads/master into fix-slots-saved-for-blocks
2020-03-25 03:05:20 +00:00
Preston Van Loon
729bd83734
Add span to HTR and skip slot cache (#5197)
* Add a bit more span data
* missing import
* Merge branch 'master' into more-spans
2020-03-25 01:15:00 +00:00
terence tsao
c63fb2cd44
Add HighestSlotState Getter for db (#5192) 2020-03-24 14:51:24 -07:00
terence tsao
78a865eb0b
Replace boltdb imports with bbolt import (#5193)
* Replaced. Debugging missing strict dependencies...
* Merge branch 'master' into bbolt-import
* Update import path
* Merge branch 'bbolt-import' of github.com:prysmaticlabs/prysm into bbolt-import
* use forked prombbolt
* Merge branch 'bbolt-import' of github.com:prysmaticlabs/prysm into bbolt-import
* fix
* remove old boltdb reference
* Use correct bolt for pk manager
* Merge branch 'bbolt-import' of github.com:prysmaticlabs/prysm into bbolt-import
* fix for docker build
* gaz, oops
2020-03-24 20:00:54 +00:00
shayzluf
6e516dabf9
Setup Slasher RPC server (#5190)
* slasher rpc server

* fix comment

* fix comment

* remove server implementation from pr

* Apply suggestions from code review

* Gazelle

* Update slasher/rpc/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update slasher/detection/detect.go

* Update slasher/detection/detect.go

* Update slasher/detection/detect.go

* Update slasher/detection/detect.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-24 14:30:21 -04:00
Nishant Das
454e02ac4f
Add Improvements to State Benchmarks (#5187)
* add improvements

* clean up
2020-03-24 09:16:07 -05:00
Preston Van Loon
35d74981a0
Correctly return attestation data for late requests (#5183)
* Add functionality to support attestation requests that are older than the head state

* lint

* gaz

* Handle nil state case

* handle underflow of first epoch

* Remove redundant and possibly wrong genesisTime struct field

* fix remaining tests

* gofmt

* remove debug comment

* use stategen.StateByRoot interchangably with beaconDB.State

* gofmt

* goimports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-23 21:30:28 -07:00
Ivan Martinez
d8bcd891c4
Change E2E config ports to be non-commonly used (#5184)
* Change config ports to be non-commonly used
2020-03-24 03:20:38 +00:00
terence tsao
2e0158d7c5
Add HighestSlotBlock Getter for db (#5182)
* Starting

* Update block getters in db

* New test

* New test for save blocks as well

* Delete blocks can clear bits tests

* Fmt
2020-03-23 18:42:41 -05:00
Ivan Martinez
bdb80f4639
Change ListAttestations to get attestations from blocks (#5145)
* Start fixing api to get from blocks

* Fix listatts tests

* Fix slasher

* Improve blocks

* Change att grouping

* Use faster att concat

* Try to fix e2e

* Change back time

* tiny e2e fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-23 16:22:37 -04:00
Preston Van Loon
0043fb0441
Shard //beacon-chain/core/state:go_default_test (#5180)
* Add shards to core state tests, some of the fuzzing tests can be very slow
* Merge refs/heads/master into shard-core-state-tests
2020-03-23 19:08:39 +00:00
Preston Van Loon
f520472fc6
Buildkite changes (#5178)
* Do not override jobs, dont print colors
* Merge branch 'master' of github.com:prysmaticlabs/prysm into buildkite-changes
* use composite flag for minimal downloads
* Add repository cache
* use hardlinks
* repository cache common
* query and build repo cache
2020-03-23 19:00:37 +00:00
Preston Van Loon
5241582ece
Add CORS preflight support (#5177)
* Add CORS preflight support

* lint

* clarify description
2020-03-23 13:17:17 -05:00
Nishant Das
b0128ad894
Add Attestation Subnet Bitfield (#4989)
* bump bitfield dep

* add new methods

* get it working

* add nil check

* add check

* one more check

* add flag

* everything works local run

* add debug log

* more changes

* ensuring p2p interface works enough for tests to pass

* all tests pass

* include proper naming and comments to fix lint

* Apply suggestions from code review

* discover by peers

* cannot figure out why 0 peers

* remove keys

* fix test

* fix it

* fix again

* remove log

* change back

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-23 09:41:47 -05:00
Preston Van Loon
5d1c3da85c
BLS: some minor improvements (#5161)
* some improvements
* gofmt
* Merge refs/heads/master into bls-improvements
* Merge refs/heads/master into bls-improvements
* Merge refs/heads/master into bls-improvements
2020-03-22 23:40:39 +00:00
terence tsao
301c2a1448
New byteutils for state gen (#5163)
* New byteutils for state gen
* Added empty slice and nil checks
* Merge branch 'master' into bit-utils
* SetBit to extend bit
* Merge branch 'bit-utils' of github.com:prysmaticlabs/prysm into bit-utils
* Comment
* Add HighestBitIndexBelow
* Test for HighestBitIndexBelow
* another test and better test fail output
* Feedback
* Merge branch 'bit-utils' of github.com:prysmaticlabs/prysm into bit-utils
* Feedback
* Preston's feedback, thanks!
* Use var
* Use var
* Merge refs/heads/master into bit-utils
2020-03-22 23:19:38 +00:00
Ivan Martinez
bc3d673ea4
Parallelize E2E Testing (#5168)
* Begin cleanup for E2E

* Parellize testing

* Add comments

* Add comment
2020-03-22 19:04:23 -04:00
Preston Van Loon
3d092d3eed
Update go-bitfield (#5162)
* Update go-bitfield from https://github.com/prysmaticlabs/go-bitfield/pull/28
2020-03-22 04:51:06 +00:00
Preston Van Loon
4df5c042d9
Use faster bitfield BitIndices to build attesting indices (#5160)
* Refactor AttestingIndices to not return any error. Add tests. Add shortcut for fully attested attestation
* attestationutil.ConvertToIndexed never returned error either
* Working with benchmark:
* fix test
* Merge branch 'attestationutil-improvements-0' into attestationutil-improvements-1
* out of bounds check
* Update after merge of https://github.com/prysmaticlabs/go-bitfield/pull/26
* remove shortcut
* Merge refs/heads/attestationutil-improvements-0 into attestationutil-improvements-1
* Merge branch 'attestationutil-improvements-0' into attestationutil-improvements-1
* Merge branch 'attestationutil-improvements-1' of github.com:prysmaticlabs/prysm into attestationutil-improvements-1
* revert test...
* Merge refs/heads/attestationutil-improvements-0 into attestationutil-improvements-1
* Merge branch 'master' of github.com:prysmaticlabs/prysm into attestationutil-improvements-1
* Merge branch 'attestationutil-improvements-1' of github.com:prysmaticlabs/prysm into attestationutil-improvements-1
* Update go-bitfield after https://github.com/prysmaticlabs/go-bitfield/pull/27
2020-03-22 01:42:51 +00:00
Preston Van Loon
d06b0e8a86
Refactor attestationutil.AttestingIndices (#5159)
* Refactor AttestingIndices to not return any error. Add tests. Add shortcut for fully attested attestation
* attestationutil.ConvertToIndexed never returned error either
* fix test
* remove shortcut
* revert test...
2020-03-22 00:23:37 +00:00
Jim McDonald
4f8d9c59dd
Replace default value for datadir (#5147) 2020-03-21 23:30:51 +08:00
Ivan Martinez
021d777b5e
Add Anti-Flake test for E2E (#5149)
* Add antiflake test

* Respond to comments

* Comment

* Change issue num
2020-03-21 14:42:51 +08:00
terence tsao
dc3fb018fe
Fix new state mgmt sync stuck in a loop (#5142) 2020-03-19 18:46:35 -07:00
Preston Van Loon
2ab4b86f9b
Allow setting flags via yaml config file. (#4878) 2020-03-19 14:46:44 -07:00
Ivan Martinez
b30a089548
Add fetching validators by indices and public keys (#5141)
* update ethereumapis with patch
* Add indices and pubkeys to ListValidators request
* Add sorting
* Merge branch 'master' into validators-by-keys-indices
* Rename to index
* Merge branch 'validators-by-keys-indices' of https://github.com/prysmaticlabs/prysm into validators-by-keys-indices
* Add comment
2020-03-19 20:30:40 +00:00
Ivan Martinez
271938202e
Improve validator logs (#5140)
* Imporve validator logging

* Update validator/client/validator_log.go
2020-03-19 13:34:50 -04:00
shayzluf
6fe814c5aa
double proposal detector (#5120)
* proposal detector

* comment fixes

* comment fixes

* raul feedback

* fix todo

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-19 17:29:35 +05:30
Preston Van Loon
a9f4d1d02d
Attestation: Add a check for overflow (#5136)
* Add a check for overflow
* gofmt beacon-chain/cache/committee_test.go
2020-03-19 04:41:05 +00:00
Preston Van Loon
7c110e54f0
Add ssz marshal and unmarshal for most data structures (#5121)
* Add ssz marshal and unmarshal for most data structures
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Merge refs/heads/master into ssz-stuff
* Update ferran SSZ
* Update ferran's SSZ
* Merge refs/heads/master into ssz-stuff
* fix tests
* Merge branch 'ssz-stuff' of github.com:prysmaticlabs/prysm into ssz-stuff
* gaz
2020-03-19 02:39:23 +00:00
Raul Jordan
3043d4722f
Attestation Dynamic Committee Subnets (#5123)
* initiate cache
* imports fix
* add in feature config flag
* utilize a dynamic set of subnets
* Merge branch 'master' into att-subnets
* add in feature config flag
* Merge branch 'att-subnets' of github.com:prysmaticlabs/prysm into att-subnets
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into att-subnets
* shift
* more changes
* gaz
* Update beacon-chain/rpc/validator/assignments.go
* Update beacon-chain/rpc/validator/assignments.go
* add flag
* Merge branch 'att-subnets' of https://github.com/prysmaticlabs/geth-sharding into att-subnets
* Merge branch 'master' into att-subnets
* Merge refs/heads/master into att-subnets
* no double flag
* Merge branch 'att-subnets' of github.com:prysmaticlabs/prysm into att-subnets
* amend committee ids to better name
* gaz
2020-03-18 23:13:37 +00:00
Ivan Martinez
c96c8b4aa3
Minor slasher fixes (#5129)
* Minor fixes

* Change log
2020-03-18 14:49:20 -05:00
Nishant Das
9f46000dba
change to batch size (#5128)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-18 17:57:20 +08:00
Nishant Das
5450b3155e
Integrate Field Tries into Current State (#5082)
* add new methods
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* new field trie
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* perform better copying
* fix bug
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* add support for variable length arrays
* get it running
* save all new progress
* more fixes
* more fixes
* more cleanup
* some more clean up
* new memory pool
* remove lock
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into improvedHTRArrays
* use wrapper
* remove redundant methods
* cleanup
* cleanup
* remove unused method
* change field
* Update beacon-chain/state/types.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Update beacon-chain/state/types.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Apply suggestions from code review

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
2020-03-18 04:52:08 +00:00
Nishant Das
1bb12c3568
Add Field Trie to State (#5118)
* add new helpers

* make zerohash public

* remove unused method

* add more tests

* cleanup

* add in new tests

* fix all tests

* Apply suggestions from code review

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-18 10:09:31 +08:00
Ivan Martinez
1be8b3aa5e
Slasher lag fix (#5124)
* Slasher fixes

* fix
2020-03-17 16:53:08 -05:00
Nishant Das
431762164e
Add New State Utils (#5117)
* add new helpers

* make zerohash public

* remove unused method

* add more tests

* cleanup

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-17 14:25:17 -05:00
Victor Farazdagi
3ec2a0f9e0
Refactoring of initial sync (#5096)
* implements blocks queue

* refactors updateCounter method

* fixes deadlock on stop w/o start

* refactors updateSchedulerState

* more tests on schduler

* parseFetchResponse tests

* wraps up tests for blocks queue

* eod commit

* fixes data race in round robin

* revamps fetcher

* fixes race conditions + livelocks + deadlocks

* less verbose output

* fixes data race, by isolating critical sections

* minor refactoring: resolves blocking calls

* implements init-sync queue

* udpate fetch/send buffers in blocks fetcher

* blockState enum-like type alias

* refactors common code into releaseTicket()

* better gc

* linter

* minor fix to round robin

* moves original round robin into its own package

* adds enableInitSyncQueue flag

* fixes issue with init-sync service selection

* Update beacon-chain/sync/initial-sync/round_robin.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* initsyncv1 -> initsyncold

* adds span

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
2020-03-17 12:27:18 -05:00
Victor Farazdagi
e96b45b29c
asserts non-nil state (#5115) 2020-03-17 07:58:16 -07:00
terence tsao
e529f5b1d6
Part 1 of integrating new state mgmt to run time (#5108) 2020-03-16 12:07:07 -07:00
Victor Farazdagi
f18bada8c9
Init sync blocks queue (#5064)
* fixes data race, by isolating critical sections

* minor refactoring: resolves blocking calls

* implements init-sync queue

* udpate fetch/send buffers in blocks fetcher

* blockState enum-like type alias

* refactors common code into releaseTicket()

* better gc

* linter

* Update beacon-chain/sync/initial-sync/blocks_queue.go

Co-Authored-By: shayzluf <thezluf@gmail.com>

* Update beacon-chain/sync/initial-sync/blocks_queue_test.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: shayzluf <thezluf@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-16 18:21:36 +03:00
terence tsao
5657535c52
Fixed saveNewValidators error log (#5109) 2020-03-15 16:21:56 -07:00
Preston Van Loon
9da9fbdfba
Fix reward and penality zero epoch bug (#5107)
* Fix reward and penality bug https://github.com/prysmaticlabs/prysm/issues/5105
* Merge branch 'master' into fuzz-fix-attestationDelta
2020-03-15 19:14:52 +00:00
Ivan Martinez
de2ec8e575
Update README for Slasher (#5106)
* Add readme
2020-03-15 18:46:21 +00:00
terence tsao
3660732f44
Resume new state mgmt (#5102) 2020-03-15 09:47:49 -07:00
Jim McDonald
8e6c16d416
Tweak validator logging (#5103)
* Tidy up logging
2020-03-15 15:46:22 +00:00
Ivan Martinez
8143cc36bc
Add Slasher to E2E (#5061)
* Start adding "inject slashing into pool"

* Attempt at slashing

* Remove unneded

* Fix

* Begin adding slasher client to e2e

* Start slasher in e2e

* Get slashing detection working

* Get slashing evaluators working

* Progress on e2e

* Cleanup e2e

* Fix slasher e2e!

* lint

* Comment

* Fixes

* Improve accuracy of balance check

* REmove extra

* Remove extra

* Make more accurate
2020-03-15 01:09:23 -04:00