Commit Graph

3734 Commits

Author SHA1 Message Date
Ivan Martinez
0982ff124e
Fix E2E test sharding (#5248) 2020-03-30 12:10:00 -04:00
Nishant Das
63df1d0b8d
Add Merkleize With Customized Hasher (#5234)
* add buffer for merkleizer
* add comment
* Merge refs/heads/master into merkleize
* Merge refs/heads/master into merkleize
* Merge refs/heads/master into merkleize
* Merge refs/heads/master into merkleize
* Merge refs/heads/master into merkleize
* Merge branch 'master' of https://github.com/prysmaticlabs/geth-sharding into merkleize
* Merge branch 'merkleize' of https://github.com/prysmaticlabs/geth-sharding into merkleize
* lint
* Merge refs/heads/master into merkleize
2020-03-29 06:13:24 +00:00
Ivan Martinez
cb9ac6282f
Separate anti flakes to prevent E2E issues (#5238)
* Separate anti flakes

* Gaz
2020-03-29 13:54:13 +08:00
terence tsao
c67b01e5d3
Check new state mgmt service is compatible with DB (#5231) 2020-03-28 18:07:51 -07:00
terence tsao
b40e6db1e5
Fix save blocks return nil (#5237)
* Fixed save blocks return nil
* Merge refs/heads/master into fix-batch-save-blocks
* Merge refs/heads/master into fix-batch-save-blocks
2020-03-28 19:05:56 +00:00
Preston Van Loon
f89d753275
Add configurable e2e epochs (#5235)
* Add configurable e2e epochs
* Merge refs/heads/master into configurable-test-epochs
* Merge refs/heads/master into configurable-test-epochs
2020-03-28 18:47:31 +00:00
Preston Van Loon
a24546152b
HashProto: Use fastssz when available (#5218)
* Use fastssz when available
* fix tests
* fix most tests
* Merge branch 'master' into faster-hash-proto
* Merge refs/heads/master into faster-hash-proto
* Merge refs/heads/master into faster-hash-proto
* Merge refs/heads/master into faster-hash-proto
* fix last test
* Merge branch 'faster-hash-proto' of github.com:prysmaticlabs/prysm into faster-hash-proto-2
* lint
* fix last test
* fix again
* Update beacon-chain/cache/checkpoint_state_test.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* Merge refs/heads/master into faster-hash-proto
2020-03-28 18:32:11 +00:00
Preston Van Loon
6bc70e228f
Prevent panic for different size bitlists (#5233)
* Fix #5232
* Merge branch 'master' into bugfix-5232
2020-03-28 06:25:49 +00:00
terence tsao
f2a3fadda7
Productionization new state service part 1 (#5230)
* Fixed last play methods

* Fixed a regression. Genesis case for state gen

* Comment

* Starting

* Update proto

* Remove boundary root usages

* Update migrate

* Clean up

* Remove unused db methods

* Kafta

* Kafta

* Update tests

* Comments

* Fix state summary tests

* Missed one pass through for kafta
2020-03-27 13:28:38 -07:00
terence tsao
6a4b17f237
Prune garbage state is not for new state mgmt (#5225)
* Prune garbage state is not for new state mgmt
* Merge branch 'master' into state-mgmt-pruning
* Merge branch 'master' into state-mgmt-pruning
* Merge branch 'master' into state-mgmt-pruning
2020-03-27 14:30:24 +00:00
Victor Farazdagi
7ebb3c1784
init-sync revamp (#5148)
* fix issue with rate limiting
* force fetcher to wait for minimum peers
* adds backoff interval
* cap the max blocks requested from a peer
* queue rewritten
* adds docs to fsm
* fix visibility
* updates fsm
* fsm tests added
* optimizes queue resource allocations
* removes debug log
* replace auto-fixed comment
* fixes typo
* better handling of evil peers
* fixes test
* minor fixes to fsm
* better interface for findEpochState func
2020-03-27 09:54:57 +03:00
Nishant Das
33f6c22607
Revert "Add Fast Copy of Trie" (#5228)
* Revert "new fixes (#5221)"

This reverts commit 4118fa5242.
2020-03-27 01:06:30 +00:00
terence tsao
1a0a399bed
Handle genesis case for blocks/states at slot index (#5224)
* Handle highest slot = 0
* TestStore_GenesisState_CanGetHighestBelow
* TestStore_GenesisBlock_CanGetHighestAt
* Merge refs/heads/master into handle-genesis
2020-03-27 00:09:14 +00:00
Preston Van Loon
c4c9a8465a
Faster hashing for attestation pool (#5217)
* use faster hash proto
* Merge branch 'master' into faster-att-pool
* gaz
* Merge branch 'faster-att-pool' of github.com:prysmaticlabs/prysm into faster-att-pool
* nil checks and failing tests
* Merge refs/heads/master into faster-att-pool
* Merge refs/heads/master into faster-att-pool
* Merge refs/heads/master into faster-att-pool
* Merge refs/heads/master into faster-att-pool
* Merge refs/heads/master into faster-att-pool
* Fix
* Merge branch 'faster-att-pool' of github.com:prysmaticlabs/prysm into faster-att-pool
* Fix tests
2020-03-26 23:55:25 +00:00
terence tsao
5e2faf1a9d
Short circuit genesis condition for new state mgmt (#5223)
* Fixed a regression. Genesis case for state gen

* Comment

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-26 14:13:45 -05:00
shayzluf
93e68db5e6
is slashable attestation endpoint implementation (#5209)
* is slashable attestation endpoint implementation
* fix todo
* comment
* Merge refs/heads/master into is_slashable_attestation
* Merge refs/heads/master into is_slashable_attestation
* Merge refs/heads/master into is_slashable_attestation
* Update slasher/rpc/server.go
* Update slasher/rpc/server.go
* Update slasher/rpc/service.go
2020-03-26 18:31:20 +00:00
Nishant Das
cdac3d61ea
Custom Block HTR (#5219)
* add custom htr

* fix root

* fix everything

* Apply suggestions from code review

* Update beacon-chain/state/stateutil/blocks.go

* Update beacon-chain/blockchain/receive_block.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* Update beacon-chain/blockchain/process_block.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>

* terence's review

Co-authored-by: terence tsao <terence@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-26 13:10:22 -05:00
Nishant Das
4118fa5242
new fixes (#5221)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-26 11:21:02 -05:00
terence tsao
2df76798bc
Add HighestSlotStatesBelow DB getter (#5213)
* Add HighestSlotStatesBelow
* Tests for HighestSlotStatesBelow
* Typos
* Comment
* Merge refs/heads/master into states-slots-saved-at
* Quick fix
* Merge branch 'states-slots-saved-at' of github.com:prysmaticlabs/prysm into states-slots-saved-at
* Prevent underflow foreal, thanks nishant!
2020-03-26 15:37:40 +00:00
Preston Van Loon
3792bf67b6
Add alpine based docker images for validator and beacon chain (#5214)
* Add alpine based images for validator and beacon chain

* Use an alpine image with glibc

* manual tags on transitional targets

* poke buildkite

* poke buildkite
2020-03-25 19:36:28 -05:00
Nishant Das
e077d3ddc9
Fix Incorrect Logging for IPV6 Addresses (#5204)
* fix ipv6 issues
* Merge branch 'master' into fixIPV6
* imports
* Merge branch 'fixIPV6' of https://github.com/prysmaticlabs/geth-sharding into fixIPV6
* Merge branch 'master' into fixIPV6
2020-03-25 17:19:11 +00:00
Preston Van Loon
2ad5cec56c
Add gRPC headers flag support for validator client side (#5203)
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-25 11:29:04 -05:00
terence tsao
6b1e60c404
Remove extra udp port log (#5205)
* Remove extra udp port log
* Merge branch 'master' into rm-log
2020-03-25 15:28:29 +00:00
terence tsao
48e984f526
Add HighestSlotBlocksBelow getter for db (#5195)
* Add HighestSlotBlockAt

* Start testing

* Apply fixes

* Typo

* Test

* Rename for clarity

* Use length

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-25 08:41:48 -05:00
Preston Van Loon
fbee94a0e9
Remove deprecated aggregator (#5200) 2020-03-25 06:14:21 -07:00
Preston Van Loon
9740245ca5
Add enable-state-field-trie for e2e (#5198)
* Add enable-state-field-trie for e2e
* Merge refs/heads/master into e2e-enable-state-field-trie
* Merge refs/heads/master into e2e-enable-state-field-trie
* fix all this
* Update shared/sliceutil/slice.go

Co-Authored-By: terence tsao <terence@prysmaticlabs.com>
* terence's review
* comment
* Merge branch 'e2e-enable-state-field-trie' of https://github.com/prysmaticlabs/geth-sharding into e2e-enable-state-field-trie
2020-03-25 06:54:56 +00:00
Preston Van Loon
48d4a8655a
Add ipv6 multiaddr support (#5199)
* Add ipv6 multiaddr support
* allow ipv6 for discv5
2020-03-25 04:03:51 +00:00
terence tsao
e15d92df06
Apply fixes to block slots methods in DB (#5194)
* Apply fixes
* Typo
* Merge refs/heads/master into fix-slots-saved-for-blocks
2020-03-25 03:05:20 +00:00
Preston Van Loon
729bd83734
Add span to HTR and skip slot cache (#5197)
* Add a bit more span data
* missing import
* Merge branch 'master' into more-spans
2020-03-25 01:15:00 +00:00
terence tsao
c63fb2cd44
Add HighestSlotState Getter for db (#5192) 2020-03-24 14:51:24 -07:00
terence tsao
78a865eb0b
Replace boltdb imports with bbolt import (#5193)
* Replaced. Debugging missing strict dependencies...
* Merge branch 'master' into bbolt-import
* Update import path
* Merge branch 'bbolt-import' of github.com:prysmaticlabs/prysm into bbolt-import
* use forked prombbolt
* Merge branch 'bbolt-import' of github.com:prysmaticlabs/prysm into bbolt-import
* fix
* remove old boltdb reference
* Use correct bolt for pk manager
* Merge branch 'bbolt-import' of github.com:prysmaticlabs/prysm into bbolt-import
* fix for docker build
* gaz, oops
2020-03-24 20:00:54 +00:00
shayzluf
6e516dabf9
Setup Slasher RPC server (#5190)
* slasher rpc server

* fix comment

* fix comment

* remove server implementation from pr

* Apply suggestions from code review

* Gazelle

* Update slasher/rpc/service.go

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* Update slasher/detection/detect.go

* Update slasher/detection/detect.go

* Update slasher/detection/detect.go

* Update slasher/detection/detect.go

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
Co-authored-by: Ivan Martinez <ivanthegreatdev@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-24 14:30:21 -04:00
Nishant Das
454e02ac4f
Add Improvements to State Benchmarks (#5187)
* add improvements

* clean up
2020-03-24 09:16:07 -05:00
Preston Van Loon
35d74981a0
Correctly return attestation data for late requests (#5183)
* Add functionality to support attestation requests that are older than the head state

* lint

* gaz

* Handle nil state case

* handle underflow of first epoch

* Remove redundant and possibly wrong genesisTime struct field

* fix remaining tests

* gofmt

* remove debug comment

* use stategen.StateByRoot interchangably with beaconDB.State

* gofmt

* goimports

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-03-23 21:30:28 -07:00
Ivan Martinez
d8bcd891c4
Change E2E config ports to be non-commonly used (#5184)
* Change config ports to be non-commonly used
2020-03-24 03:20:38 +00:00
terence tsao
2e0158d7c5
Add HighestSlotBlock Getter for db (#5182)
* Starting

* Update block getters in db

* New test

* New test for save blocks as well

* Delete blocks can clear bits tests

* Fmt
2020-03-23 18:42:41 -05:00
Ivan Martinez
bdb80f4639
Change ListAttestations to get attestations from blocks (#5145)
* Start fixing api to get from blocks

* Fix listatts tests

* Fix slasher

* Improve blocks

* Change att grouping

* Use faster att concat

* Try to fix e2e

* Change back time

* tiny e2e fix

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-23 16:22:37 -04:00
Preston Van Loon
0043fb0441
Shard //beacon-chain/core/state:go_default_test (#5180)
* Add shards to core state tests, some of the fuzzing tests can be very slow
* Merge refs/heads/master into shard-core-state-tests
2020-03-23 19:08:39 +00:00
Preston Van Loon
f520472fc6
Buildkite changes (#5178)
* Do not override jobs, dont print colors
* Merge branch 'master' of github.com:prysmaticlabs/prysm into buildkite-changes
* use composite flag for minimal downloads
* Add repository cache
* use hardlinks
* repository cache common
* query and build repo cache
2020-03-23 19:00:37 +00:00
Preston Van Loon
5241582ece
Add CORS preflight support (#5177)
* Add CORS preflight support

* lint

* clarify description
2020-03-23 13:17:17 -05:00
Nishant Das
b0128ad894
Add Attestation Subnet Bitfield (#4989)
* bump bitfield dep

* add new methods

* get it working

* add nil check

* add check

* one more check

* add flag

* everything works local run

* add debug log

* more changes

* ensuring p2p interface works enough for tests to pass

* all tests pass

* include proper naming and comments to fix lint

* Apply suggestions from code review

* discover by peers

* cannot figure out why 0 peers

* remove keys

* fix test

* fix it

* fix again

* remove log

* change back

* gaz

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2020-03-23 09:41:47 -05:00
Preston Van Loon
5d1c3da85c
BLS: some minor improvements (#5161)
* some improvements
* gofmt
* Merge refs/heads/master into bls-improvements
* Merge refs/heads/master into bls-improvements
* Merge refs/heads/master into bls-improvements
2020-03-22 23:40:39 +00:00
terence tsao
301c2a1448
New byteutils for state gen (#5163)
* New byteutils for state gen
* Added empty slice and nil checks
* Merge branch 'master' into bit-utils
* SetBit to extend bit
* Merge branch 'bit-utils' of github.com:prysmaticlabs/prysm into bit-utils
* Comment
* Add HighestBitIndexBelow
* Test for HighestBitIndexBelow
* another test and better test fail output
* Feedback
* Merge branch 'bit-utils' of github.com:prysmaticlabs/prysm into bit-utils
* Feedback
* Preston's feedback, thanks!
* Use var
* Use var
* Merge refs/heads/master into bit-utils
2020-03-22 23:19:38 +00:00
Ivan Martinez
bc3d673ea4
Parallelize E2E Testing (#5168)
* Begin cleanup for E2E

* Parellize testing

* Add comments

* Add comment
2020-03-22 19:04:23 -04:00
Preston Van Loon
3d092d3eed
Update go-bitfield (#5162)
* Update go-bitfield from https://github.com/prysmaticlabs/go-bitfield/pull/28
2020-03-22 04:51:06 +00:00
Preston Van Loon
4df5c042d9
Use faster bitfield BitIndices to build attesting indices (#5160)
* Refactor AttestingIndices to not return any error. Add tests. Add shortcut for fully attested attestation
* attestationutil.ConvertToIndexed never returned error either
* Working with benchmark:
* fix test
* Merge branch 'attestationutil-improvements-0' into attestationutil-improvements-1
* out of bounds check
* Update after merge of https://github.com/prysmaticlabs/go-bitfield/pull/26
* remove shortcut
* Merge refs/heads/attestationutil-improvements-0 into attestationutil-improvements-1
* Merge branch 'attestationutil-improvements-0' into attestationutil-improvements-1
* Merge branch 'attestationutil-improvements-1' of github.com:prysmaticlabs/prysm into attestationutil-improvements-1
* revert test...
* Merge refs/heads/attestationutil-improvements-0 into attestationutil-improvements-1
* Merge branch 'master' of github.com:prysmaticlabs/prysm into attestationutil-improvements-1
* Merge branch 'attestationutil-improvements-1' of github.com:prysmaticlabs/prysm into attestationutil-improvements-1
* Update go-bitfield after https://github.com/prysmaticlabs/go-bitfield/pull/27
2020-03-22 01:42:51 +00:00
Preston Van Loon
d06b0e8a86
Refactor attestationutil.AttestingIndices (#5159)
* Refactor AttestingIndices to not return any error. Add tests. Add shortcut for fully attested attestation
* attestationutil.ConvertToIndexed never returned error either
* fix test
* remove shortcut
* revert test...
2020-03-22 00:23:37 +00:00
Jim McDonald
4f8d9c59dd
Replace default value for datadir (#5147) 2020-03-21 23:30:51 +08:00
Ivan Martinez
021d777b5e
Add Anti-Flake test for E2E (#5149)
* Add antiflake test

* Respond to comments

* Comment

* Change issue num
2020-03-21 14:42:51 +08:00
terence tsao
dc3fb018fe
Fix new state mgmt sync stuck in a loop (#5142) 2020-03-19 18:46:35 -07:00