Commit Graph

33 Commits

Author SHA1 Message Date
Raul Jordan
bdb09ca9ea
V1Alpha1 gRPC Gateway End-to-End Test Evaluators (#9375)
* add in initial evaluator for gateway v1alpha1

* add gazelle

* build

* compare

* functional options and tool for local verification

* chain head response compare base64

* compare validators request

* add validators compare

* more compare

* passing validator comparisons

* compare attestations pool

* compare atts

* begin peers test

* e2e verifiers

* correct port

* renaming

* evaluators

* rem api compare tool

* peer compare tests

* deepsource

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-08-16 20:31:21 +00:00
Raul Jordan
c1280867ed
Add Tracing Request Sink to E2E (#9341)
* add tracing request sink

* fix struct order

* add in base64 encode and gzip

* add encoding and gzip

* tracing sink and replay tool

* post

* replay

* include latest sink and replay tool

* capture the gzout file instead

* rem time sleep

* handle err

* better handling

* add documentation

* changes

* working sync

* working

* added more logging
2021-08-11 15:38:00 +00:00
Radosław Kapka
0347b4b4af
Retrieve validators through v1 API in e2e (#9117)
* Retrieve validators through v1 API in e2e

* fix comment

* simplify boolean comparison
2021-06-29 09:47:59 -05:00
Preston Van Loon
1f6a031630
Add a test for e2e validator to run against prior release (#9042)
* Add a test for e2e validator to run against prior release

* gofmt

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2021-06-23 17:07:28 +00:00
terence tsao
a4be7c4a46
Testutil: Reset deposit cache internally (#9059)
* Reset testutil cache internally

* Fix import

* Fix test to correct error

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2021-06-18 04:05:40 +00:00
Victor Farazdagi
961a012502
E2E refactoring (#8643)
* define ComponentRunner

* update ETH1 node

* extrace BootNode

* add logger

* remove refs to t

* gazelle

* update BeaconNode

* parametrize tests

* update log formatting

* update ValidatorNode

* remove redundant test id from params

* decrease delta

* move BootNode

* update BootNode

* revert params

* update SlasherNode

* mask unused param

* update helpers

* update test runner

* go mod tidy

* remove unused fields

* re-arrange tests

* extract sync into its own method

* gazelle

* BeaconNode

* types/types.go

* BootNode

* gofmt

* remove unused argument

* remove redundant comment

* add deprecation comment

* types comment

* remove deprecated method

* BeaconNodes -> BeaconNodeSet

* make sure that slasher is required component

* update StartBeaconNodeSet() and StartValidatorNodeSet()

* update SlasherNodeSet

* gazelle
2021-03-25 22:15:58 -05:00
terence tsao
b4437e6cec
Load graffiti from file (#8041)
* Pass graffati file and   use it

* Visibility

* Parse test

* More proposal tests

* Gazelle

* Add sequential functionality

* fix length check

* Update priorities. Specified -> random -> default

* Log warn instead return err

* Comment

* E2e test

* Comment

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-12-04 23:15:12 +00:00
Radosław Kapka
dccf0992e5
Verify eth1data vote in E2E (#7551)
* add majority vote to e2e

* extract policies to a separate package

Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-10-19 19:35:34 +00:00
Victor Farazdagi
ba07ccb484
Apply testutils assertions: final cleanup (#7003)
* slasher/beaconclient tests
* slasher/db/kv tests
* Merge branch 'master' into apply-testutils-assertions-to-slasher
* fix build
* slasher/detection tests
* rest of the tests
* misc tests
* tools tests
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge branch 'master' into apply-testutils-assertions-misc
* agg tests
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge branch 'master' into apply-testutils-assertions-misc
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* updates aggregated_test
* beacon-chain/operations/attestations/kv/* tests updated
* beacon-chain/operations/attestations tests updated
* beacon-chain/operations/slashings tests updated
* Merge branch 'master' into apply-testutils-assertions-misc
* gazelle
* beacon-chain/core tests updated
* fixes test
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* beacon-chain/rpc tests updated
* beacon-chain/sync/initial-sync tests
* misc tests
* optimizes error message parsing in testutils
* Merge branch 'assertutils-optimize-processing' into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* endtoend tests
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* gazelle
* Merge refs/heads/master into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* beacon-chain/blockchain tests updated
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* beacon-chain/state/stategen tests updated
* beacon-chain all left-over tests are done
* Merge refs/heads/master into apply-testutils-assertions-misc
* validator tests updated
* slasher tests
* Merge branch 'master' into apply-testutils-assertions-misc
* gofmt
* gazelle
* Merge refs/heads/master into apply-testutils-assertions-misc
* shared upd
* end2end tests deps fixed
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
* Merge refs/heads/master into apply-testutils-assertions-misc
* misc
* all tests are updated
* Merge branch 'apply-testutils-assertions-misc' of github.com:prysmaticlabs/prysm into apply-testutils-assertions-misc
2020-08-25 15:23:06 +00:00
Ivan Martinez
492944db01
E2E: Add cpu profile and fix logs and sync time (#6872)
* Add cpu profile for beacon node to e2e

* Fix

* Fix again

* Properly kill beacon nodes

* Fix logs

* Try 100%

* Handle cp profile from pprof

* Fix

* Bring expected participation to 98%

* Only enable pprof in non-local running

* Modify sync time

* Fix

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-05 09:58:50 -05:00
Ivan Martinez
37343930dd
Change E2E config to test deposits in PR (#6824)
* Change config to allow deposit in PR e2e

* Fix comment

* Fix

* Fix

* Fix

* Fix

* Fix

* Remove pid handling

* Fix build.bazel

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-08-01 12:22:53 -05:00
Ivan Martinez
b052ab7087
Make E2E less flaky (#6536) 2020-07-09 14:08:21 -05:00
Ivan Martinez
260847e92f
Fix long running E2E deposits testing (#6208)
* Fix long running e2e deposit processing
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-long-e2e
* Remove logs
* Uncomment
* typing
* Reorder
* Add const for node count
* Merge refs/heads/master into fix-long-e2e
* Add trace logging
* Merge branch 'fix-long-e2e' of github.com:prysmaticlabs/prysm into fix-long-e2e
2020-06-11 06:38:15 +00:00
Ivan Martinez
4e96cbeae7
E2E Improvements (#6091)
* Some fixes
* Merge branch 'master' into e2e-fixes
* Add another small delay
* Merge branch 'e2e-fixes' of github.com:prysmaticlabs/prysm into e2e-fixes
* Remove genesis test, make normal e2e run longer
* Gaz
* more fixes
* Merge branch 'master' into e2e-fixes
* Merge refs/heads/master into e2e-fixes
* Fix comment
* Merge refs/heads/master into e2e-fixes
2020-06-03 18:44:13 +00:00
Ivan Martinez
213f8ef088
Fix long-running E2E (#6047)
* Fixes for long running E2E
* Merge branch 'master' of github.com:prysmaticlabs/prysm into fix-e2e
* Move metrics check up
* Merge refs/heads/master into fix-e2e
2020-05-29 16:52:38 +00:00
Ivan Martinez
0b64db5e21
Add voluntary exit processing to E2E (#6016)
* Add voluntary exit to E2E

* Fix long urnning e2e

* Fix for comments

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-28 22:51:00 -04:00
Ivan Martinez
0f7cf212a2
Prepare E2E for v0.12 (#5991)
* Prepare e2e for v0.12

* Fix e2e

* Comments

* Attempt 100% participation

* Fixes

* Fix

* Update readme

Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>
2020-05-26 14:04:42 -05:00
Ivan Martinez
c82f2d5657
Fix test setup and change participation min back to 100% (#5791)
* Increase time for test to start

* Try anothre fix

* Set minimum participation back to 100%

* Change bnode count to 2

* Give extra wiggle room for metrics

* Remove comment
2020-05-09 15:11:10 -05:00
Ivan Martinez
84e51a5236
Add prometheus metrics test into E2E (#5673)
* Progress on metrics tests
* Progress on metrics test
* Get metrics E2E working
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-metrics
* Complete most of metrics tests
* Change E2E polling to the middle of a slot, instead of at the start of the middle
* Add metrics to all E2E
* Remove extra types
* Update endtoend/evaluators/metrics.go

Co-Authored-By: Preston Van Loon <preston@prysmaticlabs.com>
* Merge branch 'master' into e2e-metrics
* Add more comments, address feedback
* Merge branch 'e2e-metrics' of https://github.com/prysmaticlabs/prysm into e2e-metrics
* Fix build
* Remove unneeded comment
* Set E2E_EPOCHS back
* Improve sync testing reliability
* Remove metrics check from slashing
* Improve time allotted to sync
* Remove possibly flaky sync test change
2020-04-29 13:42:12 +00:00
Ivan Martinez
f70635a992
Add post-genesis deposit testing to long-running E2E (#5449)
* WIP add deposits
* Modify validator component
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-add-depsoits
* Fix e2e
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-add-depsoits
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-add-depsoits
* Start running with extra deposits
* Begin adding evluator for e2e deposit
* Get deposit E2E working
* Add more rigorous testing for deposits
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-add-depsoits
* Improve policy for deposits
* Fix build
* Remove sync testing for long running e2e
* Undo shard change
* Undo unneeded changes
* Adjust for comments
* Merge branch 'master' into e2e-add-depsoits
* Fix bug where long running E2E would always run
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-add-depsoits
* Merge branch 'e2e-add-depsoits' of https://github.com/0xKiwi/Prysm into e2e-add-depsoits
* Merge branch 'master' into e2e-add-depsoits
2020-04-27 15:59:42 +00:00
Ivan Martinez
c70103bdb3
Add healthz check to E2E (#5426)
* Add healthz check to E2E
* Log fixes
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-healthz
* Merge branch 'master' of https://github.com/prysmaticlabs/prysm into e2e-healthz
* Merge branch 'master' into e2e-healthz
* Merge branch 'master' into e2e-healthz
* Merge branch 'master' into e2e-healthz
* Merge branch 'master' into e2e-healthz
2020-04-15 21:20:58 +00:00
Preston Van Loon
f89d753275
Add configurable e2e epochs (#5235)
* Add configurable e2e epochs
* Merge refs/heads/master into configurable-test-epochs
* Merge refs/heads/master into configurable-test-epochs
2020-03-28 18:47:31 +00:00
Ivan Martinez
bc3d673ea4
Parallelize E2E Testing (#5168)
* Begin cleanup for E2E

* Parellize testing

* Add comments

* Add comment
2020-03-22 19:04:23 -04:00
Ivan Martinez
021d777b5e
Add Anti-Flake test for E2E (#5149)
* Add antiflake test

* Respond to comments

* Comment

* Change issue num
2020-03-21 14:42:51 +08:00
Ivan Martinez
8143cc36bc
Add Slasher to E2E (#5061)
* Start adding "inject slashing into pool"

* Attempt at slashing

* Remove unneded

* Fix

* Begin adding slasher client to e2e

* Start slasher in e2e

* Get slashing detection working

* Get slashing evaluators working

* Progress on e2e

* Cleanup e2e

* Fix slasher e2e!

* lint

* Comment

* Fixes

* Improve accuracy of balance check

* REmove extra

* Remove extra

* Make more accurate
2020-03-15 01:09:23 -04:00
Ivan Martinez
9149c2e4f4 Replace no-genesis-delay with custom-genesis-delay (#4678)
* Change NoGenesisDelay to CustomGenesisDelay
* Implement flag
* gazelle
* Fix docs
* Merge branch 'master' of https://github.com/prysmaticlabs/Prysm into custom-genesis-delay
* Fix
* Gazelle
* Add case to fix bad math
* Merge branch 'master' into custom-genesis-delay
2020-01-28 21:07:43 +00:00
Ivan Martinez
ad01bfbcde Add sync test to E2E (#4654)
* Complete evaluator for chain consensus

* Add sync e2e test

* Cleanup

* Rename

* Add tad more offset for correct head

* Change offset to middle of slot

* Change head block root to head epoch

* comment

* Fix eth1

* Address comments

* Gazelle

* Change to use file

* Change to use reader

* Use fil
2020-01-28 13:16:00 -06:00
terence tsao
eaf7ae3774
Part 1 of block chain service refactor - move process block (#4670) 2020-01-27 13:48:16 -08:00
Ivan Martinez
127f05d531
Allow easy plugin of featureflags into E2E (#4659)
* Enable easy plugin of featureflags into E2E

* Gazelle

* Fix text

* Fix whitespace
2020-01-26 21:42:10 -05:00
terence tsao
d4bea51482
Proto array fork choice tree handler (#4658) 2020-01-26 12:25:33 -08:00
Ivan Martinez
fa2acb3632 Improve E2E to be more consistent with timing, and allow for custom flags (#4620)
* Add committees helper, benchmark, results show 62ms for 8k validators which was previously 4 minutes

* Add regression test with same data

* fix epoch conversion

* lint

* undo and lint

* Begin work on adding mainnet config benchmark

* Try more to get mainnet e2e

* Try to fix delay

* Get past chainstart on e2e

* Try to fix flaky

* Get demo config working

* Remove unneeded changes

* Change how flags are enabled

* Lower shard count

* Temp skip

* Fix e2e

* Fix testing to run until last epoch

* Fix

* Add ending time log and remove att cache flag

* Fix ordering

* Reenable flag

* Change ports from default

* Add no log for if there are no err logs

* Add block evaluator

* Try to improve evaluators

* Progress on attestation evaluator

* Remove attestation evaluator

* Fix e2e

Co-authored-by: Preston Van Loon <preston@prysmaticlabs.com>
2020-01-25 15:39:56 +08:00
Raul Jordan
ed529965af Fix Up SSZ Cache Branch Recomputation (#4558)
* e2e ssz cache busting
* use ssz cache for e2e
* caching ensure
* fix up the cache even more
* gazelle
* formatting
* formatting
* add back cache for val registry
* sync
* fix up commented item
* add attestations
* Merge branch 'master' into e2e-ssz-cache
* Merge branch 'master' into e2e-ssz-cache
* Merge branch 'e2e-ssz-cache' of github.com:prysmaticlabs/prysm into e2e-ssz-cache
* formatting
* gaz
* Merge branch 'master' into e2e-ssz-cache
* resolve comments
* Merge branch 'master' into e2e-ssz-cache
* naming of test
* Merge refs/heads/master into e2e-ssz-cache
* Merge refs/heads/master into e2e-ssz-cache
2020-01-16 21:40:09 +00:00
Ivan Martinez
68edad13bc End To End Tests for Demo and Minimal config (#3932)
* Begin working on end to end tests using geth dev chain

* Start on beacon node set up

* More progress on bnode setup

* Complete flow until chainstart, begin work on evaluators

* More progress on evaluators

* Start changing bazel run to direct binary

* Move endtoend to inside beacon-chain

* use bazel provided geth, use bazel test

* tempdir

* use fork rules_go

* Change to use UUID dir and bazel binaries

* Truncate UUID a bit

* Get full run from chainstart to evaluating

* Rewrite to react to logs rather than arbitrarily wait

* Fix export

* Move evaluators to evaluators.go

* Add peer check test

* Add more comments

* Remove unneeded exports

* Check all nodes have the correct amount of peers

* Change name to onGenesisEpoch

* Remove extra wait times where not needed

* Cleanup

* Add log for beacon start

* Fix deposit amount

* Make room for eth1follow distnce

* Cleanup and fix minimal test

* Goimports

* Fix imports

* gazelle and minimal

* manual

* Fix for comments

* Make timing rely on reading logs, and cleanup

* Fix for comments

* Fix workspace

* Cleanup

* Fix visibility

* Cleanup and some comments

* Address comments

* Fix for v0.9

* Modify for v0.9

* Move to own package outside of beacon-chain

* Gazelle

* Polishing, logging

* Fix filenames

* Add more logs

* Add flag logging

* Cover for page not having libp2p info

* Improve multiAddr detection

* Add more logs

* Add missing flags

* Add log printing to defer

* Get multiAddr from logs

* Fix logging and detection

* Change evaluators to rely on EpochTimer

* Add evaluator for ValidatorParticipation

* Fix validator participation evaluator

* Cleanup, comments and fix participation calculation

* Cleanup

* Let the file searcher search for longer

* Change participation to check for full

* Log out file contents if text isnt found

* Split into different files

* Disable IPC and use RPC instead, change tmp dir to bazel dir

* Change visibility

* Gazelle

* Add e2e tag

* new line
2019-11-15 13:56:26 -05:00