prysm-pulse/testing/endtoend
Preston Van Loon ecad5bbffc
e2e: Provide e2e config yaml to web3signer (#10123)
* e2e: Provide e2e config yaml to web3signer

* fix build for //testing/endtoend:go_default_test

* Update with web3signer with teku fixes

* buildifier

* Add slasher case for web3signer

* Update testing/endtoend/minimal_e2e_test.go

* Update web3signer to 21.10.6

* Revert "Update web3signer to 21.10.6"

This reverts commit bdf3c408f2a0432379fa1a735a2e3f70c62d11cc.

* Remove slasher part of web3signer e2e tests

* Revert "Remove slasher part of web3signer e2e tests"

This reverts commit 24249802ae34fb7727b83dbfa187d73c1d97d2f8.

* fix slasher web3signer test

* fixing build

* updating yaml to match testnet_e2e_config.go

* trying a different order to the e2e test and adding a log

* trying different way to kill process

* handling unhandled error

* testing changes to config WIP

* fixing bazel WIP

* fixing build

* ignoring test for now to test

* fixing bazel

* Test only web3signer e2e

* rolling back some commits to test

* fixing bazel

* trying an updated web3signer version

* changing flag to match version

* trying current version of develop for web3signer

* testing not using the --network property

* addressing build error

* testing config change

* reverting to go bakc to using the network file

* testing adding epochs per sync committee period

* rolling back configs

* removing check to test

* adding log to get sync committee duties and changing bellatrix fork epoch to something large and altair to epoch 1

* fixing bazel

* updating epoch in config file

* fixing more descrepencies between the configurations

* removing un used yaml

* removing goland added duplicates

* reverting using network minimal

* fixing bug

* rolling back some changes

* rolling back some changes

* rolling back changes

* making sure web3signer test doesn't make it to bellatrix fork yet

* reverting changes I did not touch

* undo comment

* Update testing/endtoend/deps.bzl

* Apply suggestions from code review

* rm nl

* fix //testing/endtoend:go_mainnet_test

* Remove unnecessary dep

* fix //testing/endtoend:go_mainnet_test

* addressing review comments

* fixing build and internal conflict

* removing web3signer slasher test as it's unneeded due to the interface nature of key signing, the regular slashing test is enough

* fix: The validator we fetch from the binary can only run before altair, if you add that and the web3signer then these things can never run together as the web3signer sets it to before bellatrix

Co-authored-by: james-prysm <90280386+james-prysm@users.noreply.github.com>
Co-authored-by: James He <james@prysmaticlabs.com>
Co-authored-by: Nishant Das <nishdas93@gmail.com>
Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>
2022-04-12 16:57:46 +00:00
..
components e2e: Provide e2e config yaml to web3signer (#10123) 2022-04-12 16:57:46 +00:00
evaluators Bellatrix evaluators (#10506) 2022-04-11 13:45:22 +00:00
helpers Refactor E2E port registration (#10306) 2022-03-04 09:26:28 +00:00
params Bellatrix evaluators (#10506) 2022-04-11 13:45:22 +00:00
policies Move EndtoEnd Tests to Testing/ Folder (#9586) 2021-09-15 14:42:05 +00:00
static-files/eth1 Enable Bellatrix E2E (#10437) 2022-04-05 14:02:46 +00:00
types Enable Bellatrix E2E (#10437) 2022-04-05 14:02:46 +00:00
BUILD.bazel e2e: Provide e2e config yaml to web3signer (#10123) 2022-04-12 16:57:46 +00:00
deps.bzl e2e: Provide e2e config yaml to web3signer (#10123) 2022-04-12 16:57:46 +00:00
endtoend_test.go e2e: Provide e2e config yaml to web3signer (#10123) 2022-04-12 16:57:46 +00:00
geth_deps.go Move EndtoEnd Tests to Testing/ Folder (#9586) 2021-09-15 14:42:05 +00:00
lighthouse.BUILD Multiclient E2E With Lighthouse (#10020) 2022-02-02 19:13:52 +00:00
mainnet_e2e_test.go Bellatrix evaluators (#10506) 2022-04-11 13:45:22 +00:00
minimal_e2e_test.go e2e: Provide e2e config yaml to web3signer (#10123) 2022-04-12 16:57:46 +00:00
minimal_slashing_e2e_test.go control test (#10323) 2022-03-08 07:58:19 -08:00
README.md e2e: Add web3signer component (#10088) 2022-01-19 16:22:23 +00:00
slasher_simulator_e2e_test.go Refactor mock test helpers (#10133) 2022-01-26 14:48:20 +00:00
web3signer.BUILD e2e: Add web3signer component (#10088) 2022-01-19 16:22:23 +00:00

End-to-end Testing Package

This is the main project folder of the end-to-end testing suite for Prysm. This performs a full end-to-end test for Prysm, including spinning up an ETH1 dev chain, sending deposits to the deposit contract, and making sure the beacon node and its validators are running and performing properly for a few epochs. It also performs a test on a syncing node, and supports featureflags to allow easy E2E testing of experimental features.

How it works

Through the end2EndConfig struct, you can declare several options such as how many epochs the test should run for, and what BeaconConfig the test should use. You can also declare how many beacon nodes and validator clients are run, the E2E will automatically divide the validators evently among the beacon nodes.

In order to "evaluate" the state of the beacon chain while the E2E is running, there are Evaluators that use the beacon chain node API to determine if the network is performing as it should. This can evaluate for conditions like validator activation, finalization, validator participation and more.

Evaluators have 3 parts, the name for it's test name, a policy which declares which epoch(s) the evaluator should run, and then the evaluation which uses the beacon chain API to determine if the beacon chain passes certain conditions like finality.

Current end-to-end tests

  • Minimal Config - 2 beacon nodes, 256 validators, running for 8 epochs
  • Minimal Config Slashing Test - 2 beacon nodes, 256 validators, tests attester and proposer slashing

Instructions

Note: Java 11 or greater is required to run web3signer.

If you wish to run all the minimal spec E2E tests, you can run them through bazel with:

bazel test //testing/endtoend:go_default_test --test_output=streamed