73443208a1
* Remove native state flag and use native state in spectests * remove feature from tests * use e2e config in slasher simulator * use params.BeaconConfig in testutil * use correct function * use minimal config in go_test * fix TestListValidators * parameterize sync committee bits and aggregation bits * Fix TestServer_ListIndexedAttestations_GenesisEpoch (cherry picked from commit 254ab623dde08ae8886b152facdbbd8889ed79db) * fix more tests * fix even more * moreeee * aaaand more * one more fix * one more * simplify TestGetAltairDuties_UnknownPubkey * comment out problematic test * one more fix * one more * aaaand one more * another * use fieldparams in HydrateBlindedBeaconBlockBodyBellatrix * create new package for mainnet tests * TestServer_GetBellatrixBeaconBlock * change slashed validator index * clear cache in reward_test.go * deprecate flag * create bazel mainnet target * move attester mainnet test to mainnet target * "fix" proposer tests * use minimal config in TestServer_circuitBreakBuilder * fix TestProposer_ProposeBlock_OK * more fixes in validator package * more fixes * more fixes * test code * move TestProposer_GetBeaconBlock_BellatrixEpoch to minimal * finally * remove proposer_bellatrix_mainnet_test.go * fix TestServer_GetBellatrixBeaconBlock_HappyCase * fix TestServer_GetBellatrixBeaconBlock_BuilderCase * Preston needs to fix this! * Revert "Preston needs to fix this!" This reverts commit b03d97a16e3080e254c7b19d7f193d3c600ca869. * remove proto state tests * fix migration tests * static analysis fix * review * remove proto state * swap state in tests * fix BUILD file in /proto/testing * remove metrics test with nil state |
||
|---|---|---|
| .. | ||
| benchmark_files | ||
| BUILD.bazel | ||
| pregen_test.go | ||
| pregen.go | ||
| README.md | ||
Benchmarks for Prysm State Transition
This package contains the functionality needed for benchmarking Prysm state transitions, this includes its block processing (with and without caching) and epoch processing functions. There is also a benchmark for HashTreeRoot on a large beacon state.
Benchmark Configuration
The following configs are in config.go:
ValidatorCount: Sets the amount of active validators to perform the benchmarks with. Default is 16384.AttestationsPerEpoch: Sets the amount of attestations per epoch for the benchmarks to perform with, this affects the amount of attestations in a full block and the amount of attestations per epoch in the state for theProcessEpochandHashTreeRootbenchmark. Default is 128.
Generating new SSZ files
Due to the sheer size of the benchmarking configurations (16384 validators), the files used for benchmarking are pregenerated so there's no wasted computations on generating a genesis state with 16384 validators. This should only be needed if there is a breaking spec change and the tests fail from SSZ issues.
To generate new files to use for benchmarking, run the below command in the root of Prysm.
bazel run //tools/benchmark-files-gen -- --output-dir $PRYSMPATH/testing/benchmark/benchmark_files/ --overwrite
Running the benchmarks
To run the ExecuteStateTransition benchmark:
bazel test //beacon-chain/core/state:go_default_test --test_filter=BenchmarkExecuteStateTransition_FullBlock --test_arg=-test.bench=BenchmarkExecuteStateTransition_FullBlock
To run the ExecuteStateTransition (with cache) benchmark:
bazel test //beacon-chain/core/state:go_default_test --test_filter=BenchmarkExecuteStateTransition_WithCache --test_arg=-test.bench=BenchmarkExecuteStateTransition_WithCache
To run the ProcessEpoch benchmark:
bazel test //beacon-chain/core/state:go_default_test --test_filter=BenchmarkProcessEpoch_2FullEpochs --test_arg=-test.bench=BenchmarkProcessEpoch_2FullEpochs
To run the HashTreeRoot benchmark:
bazel test //beacon-chain/core/state:go_default_test --test_filter=BenchmarkHashTreeRoot_FullState --test_arg=-test.bench=BenchmarkHashTreeRoot_FullState
To run the HashTreeRootState benchmark:
bazel test //beacon-chain/core/state:go_default_test --test_filter=BenchmarkHashTreeRootState_FullState --test_arg=-test.bench=BenchmarkHashTreeRootState_FullState
Extra flags needed to benchmark properly:
--nocache_test_results --test_arg=-test.v --test_timeout=2000 --test_arg=-test.cpuprofile=/tmp/cpu.profile --test_arg=-test.memprofile=/tmp/mem.profile --test_output=streamed
Current Results as of January 2020
BenchmarkExecuteStateTransition_FullBlock-4 20 2031438030 ns/op
BenchmarkExecuteStateTransition_WithCache-4 20 1857290454 ns/op
BenchmarkHashTreeRoot_FullState-4 50 297655834 ns/op
BenchmarkHashTreeRootState_FullState-4 50 155535883 ns/op