lighthouse-pulse/testing/ef_tests/check_all_files_accessed.py
Eitan Seri-Levi e4d4e439cb
Add Capella & Deneb light client support (#4946)
* rebase and add comment

* conditional test

* test

* optimistic chould be working now

* finality should be working now

* try again

* try again

* clippy fix

* add lc bootstrap beacon api

* add lc optimistic/finality update to events

* fmt

* That error isn't occuring on my computer but I think this should fix it

* Merge branch 'unstable' into light_client_beacon_api_1

# Conflicts:
#	beacon_node/beacon_chain/src/events.rs
#	beacon_node/http_api/src/lib.rs
#	beacon_node/http_api/src/test_utils.rs
#	beacon_node/http_api/tests/main.rs
#	beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs
#	beacon_node/lighthouse_network/src/rpc/methods.rs
#	beacon_node/lighthouse_network/src/service/api_types.rs
#	beacon_node/network/src/beacon_processor/worker/rpc_methods.rs
#	beacon_node/tests/test.rs
#	common/eth2/src/types.rs
#	lighthouse/src/main.rs

* Add missing test file

* Update light client types to comply with Altair light client spec.

* Fix test compilation

* Merge branch 'unstable' into light_client_beacon_api_1

* Support deserializing light client structures for the Bellatrix fork

* Move `get_light_client_bootstrap` logic to `BeaconChain`. `LightClientBootstrap` API to return `ForkVersionedResponse`.

* Misc fixes.
- log cleanup
- move http_api config mutation to `config::get_config` for consistency
- fix light client API responses

* Add light client bootstrap API test and fix existing ones.

* Merge branch 'unstable' into light_client_beacon_api_1

* Fix test for `light-client-server` http api config.

* Appease clippy

* Add Altair light client SSZ tests

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into light_client_beacon_api_1

* updates to light client header

* light client header from signed beacon block

* using options

* implement helper functions

* placeholder conversion from vec hash256 to exec branch

* add deneb

* using fixed vector

* remove unwraps

* by epoch

* compute merkle proof

* merkle proof

* update comments

* resolve merge conflicts

* linting

* Merge branch 'unstable' into light-client-ssz-tests

# Conflicts:
#	beacon_node/beacon_chain/src/beacon_chain.rs
#	consensus/types/src/light_client_bootstrap.rs
#	consensus/types/src/light_client_header.rs

* superstruct attempt

* superstruct changes

* lint

* altair

* update

* update

* changes to light_client_optimistic_ and finality

* merge unstable

* refactor

* resolved merge conflicts

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types

* block_to_light_client_header fork aware

* fmt

* comment fix

* comment fix

* include merge fork, update deserialize_by_fork, refactor

* fmt

* pass by ref to prevent clone

* rename merkle proof fn

* add FIXME

* LightClientHeader TestRandom

* fix comments

* fork version deserialize

* merge unstable

* move fn arguments, fork name calc

* use task executor

* remove unneeded fns

* remove dead code

* add manual ssz decoding/encoding and add ssz_tests_by_fork macro

* merge deneb types with tests

* merge ssz tests, revert code deletion, cleanup

* move chainspec

* update ssz tests

* fmt

* light client ssz tests

* change to superstruct

* changes from feedback

* linting

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types

* test fix

* cleanup

* Remove unused `derive`.

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types

* beta compiler fix

* merge
2024-03-25 11:01:56 +00:00

87 lines
3.0 KiB
Python
Executable File

#!/usr/bin/env python3
# The purpose of this script is to compare a list of file names that were accessed during testing
# against all the file names in the consensus-spec-tests repository. It then checks to see which files
# were not accessed and returns an error if any non-intentionally-ignored files are detected.
#
# The ultimate goal is to detect any accidentally-missed spec tests.
import os
import re
import sys
# First argument should the path to a file which contains a list of accessed file names.
accessed_files_filename = sys.argv[1]
# Second argument should be the path to the consensus-spec-tests directory.
tests_dir_filename = sys.argv[2]
# If any of the file names found in the consensus-spec-tests directory *starts with* one of the
# following regular expressions, we will assume they are to be ignored (i.e., we are purposefully
# *not* running the spec tests).
excluded_paths = [
# Eth1Block and PowBlock
#
# Intentionally omitted, as per https://github.com/sigp/lighthouse/issues/1835
"tests/.*/.*/ssz_static/Eth1Block/",
"tests/.*/.*/ssz_static/PowBlock/",
# light_client
"tests/.*/.*/light_client",
# LightClientStore
"tests/.*/.*/ssz_static/LightClientStore",
# LightClientSnapshot
"tests/.*/.*/ssz_static/LightClientSnapshot",
# One of the EF researchers likes to pack the tarballs on a Mac
".*\.DS_Store.*",
# More Mac weirdness.
"tests/mainnet/bellatrix/operations/deposit/pyspec_tests/deposit_with_previous_fork_version__valid_ineffective/._meta.yaml",
# bls tests are moved to bls12-381-tests directory
"tests/general/phase0/bls",
# some bls tests are not included now
"bls12-381-tests/deserialization_G1",
"bls12-381-tests/deserialization_G2",
"bls12-381-tests/hash_to_G2",
"tests/.*/eip6110",
"tests/.*/whisk"
]
def normalize_path(path):
return path.split("consensus-spec-tests/")[1]
# Determine the list of filenames which were accessed during tests.
passed = set()
for line in open(accessed_files_filename, 'r').readlines():
file = normalize_path(line.strip().strip('"'))
passed.add(file)
missed = set()
accessed_files = 0
excluded_files = 0
# Iterate all files in the tests directory, ensure that all files were either accessed
# or intentionally missed.
for root, dirs, files in os.walk(tests_dir_filename):
for name in files:
name = normalize_path(os.path.join(root, name))
if name not in passed:
excluded = False
for excluded_path_regex in excluded_paths:
if re.match(excluded_path_regex, name):
excluded = True
break
if excluded:
excluded_files += 1
else:
print(name)
missed.add(name)
else:
accessed_files += 1
# Exit with an error if there were any files missed.
assert len(missed) == 0, "{} missed files".format(len(missed))
print("Accessed {} files ({} intentionally excluded)".format(
accessed_files, excluded_files))