Add smoke tests

- extend `pytest` with an option to run only the minimum number
  of (unit) test cases to just keep the coverage at 100%
- rationale:
  + many of the unit test cases partly overlap with
    respect to the lines of source code executed
  + also, integration tests, by definition, do not
    contribute to a higher test coverage
- implementation: mark "redundant" test cases as one of:
  + `pytest.mark.integration_test`
    => code usage from the perspective of the end user
  + `pytest.mark.overlapping_test`
    => tests not contributing to the 100% coverage
  + `pytest.mark.sanity_test`
    => tests providing confidence in the test data
- add `tests.conftest` module
  => programatically convert the above markers into
     `@pytest.mark.no_cover` and collect the non-"redundant" tests
- add nox session "test-fast" to run only the minimum
  number of (unit) test while holding coverage at 100%
- refactor some test modules
  + wrap some test cases in a class
  + move sanity tests to the end of the files
This commit is contained in:
Alexander Hess 2024-10-15 01:49:32 +02:00
commit 7e3e67c300
Signed by: alexander
GPG key ID: 344EA5AB10D868E0
11 changed files with 182 additions and 35 deletions

View file

@ -254,6 +254,14 @@ def test(session: nox.Session) -> None:
args = posargs or (
"--cov",
"--no-cov-on-fail",
*( # If this function is run via the "test-fast" session,
( # the following arguments are added:
"--cov-fail-under=100",
"--smoke-tests-only",
)
if session.env.get("_smoke_tests_only")
else ()
),
TEST_RANDOM_SEED,
TESTS_LOCATION,
)
@ -339,6 +347,18 @@ def test_docstrings(session: nox.Session) -> None:
session.run("xdoctest", "src/lalib")
@nox.session(name="test-fast", python=MAIN_PYTHON)
def test_fast(session: nox.Session) -> None:
"""Test code with `pytest`, selected (smoke) tests only.
The (unit) test cases are selected such that their number
is minimal but the achieved coverage remains at 100%.
"""
# See implementation notes in `pre_commit_test_hook()` below
session.env["_smoke_tests_only"] = "true"
test(session)
@nox.session(name="clean-cwd", python=MAIN_PYTHON, venv_backend="none")
def clean_cwd(session: nox.Session) -> None:
"""Remove (almost) all glob patterns listed in git's ignore file.