Skip to main content

Transformation: Python

Open Repo Download ZIP

git clone https://github.com/AEEF-AI/aeef-transform.git

The Transformation tier extends the Quick Start Python setup with a full 7-stage CI pipeline, mutation testing via mutmut, strict type checking with mypy, and provenance generation. This guide covers every addition.

Full Pipeline Walkthrough​

The CI pipeline runs seven stages:

ruff-check --> ruff-format --> mypy --> pytest-cov --> mutmut --> SAST --> SCA+license

Stage 1: Ruff Lint​

- name: Lint
  run: ruff check .

Ruff checks for code quality issues, import ordering, naming conventions, and security anti-patterns. Rules are configured in pyproject.toml under [tool.ruff].

Stage 2: Ruff Format​

- name: Format Check
  run: ruff format --check .

Verifies code formatting consistency. Fails if any file would be reformatted.

Stage 3: Type Check (mypy)​

- name: Type Check
  run: mypy app/

mypy enforces strict type checking. The configuration requires type annotations on all function signatures:

# mypy.ini
[mypy]
python_version = 3.12
strict = true
warn_return_any = true
warn_unused_configs = true
disallow_untyped_defs = true
disallow_incomplete_defs = true

Stage 4: Test with Coverage​

- name: Test
  run: pytest

pytest runs with the pytest-cov plugin enforcing an 80% coverage floor. Coverage options are configured in pyproject.toml under [tool.pytest.ini_options].

Stage 5: Mutation Testing (mutmut)​

- name: Mutation Testing
  run: |
    mutmut run --no-progress || true
    mutmut results

mutmut introduces mutations to source code and checks that tests detect them. A custom script validates the mutation score meets the 70% threshold.

The check_mutation_score.py script:

import argparse
import subprocess
import json

def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--threshold", type=float, default=70.0)
    args = parser.parse_args()

    result = subprocess.run(
        ["mutmut", "results", "--json"],
        capture_output=True, text=True, check=True,
    )
    data = json.loads(result.stdout)
    killed = data["killed"]
    total = data["total"]
    score = (killed / total) * 100 if total > 0 else 0

    if score < args.threshold:
        raise SystemExit(
            f"Mutation score {score:.1f}% below threshold {args.threshold}%"
        )
    print(f"Mutation score: {score:.1f}% (threshold: {args.threshold}%)")

if __name__ == "__main__":
    main()

Stage 6: SAST (Semgrep)​

- name: SAST
  run: |
    semgrep --config .semgrep/ --config p/python app/
    bandit -r app/ -c pyproject.toml

Runs Semgrep with AEEF custom rules and the community Python ruleset.

Stage 7: SCA and License Check​

- name: SCA
  run: |
    pip-audit --strict --desc

Checks for known vulnerabilities and verifies dependency licenses.

Ruff Configuration​

The Transformation tier Ruff config adds stricter rules:

# In pyproject.toml under [tool.ruff]
[tool.ruff]
target-version = "py312"
line-length = 100

[tool.ruff.lint]
select = [
    "E",    # pycodestyle errors
    "W",    # pycodestyle warnings
    "F",    # pyflakes
    "I",    # isort
    "N",    # pep8-naming
    "UP",   # pyupgrade
    "B",    # flake8-bugbear
    "S",    # flake8-bandit
    "A",    # flake8-builtins
    "C4",   # flake8-comprehensions
    "DTZ",  # flake8-datetimez
    "T20",  # flake8-print
    "SIM",  # flake8-simplify
    "TCH",  # flake8-type-checking
    "RUF",  # ruff-specific
]
ignore = ["S101"]  # allow assert in tests

[tool.ruff.lint.per-file-ignores]
"tests/**" = ["S101", "S106"]

pytest-cov Thresholds​

Coverage thresholds are configured in pyproject.toml:

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "--cov=app --cov-report=term-missing --cov-report=html --cov-report=json --cov-fail-under=80"

[tool.coverage.run]
branch = true
source = ["app"]

[tool.coverage.report]
exclude_lines = [
    "pragma: no cover",
    "if TYPE_CHECKING:",
    "if __name__ == .__main__.:",
]

Provenance Generation​

The CI pipeline generates a provenance record after all stages complete. A post-job step collects stage outcomes and writes a JSON provenance file validated against the AEEF schema:

- name: Generate Provenance
  if: always()
  run: python scripts/generate_provenance.py

Next Steps​

Last updated on by BalrogEg