- check_budget.py escalated a hard index-row violation to 'warn' on both branches, so INDEX_ROW_HARD could never fail anything. It fails now, and tests/unit/test_budget_measurement.py pins both thresholds. - The corpus note counted scripts/__pycache__/*.pyc, so the same commit reported 27 scripts after the suite had run and 16 on a clean checkout. - TOKENS_PER_CHAR is re-derived from a measurement (26,795 chars -> 6,065 tokens = 4.418); the comment's old figure and its arithmetic were both wrong for this file. - check_repo.py's yaml sentinel carries the type ignore, so mypy is clean with PyYAML installed as well as without it. - The CPUID test now skips with the probe's own note on a host without CPUID instead of failing, and pytest.ini points at tests/README.md, which exists, instead of an EXTENSION file that never did. Tooling: .flake8 (a measured 120-column limit), .gitattributes, .gitignore and .github/workflows/ci.yml (the five gates, the suite on 3.11-3.14 and a lint job), which is what CONTRIBUTING.md already claimed CI runs. The pull-request template it refers to now exists. Content: pitfalls P26-P34 -- the analysis-level failures the catalogue was missing (sampling read as observation, a self-built parser trusted, a verification that could not have failed, a shell-mangled command line) -- registered in the symptom index and its routing mirror. FINDINGS F11-F15 record the defects above. Tests: the coverage guard derives its subject from scripts/ rather than a hand-written list, pe_triage.py and pe_diff.py gain their first tests, and plan.py / preflight.py are declared uncovered with reasons.
126 lines
5.3 KiB
YAML
126 lines
5.3 KiB
YAML
# CI for the exe-reverse skill repository.
|
|
#
|
|
# WHAT THIS RUNS
|
|
# * the repository's own structural gates: check_repo.py, check_refs.py, check_budget.py,
|
|
# check_routing.py, and check_commands.py
|
|
# * the pytest suite: tests/unit, tests/cli, tests/integration
|
|
# * lint: mypy and flake8 over the gate scripts and the shipped skill scripts
|
|
#
|
|
# WHAT THIS DOES NOT RUN, ON PURPOSE
|
|
# * anything needing a debugger, a driver, a kernel component, a live target process, a hypervisor,
|
|
# test signing or a network peer. Those routes are recorded experiments in tests/benchmark.md,
|
|
# and what the suite deliberately does not cover is stated in tests/README.md.
|
|
# * anything about agent behaviour. Whether a model routes, classifies or refuses correctly is not
|
|
# observable from a unit test; skills/exe-reverse/evals/evals.json is the skeleton for that, and
|
|
# it has never been executed.
|
|
# * the hardware-observation routes. They need particular silicon and privilege; hwprobe.py
|
|
# reports what a runner lacks and the suite skips with that reason instead of failing.
|
|
#
|
|
# The runner is Linux while the skill targets Windows. That is deliberate: the suite observes the
|
|
# scripts' behaviour -- parsing, patching, gate logic, exit codes -- and every check that needs a
|
|
# Windows fact asks the probe or skips. A green run here is a statement about the *code*, never
|
|
# about a route that needs a Windows host.
|
|
#
|
|
# Coverage is therefore: static structure, documentation anchors, budget, documented commands,
|
|
# routing coherence, pure parsers, CLI contracts, fixture construction, typing/style lint and
|
|
# no-debugger flows. Nothing more is claimed.
|
|
name: ci
|
|
|
|
on:
|
|
push:
|
|
pull_request:
|
|
#
|
|
# NOTE ON TRIGGERS: no `paths:` filter, deliberately. A filter that watches only `*.py` and the lint
|
|
# configuration would skip the structural gates on exactly the changes they exist to catch -- a
|
|
# reference index or a documented command edited without touching a script. check_refs.py and
|
|
# check_routing.py are documentation gates, so they have to run on documentation commits.
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
test:
|
|
name: python ${{ matrix.python-version }}
|
|
runs-on: ubuntu-latest
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
python-version: ["3.11", "3.12", "3.13", "3.14"]
|
|
|
|
env:
|
|
PYTHONIOENCODING: utf-8
|
|
PYTHONDONTWRITEBYTECODE: "1"
|
|
# The gates spawn sibling scripts and decode their output. On a Windows runner the default
|
|
# codec is the console's (GBK here), and one script's --help text is not GBK-decodable, which
|
|
# is reported as a crash in that script rather than as a harness decoding problem. Pinning
|
|
# UTF-8 removes the whole class of failure on every platform.
|
|
PYTHONUTF8: "1"
|
|
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
|
|
- uses: actions/setup-python@v7
|
|
with:
|
|
python-version: ${{ matrix.python-version }}
|
|
|
|
- name: Install test dependencies
|
|
# `tiktoken` is optional for the suite but not for the budget gate's *measurement*: without
|
|
# it, check_budget.py falls back to characters/constant, and the two paths must agree.
|
|
# Installing it here means CI exercises the real tokenizer path, and the unit tests
|
|
# comparing both paths run instead of skipping.
|
|
# Not upgrading pip itself: it is not a goal here, and a pip self-upgrade failing would be
|
|
# reported as this repository's regression.
|
|
run: python -m pip install --upgrade pytest tiktoken
|
|
|
|
- name: Structural gate - repository layout
|
|
run: python check_repo.py
|
|
|
|
- name: Structural gate - documentation anchors
|
|
run: python check_refs.py
|
|
|
|
- name: Structural gate - context budget
|
|
run: python check_budget.py
|
|
|
|
- name: Structural gate - documented commands
|
|
run: python check_commands.py
|
|
|
|
- name: Structural gate - routing inventory matches the entry point
|
|
run: python check_routing.py
|
|
|
|
- name: Regression suite
|
|
run: python -m pytest -q
|
|
|
|
- name: Summarise skipped coverage
|
|
if: always()
|
|
run: python -m pytest -q -ra --no-header -p no:cacheprovider 2>&1 | tail -n 40
|
|
|
|
lint:
|
|
# Static checks over the same tracked surface the local commands cover, so what passes locally
|
|
# cannot drift from what CI enforces. Both tools read their configuration (mypy.ini, .flake8)
|
|
# from the repository root automatically.
|
|
#
|
|
# The environment is deliberately bare -- only mypy and flake8, none of PyYAML, capstone,
|
|
# tiktoken or pyelftools. That is the poorest environment the configuration must keep working
|
|
# in, which is what mypy.ini's ignore_missing_imports overrides exist for: CI exercises the
|
|
# fallbacks instead of a machine-local install hiding a regression.
|
|
#
|
|
# One interpreter, not the matrix: the checks are static, and the matrix's job is runtime
|
|
# behaviour, which lint does not observe.
|
|
name: lint (mypy + flake8)
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: actions/checkout@v7
|
|
|
|
- uses: actions/setup-python@v7
|
|
with:
|
|
python-version: "3.12"
|
|
|
|
- name: Install lint dependencies
|
|
run: python -m pip install --upgrade mypy flake8
|
|
|
|
- name: mypy
|
|
run: mypy *.py skills/exe-reverse/scripts/*.py
|
|
|
|
- name: flake8
|
|
run: flake8 *.py skills/exe-reverse/scripts/*.py
|