Fix the budget gate and add CI, lint and coverage guards
- check_budget.py escalated a hard index-row violation to 'warn' on both branches, so INDEX_ROW_HARD could never fail anything. It fails now, and tests/unit/test_budget_measurement.py pins both thresholds. - The corpus note counted scripts/__pycache__/*.pyc, so the same commit reported 27 scripts after the suite had run and 16 on a clean checkout. - TOKENS_PER_CHAR is re-derived from a measurement (26,795 chars -> 6,065 tokens = 4.418); the comment's old figure and its arithmetic were both wrong for this file. - check_repo.py's yaml sentinel carries the type ignore, so mypy is clean with PyYAML installed as well as without it. - The CPUID test now skips with the probe's own note on a host without CPUID instead of failing, and pytest.ini points at tests/README.md, which exists, instead of an EXTENSION file that never did. Tooling: .flake8 (a measured 120-column limit), .gitattributes, .gitignore and .github/workflows/ci.yml (the five gates, the suite on 3.11-3.14 and a lint job), which is what CONTRIBUTING.md already claimed CI runs. The pull-request template it refers to now exists. Content: pitfalls P26-P34 -- the analysis-level failures the catalogue was missing (sampling read as observation, a self-built parser trusted, a verification that could not have failed, a shell-mangled command line) -- registered in the symptom index and its routing mirror. FINDINGS F11-F15 record the defects above. Tests: the coverage guard derives its subject from scripts/ rather than a hand-written list, pe_triage.py and pe_diff.py gain their first tests, and plan.py / preflight.py are declared uncovered with reasons.
This commit is contained in:
+17
-8
@@ -78,9 +78,10 @@ NAV_LINES = 100 # above this, a reference must open navigably
|
||||
CORPUS_FILES_NOTE = 110 # references + scripts, reported as a note past this
|
||||
CORPUS_LINES_NOTE = 26000 # total lines across the same corpus, same treatment
|
||||
# Characters per token, for the no-tiktoken fallback. **Calibrated on SKILL.md against
|
||||
# tiktoken/cl100k_base** (47,629 chars -> 11,114 tokens = 4.285), so both measurement paths reach
|
||||
# the same verdict. Getting this wrong made the gate answer differently on CI than locally.
|
||||
TOKENS_PER_CHAR = 4.285
|
||||
# tiktoken/cl100k_base** (26,795 chars -> 6,065 tokens = 4.418, re-measured 2026-10-24), so both
|
||||
# measurement paths reach the same verdict. Getting this wrong made the gate answer differently on
|
||||
# CI than locally.
|
||||
TOKENS_PER_CHAR = 4.418
|
||||
# A navigable head: one of these must appear in the first NAV_HEAD_LINES lines.
|
||||
NAV_HEAD_LINES = 40
|
||||
HEDGE_RE = re.compile(
|
||||
@@ -145,9 +146,10 @@ def token_estimate(text):
|
||||
everywhere in this repository, including in this file. When it is not, it falls back to
|
||||
`characters / TOKENS_PER_CHAR`.
|
||||
|
||||
The fallback constant is **calibrated against that measurement** rather than guessed: a
|
||||
markdown-heavy document tokenizes worse per character than the usual /4 rule of thumb, and this
|
||||
file measured 0.30 characters-to-tokens on SKILL.md (67,011 chars -> 19,929 tokens).
|
||||
The fallback constant is **calibrated against that measurement** rather than guessed: this
|
||||
SKILL.md measures 4.418 characters per token (26,795 chars -> 6,065 tokens, cl100k_base). Agreeing
|
||||
with the "about four" rule of thumb to within a few percent is not close enough to rely on, so
|
||||
the number is re-derived when the document changes rather than assumed.
|
||||
|
||||
Why the value matters more than the method: the first version used 3.6 and the gate therefore
|
||||
reached *different verdicts on different machines* -- `WARN over budget` where tiktoken was
|
||||
@@ -198,7 +200,11 @@ def check_index_rows(lines):
|
||||
out = []
|
||||
for section, key, size in long_rows:
|
||||
hard = size > INDEX_ROW_HARD
|
||||
out.append(('warn' if hard else 'warn',
|
||||
# `hard` must actually escalate. This read `'warn' if hard else 'warn'` -- both branches the
|
||||
# same -- so INDEX_ROW_HARD was defined, computed and never able to fail anything: an index
|
||||
# row could grow without limit and CI stayed green. Found by injecting an over-long row and
|
||||
# asking the gate to fail, which is the only way this class of defect shows up.
|
||||
out.append(('fail' if hard else 'warn',
|
||||
'%s: row for %s is %d chars (soft limit %d)%s' %
|
||||
(section, key, size, INDEX_ROW_SOFT,
|
||||
' -- that is a paragraph, not an index entry' if hard else '')))
|
||||
@@ -305,8 +311,11 @@ def check_corpus():
|
||||
green always-loaded count.
|
||||
"""
|
||||
refs = sorted(glob.glob(os.path.join(REFS, '**', '*.md'), recursive=True))
|
||||
# A bytecode cache is not corpus. It is untracked, it appears the first time the suite runs, and
|
||||
# counting it made this note report 27 scripts where the directory ships 16 -- a measurement that
|
||||
# changes when a *reader* runs a test is not a measurement of the repository.
|
||||
scripts = sorted(p for p in glob.glob(os.path.join(SKILL_DIR, 'scripts', '**', '*'), recursive=True)
|
||||
if os.path.isfile(p))
|
||||
if os.path.isfile(p) and '__pycache__' not in p.split(os.sep))
|
||||
|
||||
def count(paths):
|
||||
total = 0
|
||||
|
||||
Reference in New Issue
Block a user