exe-reverse/docs/extending.md

Extending beyond a file

What this answers: how to change this repository's own machinery, rather than add another document to it: a new class of artifact, a reference family, the frontmatter or the metadata contract, the budget that governs the always-loaded file, a new gate and its CI wiring, and the procedure for demoting a claim when a measurement disagrees with it.

When to load this: when the change you want is not covered by the extension points in CONTRIBUTING.md, or when it changes what a green build means.

Everything here rests on one distinction. Part of this repository's contract is enforced by scripts at the root; the rest can only be judged by a reader. A change that moves a rule from the second half into the first is a real improvement, and a change that looks machine-checked but is not is how a guarantee rots. Where a class is maintained by a reader, this document says so.

Ordinary, or a discussion first

Change	Status	Why
A reference, a script, a pitfall entry, a case precedent, a measurement record, a capability row	Ordinary. Follow CONTRIBUTING.md	Each has a registration point and a gate that notices if the registration is missing
A new gate script, additive and read-only	Ordinary, if it cannot fail a change that used to pass	A gate that only adds a signal is cheap to review
Wiring an existing gate into CI	Ordinary	The sequence is in .github/workflows/ci.yml, and a missing gate is a visible gap
Recording a downgrade of a claim	Ordinary, and expected	The repository states that a measurement which disagrees re-opens the conclusion
A reference family (a subdirectory with its own index)	Ordinary, with one condition stated below	Its files are not resolved by any gate, so its index is the only entry point that exists
A whole new artifact class (a new tree, or a new format the tooling reads)	Discussion first	It changes what a reader must check by hand, and possibly what an installed skill ships
Adding a key to the metadata frontmatter block	Ordinary	No gate resolves those paths, so nothing depends on the key existing
Changing what an existing metadata key means, or the strength-label vocabulary	Discussion first	Every document in the corpus grades against that vocabulary
Changing a budget threshold, or the method that measures it	Discussion first, with a measurement	A threshold moved without a measurement is a preference with a number on it
Adding a new failure mode for contributors	Discussion first	A gate that blocks a reasonable change is bypassed, and then it protects nothing

What the tooling actually reads

Five facts about the layout decide how a new artifact is discovered, and none of them is stated in the file you are adding.

1. check_repo.py discovers every skills/*/SKILL.md up to three levels deep, and a shallower SKILL.md shadows anything nested below it, which is the installer's own rule. Adding a second skill needs no change to the gates.
2. check_repo.py and check_routing.py look at references/*.md and scripts/*.py|.js non-recursively. A file in a subdirectory of either is invisible to the routing comparison.
3. check_budget.py counts the corpus recursively: references/**/*.md, plus every file under scripts/. The corpus note fires at 110 files or 26,000 lines, so a family is visible to the budget before it is visible to routing.
4. The path resolver patterns match references/<name>.md and scripts/<name>.py with no separator anywhere in the name, so a nested path such as references/coverage/route-inventory.md is not resolved by any gate. A dangling nested pointer fails nothing.
5. check_commands.py scans README.md and every .md under skills/ and docs/, skipping directories named fixtures. A command in a nested file is still validated; a command in a file that no walker reaches is not.

The consequence is the rule for everything below: a new class needs one file that an existing gate already walks, and that file must state honestly what is checked and what is not.

Adding a whole new artifact class

A class is a set of files with a shared shape that a reader is expected to keep consistent: the evidence companions, the coverage inventory and the case library are three examples of what is already here.

1. Decide whether it ships. Inside skills/exe-reverse/ is shipped to anyone who installs the skill; the repository root is maintenance tooling. README.md's structure block states the split, and a class that ships belongs in that block with an explicit skills/exe-reverse/... path, because the README's path check fails a bare relative path.
2. Find out whether any gate walks it. If one does, its completeness is machine-checked and the class can rely on that. If none does, the class's own index must say so in a sentence a reader can act on, because a machine-checked guarantee that does not exist is the failure this repository keeps recording.
3. Give the class an index that is discovered. The cheapest reliable form is a top-level references/<class>.md that opens with the head shape (the title, a goal line, and ## When to load this), registers itself in routing.md's Reference index, and is named in routing.md's Sub-indexes table as the class's index, which is how the coverage inventory and the case library are already registered.
4. State the row contract in the index, not in the individual files: what each entry must carry, which document holds the claim it supports, and which grade applies.
5. Add the class's own test only if it has a mechanical part. A class of prose files has nothing to test, and a test that asserts a file exists is not a guarantee about its content.

Two things to avoid. A class whose only index is named inside another document's prose is not reachable, because nothing compares prose to a directory listing. And a class that duplicates a registration duty that routing already owns will drift from it.

Adding a reference family

A family is a directory of related files with an index, and it is worth the extra structure exactly when the files are read together and would be misleading in the flat library.

1. Create skills/exe-reverse/references/<family>/ and write README.md as the index: what the family is for, when to load it, the register or table of its members, and the template a new member follows. Copy the shape of references/precedents/README.md, which is both an index and a template.
2. Register the family index in routing.md, in the Sub-indexes table, as references/<family>/README.md, saying what the file holds.
3. If the family needs a top-level touchpoint as well (a symptom, a route), add that row to SKILL.md and to the mirror in routing.md in the same pass; the routing gate compares those two tables row by row and fails when one side has a row the other lacks.
4. Add each member to the family's own register at the same time you add the file, never later. The register is the only reachability that exists for a nested file.
5. Say in the index that the members are not resolved by any gate, so a reviewer knows the index must be read against the directory. This is a one-sentence honesty cost that makes the gap visible instead of implied.

Members of a family are still covered by the checks that scan every markdown file under skills/: a python <script>.py command in one of them is validated against the script's argument table, and a cross-file section reference is resolved against the target's real headings. check_refs.py reads README.md and skills/ only, so the same content placed under docs/ is not checked that way.

Evolving the frontmatter or the metadata contract

What is enforced. check_repo.py parses the frontmatter with a real YAML parser, because frontmatter that satisfies a pattern can still be rejected by the parser the installer uses. It then requires name to equal the skill's directory name and to match lowercase alphanumerics joined by single hyphens, at most 64 characters, and description to be between 40 and 1024 characters. A description whose value contains a colon followed by a space must be quoted; an unquoted one makes the whole skill invisible to the installer. Where PyYAML is not installed the gate prints a warning and falls back to a colon test, so the check degrades rather than disappearing.

What is not enforced. The metadata block. capability_registry, host_probe, evidence_summary and strength_labels are promises about paths and vocabulary, and no gate resolves any of them. A renamed file whose path still appears there is drift nothing will report, so a rename has to be carried by hand and reviewed. Treat the block as documentation that happens to live inside a parsed structure.

The description is a loading decision, not a summary. It is the only text an agent sees when deciding whether this skill applies to its task. Adding a route to it is a budget decision against the 1024-character ceiling, and the routes that earn their characters are the ones a wrong answer would otherwise be given for.

Changing the strength-label vocabulary needs a discussion, because every reference, every evidence row and every case grades against observed | inferred | unverified. The burden of proof is that the existing three cannot express a distinction that changes a decision a reader makes. If a new label is genuinely needed, the change is mechanical in its extent and semantic in its risk: every document that states a grade has to be revisited in the same pass, and the frontmatter key that names the vocabulary has to be updated with them.

Changing the always-loaded budget

SKILL.md loads in full whenever the skill activates, so the budget is measured on the whole file, including its index tables. check_budget.py reports:

Measure	Soft	Hard	Fails the gate?
SKILL.md lines	460	560	at the hard limit
SKILL.md tokens	13000	16000	at the hard limit
Index-row description length	250 characters	400	never: both branches are warnings
Long reference with no navigable head	100 lines, head checked in the first 40	not applicable	never: a warning
Corpus size	110 files, 26000 lines	not applicable	never: a note

The token count comes from tiktoken's cl100k_base encoding when it is installed, and otherwise from characters / 4.285. That fallback constant is calibrated against the real measurement rather than guessed, and it exists for a stated reason: an earlier value made the gate reach different verdicts on different machines, reporting an over-budget file where the tokenizer was absent and a compliant one where it was present. The gate's answer must not depend on what the runner happens to have installed, which is also why CI installs the tokenizer so the real path is exercised.

To change a threshold: produce the measurement that justifies the new number, keep the two measurement paths agreeing, and update the numbers and the calibration note in the docstring in the same change, because the docstring is the only place the calibration is recorded. A threshold that moves because a document outgrew it is not a change to the budget; it is the budget being abandoned.

The ordinary answer instead: move the detail into a reference and leave a pointer. Two things stay in the always-loaded body, and both are argued in the gate's own comment: the symptom index, because a symptom-to-file lookup must stay one hop, and the gates with their pass criteria, because "the idea is understood" does not clear a gate.

Verify with both modes: python check_budget.py reports the measured values and the notes; python check_budget.py --strict turns every warning into a failure, which is how a warn-only rule is checked when someone chooses to hold the line.

One known wording defect, reported here rather than copied: the docstrings of check_routing.py and check_budget.py describe "the four gates with their pass criteria", while the entry point defines five (G1 to G5). Comment text is checked by no gate. If you touch either file, correct the wording instead of reproducing it.

Adding a new gate script and wiring it into CI

The shape a gate has here. A root-level check_<name>.py, runnable from the repository root with no arguments as check_<name>.py under the interpreter, printing a human-readable summary and, where a caller needs to branch on the verdict, a final RESULT=<token> line. Exit 0 when the tree satisfies the rule and non-zero when it does not. --json is the convention for machine-readable output, and the one place it is deliberately absent from the text stream is a JSON payload: a RESULT= line appended to JSON makes the stream unparseable for the caller that asked for JSON, so there the status is a field and the exit code carries the meaning. check_repo.py and check_refs.py take no arguments at all; check_routing.py takes only --json; check_budget.py adds --strict; check_commands.py adds --path, --fix-report, --include-workbench and --quiet.

Rules a gate follows here:

1. Read-only. A gate must not modify the tree. The only root tool that writes is build_scripts.py, and it writes only under --apply, after an audit pass that is the default.
2. Report the boundary of the check. check_commands.py counts the invocations it could not verify rather than skipping them silently, because "no finding" from an interface it cannot read would be indistinguishable from coverage. A new gate that cannot see part of its surface should say so in its own output.
3. Say when the surface is empty. A gate that found nothing to check has not passed; it has run against nothing. scan_leaks.py has a dedicated exit code for exactly this.
4. Prefer a narrow rule with a stated exemption over a broad rule that is wrong. The gates here carry their exemptions in a table with a reason per entry, because a gate that gets switched off protects nothing.
5. Do not restate another gate's rule without saying why. Measured duplication exists here: the routing gate and the budget gate both check that a reference is reachable. The second copy is justified in a comment, because a dead reference is invisible in a way a broken path is not.
6. Warn, do not fail, when the gate cannot judge. Hedge detection, restatement and corpus size are notes; index-row length and navigability are warnings; only the ceilings fail. --strict is the opt-in that makes warnings fatal.

Every gate needs a test. A gate gets a subprocess contract test under tests/cli/: the flags it accepts, the exit code per verdict, and the RESULT= token where it prints one. A mis-typed marker is a harness failure rather than a warning, because the suite runs with --strict-markers.

Lint and typing are free. The CI lint job runs mypy and flake8 over *.py at the root plus the skill's scripts, so a new root gate is checked without a configuration change. Both tools read their configuration from the repository root, and the job deliberately runs on an interpreter with none of the optional dependencies installed, so a new gate must import them lazily or behind a fallback with a stated behaviour.

Wiring it in. Add the step to the test job in .github/workflows/ci.yml, in the gate sequence and before the regression suite. There is deliberately no paths: filter on the workflow: a filter that watches only source files would skip the structural gates on exactly the commits they exist to catch, which are documentation commits. A gate that can only run where a device, a debugger or a network peer exists does not belong in CI; record that route in the measurement record instead and let the test self-skip with a printed reason.

Document it. README.md's repository-maintenance list names every root tool with what it enforces, one line each. A tool that is not in that list is invisible to a contributor who never opens the root directory.

What needs a discussion first: a gate that can fail a change which used to pass, a gate whose rule is a matter of taste rather than a checkable property, and a gate that duplicates a rule another gate already owns.

Recording a downgrade when a measurement disagrees

The stop condition is stated in the entry point: a measurement that disagrees with a conclusion recorded as settled re-opens the conclusion, and the measurement is not explained away. That is a rule about behaviour; this is the procedure that makes it a change a repository can hold.

1. Freeze the measurement first. The command as it was run, its output, the artifact digest, the tool and its version, and the host build. Without those pins the disagreement is an anecdote, and it cannot raise or lower anything.
2. Name the link that was weaker than recorded. A chain is graded by its weakest link, so the question is not "was the conclusion wrong" but "which step was assumed". A tool that reports success is evidence about the tool; it was never evidence about the target.
3. Move every document that states the claim, in one pass. The coverage row (references/coverage-and-limits.md: the direction, the strongest defensible grade, what it rests on, and the dependency that is not shipped here), the evidence row (references/evidence-summary.md), and the machine-readable row under skills/exe-reverse/evidence/. A downgrade that lands in one of the three leaves the other two asserting the stronger grade, and the stronger grade is the one a reader will find.
4. Record the negative result. docs/tool-verification/ is where the pass goes, with the command and its output, and with the scenarios the target could not exercise named. A route measured and failed on a named host is more useful than a route nobody ran.
5. Add a pitfall entry only if the downgrade has a mechanism. The catalogue's value is the mechanism that made the failure invisible, not the failure. If the earlier claim looked right for a reason a reader could hit, that reason is an entry; if the claim was merely optimistic, it is not. Cite the entry by number from anywhere else rather than restating it.
6. Describe the demotion in the change. The repository's own rule is that a corrected conclusion is described rather than quietly edited, and that a record keeps the wrong version beside the right one, because the retraction is the transferable part.

Restate the grade for the whole chain, not for the corrected step alone. A chain that ends in a step nobody executed is inferred at best, and a chain whose weakest link was just replaced is entitled to be re-graded only as far as the new weakest link.

A downgrade is a result, not a failure. The coverage file commits to this: a negative measurement raises a row's honesty even while it lowers its grade, and a row with no measurement behind it is worse than a row with a disappointing one. What a downgrade must not do is delete the earlier claim, edit a conclusion without saying so, or leave a downstream document stating the grade that was just withdrawn.

There is a precedent in this repository for the kind of error that forces a downgrade: the budget gate's token fallback constant was first set to a value that made the gate reach different verdicts on machines with and without the tokenizer, which is the shape of a claim that looks measured and is not. Its docstring now records the calibration and the reason, which is what a downgrade should leave behind.