exe-reverse/tests

tests/ — executable regression suite for the exe-reverse skill

The suite that proves the shipped scripts still do what their docstrings say, offline, with only the standard library and pytest.

When to load this

Read it before changing anything under skills/exe-reverse/scripts/, and read it before quoting a coverage claim about this skill. It is the boundary between "the script was written to do X" and "the script was measured doing X on this host".

What it runs

File	What it asserts
conftest.py	Shared fixtures: builds the synthetic PE images at test time, loads a script by path, runs a script as a subprocess with a timeout, and the two synthetic scan corpora.
unit/test_peutil_reader.py	Header fields against the fixture's declared metadata, RVA to file-offset round trips including the virtual-only tail, the PE32 versus PE32+ optional-header difference, the checksum against a second implementation, the resource tree, named and ordinal imports, the TLS callback, the RSDS/PDB record, relocation blocks, entropy and flag decoding, and the parser's rejections.
unit/test_pepatch_patcher.py	Pattern parsing and matching, the ambiguity refusal, the expect-mismatch refusal, the length-unproven and length-change cases, the resolver's refusals, the re-read verification, and the CLI-level facts that a dry run writes nothing and a real run leaves its input byte-identical.
unit/test_aob_scan.py	The verdict boundaries at both sides of each threshold, hit reporting with offsets, RVAs and sections, section filtering, context windows, and every accepted shape of a pattern file.
unit/test_scan_leaks.py	Every rule firing on a shaped sample, every exemption class staying silent, the three measured deviations from the exemption list, scan position reporting, and the file walk's skips.
unit/test_capabilities_registry.py	The capability table: every atom's kind, basis and probe name, the transitive closure against an independent walk, the dependency graph's acyclicity, resolve on missing atoms and on file: atoms that are and are not on disk, the declaration parser including the documentation template it must reject, and the declared / mapped / unknown classification.
unit/test_doctor.py	The static dependency pass (stdlib, siblings and a third-party name separated without executing the file), discover_scripts scanning rather than listing, the per-script table with an unclassified script still counted, and the environment facts with their keys and severity labels.
unit/test_dart_snapshot_strings.py	The ELF reader against a generated container, both literal encodings recovered at the offsets the fixture placed them at, the length and limit gates, and the pool counts -- zero on an image carrying neither idiom and non-zero on hand-built bytes carrying each one.
cli/test_cli_surface.py	Help and usage exit codes for every script, the documented exit codes for each script's error paths, and the negative case that inspect-only runs write nothing into the working directory.
cli/test_cli_json.py	Each script's machine-readable output: keys, values, and the conventions that are not what a reader would assume.
cli/test_cli_registry_surface.py	The registry and doctor subprocess contracts: --list / --json / single-capability exit codes and RESULT= tokens, the doctor's three blocks and its 0-versus-3 verdict, and the view flags that print one block only.
cli/test_cli_dart_snapshot.py	The snapshot reader's CLI: the refusals and their exit codes, --find case-insensitivity and the exit code a miss produces, --at by file offset and by virtual address, and the --json document.
integration/test_pipeline_end_to_end.py	Build, locate, patch, re-read: the patched bytes, the untouched control site, the recomputed checksum, and a control run in which nothing changes.
integration/test_across_builds.py	Pattern hit counts compared between two builds, including the case where a patch removes the very byte the pattern was located with.
run_tests.py	The one-command entry point; prints the install line and exits 3 when pytest is missing.
benchmark.md	The regression matrix: dimension, target, scripts, measured result, strength label.

How to run it

python -m pytest -q                     # from the repository root: the whole suite
python -m pytest -q -m unit             # one layer: unit, cli or integration
python tests/run_tests.py               # same thing, with a clear message if pytest is missing
python tests/run_tests.py -q -m unit    # arguments are forwarded to pytest

If pytest is not installed: python -m pip install pytest.

What it deliberately does not do

It does not attach a debugger, load a driver, open a socket, read a device, or execute a target. Those routes are conditional on host hardware and policy, and a suite that asserts them is either fragile or dishonest. The recorded experiments for them live in benchmark.md with the strength label that says whether they were reproduced here.

It does not commit or download a sample. Every PE input is generated into a temporary directory by make_fixture_pe.py on the way in and removed with the session, so the suite carries no binary, no licence question and no dependency on a target that changes underneath it.

It does not assert on timing. The only time in the suite is a generous per-process timeout, which exists to turn a hang into a failure rather than to measure throughput. The timings in benchmark.md were taken by hand and are recorded as measurements, not enforced as limits.

It does not test the reference documents' prose. Cross-reference, budget and routing checks belong to the repository's own gates.

Fixtures

conftest.py builds four images per session, addressing make_fixture_pe.py's own recipes rather than duplicating their shapes:

Name	Recipe	Why it is in the suite
pe64	default	PE32+ with every structure the builder can emit, five sections and the planted pair.
pe32	pe32	The 32-bit optional-header layout, whose field offsets differ.
bare	bare	No exports, resources, TLS or debug directory, so the absent cases are representable.
fatalign	fat-align	A 4 KiB file alignment, large enough that .rdata has no virtual-only tail.

The builder's no-virtual-tail recipe is deliberately not used: with a zero virtual pad its own guard rejects the layout it names, so the recipe exits 1 and writes nothing. fatalign covers the same property. cli/test_cli_surface.py pins that failure so the recipe cannot quietly stay broken.

conftest.build_synthetic_elf builds a second kind of input, because dart_snapshot_strings.py reads an ELF shared object rather than a PE and no sample of any kind may be committed: a valid ELF64 header, a .text of sixteen bytes carrying neither idiom the pool observer counts, a .rodata holding one one-byte (ASCII) literal and one two-byte (UTF-16LE) literal at known offsets, and a section-name string table. It is a container to hold the reader against, not a Flutter snapshot: the counts the script's docstring records came from a real 26.8 MB app.so and cannot be reproduced from a generated file, which is why the counting logic is asserted in both directions -- zero on the fixture, non-zero on hand-built bytes -- and the real measurement stays a recorded observation.

Conventions

Marker is the layer: unit for pure functions and parsers, cli for subprocess contracts, integration for the no-debugger flows. All three are declared in pytest.ini, and --strict-markers is on, so an undeclared marker is an error rather than a warning. There is no slow or deselected tier and therefore no --runslow switch: nothing in the suite needs one, and declaring a marker that no test carries would be decoration.

A missing prerequisite skips with a printed reason: an interpreter older than the 3.9 the scripts declare, or a local tool a check needs. A wrong result always fails. Nothing is skipped silently, and no test was weakened to pass.

Untrusted sample text is assembled from fragments in conftest.py rather than quoted whole. The suite has to contain strings shaped like pasted credentials and profile paths in order to prove the scanner catches them; written whole they would also be caught by the repository's own pre-publish scan. The runtime values are identical, and the source text matches nothing.

Deviations the suite pins

Each of these is a measured behaviour that differs from what the corresponding docstring implies. They are asserted as observations so a fix shows up as a test change instead of as a quietly gained or lost result, and each is also recorded in benchmark.md.

Where	Measured behaviour
make_fixture_pe.py	The no-virtual-tail recipe contradicts its own guard and cannot be built.
peutil.py, pepatch.py	A file truncated mid-header raises struct.error and prints a traceback instead of using the documented exit code; a fully non-PE input does use it.
pepatch.py	A missing --spec file reaches an unguarded open() and prints a traceback.
aob_scan.py	A bare JSON array of patterns raises AttributeError; only the object form is accepted.
scan_leaks.py	Exit code 3 is documented for an empty scan surface and is unreachable: an empty root reports RESULT=clean and exits 0.
scan_leaks.py	The bare-GUID exemption matches inside the machine-guid rule's own match text, so that rule cannot produce a finding.
scan_leaks.py	The .NET entry matches any .net TLD, so a hit inside such a host is suppressed as a platform term.
scan_leaks.py	The <[a-z_]+> placeholder alternative is wrapped in \b, so it cannot match bracketed prose.
hwprobe.py	Two route rows answer with a sentence instead of one of the three documented states.
pepatch.py	--allow-resize writes in place, so the file does not grow and the bytes after the site are overwritten rather than shifted.

Limits

Coverage is bounded by the fixture, not by the reader. A synthetic image exercises every structure the builder emits and nothing it does not; a route that needs a device, a debugger, a driver or a network is out of scope here and is listed as not run in benchmark.md. The suite also cannot say whether a pattern will survive a build it has never seen, which is why the comparisons it does make are between two images it built itself.

The scripts under skills/exe-reverse/scripts/ were being edited while this suite was written, and several assertions moved with them. Re-run the suite after any change to those files rather than trusting a count recorded earlier.