Selectivity Keeps a Useful Checkpoint When the Test Budget Runs Out

A reduction can be useful before it is finished. That matters when the predicate is expensive, a CI job has a fixed allowance, or a developer wants to make a first pass without committing to thousands of executions. Selectivity’s execution budget puts a bound on how many predicate processes can start. Reaching that limit leaves an incomplete run and the latest accepted checkpoint, rather than silently promoting unfinished work to a completed result.

The generated example starts with a 256-line Python file totaling 3866 bytes. It contains ordinary assignments and one break statement outside a loop. The predicate compiles the candidate without executing it and recognizes that exact SyntaxError message. A different syntax error returns unresolved status. A candidate that compiles successfully returns uninteresting status. This is a controlled diagnostic example, not a claim of a Python compiler defect.

In one run of the released Selectivity 0.1.1 binary on Linux x86_64 with Python 3.12.14, a budget of 12 predicate executions leaves a 64-line checkpoint totaling 1014 bytes. Every candidate uses three observations and a five-second timeout, so the budget allows only four complete candidate checks. The audit also records the next attempted check as incomplete without starting another predicate process. The command exits with code 2, and the summary says incomplete with locally_minimal set to false.

That checkpoint was accepted under the repeated predicate observations available during the search. It has not received the final verification that belongs to a completed run. A CI wrapper should preserve that distinction when collecting artifacts. Reporting a failed or incomplete reduction as completed would discard information the engine went to some trouble to keep.

The follow-up starts a fresh reduction from the checkpoint, with a new output directory and the normal execution budget. It does not restore the earlier search state or reuse its candidate history. In the recorded run, that new search reduces the 1014-byte checkpoint to the six-byte input break followed by a newline. It uses another 36 predicate executions across 12 checks, completes final verification and reports local deletion minimality at line granularity. The original 256-line file remains unchanged throughout both runs.

A practical invocation for that second pass looks like this, with the predicate path replaced by its absolute location:

./selectivity --input budget/reduced --out continued --repeats 3 \
  -- python3 /absolute/path/syntax_predicate.py '{input}'

The runnable examples generate the source file, enforce the 12-execution first budget and check both expected exit codes. The saved evidence keeps the incomplete and completed summaries separately. The lesson is fairly practical: an execution limit can produce a useful smaller starting point, provided the report remains honest about which checks have actually finished.