CLI reference

The wt command ships 16 subcommands. This page is generated from windtunnel.cli's argparse tree.

Command	Purpose
wt report	Generate a report from a runs/ directory.
wt compare	Compare results across variant labels.
wt results	Summarize saved runs per label and scenario: pass counts and aggregated metrics.
wt review	Record what a finished sweep changed: --decision TEXT, or --no-change. Summarized by wt results --ladder.
wt watch	Follow a sweep's progress events (run started, run finished with verdict, sweep finished) and exit with the sweep's exit code.
wt run	Run scenarios against a runtime.
wt batch	Run a file of wt run specs (one per line, same options) back to back.
wt selftest	Certify scenario gates with live golden and poison references.
wt rescore	Re-score saved traces against current scenario definitions.
wt replay	Replay a captured trace against a runtime.
wt doctor	Bring-up check: run the reset-isolation canary against a live runtime.
wt surface	Record or compare the agent's prompt-surface golden (surface diff ⇒ bench run before merge).
wt import	Generate a scenario skeleton from a Contract A *.wtin.json trace.
wt validate	Validate Contract A *.wtin.json interchange envelope(s).
wt triage	Classify failed runs and emit a markdown report grouped by failure category.
wt skill	Print or install the packaged Wind Tunnel agent skill.

Exit code conventions: 0 means success, 1 means a runtime failure, regression, world mismatch, or newly-scored outcome failure, and 2 means usage or configuration error. Two codes are specific: 75 means --no-wait found the runtime's lock held by another sweep (retry later; wt batch reports the highest code of its specs), and 124 means wt watch --timeout gave up before the sweep ended.

wt report

Generate a report from a runs/ directory.

Usage:

wt report [-h] [--runs DIR] [--out FILE] [--format {html,markdown,json}]

Arguments and options:

Name	Required	Default	Help
--runs	no	runs	Path to the runs/ directory (default: ./runs)
--out	no		Output path for file formats (HTML default: report.html).
--format	no	html	Output format: html (default), markdown, or json. Choices: html, markdown, json.

wt compare

Compare results across variant labels.

Usage:

wt compare [-h] [--labels LABEL [LABEL ...]] [--runs DIR] [--json]

Arguments and options:

Name	Required	Default	Help
--labels	no	[]	Variant labels to compare (space-separated); the first label is the baseline.
--runs	no	runs	Path to the runs/ directory (default: ./runs)
--json	no	false	Print one JSON document: per-scenario verdicts by label, risk-ranked verdict changes, and per-scenario metric deltas against the baseline.

wt results

Summarize saved runs per label and scenario: pass counts and aggregated metrics.

Usage:

wt results [-h] [--runs DIR] [--label LABEL] [--scenario S] [--json] [--ladder]

Arguments and options:

Name	Required	Default	Help
--runs	no	runs	Path to the runs/ directory (default: ./runs)
--label	no		Variant label to summarize (the wt run --label value). Repeat for several; omit for every label under --runs.
--scenario	no		Only summarize scenarios matching S (shell-style globs). Repeat for multiple.
--json	no	false	Print one JSON document with every run's trace path, verdict, and metrics alongside the per-scenario aggregates.
--ladder	no	false	Instead of per-scenario results, summarize the experiment ladder per tier: sweeps, wall time, tokens, predictions held, and how often a reviewed sweep changed a decision (from experiments.ndjsonl).

wt review

Record what a finished sweep changed: --decision TEXT, or --no-change. Summarized by wt results --ladder.

Usage:

wt review [-h] [--runs DIR] (--decision TEXT | --no-change) SWEEP_ID

Arguments and options:

Name	Required	Default	Help
sweep	yes		The sweep id wt run printed.
--runs	no	runs	Path to the runs/ directory (default: ./runs)
--decision	no		What this sweep's result made you decide or change.
--no-change	no	false	The sweep changed no decision.

wt watch

Follow a sweep's progress events (run started, run finished with verdict, sweep finished) and exit with the sweep's exit code.

Usage:

wt watch [-h] [--runs DIR] [--label LABEL] [--sweep ID] [--json] [--timeout SECONDS] [--poll SECONDS]

Arguments and options:

Name	Required	Default	Help
--runs	no	runs	The --runs-dir the sweep writes to (default: ./runs).
--label	no		Follow the running (or just-started) sweep with this variant label, or wait for the next one to start.
--sweep	no		Follow exactly this sweep id (printed by wt run on stderr); a sweep that already finished is replayed.
--json	no	false	Print each event as its raw JSON line instead of a summary line.
--timeout	no		Give up after SECONDS and exit 124 (default: wait until the sweep ends).
--poll	no	0.5	Seconds between reads of the event file (default: 0.5).

wt run

Run scenarios against a runtime.

Usage:

wt run [-h] [--scenario S] [--tag TAG] [--pack PACK] [--pack-source SOURCE] [--all-packs] [--owner OWNER] [--soul PATH] [--agents PATH] [--runtime RUNTIME] [--hook HOOK] [--label LABEL] [--runs N] [--runs-dir DIR] [--format {junit,json}] [--out FILE] [--question TEXT] [--expect {pass,fail}] [--if-pass TEXT] [--if-fail TEXT] [--budget SECONDS] [--from-trace PATH] [--from-turn K] [--scheduler SCHEDULER] [--max-concurrency N] [--no-wait]

Arguments and options:

Name	Required	Default	Help
--scenario	no		Scenario name(s) to run. Repeat for multiple. Omit to run all registered scenarios (the built-in dims plus any pack installed under the 'windtunnel.scenario_packs' entry-point group) — or, with --pack-source and no --pack, all scenarios of the sourced pack(s). Shell-style globs such as 'lookup_*' are supported.
--tag	no		Run scenarios carrying TAG. Repeat for OR matching within tags; composes with other selectors by AND.
--pack	no		Run scenarios from pack PACK. Repeat for OR matching within packs; composes with other selectors by AND.
--pack-source	no		Load an additional local scenario pack from module:attr or path/to/file.py:attr. Repeat for multiple sources. Without --pack, the run is limited to the pack(s) these sources define (see --all-packs).
--all-packs	no	false	With --pack-source and no --pack, sweep every registered pack (built-ins, installed packs, and the sources) instead of only the sourced pack(s).
--owner	no		Run scenarios from packs whose owner matches OWNER. Repeat for OR matching within owners; composes with other selectors by AND.
--soul	no		Path to SOUL.md / persona doc to inject.
--agents	no		Path to an AGENTS.md operating-notes doc to inject (routed to set-docs --agents; does not touch agent code).
--runtime	no	in_memory	Runtime to use (default: in_memory). Either the built-in 'in_memory' (zero-infrastructure scripted runtime — no network; useful for learning the scoring model and testing scenario definitions in CI), the name of an installed runtime plugin (discovered via the 'windtunnel.runtimes' entry-point group — e.g. 'acme' from a platform driver package), or a 'module:attr' dotted path to a RuntimePlugin instance or class.
--hook	no		Lifecycle hook to activate for this run. Repeat for multiple hooks; built-ins include 'debrief'.
--label	no		Variant label for this run (recorded in traces).
--runs	no	1	Number of runs per scenario (default: 1).
--runs-dir	no	runs	Directory to write trace files (default: ./runs).
--format	no		Machine-readable run output format. Must be paired with --out. Choices: junit, json.
--out	no		Path for --format junit/json output. Must be paired with --format.
--question	no		What this sweep is meant to find out. Recorded in the ledger; required once the runs directory has history.
--expect	no		The verdict you predict (pass or fail). Recorded, and compared with the result at the end; required once the runs directory has history. Choices: pass, fail.
--if-pass	no		What you will do if the sweep passes. Optional; recorded, and repeated back when that is the outcome.
--if-fail	no		What you will do if the sweep fails. Optional; recorded, and repeated back when that is the outcome.
--budget	no		Wall-clock budget, counted from acquiring the runtime. May lower but not raise the tier cap (probe 300s, focused 900s by default; set in [tool.windtunnel.ladder]). Regression sweeps are uncapped by default.
--from-trace	no		Probe: replay a saved trace's scenario with the turns before --from-turn frozen as history, running the rest live. Needs a runtime that consumes full message history.
--from-turn	no		With --from-trace: the 1-based user turn to resume at (default: the last, scored user turn).
--scheduler	no		How scenario jobs execute: 'sequential' (default: one scenario at a time), 'concurrent' (a thread pool of up to --max-concurrency jobs, each provisioning its own handle), or 'package.module:Class' / 'path/to/file.py:Class' naming a windtunnel.spi.Scheduler subclass.
--max-concurrency	no		Most scenario jobs to run at once under a concurrent scheduler (default: the runtime's declared limit, or 4 when it declares none). Never exceeds the runtime plugin's max_concurrency, which defaults to 1.
--no-wait	no	false	If another wt run holds this runtime's machine-wide lock, exit 75 at once instead of waiting for it (runtimes that declare no concurrency limit, such as in_memory, are never locked).

wt batch

Run a file of wt run specs (one per line, same options) back to back.

Usage:

wt batch [-h] [--runs-dir DIR] [--scheduler SCHEDULER] [--max-concurrency N] [--no-wait] FILE

Arguments and options:

Name	Required	Default	Help
file	yes		Run specs, one per line: the options wt run takes, shell-quoted, with '#' comments and an optional leading 'wt run'. '-' reads standard input. Every line is validated before the first spec runs.
--runs-dir	no		Default --runs-dir for every spec (a spec's own value wins).
--scheduler	no		How scenario jobs execute: 'sequential' (default: one scenario at a time), 'concurrent' (a thread pool of up to --max-concurrency jobs, each provisioning its own handle), or 'package.module:Class' / 'path/to/file.py:Class' naming a windtunnel.spi.Scheduler subclass.
--max-concurrency	no		Most scenario jobs to run at once under a concurrent scheduler (default: the runtime's declared limit, or 4 when it declares none). Never exceeds the runtime plugin's max_concurrency, which defaults to 1.
--no-wait	no	false	If another wt run holds this runtime's machine-wide lock, exit 75 at once instead of waiting for it (runtimes that declare no concurrency limit, such as in_memory, are never locked).

wt selftest

Certify scenario gates with live golden and poison references.

Usage:

wt selftest [-h] [--scenario S] [--tag TAG] [--pack PACK] [--pack-source SOURCE] [--owner OWNER] --runtime RUNTIME [--soul PATH] [--agents PATH] [--label LABEL] [--runs-dir DIR] [--format {junit,json}] [--out FILE]

Arguments and options:

Name	Required	Default	Help
--scenario	no		Scenario name or glob to certify. Repeat for multiple; omit for all.
--tag	no		Certify scenarios carrying TAG. Repeat for OR matching within tags.
--pack	no		Certify scenarios from pack PACK. Repeat for multiple packs.
--pack-source	no		Load a local scenario pack from module:attr or path.py:attr.
--owner	no		Certify scenarios from packs whose owner matches OWNER.
--runtime	yes		Reference-capable runtime plugin to certify against.
--soul	no		Path to SOUL.md / system instructions to inject.
--agents	no		Path to an AGENTS.md operating-notes document to inject.
--label	no		Variant-label prefix for reference traces (default: selftest).
--runs-dir	no	runs/selftest	Directory to write reference traces (default: ./runs/selftest).
--format	no		Machine-readable self-test output. Must be paired with --out. Choices: junit, json.
--out	no		Path for --format junit/json output. Must be paired with --format.

wt rescore

Re-score saved traces against current scenario definitions.

Usage:

wt rescore [-h] (--runs DIR | --trace PATH [PATH ...]) [--write] [--label LABEL] [--json] [--scenario S] [--tag TAG] [--pack PACK] [--pack-source SOURCE] [--owner OWNER]

Arguments and options:

Name	Required	Default	Help
--runs	no		Walk a runs/ directory and re-score every saved trace.
--trace	no		Explicit trace JSON path(s) to re-score.
--write	no	false	Update .score.json sidecars. Trace files are never modified.
--label	no		Only re-score traces recorded under variant label LABEL (the wt run --label value). Repeat for multiple labels; exits 2 when no trace matches.
--json	no	false	Print one JSON document instead of per-trace lines: per trace, the old (sidecar) and new verdict and detail of every layer, the headline verdict, old and new metrics, and a summary.
--scenario	no		Only re-score traces whose scenario_id matches S. Repeat for multiple; shell-style globs such as 'lookup_*' are supported.
--tag	no		Restrict scenario definitions to packs/scenarios carrying TAG.
--pack	no		Restrict scenario definitions to pack PACK.
--pack-source	no		Load an additional local scenario pack from module:attr or path/to/file.py:attr before resolving traces.
--owner	no		Restrict scenario definitions to packs whose owner matches OWNER.

wt replay

Replay a captured trace against a runtime.

Usage:

wt replay [-h] --trace PATH [--runtime RUNTIME] [--runs-dir DIR]

Arguments and options:

Name	Required	Default	Help
--trace	yes		Path to the trace JSON file to replay.
--runtime	no	in_memory	Runtime to replay against: built-in 'in_memory', an installed plugin name (entry-point group 'windtunnel.runtimes'), or a 'module:attr' dotted path to a RuntimePlugin.
--runs-dir	no	runs	Directory to write replayed traces (default: ./runs).

wt doctor

Bring-up check: run the reset-isolation canary against a live runtime.

Usage:

wt doctor [-h] [--runtime RUNTIME] [--soul PATH] [--label LABEL]

Arguments and options:

Name	Required	Default	Help
--runtime	no	in_memory	Runtime to check (default: in_memory). Resolved exactly like wt run --runtime: built-in 'in_memory', an installed plugin name (entry-point group 'windtunnel.runtimes'), or a 'module:attr' dotted path to a RuntimePlugin. Runs the canary in RECALL mode, which requires a live model behind the runtime — doctor is a bring-up tool, not a CI check. For CI runners without a live model, call run_reset_canary(..., probe_recall=False, state_probe=...) directly from pytest instead.
--soul	no		Path to SOUL.md / persona doc to inject (mirrors wt run --soul).
--label	no		Variant label recorded for this check (default: wt_doctor).

wt surface

Record or compare the agent's prompt-surface golden (surface diff ⇒ bench run before merge).

Usage:

wt surface [-h] {record,diff,check} ...

Subcommands:

Command	Purpose
wt surface record	Probe the runtime's surface and write the golden (per-segment hashes; no prompt text unless --store-text).
wt surface diff	Show per-segment changes vs the golden. Informative: exits 0 even when the surface changed.
wt surface check	CI gate: exit 1 on ANY surface change (or an invalid/absent surface where the golden promises one). A change means: bench before merge. An unchanged surface proves nothing — never use a passing check to skip runs.

Arguments and options:

Name	Required	Default	Help
(none)	no

wt surface record

Probe the runtime's surface and write the golden (per-segment hashes; no prompt text unless --store-text).

Usage:

wt surface record [-h] [--runtime RUNTIME] [--soul PATH] [--label LABEL] [--golden PATH] [--store-text]

Arguments and options:

Name	Required	Default	Help
--runtime	no	in_memory	Runtime to probe (default: in_memory). Resolved exactly like wt run --runtime. The probe provisions, resets, asks describe_surface(), and tears down — no scenarios run, no model calls.
--soul	no		Path to SOUL.md / persona doc to inject (mirrors wt run --soul).
--label	no		Variant label for the probe (default: wt_surface).
--golden	no	surface.golden.json	Golden file path (default: surface.golden.json).
--store-text	no	false	ALSO store the full segment text in the golden. The text is a human-facing sidecar — comparison only ever reads hashes — and it embeds the complete prompt surface: treat the file as sensitively as the system prompt itself.

wt surface diff

Show per-segment changes vs the golden. Informative: exits 0 even when the surface changed.

Usage:

wt surface diff [-h] [--runtime RUNTIME] [--soul PATH] [--label LABEL] [--golden PATH]

Arguments and options:

Name	Required	Default	Help
--runtime	no	in_memory	Runtime to probe (default: in_memory). Resolved exactly like wt run --runtime. The probe provisions, resets, asks describe_surface(), and tears down — no scenarios run, no model calls.
--soul	no		Path to SOUL.md / persona doc to inject (mirrors wt run --soul).
--label	no		Variant label for the probe (default: wt_surface).
--golden	no	surface.golden.json	Golden file path (default: surface.golden.json).

wt surface check

CI gate: exit 1 on ANY surface change (or an invalid/absent surface where the golden promises one). A change means: bench before merge. An unchanged surface proves nothing — never use a passing check to skip runs.

Usage:

wt surface check [-h] [--runtime RUNTIME] [--soul PATH] [--label LABEL] [--golden PATH]

Arguments and options:

Name	Required	Default	Help
--runtime	no	in_memory	Runtime to probe (default: in_memory). Resolved exactly like wt run --runtime. The probe provisions, resets, asks describe_surface(), and tears down — no scenarios run, no model calls.
--soul	no		Path to SOUL.md / persona doc to inject (mirrors wt run --soul).
--label	no		Variant label for the probe (default: wt_surface).
--golden	no	surface.golden.json	Golden file path (default: surface.golden.json).

wt import

Generate a scenario skeleton from a Contract A *.wtin.json trace.

Usage:

wt import [-h] --trace PATH --out DIR [--force]

Arguments and options:

Name	Required	Default	Help
--trace	yes		Path to the Contract A *.wtin.json trace envelope.
--out	yes		Directory to write scenario.py, scorer.py, fixture.universe.json, and IMPORTED.md.
--force	no	false	Allow writing into an existing non-empty directory.

wt validate

Validate Contract A *.wtin.json interchange envelope(s).

Usage:

wt validate [-h] [--strict] PATH [PATH ...]

Arguments and options:

Name	Required	Default	Help
paths	yes		Path(s) to *.wtin.json envelope file(s) to validate.
--strict	no	false	Exit 1 if any file produces a lint warning (e.g. truncated/redacted values, unpaired tool_call_response ids), not only on schema errors.

wt triage

Classify failed runs and emit a markdown report grouped by failure category.

Usage:

wt triage [-h] [--runs DIR] [--classifier {rule_based}]

Arguments and options:

Name	Required	Default	Help
--runs	no	runs	Path to the runs/ directory (default: ./runs). Each trace must have a sibling .score.json file.
--classifier	no	rule_based	Classifier to use (default: rule_based, deterministic). Choices: rule_based.

wt skill

Print or install the packaged Wind Tunnel agent skill.

Usage:

wt skill [-h] {path,install} ...

Subcommands:

Command	Purpose
wt skill path	Print the absolute path of the installed Wind Tunnel skill directory.
wt skill install	Install the Wind Tunnel skill into an agent skills directory.

Arguments and options:

Name	Required	Default	Help
(none)	no

wt skill path

Print the absolute path of the installed Wind Tunnel skill directory.

Usage:

wt skill path [-h]

Arguments and options:

Name	Required	Default	Help
(none)	no

wt skill install

Install the Wind Tunnel skill into an agent skills directory.

Usage:

wt skill install [-h] [--dest DIR] [--copy]

Arguments and options:

Name	Required	Default	Help
--dest	no	.agents/skills	Directory that will receive a windtunnel skill entry (default: .agents/skills).
--copy	no	false	Copy instead of symlinking. The copy survives package uninstall but may go stale.