CLI

The crucible binary carries three subcommands.

crucible run <scenario>

One-shot A/B session. Blocks until the judge returns, prints the verdict JSON on stdout.

crucible run scenarios/asd-vs-baseline.md \
    [--model openai/gpt-4o-mini]           # override every arm's model
    [--judge-model openai/gpt-4o-mini]     # default: gpt-4o-mini
    [--out path/to/verdict.json]           # write stdout to a file too
    [--state-dir ~/.crucible-state]        # persist to sqlite + JSON summary

--model is what you use when the bundled scenarios pin Anthropic but you only have OpenAI credentials on hand. Applies to both arms.

Without --state-dir the run is memory-backed (drops on exit). With, it lands in the same layout crucible serve reads:

<state_dir>/
├── sessions.db                        sqlite ASG store
└── sessions/
    └── session-<scenario>-<ts>.json   cached SessionOutcome

crucible serve

Starts the HTTP API and (optionally) serves the UI on the same port.

crucible serve \
    [--addr 127.0.0.1:8080] \
    [--scenarios-dir scenarios] \
    [--state-dir <path>] \                 # required for /api/sessions*, POST persists
    [--web-dir path/to/web/build]          # SvelteKit static bundle → SPA at /

Endpoints are documented on HTTP API. OpenAPI schema at /api/openapi.json.

Live-view example:

# Terminal 1: server
crucible serve --state-dir ~/.crucible-state --web-dir ~/crucible/web/build

# Terminal 2: kick an async run + tail the SSE stream
SID=$(curl -s -X POST http://127.0.0.1:8080/api/runs/async \
    -H 'content-type: application/json' \
    -d '{"scenario":"asd-vs-baseline","model_override":"openai/gpt-4o-mini"}' \
    | jq -r .session_id)
curl -N "http://127.0.0.1:8080/api/runs/$SID/stream"

Or open the UI at http://127.0.0.1:8080 and use /runs/new.

crucible mcp

Stdio MCP server. Registered with a client (Claude Code, Cursor, Codex) it exposes 7 Crucible tools alongside whatever else the client has hooked up.

crucible mcp \
    [--scenarios-dir scenarios] \
    [--state-dir <path>]                   # required for the live tools

See MCP tools for the full tool table and sample agent prompts.

crucible --version

Reports the version stamped at build time. 0.1.6 at the time of writing.