Architecture

Crucible is a small Rust workspace on top of the AgentStateGraph (ASG) substrate. Six crates, one binary, one static UI, one MCP server.

Layout

crates/
├── crucible-llm      OpenAI + Anthropic adapters. One-line switch by model prefix.
├── crucible-scorer   Reference-fix similarity metric. Parser + score, no LLM calls.
├── crucible-core     Scenarios, session store, agent loop, judge, runner, doctor.
├── crucible-server   axum HTTP API + SSE stream, serves the UI as a fallback.
├── crucible-mcp      rmcp stdio server exposing crucible_run, crucible_judge, etc.
├── crucible-cli      The `crucible` binary. Subcommands: run, serve, mcp.
└── crucible-llm-proxy Optional: OpenAI-compatible HTTP shim over the llm crate.
web/                  SvelteKit + @agentstate/lens-core UI. Static bundle at web/build.

Sessions live on ASG

A session is one ASG namespace, keyed by scenario + start timestamp:

namespace: session-<scenario>-<yyyymmddThhmmss>
├── branch: main
│   ├── /meta               scenario + started_at + fixture_path
│   ├── /meta/arms/<id>     arm config (model, system prompt, extra tools)
│   ├── /meta/budget        per-run caps
│   └── /verdict            judge output, set at finalize time
└── branch: arm-<id>        one branch per arm
    ├── /decisions/seq-0001 per-decision payload (model_turn / tool_call / policy_check / note)
    ├── /final_diff         git diff of the workdir after the arm completes
    └── /test_outcome       exit code + tails from running scenario.test_command

Every decision the agent emits — a model turn, a tool call, a policy check — is a Commit with a structured Intent payload. That means ASG’s blame, query, and epoch tooling work over Crucible runs without any special casing. Anything ASG can ask, so can you.

Storage

By default, sessions are memory-backed — hermetic, fast, drops on exit. Pass --state-dir <path> and sessions land in a real sqlite ASG store at <state_dir>/sessions.db, with a JSON summary of the outcome in <state_dir>/sessions/<session_id>.json for cheap list and detail queries by the HTTP API.

The run loop

run_session in crucible-core::runner:

1. Loads the scenario, applies any model override.
2. Opens a SessionStore (memory or sqlite) → gets a namespace.
3. For each arm:
   • Copies the fixture into a fresh tempdir.
   • Runs setup (asd index build if asd is in extra_tools).
   • git init + baseline commit so subsequent git diff captures only what the agent changed.
   • Assembles tool pack (baseline vs baseline+asd+ctx).
   • Loads the scenario’s policies into a PolicyEngine.
   • Runs the agent loop: Completer.complete → policy check per tool call → commit a Decision to the arm branch → append tool result. Stops on Final (no tool calls) or when the budget tripwires (turns/tokens/wall-clock).
   • Executes scenario.test_command in the workdir, captures pass/fail.
   • Computes git diff HEAD and scores it against the reference patch (if present).
   • Fires arm_finished SSE event.
4. Calls the judge with both RunSummarys. Judge sees a rubric plus AUTHORITATIVE_FACTS (tests passed, ref-fix score, budget flags).
5. Verdict lands on the session’s main branch. Fires verdict_ready.

Surfaces

Everything shares one code path:

• CLI (crucible run) — blocking, prints JSON to stdout, optional --out <path>.
• HTTP (crucible serve) — POST /api/runs blocks and returns the outcome; POST /api/runs/async returns a session id and pushes DecisionEvents over GET /api/runs/:id/stream (SSE) as commits land.
• MCP (crucible mcp) — 7 tools an agent can call over stdio. crucible_run_session is the same runner; live decisions aren’t streamed since MCP doesn’t have a stream primitive today.

The judge is separately reusable via POST /api/verdicts (or the crucible_rejudge MCP tool): pass two {session_id, arm_id} refs from the same scenario and it returns a fresh verdict without re-running the agents.