Live Smoke Regression

언어: 🇺🇸 English

This procedure runs the same small request in real agent runtimes to verify that hook, gate, permission, and completion contracts are alive. Automated unit tests cover hook payload and file contracts; this smoke closes the final connection on the actual session surface.

Purpose
Standard Input
Target Runtimes
Expected Signals
Verification Layers
Smoke Record
Blind Behavior Record
Failure Triage
Automated Verification Boundary

Purpose

Run the same input on the claimed installed Claude Code and Codex targets. Consult skill-catalog/platform-adapters.json for the supported adapter set; terminal-only is a manual fallback.
Keep the request small and reversible, such as reading the first 10 README lines.
Verify runtime gate observability, not the quality of the summary itself.
Save experiment notes as a smoke record, but do not store raw prompts, secrets, or full transcripts.

Standard Input

Open the repo root in each runtime and send this request:

Read the first 10 lines of README and summarize what Ghost-ALICE OS is in one paragraph. Apply verification-before-completion before completion.

Target Runtimes

Platform	Preparation	Observed surface
Claude Code	Fresh session after Ghost-ALICE install	Skill permission, SessionStart/UserPromptSubmit hook, completion reminder
Codex	Fresh session after Ghost-ALICE install	`AGENTS.md` bootstrap, `~/.codex/hooks.json`, `~/.codex/config.toml`, `SKILL.md` read records, observed hook payload firing when the runtime supports hooks; otherwise explicit hookless/manual fallback wording
Codex native Windows	Fresh session after Ghost-ALICE install in a Windows native environment	actual hook payload firing, `~/.codex/hooks.json`, `~/.codex/config.toml`, `SKILL.md` read records

Expected Signals

Each smoke record captures whether these signals were observed:

agent-run result: the agent process exits 0 within the timeout and writes a non-empty last-message artifact.
tool/runtime errors: the log has no tool router errors, hook failures, tracebacks, runtime panics, or cache initialization errors.
harness validity: the prompt permits the read-only tool action needed to satisfy the request. A contradictory prompt is invalid-harness, not pass.
task-router: routing is visible before the first tool action.
session-intent-analyzer: work with a new or changed interpretation records interpreted-as: before its first tool call, without raw prompt storage.
boundary-contract: follows the current routing-surface decision; executed read-only verification may require a boundary contract. Record n/a only when that route does not require one.
tool checkpoint: follows the common contract’s io-trace Transparency Rule; a routine read-only call needs no checkpoint.
skill activation permission: Claude Code is not blocked by a shortened allowlist that covers only core gates.
verification-before-completion: the workflow connects each closure claim to applicable evidence, preserving the original source and time for valid unchanged proof.
internal audit: tool access remains auditable without requiring printed [io-trace].
pending merge precheck: before changing an installed skill, re-check the current platform manifest; the session-start observation does not replace that check.

Verification Layers

Layer	Purpose	Subject visibility	Release role
Skill pressure RED/GREEN	Check whether written methodology changes behavior under disclosed pressure.	The skill and pressure purpose are visible.	Authoring evidence only; never installed-behavior release evidence.
Governance smoke	Check hooks, gates, process health, and required control-surface markers.	The governance request and markers are visible.	Runtime plumbing evidence, not an AI behavior judgment.
Automated blind screening	Evaluate an installed subject on a held-out authentic prompt in a fresh session.	Only the prompt and ordinary runtime context are visible.	Repeatable behavior screening; evaluator purpose, rubric, expected answer, pass criteria, prior output, and experiment label stay private.
Manual clean-terminal acceptance	A human runs the held-out case in a newly opened terminal after canonical installation.	Only the authentic request is given to the subject.	Final live acceptance gate. Computer Use must not automate this terminal.

Automated blind screening uses only direct-process transport. Its controller and subject packets are structurally separate, and subject output stays in memory until a separate evaluator returns a verdict. Missing or invalid evaluator state is a non-pass. The controller loads the sealed case file itself, computes a digest of the exact loaded bytes plus a canonical hash of the validated case, and rejects caller-supplied freshness, isolation, or digest fields. Installed provenance comes only from the schema-version-1 installer state at ~/.ghost-alice/install-state/<platform>.json; caller or subject self-attestation is not provenance.

Smoke Record

When storing a record in the repo, use only .tmp/ or local scratch. Do not mutate the remote Wiki or user home settings.

platform:
date:
repo_ref:
input_case: README first 10 lines
observed:
  task-router:
  session-intent-analyzer:
  tool checkpoint:
  skill activation permission:
  verification-before-completion:
  internal-audit:
failure triage:
  status: pass | fail | invalid-harness | inconclusive
  reason:
  next owner:

Minimum machine-readable fields for a local scratch summary:

platform:
case:
status: pass | fail | invalid-harness | inconclusive
agent_command:
exit_code:
timed_out:
log_file:
output_file:
reasons:

The blind record is a separate schema from the governance-smoke summary. It may contain only the case id/hash, exact-file suite digest, source/version, installed provenance, mode, verdict/dimensions, sanitized reason, and minimal process status (exit_code and timed_out). Do not store the raw prompt, rubric, expected answer, subject response or full transcript, raw stdout/stderr logs, pass criteria, prior output, experiment label, credentials, or evaluator-private notes.

Persisted dimension keys are constrained opaque ids. Criterion text remains evaluator-private and is never used as a durable dimension key.

_shared/blind_behavior.py is the dedicated automated screening path; blind evaluation is not a mode of _shared/live_agent_smoke.py. The controller validates the loaded case and evaluator-private types, then delegates subject execution to _shared/fresh_agent_session.py, which alone constructs the Claude no-persistence or Codex ephemeral command, rejects resume/session arguments, and creates the empty run cwd. It sends only the authentic prompt over subject stdin, adds no case/private argv, and passes only ordinary runtime values plus CLAUDE_CONFIG_DIR and CODEX_HOME when configured; rubric, purpose, expected-answer, and controller-private environment values are stripped. The controller holds the response in memory and sends it plus private state to a separate evaluator process.

Subject or evaluator timeout, non-zero exit, empty subject response, malformed evaluator output, invalid evaluator state, and overall/dimension disagreement all fail closed and emit no conduct-feedback candidate. Only an evaluator-confirmed behavioral failure emits exactly one canonical report-only candidate. An optional record write is atomic and persists only the sanitized record, never the candidate or controller-private material.

Failure Triage

Symptom	Judgment	Next action
`task-router` is not visible before the first tool action	gate routing failure	Inspect platform bootstrap or hook payload.
Completion claim appears without `verification-before-completion`	completion gate failure	Inspect completion reminder and Codex hook evidence/fallback wording.
routine tool-checkpoint output requires recovery-cost or recovery-note fields	tool-checkpoint surface failure	Synchronize `docs/policies/session-gate-matrix.md`, platform bootstrap, and hook message wording.
skill activation permission allows only core gates	permission scope failure	Inspect installer hook permission sync and platform policy files.
smoke record stores raw prompt or transcript	audit hygiene failure	Fix the session-intent-analyzer storage contract first.
agent command times out	runtime smoke failure	Inspect the affected failure and follow the bounded repair conditions in the session gate matrix. Reinstall only affected runtime assets; retry with a distinct hypothesis and remaining authority/resources.
log contains `ERROR codex_core::tools::router`, hook failure, traceback, panic, or cache initialization error	runtime smoke failure	Treat the run as failed even if a later, different prompt passes. Reproduce or intentionally retire the failing case with evidence.
output file is missing or empty	runtime smoke failure	Do not infer success from partial log activity. Rerun after fixing the agent command or output path.
Windows resolves `codex` to different shim or binary across PowerShell, CMD, and automation	harness drift	Record `agent_command` and prefer the intended `codex.cmd` shim or an explicit absolute path for Codex smoke.
prompt forbids the only available read-only method needed by the task	invalid harness	Fix the harness and rerun. This is not product pass evidence.
a later bounded pass follows an earlier unresolved failure in the same loop	partial status	Retain the unresolved case and report partial status. Continue only eligible authorized repair under the session gate matrix; a bounded later pass does not close the earlier criterion.

Automated Verification Boundary

Runtime smoke records are manual evidence, not a keyword-presence unit test target. Static hook payload and gate wording contracts are covered by scripts/check_skill_gate_contract.py, scripts/validate_entrypoints.py, and _shared.test_install_hooks. _shared/live_agent_smoke.py may classify process-level evidence and run Codex fresh-session smoke cases, but it does not replace adversarial review of the output against this policy and the design documents. Automated blind screening strengthens that evidence but does not replace the human-operated clean-terminal final acceptance gate.