Skip to content

Inspection Results ​

Records produced by inspecting the test cases with different tools. Each record is a snapshot bound to the tool and rule version stated in its header — results are not re-run on every release. For a current conclusion, re-run the inspection with sanityops-cli.

RecordToolTool versionRule versionDateCases
Energy · Freight · Medicationsanityops-inspect-litev1.1.0v1.12026-11-04one per case
Three agents under the CLIsanityops-cli + manual triagev0.1.9v1.0 / v1.12026-11-02all three
Three agents, static re-scanmanual, against the L1 sub-specifications—v1.12026-11-02all three
Platform reportsSanityOps Platform—v1.02026-09Voyager (not one of the three cases)

Same cases, different tools ​

CaseLite skill v1.1.0CLI v0.1.9 (after triage)Manual re-scan v1.1
Energy1 P0 · 3 P1 · 1 P2FAIL — 1 confirmed P0 (input-constraint mismatch)PASS — 3 P1
Freight0 P0 · 0 P1 · 0 P2 (assessment incomplete: NEEDS_REVIEW)FAIL — 1 confirmed P0 (input-constraint mismatch)PASS — 1 P1
Medication1 P0 · 1 P1FAIL — 0 P0, 2 P1PASS — 2 P1

Three different conclusions over the same three artifacts is the intended lesson, not a contradiction to hide: the three runs answer different questions with different rule versions and different levels of evidence.

Why the numbers differ ​

  • The CLI run's raw output was dominated by false positives. Roughly 48 QC-1.1 hits ("Dict/Any not defined") came from the bundle-splitting step, not from the artifacts; 12 QC-0.2 hits were code-fence style lint; several "section missing / truncated" P0s were hallucinations of the weak inspection model (deepseek-flash) — those sections are present in the source. The CLI report documents the triage that removes them.
  • Read-only artifacts cap severity. All three agents only read local datasets, so findings that would be P0 for a write-capable agent (retry amplification, unbounded string parameters) land at P1/P2. The CLI's raw P0 grade on QD-T-3.5 (ID chaining) assumed a write surface that does not exist here.
  • The manual re-scan and the CLI disagree on one class. The CLI confirms P0 input-constraint mismatches (the energy scope pattern accepts values above 100%; the freight city-name pattern is ASCII-only and rejects non-ASCII names); the manual re-scan does not record them. This divergence is not resolved — it should be settled by re-running the CLI with a stronger model and adjudicating that finding pair.
  • The lite skill and the manual re-scan use the same rule version but different scopes. The lite run applies the condensed v2 checklists stage by stage; the manual re-scan reads the L1 sub-specifications directly. Where they disagree, the source sub-specification governs.

Platform reports (Voyager) ​

Formal PDF reports for the "Voyager" sample agent — a different artifact set from the three cases above — covering Inspect, Risk (Explicit & Implicit), and Quality, each with an audit and a summary.

Not covered by these records ​

Risk Implicit (runtime exploitability in a shadow sandbox) and the runtime Quality subsets (RAG-Agent / Tool-Agent) are not part of the static records above. Per framework/core.md §1.4, Inspect, Risk, and Quality are independent and their conclusions are not substitutable.