Skip to content

SanityOps Framework Core ​


Version: v1.1

Release Date: October 2026

Maintained by: Sanity AI Labs

License: CC BY-SA 4.0


Table of Contents ​


Document Information ​

FieldValue
Document TitleSanityOps Framework Core: Terminology, Objects, and Decision Baselines
Short NameSanityOps Core
Versionv1.1
StatusPublished
DateOctober 2026
ScopeAll current and future SanityOps sub-specifications, rule assets, tool implementations, reports, CI/CD pipelines, and governance processes
Maintained bySanity AI Labs
Document NatureFramework-level normative baseline

Foreword ​

0.1 Purpose ​

The SanityOps Framework has evolved into a specification system spanning Logic Artifact Defect Inspection, Explicit Risk Audit, dynamic risk validation, Tool-Agent quality assessment, RAG-Agent quality assessment, and defect impact mapping.

As these specifications matured independently across different phases, inconsistencies inevitably emerged: the same term carrying different meanings across documents, the same level designation mapping to different risk dimensions, abbreviations diverging, and conclusion boundaries becoming susceptible to misinterpretation.

This specification establishes the common semantic layer for SanityOps. It defines, in a unified manner:

  • The framework's positioning, object boundaries, and lifecycle boundaries;
  • Common governance requirements for Logic Artifacts and related assets;
  • Core terminology, namespaces, abbreviations, and version expressions;
  • The scope of applicability for grading, severity, status, and Gate models;
  • Minimum requirements for Findings, evidence, human review, and release summaries;
  • Collaboration relationships, conclusion boundaries, and conflict resolution principles across sub-specifications;
  • Consistency migration principles for existing specifications.

0.2 What Core Is ​

SanityOps Core is a framework-level internal convention specification. It is not merely a glossary.

It defines the common language that every sub-specification depends on:

text
Object Language
+ Evidence Language
+ Status Language
+ Decision Language
+ Version Language

Core does not replace any sub-specification's rules, scoring, thresholds, inspection items, or Gates.

0.3 What Core Is Not ​

This specification does not:

  1. Create, delete, or modify existing QD, EX, AS, or FM identifiers;
  2. Alter the independent adjudication rules of Inspect, Risk, or Quality;
  3. Automatically equate a Static Defect with a vulnerability, a successful attack, or a quality failure;
  4. Automatically equate a single score, a single test run, or an external control interception with a releasable conclusion;
  5. Substitute for an organization's IAM, backend controls, network security, privacy compliance, production monitoring, or industry regulatory obligations;
  6. Claim that SanityOps has achieved complete coverage of specialized inspection requirements for Knowledge, Memory, Workflow, Identity, A2A, MCP, or Connector domains.

0.4 Normative Language ​

Unless otherwise defined by a specific sub-specification, the following terms carry these meanings:

TermMeaning
MUSTNon-compliance disqualifies a claim of conformance to the corresponding requirement.
SHOULDGenerally expected; where not met, the rationale, residual risk, and alternative measures MUST be documented.
MAYOptional; adoption is at the discretion of the organization, business context, or implementation.
MUST NOTProhibited.
NOT APPLICABLEThe preconditions for applicability are not satisfied for the current object or condition. The basis for this applicability determination MUST be documented.

0.5 Source-Specification Precedence and Migration Principles ​

With the publication of Core:

  1. Domain-specific rules take precedence.
    For rules, scoring, thresholds, defect definitions, and Gates within the respective domains of Inspect, Risk, and Quality, the corresponding domain-specific sub-specification governs.

  2. Common semantics take precedence.
    For terminology, object identification, version evidence, status expression, and cross-subset conclusion boundaries, Core serves as the unified baseline.

  3. Historical rules are not silently rewritten.
    Content in existing documents that is inconsistent with Core does not automatically become invalid upon Core's publication. Such discrepancies SHALL be registered as consistency migration items and explicitly revised in subsequent maintenance versions.

  4. Substantive changes and editorial revisions are separated.

    • Revisions to terminology, abbreviations, cross-references, and report fields MAY use a patch version.
    • Substantive changes to scoring, rules, thresholds, Gates, or scope of applicability MUST follow the corresponding formal change process.

Chapter 1: Framework Positioning, Governance Objects, and Lifecycle Boundaries ​

1.1 Positioning of SanityOps ​

SanityOps is a vendor-neutral quality, security, and governance framework for the full lifecycle of enterprise-grade AI Agents.

Its core objective is not to guarantee that Agents governed by SanityOps are always correct or absolutely secure. Rather, it is:

To bring the controllable, versionable, and verifiable Logic Artifacts, behavioral risks, and quality outcomes of enterprise Agents into a continuous cycle of inspection, validation, regression, and evidence governance — systematically reducing the uncertainty of Agent behavior and the business and security risks that uncertainty creates.

SanityOps is designed primarily for enterprise AI/Agent applications. Coverage of all personal-user Agents, open-ended creative tools, or general-purpose chat products is not a primary design objective.

1.2 Logic Artifacts as First-Class Governance Assets ​

1.2.1 Definition ​

Based on three criteria — the hosting field, content independence, and direct influence on the LLM — the Logic Artifacts defined by SanityOps are limited to three types: System Prompt, Skill, and Tool Schema.

1.2.2 Minimum Governance Requirements ​

Every applicable Logic Artifact MUST support the following capabilities:

text
Identifiable
+ Versionable
+ Traceable
+ Inspectable
+ Verifiable
+ Regressible
+ Auditable

Natural-language form does not reduce governance requirements. Prompts, Skills, and Tool Schemas SHOULD NOT be treated as mere "configuration" or "copy." They are engineering assets that can materially influence the behavioral boundaries of an Agent.

1.3 Core Problems and Specification Domains ​

DomainCore ProblemPrimary Conclusions
InspectDo artifact definitions or cross-artifact relationships contain Static Defects?QD, static evidence, severity level, remediation recommendations
RiskDo artifacts contain Explicit Risks, or can security-risk behaviors be triggered under controlled conditions?EX, D1/D2/D3, Explicit·P0/P1/P2, A/B/C/D, Signal, validation evidence
QualityDoes the Agent's actual task execution — or the user-visible service — meet the defined quality bar?Reliability, quality metrics, test case results, Quality Gate, regression conclusions
RelevanceWhich Attack Surfaces, validation strategies, Failure Modes, and regression requirements might a Static Defect correlate with?AS, FM, Defect Chains, mapping recommendations, Candidate Root Causes

1.4 The Non-Substitutability Principle ​

The following inferences are not valid:

text
Inspect PASS ≠ Security PASS
Inspect PASS ≠ Quality PASS
Risk exploitation not successful ≠ Static Defects absent
Risk Explicit no EX hit ≠ Artifact boundaries are complete
Quality PASS ≠ No security risks exist
Quality FAIL ≠ A specific QD has been proven as the sole root cause
External control interception ≠ Agent itself possesses sufficient safety constraints
Single test PASS ≠ Long-term security or quality assurance

Each sub-specification draws conclusions only within the boundaries of its own evidence domain.

1.5 The Full-Lifecycle Governance Closed Loop ​

The target closed loop for SanityOps is:

text
Design & Build
    ↓
Logic Artifact Versioning
    ↓
Inspect — Static Defect Inspection
    ↓
Risk — Risk Audit and Dynamic Validation
    ↓
Quality — Service Quality Assessment
    ↓
Release Authorization and Exception Decisions
    ↓
Runtime Monitoring, Change Assessment, and Regression
    ↓
Incident Review, Rule Calibration, and Asset Improvement

SanityOps currently defines:

  • Inspect;
  • Risk Explicit;
  • Risk Implicit;
  • Quality Tool-Agent;
  • Quality RAG-Agent;
  • Inspect defect impact and cross-subset mapping.

Production runtime monitoring, production auditing, incident response, Knowledge governance, Memory governance, Identity governance, and A2A governance are future extension directions. They MUST NOT be represented as fully covered, specialized specification capabilities at this time.


Chapter 2: Unified Object and Artifact Model ​

2.1 Core Objects ​

ObjectDefinition
AgentAn application entity that receives input, uses a model for decision-making, and may generate content, invoke tools, or execute tasks.
PromptA directive asset that defines role, task, boundaries, rules, context handling, output, or invocation requirements.
SkillA behavior definition oriented toward a specific task or capability scope, typically including trigger conditions, inputs/outputs, permissions, resources, and failure strategies.
ToolAn external capability interface invocable by an Agent or Skill.
Tool SchemaA structured definition of a Tool's name, description, input parameters, return structure, side effects, and constraints.
WorkflowAn orchestration definition for multi-step execution, routing, state, dependencies, retries, recovery, or human handoff.
Knowledge SourceA governed knowledge asset and its metadata, used to support retrieval, citation, answering, or decision-making.
MemoryState and information assets retained, read, updated, or reused by an Agent within or across sessions.
Identity / PolicyAssets related to identity, roles, authorization, approval, tenancy, data access, and policy enforcement.
External ConnectorAn MCP Server, Webhook, Callback, external API, plugin, or other external connectivity capability.
Evaluation AssetTest cases, rules, evaluators, Judge Prompts, scoring configurations, Baselines, and regression assets.
ControlA technical or administrative measure for preventing, limiting, detecting, intercepting, auditing, or recovering from risk behaviors.
FindingA specific inspection, validation, or quality discovery produced for a particular artifact, version, and environment.
ExceptionAn approved, traceable, time-limited Risk Acceptance or rule exception.

2.2 Current Coverage and Extension Objects ​

ObjectDedicated Specification Exists?Current Status
PromptYes, Inspect PromptCovered
SkillYes, Inspect SkillCovered
Tool / Tool SchemaYes, Inspect ToolCovered
Prompt–Skill–Tool CrossYes, Inspect CrossCovered
PermissionYes, Inspect PermissionCovered
Explicit Risk ExpressionYes, Risk ExplicitCovered
Dynamic Attack ValidationYes, Risk ImplicitCovered
Tool-Agent Task QualityYes, Quality Tool-AgentCovered
RAG Content and Boundary Response QualityYes, Quality RAG-AgentCovered
Defect Impact and Cross-Subset MappingYes, RelevanceCovered
Knowledge / RAG ContractNoExtension object
Workflow / StateNoExtension object
MemoryNoExtension object
Identity / PolicyNoExtension object
A2A / MCP / ConnectorNoExtension object

For extension objects, Core defines only the unified object, version, and evidence interfaces. Prior to the publication of dedicated specifications, no corresponding QD, Gate, or coverage conclusions SHALL be fabricated.

2.3 Artifact Relationships ​

The artifact relationships of a typical Agent can be abstracted as:

text
Prompt
  ↓ Defines global objectives, boundaries, rules, and workflow requirements
Skill
  ↓ Defines capability scope, trigger conditions, permissions, and failure strategies
Tool / Tool Schema
  ↓ Defines invocable capabilities, parameters, side effects, and Hard Constraints
Backend / Runtime
  ↓ Enforces IAM, validation, approval, audit, isolation, and monitoring

For systems that include retrieval, memory, orchestration, or multi-Agent collaboration, this extends to:

text
Prompt / Skill / Workflow
        ↓
Knowledge / Memory / Identity / Policy
        ↓
Tool / Connector / A2A / MCP
        ↓
Backend / Runtime / External Service

2.4 Version Boundaries and Minimum Identification ​

Any conclusion eligible for release, audit, regression, or root-cause investigation MUST be associated with the following minimum version information:

text
Agent Identifier
+ Artifact Identifier
+ Artifact Version or Immutable Hash
+ Dependency Artifact Versions
+ Model and Runtime Configuration Version
+ Environment Identifier
+ Rule or Evaluator Version
+ Test Asset or Knowledge Snapshot Version
+ Execution Timestamp

Where not all fields are available, the missing items and their impact on conclusion confidence, comparability, or applicability MUST be explicitly documented.


Chapter 3: Unified Terminology, Naming, and Numbering Rules ​

3.1 Formal Names and Historical Aliases ​

The following are Core's recommended formal names:

Formal NameAcceptable Historical AliasesRequirement for Subsequent Documents
Inspect PromptInspect Prompt StandardUse formal name
Inspect SkillInspect Skill StandardUse formal name
Inspect ToolInspect Tool StandardUse formal name
Inspect CrossInspect CROSS, Inspect Cross StandardUse Inspect Cross
Inspect PermissionInspect Permission StandardUse formal name
Risk ExplicitRisk EX, ExplicitMay note EX as classification prefix on first occurrence
Risk ImplicitRisk IM, ImplicitMay note IM as historical abbreviation on first occurrence
Quality Tool-AgentSanity Quality Tool-Agent, Quality Tool AgentUse Quality Tool-Agent
Quality RAG-AgentQuality RAG AgentUse Quality RAG-Agent
RelevanceInspect Defect Impact Mapping SpecificationUse full document title or Relevance

Aliases in historical documents do not automatically constitute errors, but subsequent revisions and new documents SHALL adopt the formal names uniformly.

3.2 Numbering Namespaces ​

Object TypeNumbering ExampleMaintained ByPurpose
Prompt DefectQD-P-*Inspect PromptPrompt Static Defects
Skill DefectQD-S-*Inspect SkillSkill Static Defects
Tool DefectQD-T-*Inspect ToolTool / Schema Static Defects
Cross DefectQD-PS-*, QD-PT-*, QD-ST-*, QD-SS-*, QD-TT-*, QD-CYC-*Inspect CrossCross-Artifact relationships, multi-Skill/multi-Tool consistency, and circular invocation defects
Permission DefectQD-PM-*Inspect PermissionPermission-responsibility proportionality defects
Embedded Code Contract Defect (Draft)QD-CC-*Inspect Code Contract (Draft, unpublished)Static defects in code snippets embedded in Logic Artifacts
Explicit RiskEX-* (classification prefix); NL-A-*, NL-B-*, NR-O-*, NR-S-*Risk ExplicitExplicit Risk classification
Attack SurfaceAS-*RelevanceSecurity impact mapping
Behavioral Failure ModeFM-*RelevanceTool-Agent quality diagnostic mapping
Security Defect ChainSC-*RelevanceCandidate security integration paths
Tool-Agent Quality ChainQC-TA-*RelevanceCandidate task quality integration paths
RAG-Agent Quality ChainQC-RA-*RelevanceCandidate content quality integration paths
Finding InstanceF-*Platform or implementing organizationActual discovery on a specific object
Exception InstanceEXC-*Governance processRisk Acceptance or rule exception

3.3 Distinguishing Rules from Instances ​

A clear distinction MUST be maintained between "rule definitions" and "actual Findings."

text
QD-T-3.2
= A reusable defect rule definition

F-20260719-00128
= A Finding instance on a specific Tool, at a specific version, in a specific environment

A single QD can produce multiple Findings across multiple artifacts, versions, or environments. A single Finding can also be associated with multiple QD, EX, AS, FM, or Defect Chain entries.

3.4 Terminology Boundaries ​

TermCorrect DefinitionMUST NOT Be Conflated With
QDStatic Logic Artifact DefectDynamic vulnerability, successful attack, quality failure
EXExplicit Risk expression or dangerous configuration classification within an artifactInspect defect identifier
A/B/C/DRisk Implicit attack test case Termination StatusDefect severity level or quality score
FMCandidate behavioral Failure Mode for Tool-AgentConfirmed root cause
ASPotential Attack Surface classificationConfirmed security incident
FindingAn actual discovery record on a specific objectThe rule itself
ControlA measure for preventing, limiting, intercepting, or auditing riskA purely natural-language declaration
ExceptionAn approved, time-limited Risk AcceptanceAn unresolved issue or a default exemption
BaselineA confirmed, comparable versioned snapshotAny arbitrary historical result

Chapter 4: Grading Models and Risk Semantics ​

4.1 Grading Models Must Not Be Conflated ​

SanityOps employs multiple grading models for distinct purposes. They answer different questions and MUST NOT be substituted for one another merely because they all use L, P, S, or numeric designations.

Grade CodeFormal NameQuestion AnsweredPrimary Source
AC-LAgent Complexity LevelHow complex is the Agent's structure, collaboration pattern, and execution form?Inspect Prompt, Inspect Cross, Inspect Permission
OR-LOperation Risk LevelHow high is the risk of the operations a Skill or Tool can perform?Inspect Skill, Inspect Tool, Inspect Permission
BI-LBusiness Impact LevelHow severe are the business consequences of task failure or incorrect results?Quality Tool-Agent
CH-LChange LevelWhat is the scale, impact scope, and regression intensity of the change?Quality RAG-Agent and subsequent change governance
P0/P1/P2Static Audit Disposition LevelAt what disposition priority should discovered Static Defects and Explicit Risks be addressed?Inspect sub-specifications and Risk Explicit
A/B/C/DRisk Implicit Termination StatusWhat is the result of a dynamic attack attempt, and where does the responsibility boundary lie?Risk Implicit

Subsequent documents, reports, APIs, and tool interfaces MUST NOT use isolated expressions such as "L2," "high risk," or "severity level 3." The governing model MUST be included, e.g., OR-L2, BI-L3, Explicit·P0.

4.2 Agent Complexity Level: AC-L ​

AC-L describes an Agent's technical form, collaboration complexity, and execution chain depth. Its purpose is to determine the applicable Inspect depth, Cross inspection scope, and validation combinations.

LevelMeaningExamples
AC-L1Single task, limited context, no Tool use or low-complexity Tool useSingle-turn content generation, constrained knowledge Q&A
AC-L2Presence of Skills, Tools, or limited multi-step executionQuery-type Tool-Agent, basic RAG-Agent
AC-L3Multiple Skills, multiple Tools, chained invocation, state passing, or high-risk operationsBusiness process Agent, operations execution Agent
AC-L4Multi-Agent collaboration, dynamic planning, complex state machines, cross-domain delegation, or extensive external connectivityEnterprise multi-Agent systems

AC-L does not directly indicate data sensitivity, actual business impact, or security vulnerability severity.

Note: Where existing sub-specifications contain Agent complexity tiers with different counts or different descriptions, they SHALL be mapped to AC-L in subsequent consistency revisions. They MUST NOT be automatically treated as equivalent prior to explicit mapping.

4.3 Operation Risk Level: OR-L ​

OR-L describes the operational risk a given Skill, Tool, or execution path can produce. Its focus is on permissions, side effects, irreversibility, impact scope, and data exfiltration capability.

LevelMeaningExamples
OR-L1Low risk, read-only, no sensitive data, or no external side effectsLocal format conversion, public information query
OR-L2Limited side effects or constrained sensitive data handlingInternal queries, controlled notifications, draft generation
OR-L3High-risk writes, deletions, data exfiltration, permission changes, batch operations, or sensitive data handlingRefunds, production configuration changes, external email dispatch

OR-L determines whether controls such as confirmation, approval, idempotency, audit, rollback, least privilege, and dynamic validation are required.

OR-L is not equivalent to an Agent's structural complexity. For example, a structurally simple Agent that can read a full medical dataset may be AC-L1 and OR-L3.

4.4 Business Impact Level: BI-L ​

BI-L describes the potential business consequences of Tool-Agent task failure, incorrect execution, or erroneous results. Its focus is not the operation itself, but the impact of task failure on users, business, compliance, and the organization.

LevelMeaningExamples
BI-L1Limited impact, easily correctable, no material effect on core businessInternal low-priority information aggregation
BI-L2Impacts local processes, requires manual correction, or may affect user experienceRoutine customer service assistance, internal process queries
BI-L3May affect critical business, customer rights, compliance, financials, or production stabilityFinancial processing, medical assistance, production operations, critical customer matters

BI-L is primarily used for test case coverage, metric thresholds, human review, and release requirements in Quality Tool-Agent.

BI-L is not equivalent to OR-L. A read-only query Tool may be OR-L1, but if used for a high-stakes decision, its task error could be BI-L3.

4.5 Change Level: CH-L ​

CH-L describes the potential regression impact scope of a change, to determine the minimum depth of re-inspection, re-validation, and quality regression.

LevelMeaningTypical Changes
CH-L1Local, low-impact changeCopy, non-decision-affecting examples, low-risk format adjustments
CH-L2Change affecting a single artifact or limited execution pathPrompt rules, Tool parameters, Skill trigger condition adjustments
CH-L3Change affecting multiple artifacts, permissions, data flows, workflows, or critical behavioral pathsNew Tool, high-risk operation modification, retrieval strategy adjustment
CH-L4Systemic or infrastructure-level changeModel switch, Agent architecture adjustment, identity policy restructure, cross-Agent delegation change

CH-L is Core's unified change semantic. Which specific CH-L levels require re-execution of which sub-specifications is governed by Chapter 8 and the regression requirements of each domain-specific specification.

4.6 Static Audit Disposition Level: P0/P1/P2 ​

P0/P1/P2 is the unified disposition level of static auditing, covering both the Inspect and Risk Explicit domains. Specific defect levels, risk grades, and blocking rules remain governed by the corresponding sub-specification.

Reports MUST carry the domain prefix, written as Inspect·P0 or Explicit·P0; evidence and adjudication rationale MUST be recorded separately by domain.

LevelMeaningMinimum Disposition Principle
P0Blocking Defect or unacceptable Static RiskMUST be fixed. May only proceed under an Exception with explicit, time-limited, and compensating-control-backed approval, and only when the corresponding sub-specification and organizational risk policy explicitly permit exceptions. Undisposed P0 items SHALL NOT pass the corresponding Gate.
P1Major Defect or medium-to-high risk issueSHOULD be fixed before release. Where a reasonable exception exists, residual risk, disposition plan, approval, and expiration date MUST be documented.
P2Improvement Item, low-priority issue, or limited-applicability issueSHOULD be incorporated into an improvement plan or risk register. Whether it blocks release is determined by the domain-specific specification and business policy.

Core unifies only "P0 = block release." The disposition of P1/P2 is governed by each domain specification; the P1/P2 "Minimum Disposition Principle" above is retained as a general floor and MUST NOT be made stricter or looser.

P0 is not equivalent to:

  • Risk Implicit attack success status A;
  • OR-L3 or BI-L3;
  • Highest business impact or highest legal risk.

These may be correlated, but they MUST be recorded separately.

4.7 Risk Explicit Grading: Explicit·P0/P1/P2 + Trigger Condition Label ​

Risk Explicit grades each discovered risk item by its worst possible consequence into three fixed tiers, expressed with the domain prefix as Explicit·P0 / Explicit·P1 / Explicit·P2. When multiple grades match, the highest applies. A static audit MUST NOT downgrade a grade on the grounds that the risk "might not trigger"; in such cases the grade is unchanged and a "to-be-verified" tag is appended, handing reachability to Risk Implicit. A binary trigger condition label (conditional / unconditional) is used only to order risk items within the same grade. The full criteria table and determination rules are governed by Risk Explicit (see its "Severity Determination" part) and MUST NOT be restated here.

4.8 Risk Implicit Termination Status: A/B/C/D ​

A/B/C/D records the Termination Status of a dynamic attack attempt and the corresponding responsibility boundary.

StatusMeaningSupportable Conclusion Boundary
AAttack succeeded, or target risk behavior was observedDynamic risk evidence exists for this test case, artifact version, environment, and preconditions
BBlocked by external control or runtime guardrailThe external control is effective under this test case; does not automatically prove the Agent's own artifacts are safe
CAgent / model refused or did not comply with attack directivesTarget risk behavior was not observed in this test; does not automatically prove the absence of other bypass paths
DInsufficient conditions, environment unavailable, dependencies missing, or result indeterminableDoes not constitute a security pass or fail conclusion; missing conditions MUST be documented

A/B/C/D does not indicate severity, remediation priority, or quality level.

4.9 Composite Risk Profile Expression ​

For release, audit, or governance reports, different risk dimensions SHOULD be expressed side-by-side rather than compressed into a sourceless "aggregate risk level."

yaml
risk_profile:
  agent_complexity: AC-L3
  operation_risk: OR-L3
  business_impact: BI-L2
  change_level: CH-L3
  highest_inspect_priority: "Inspect·P0"
  highest_explicit_risk: "Explicit·P1"
  implicit_validation_status: B

If an organization requires an aggregate risk score, a separate scoring model — with weights, applicability conditions, versioning, and review mechanisms — MUST be defined. The multiple grading dimensions in Core MUST NOT be mechanically summed or automatically converted into one another.


Chapter 5: Unified Finding, Evidence, and Traceability Model ​

5.1 Definition of a Finding ​

A Finding is a traceable conclusion record produced against a specific object, version, environment, and rule or evaluation condition.

Findings may originate from:

  • Inspect static inspection;
  • Risk Explicit Explicit Risk Audit;
  • Risk Implicit dynamic attack validation;
  • Quality assessment;
  • Human review, incident post-mortem, or runtime monitoring.

A Finding is not the rule itself, not a risk level in itself, and not equivalent to a final release decision.

5.2 Minimum Finding Record ​

Every Finding eligible for entry into reports, Gates, Exception processes, regression, or audit scope MUST record at least the following fields:

yaml
finding_id: F-<organization>-<unique-id>
finding_type: inspect | explicit_risk | implicit_validation | quality | review
status: open | confirmed | mitigated | accepted | closed | invalid

source:
  standard: ""
  standard_version: ""
  rule_id: ""
  rule_version: ""

subject:
  agent_id: ""
  artifact_type: ""
  artifact_id: ""
  artifact_version: ""
  dependency_versions: []

execution_context:
  environment: ""
  model_or_runtime_version: ""
  evaluator_or_tool_version: ""
  test_asset_or_baseline_version: ""
  executed_at: ""

determination:
  summary: ""
  severity_or_result: ""
  applicability: applicable | not_applicable | uncertain
  confidence: low | medium | high
  review_status: unreviewed | reviewed | validated | rejected | needs_more_evidence

evidence:
  - evidence_id: ""
    evidence_type: ""
    location_or_reference: ""
    excerpt_or_hash: ""
    collected_at: ""

links:
  related_findings: []
  related_qd: []
  related_ex: []
  related_as: []
  related_fm: []
  related_exception: []

ownership:
  owner: ""
  reviewer: ""
  due_date: ""

Implementations MAY add organization-specific fields, but MUST NOT remove key fields that enable locating the object, version, evidence source, and basis of determination.

5.3 Finding Statuses ​

StatusMeaning
openGenerated; not yet confirmed or disposed.
confirmedConfirmed via the prescribed automated or human review process.
mitigatedFix or Compensating Control implemented; awaiting or undergoing regression confirmation.
acceptedValid Risk Acceptance or Exception approval obtained.
closedFix, regression, and closure conditions all satisfied.
invalidDetermined upon review to be a false positive, not applicable, or insufficiently evidenced to stand.

accepted does not mean the risk has disappeared. mitigated does not mean the control has been dynamically validated. closed MUST retain historical evidence and the basis for closure.

5.4 Evidence Tiers ​

Evidence TypeTypical ContentConclusions It Can SupportConclusions It Cannot Support Alone
Static EvidencePrompt fragments, Skill definitions, Schema, configurations, rule hitsStatic Defects, Explicit Risks, contract inconsistenciesDynamic exploitability, production impact has occurred
Dynamic EvidenceInvocation traces, request parameters, returned results, logs, state changesAttack results under specific test conditions, control interception, runtime behaviorPermanent safety across all scenarios
Quality EvidenceTest case inputs, actual outputs, scores, Judge results, metricsQuality results under specified Baselines and versionsComplete security or sole root cause
Control EvidenceIAM policies, server-side validation, approval records, gateway rules, audit configurationsControl existence, control design and configuration stateControl effectiveness across all attack scenarios
Human EvidenceReview opinions, approvals, root-cause analyses, Exception decisionsHuman confirmation, governance decisions, and responsibility attributionSubstitution for required technical or test evidence

5.5 Evidence Sufficiency and Conclusion Strength ​

Conclusion strength MUST match the evidence type:

text
Static Evidence
→ Can confirm "definition-layer defect or risk expression exists"

Dynamic Evidence
→ Can confirm "behavioral result observed under this environment, path, and precondition"

Quality Evidence
→ Can confirm "quality requirement met or not met under this evaluation set and threshold"

Control Evidence
→ Can confirm "control has been declared, configured, or deployed"

Multiple evidence types combined, with human review
→ Can support stronger judgments on root cause, fix effectiveness, or release decisions

No report SHALL use language exceeding the ceiling of the available evidence.

5.6 Confidence and Review Status ​

5.6.1 Confidence ​

ConfidenceMeaning
lowAutomated determination, insufficient evidence fragments, heavy reliance on assumptions, or presence of significant alternative explanations.
mediumEvidence substantially complete, but lacking human confirmation, dynamic validation, or critical context.
highEvidence sufficient, artifact and environment locatable, and reviewed or validated per the applicable process.

Confidence describes the credibility of the current determination, not the severity of the risk.

5.6.2 Review Status ​

StatusMeaning
unreviewedThe prescribed automated or human review has not yet been completed.
reviewedInitial review completed.
validatedFurther confirmed via applicable dynamic validation, quality assessment, or control testing.
rejectedDetermined upon review to be unfounded, a false positive, or not applicable.
needs_more_evidenceCannot confirm; requires supplementary version, environment, invocation chain, or test evidence.

5.7 Not Applicable: N/A ​

N/A can only indicate that the applicability preconditions for the current inspection item are not satisfied. It MUST NOT be used as a means to evade inspection, lower the release bar, or substitute for an Exception process.

Each N/A determination MUST at minimum record:

text
Rule or Inspection Item
+ Object and Version
+ Rationale for Non-Applicability
+ Applicability Assessor
+ Assessment Date
+ Re-evaluation Trigger Point upon Condition Change

If a determination cannot be made due to missing evidence, uncertain or needs_more_evidence SHALL be used. N/A MUST NOT be applied.


Chapter 6: Conclusions, Gates, and Release Decision Boundaries ​

6.1 Gate Independence ​

The Gates of Inspect, Risk Explicit, Risk Implicit, and Quality serve independent purposes and rest on independent evidence foundations.

GatePrimary BasisQuestion Answered
Inspect GateQD Findings, severity levels, static evidenceDo artifact definitions contain Blocking Defects?
Risk Explicit GateEX, risk levels, explicit evidenceDo artifacts explicitly express dangerous objectives, dangerous rules, or dangerous configurations?
Risk Implicit GateDynamic attack results, control status, environmental evidenceCan target risk behavior be triggered under controlled conditions?
Quality GateTest cases, metrics, thresholds, Baselines, and regression evidenceDoes actual task quality meet requirements?

The passing of any single Gate SHALL NOT automatically substitute for any other applicable Gate.

6.2 Release Summary Status ​

Organizations, platforms, or final reports MAY use the following composite release statuses, while preserving the original conclusions:

Composite StatusMeaning
PASSAll applicable mandatory Gates passed; no undisposed Blocking Defects exist.
PASS_WITH_EXCEPTIONAn approved, traceable, unexpired Exception exists; residual risk, Compensating Controls, and responsible parties are clearly identified.
FAILAny undisposed and unacceptable blocking conclusion exists, or a mandatory Gate has not passed.
NEEDS_REVIEWUnresolved conflicts exist in applicability, evidence, conclusions, or responsibilities, preventing a reliable release judgment.
ERRORArtifacts, versions, environments, dependencies, evaluators, or inputs are incomplete, rendering inspection or assessment incapable of valid execution.

This status is a release summary wrapper; it does not replace the original conclusions of any sub-specification.

For example:

yaml
release_summary:
  status: NEEDS_REVIEW
  source_results:
    inspect_gate: PASS
    risk_explicit_gate: PASS
    risk_implicit_gate: NEEDS_REVIEW
    quality_tool_agent_gate: PASS
  exceptions:
    - EXC-20260719-003

6.3 Minimum Evidence Requirements for Release Conclusions ​

A release summary MUST, at minimum, be able to answer:

  1. Which Agent and Logic Artifact versions are included in this release;
  2. Which sub-specifications are applicable, which were determined N/A, and the rationale;
  3. Which rules, evaluators, Baselines, knowledge snapshots, and environments were used;
  4. Which unclosed Findings exist, their severity, and their owners;
  5. Whether any Exceptions exist, along with Compensating Controls, approvers, and expiration dates;
  6. Which dynamic validations or quality regressions were not executed, failed, or had insufficient evidence;
  7. The final release decision-maker and decision time.

6.4 Constraints on PASS_WITH_EXCEPTION ​

PASS_WITH_EXCEPTION MAY only be used when all of the following conditions are simultaneously met:

  • The Exception has a unique EXC-* identifier;
  • The corresponding Finding, artifact version, residual risk, and business justification are traceable;
  • Compensating Controls have been assessed and documented;
  • The Exception has a clearly identified approver, expiration date, and review trigger conditions;
  • No legal, regulatory, contractual, or organizationally non-waivable security requirements are violated;
  • Dynamic attack success, major quality failures, or incomplete mandatory validations have not been simply converted into Exceptions to bypass disposition.

Whether P0 or dynamic attack success can be accepted as an Exception SHALL be separately governed by the organization's risk policy and the corresponding sub-specification. Core does not confer automatic exemption eligibility.

6.5 Conflicts and Insufficient Evidence ​

When different sub-specifications or different evidence sources produce superficially contradictory conclusions:

  1. Preserve original conclusions. Do not overwrite one result with another.
  2. Verify that the object, version, environment, and test preconditions are consistent.
  3. Identify differences in conclusion domains. For example, Static Defects, dynamic exploitability, and quality performance are not the same judgment.
  4. Establish linked Findings or root-cause investigation records.
  5. Until conflicts are resolved, the composite status SHALL be NEEDS_REVIEW, unless a domain-specific specification or organizational policy stipulates a stricter blocking requirement.

Example:

text
Inspect hits QD-T-3.2 (Raw Input Passthrough)
+ Risk Implicit yields B (external interception) in the current gateway environment

This indicates:

  • The Static Defect SHALL still be disposed per Inspect rules;
  • The gateway provided external protection evidence for the current attack test case;
  • The QD MUST NOT be deleted on this basis, nor the Agent's own artifacts declared free of risk;
  • Whether release can proceed depends on the corresponding Gates, Compensating Controls, business risk, and Exception policy.

6.6 Precedence in Sub-Specification Conflicts ​

When a conflict in expression exists between Core and a domain-specific sub-specification:

  1. For domain-specific rules, thresholds, scoring, defect definitions, and Gates: the domain-specific sub-specification governs.
  2. For terminology, object identification, version evidence, conclusion boundaries, and summary expression: Core governs.
  3. For conflicts that cannot be clearly classified: register as a consistency migration item and adjudicate via the specification maintenance process.
  4. Prior to adjudication, the historical adjudication results of existing rules MUST NOT be silently altered by an interpretation of Core.

Chapter 7: Cross-Subset Integration and Causal Boundaries ​

7.1 Standard Integration Closed Loop ​

The sub-specifications of SanityOps SHALL form the following closed loop:

text
Logic Artifacts and Their Versions
    ↓
Inspect: Discover Static Defects (QD)
    ↓
Relevance: Map candidate Attack Surfaces, Failure Modes, test and remediation directions
    ↓
Risk: Confirm Explicit Risks or dynamic behavioral evidence under controlled conditions
    ↓
Quality: Assess actual task, output, and service quality
    ↓
Human Review: Judge root cause, control effectiveness, remediation scope, and residual risk
    ↓
Fix, Re-Inspect, Regress, Release, or Exception Decision

This closed loop is designed to establish traceable candidate associations and validation paths — not to automatically derive a definitive conclusion in one sub-specification from the result of another.

7.2 Scope of Relevance ​

Relevance connects Inspect, Risk, and Quality, but its mapping conclusions constitute risk associations and validation recommendations.

Relevance can support:

  • Deriving candidate Attack Surfaces (AS) from QD items;
  • Recommending Risk Explicit audit directions from QD items;
  • Recommending Risk Implicit attack validation directions from QD items;
  • Deriving candidate quality Failure Modes (FM) from QD items;
  • Identifying Security Defect Chains (SC-*) that may jointly amplify risk;
  • Identifying Quality Chains (QC-TA-*, QC-RA-*) that may jointly affect task or content quality;
  • Recommending the validation and regression scope to be executed after remediation.

Relevance MUST NOT be used alone to support:

  • Claiming a QD as a successfully exploited vulnerability;
  • Claiming an AS as a confirmed security incident;
  • Claiming an FM as a confirmed quality root cause;
  • Directly converting an association mapping into a Risk or Quality FAIL;
  • Treating a Defect Chain as a confirmed complete attack chain or failure chain.

7.3 Prohibited Inferences ​

The following inferences are prohibited:

Invalid InferenceCorrect Expression
"A QD hit means a vulnerability exists.""A QD hit indicates a Static Defect exists, which may form a corresponding Attack Surface."
"An EX hit means the attack has already succeeded.""An EX hit indicates the artifact contains an Explicit Risk expression requiring audit."
"Risk B means the Agent is safe.""An external control intercepted under the current environment and test case."
"Quality FAIL means the Prompt is the root cause.""Quality FAIL may trigger a candidate root-cause investigation of the Prompt and associated artifacts."
"A control is configured, therefore it is effective.""Control configuration proves the control exists; effectiveness still requires applicable test or runtime evidence."
"One regression pass means the risk is permanently eliminated.""No related issues were observed within the scope of the current version, environment, test cases, and evidence."

7.4 Finding Association Types ​

Cross-subset Finding associations SHALL be annotated with a relationship type. Bare links that imply causation are not sufficient.

Relationship TypeMeaning
maps_toA rule or Finding maps to a candidate Attack Surface, Failure Mode, or test direction.
suggests_validationRecommends executing a specific Risk or Quality validation.
correlates_withCorrelated across the same object, version, environment, or test case, but causation is not confirmed.
candidate_root_cause_forIdentified as a Candidate Root Cause for a problem, pending confirmation.
confirmed_root_cause_forRoot cause confirmed through sufficient technical and human evidence.
mitigatesA control or fix is designed to reduce the risk of a specific Finding.
validated_mitigation_forControl or fix has been confirmed effective through applicable validation.
supersedesA Finding is superseded by a subsequent Finding due to artifact version or rule changes.
duplicatesTwo Findings are confirmed to be duplicate records of the same issue.

7.5 Single Defects, Defect Chains, and Common-Cause Issues ​

7.5.1 Single Defects ​

When an issue is independently identified by a single rule in a single specific artifact, and there is insufficient evidence of a cross-artifact amplification relationship, it SHALL be retained as an independent Finding.

7.5.2 Defect Chains ​

When multiple defects are mutually dependent within the same execution path, jointly amplify risk, or may form a risk path, a Defect Chain MAY be established.

Establishing a Defect Chain requires at minimum:

text
Chain Identifier
+ Associated Findings
+ Involved Artifacts and Versions
+ Connection Relationships
+ Potential Security or Quality Consequences
+ Evidence Level
+ Validation Status
+ Remediation Priority Order

A Defect Chain is not a replacement for its member Findings. Each member Finding SHALL still independently retain its rule, evidence, and disposition status.

7.5.3 Common-Cause Issues ​

When multiple Findings, upon analysis, may stem from the same design, configuration, process, or organizational cause, a "common-cause candidate" MAY be established.

For example:

text
Multiple Tools simultaneously lack input boundaries
→ Candidate common cause: Tool Schema design template does not require input trust boundary declarations

Prior to root-cause confirmation, candidate_root_cause_for SHALL be used. Original Findings MUST NOT be directly closed or merged.

7.6 Remediation and Regression Integration Principles ​

When remediating a Finding, the impact on associated objects SHALL be assessed:

Remediation LocationMinimum Integration Scope to Assess
PromptPrompt itself, associated Skills, output constraints, Quality regression
SkillTrigger conditions, permissions, failure strategies, associated Tools, Cross inspection
Tool / SchemaParameter contracts, backend implementation, permissions, approval, callers, dynamic validation
Knowledge / RetrievalKnowledge snapshots, citation quality, RAG evaluation, and contamination risk validation
Workflow / StateRouting, state transitions, retries, recovery, cross-artifact invocation chains
Identity / PolicyRoles, tenants, least privilege, approval, audit, and associated Tools
Runtime / GuardrailExternal interception effectiveness, degradation paths, whether the Agent's own defects persist

Remediation SHALL follow the principle of least-risk remediation: prioritize constraints at locations that can be reliably enforced, have clearly defined impact scope, and are auditable and verifiable. Prompt text supplementation SHALL NOT substitute for backend controls on high-risk operations.


Chapter 8: Versioning, Change, Baseline, and Compatibility ​

8.1 Version Objects Must Be Separated ​

SanityOps requires at minimum the following version objects to be distinguished:

Version TypeExampleDescription
Specification VersionInspect Tool v1.1The document version defining rules, terminology, and adjudication requirements.
Rule or Mapping Library Versionrelevance-map-1.0.0Machine-executable rule, detector, mapping, or template versions.
Artifact VersionGit commit, immutable hash, release package versionThe version of the Logic Artifact being inspected, validated, or assessed.
Runtime VersionModel version, Agent Runtime, framework, Connector versionThe runtime environment version influencing actual behavior.
Evaluation Asset VersionTest case set, Judge Prompt, scorer, threshold configuration versionThe evaluation assets underpinning Quality or Risk.
Data Snapshot VersionKnowledge base, index, memory data, policy data snapshotThe data version influencing retrieval, decision, and permission outcomes.

A single "version number" MUST NOT be used to vaguely represent all of the above.

8.2 Definition of a Baseline ​

A Baseline is a confirmed, reproducible, comparable, versioned result snapshot used to identify quality degradation, validation differences, control changes, or rule changes.

A valid Baseline requires at minimum:

text
Object and Artifact Version
+ Runtime Environment and Model Version
+ Knowledge or Data Snapshot
+ Rule, Evaluator, and Test Asset Versions
+ Original Results and Summary Conclusions
+ Generation Timestamp
+ Review or Confirmation Status

Historical results that lack records of critical environment or asset versions MAY serve as references, but SHALL NOT serve as strict regression Baselines.

8.3 Change Level Determination ​

Changes SHALL be assigned a CH-L based on the affected objects, behavioral boundaries, data flows, permissions, and external dependencies. The determination SHALL NOT be based solely on the number of changed files or the size of textual diffs.

The following circumstances are typically at least CH-L3:

  • Adding, replacing, or removing a high-risk Tool;
  • Expanding Tool permissions, data scope, tenant scope, or data exfiltration capability;
  • Modifying approval, confirmation, IAM, policy, or audit chains;
  • Modifying retrieval strategy, knowledge source scope, or memory write policy;
  • Modifying critical business processes, state machines, routing, or delegation rules;
  • Modifying the model, structured output mode, context assembly, or system-level Prompt;
  • Modifying external Connectors, MCP Servers, Webhooks, or Callbacks.

8.4 Minimum Re-Inspection and Regression Principles ​

Change LevelMinimum Requirement
CH-L1Assess applicable localized Inspect; where user-visible output is affected, execute relevant Quality smoke checks.
CH-L2Re-run Inspect, Cross inspection, and relevant Quality regression for affected artifacts; decide whether to supplement Risk based on risk assessment.
CH-L3Re-run Inspect, Cross, applicable Risk, and Quality regression for affected artifacts and dependency paths; review related Exceptions and Baselines.
CH-L4Re-perform system-level applicability assessment; execute the full, or an approved tailored, Inspect, Risk, and Quality suite; rebuild or re-confirm Baselines.

The above are minimum principles. Where OR-L, BI-L, regulatory requirements, or organizational policy are higher, the more stringent requirement SHALL apply.

8.5 Rule or Evaluator Changes ​

Changes to rule libraries, mapping libraries, detectors, Judge Prompts, scorers, thresholds, or test assets may alter the comparability of historical conclusions.

When such changes occur, the following SHALL be documented:

text
Changed Object and Version
+ Reason for Change
+ Expected Impact on Rules, Findings, or Metrics
+ Whether Historical Baselines Need Re-Running
+ Result Comparability Statement
+ Effective Date and Rollback Method

The following circumstances typically require re-evaluation of affected objects:

  • Addition or adjustment of P0 rules;
  • Fixing systematic false negatives or false positives in detectors;
  • Changing Quality Gate thresholds;
  • Changing core evaluation test cases, Judges, or scoring logic;
  • Changing attack test payloads, success criteria, or control attribution logic;
  • Changing impact strength or recommended validation paths in Relevance mappings.

8.6 Compatibility Principles ​

Compatibility relationships SHALL be maintained between sub-specifications, rule libraries, tool implementations, and report templates.

SituationCompatibility Handling
Only non-breaking fields, aliases, or clarifications addedBackward compatibility MAY be maintained.
Formal terminology changed but historical mapping preservedAlias cross-reference and migration notes SHALL be provided.
Rule semantics, scoring, thresholds, or Gates modifiedTreated as a substantive change; incompatible scope SHALL be identified.
Rule or identifier deprecatedIdentifier MUST NOT be reused; deprecation note, replacement rule, and historical result interpretation SHALL be preserved.
New tool version cannot interpret old resultsIncompatibility SHALL be explicitly declared; re-run recommendations or conversion tools SHALL be provided.

8.7 Document Dependencies and Applicable Versions ​

Each SanityOps sub-specification SHALL declare in its document information or appendix:

text
Dependent Core Version
+ Dependent Other Specification Versions
+ Corresponding Rule / Mapping Library Version
+ Whether Known Incompatibilities Exist

Before a complete compatibility matrix is established, reports SHALL at minimum record the actual versions of each specification used. A bare "conforms to SanityOps" statement is not sufficient.


Chapter 9: Governance Roles, Remediation, and Exceptions ​

9.1 Minimum Role Model ​

Organizations MAY merge roles according to their scale, but MUST NOT eliminate separation of duties, approval traceability, or review requirements.

RoleCore Responsibilities
Artifact OwnerMaintains Prompt, Skill, Tool, Workflow, Knowledge, and other artifacts; implements fixes and provides version evidence.
Agent / Application OwnerResponsible for the Agent's business objectives, architecture, dependencies, release scope, and lifecycle.
Security OwnerReviews security risks, control design, dynamic validation, Compensating Controls, and residual risk.
Quality OwnerMaintains quality test cases, Baselines, thresholds, regression, and quality conclusions.
Business OwnerConfirms business impact, acceptable risk, human handoff, and business continuity requirements.
Release OwnerMakes the release decision after confirming applicable Gates, Findings, Exceptions, and release summary status.
Exception ApproverApproves, rejects, renews, or revokes Risk Acceptances within authorized scope.
Specification / Platform OwnerMaintains rules, mappings, tool versions, audit records, and compatibility information.

A single person MAY hold multiple roles. However, for high-risk changes, P0, dynamic attack success, or high-business-impact scenarios, the Artifact Owner and Exception Approver SHALL NOT be the same person.

9.2 Remediation Responsibility and Closed Loop ​

Every confirmed Finding MUST have at minimum:

text
Owner
+ Disposition Path
+ Target Version or Completion Date
+ Fix or Compensating Control Description
+ Re-Inspection / Regression Requirements
+ Closure Criteria

Typical disposition paths include:

DispositionApplicable Circumstances
fixModify the artifact, backend implementation, policy, or configuration to eliminate the root cause.
mitigateAdd an auditable Compensating Control when the root cause cannot be immediately eliminated.
acceptAccept residual risk within authorized scope and time limits.
deferNon-blocking issue enters an explicit version plan; MUST NOT be used to bypass risks requiring immediate disposition.
invalidateConfirmed upon review to be a false positive, not applicable, or insufficiently evidenced.

After remediation is complete, a Finding SHALL NOT be closed based solely on code or configuration changes. Re-inspection, regression, or dynamic validation commensurate with the risk and Change Level MUST be executed.

9.3 Exceptions and Risk Acceptance ​

An Exception is a time-limited and traceable Risk Acceptance for a specific Finding or requirement. It is not a permanent exemption.

Each Exception requires at minimum:

yaml
exception_id: EXC-<organization>-<unique-id>
status: proposed | approved | rejected | expired | revoked | closed

scope:
  finding_ids: []
  rule_ids: []
  agent_id: ""
  artifact_versions: []
  environment: ""

risk:
  original_determination: ""
  residual_risk: ""
  business_justification: ""

controls:
  compensating_controls: []
  validation_evidence: []
  monitoring_requirements: []

governance:
  requested_by: ""
  reviewed_by: ""
  approved_by: ""
  approved_at: ""
  effective_from: ""
  expires_at: ""
  review_trigger: []

closure:
  remediation_plan: ""
  revalidation_requirement: ""
  closure_evidence: []

9.4 Minimum Conditions for Exception Approval ​

An Exception MUST simultaneously satisfy:

  1. Scope is clearly bounded to the Agent, artifact version, environment, and associated Findings;
  2. Original risk and residual risk are documented;
  3. The business or technical reason why immediate remediation is not possible is stated;
  4. Feasible Compensating Controls and their limitations have been identified;
  5. Owner, approver, expiration date, and early review trigger conditions are clearly defined;
  6. A remediation plan and post-expiration re-validation requirements are defined;
  7. No legal, regulatory, contractual, data protection, or organizationally non-waivable requirements are violated.

The following SHALL NOT be treated as "default exceptions":

  • Object or version cannot be located;
  • Minimum evidence is not available;
  • Exception has no expiration date;
  • Exception lacks authorized approval;
  • Repeated renewal substitutes for a remediation plan;
  • A bare Prompt declaration substitutes for executable controls required for high-risk operations.

9.5 Exception Review, Expiry, and Revocation ​

An Exception MUST be reviewed, and SHALL automatically become invalid if necessary, when any of the following occurs:

  • A substantive change to the associated artifact, model, Tool, permissions, knowledge source, or runtime environment;
  • A related security incident, quality incident, customer complaint, or audit finding;
  • A Compensating Control fails, is bypassed, or evidence cannot be obtained;
  • expires_at is reached;
  • Rules, regulatory requirements, or organizational risk policies change;
  • A CH-L3 or CH-L4 change affects the original Exception scope.

Upon expiry or revocation, the associated release status SHALL NOT continue to use PASS_WITH_EXCEPTION unless re-approved.

9.6 Risk Acceptance and Release Responsibility Boundaries ​

Risk Acceptance does not alter the factual Finding, nor does it reduce its original severity.

text
P0 Finding + Approved Exception
≠ P0 disappears

Explicit·P0 + Approved Exception
≠ Explicit·P0 is automatically downgraded; the grade remains P0 and the exemption is recorded

Dynamic attack success + Temporary Compensating Control
≠ The attack path has been permanently eliminated

Risk Acceptance only represents that an authorized entity accepts the residual risk within the specified scope, duration, and Compensating Control conditions. The Release Owner remains responsible for making an independent release decision based on the applicable Gates, business impact, control evidence, and Exception status.


Chapter 10: Implementation Maturity and Extension Roadmap ​

10.1 Implementation Principles ​

SanityOps MAY be adopted in phases. Organizations are not required to implement all capabilities at once in the initial phase, but they SHOULD:

  • Clearly state the scope of specifications currently adopted;
  • Document unadopted capabilities and their risk boundaries;
  • Prioritize more complete controls for high OR-L, high BI-L, or high CH-L scenarios;
  • Avoid representing "not yet built" as "passed" or "not applicable."

10.2 Implementation Phases ​

PhaseObjectiveMinimum Capabilities
Foundation GovernanceMake Logic Artifacts visible and locatableArtifact inventory, versioning, ownership, basic Inspect, Finding ledger
VerifiableMake critical risk and quality results reproducibleRisk controlled environment, Quality test cases, Baselines, regression records
GovernableMake release and risk decisions auditableUnified Findings, Gate summaries, Exceptions, approvals, and evidence retention
Sustainable OperationsClose the loop between change and production feedbackChange tiering, continuous regression, runtime monitoring, incident review, rule calibration
Scalable GovernanceCover complex enterprise Agent architecturesKnowledge, Workflow, Memory, Identity, A2A, Connector specialized specifications and controls

These phases are for construction guidance. They do not constitute an organizational maturity rating, certification level, or compliance statement.

10.3 Future Specialized Specification Extension Order ​

Subsequent Inspect or governance specifications are recommended to extend in the following order:

text
Inspect RAG / Knowledge Contract
→ Inspect Workflow / State
→ Inspect Memory
→ Inspect Identity & Policy
→ Inspect A2A / MCP / Connector
→ Runtime Monitoring, Production Audit, and Incident Review

The priority is based on: the prevalence of the object in enterprise Agents, its impact on quality and security, its dependency relationship with the existing Prompt–Skill–Tool system, and implementability.

10.4 Current Coverage Statement ​

All external materials, product interfaces, or evaluation reports SHALL accurately describe SanityOps' current coverage status:

  • Objects with existing dedicated specification coverage MAY be declared as inspected, validated, or assessed per the corresponding specification;
  • Objects without dedicated specification coverage MAY only be declared as incorporated into the unified object, version, and evidence model;
  • The fact that an object is defined in Core does NOT justify a claim that the object has been fully inspected or certified by SanityOps.

Appendix A: Core Terminology Quick Reference ​

TermDefinition
AgentAn application entity that uses a model to process input and may generate content, invoke tools, or execute tasks.
Logic ArtifactA versioned asset that defines, constrains, guides, invokes, orchestrates, authorizes, remembers, retrieves, or evaluates Agent behavior.
InspectThe specification domain for Static Defect Inspection of Logic Artifacts and their cross-artifact relationships.
Risk ExplicitThe specification domain for auditing Explicit Risk expressions, objectives, or configurations within Logic Artifacts.
Risk ImplicitThe specification domain for validating dynamic attack paths, risk behaviors, or control interception under controlled conditions.
QualityThe specification domain for assessing task, output, or service quality, threshold adjudication, and regression analysis.
RelevanceThe specification linking Inspect, Risk, and Quality through defect impact mapping and cross-subset integration.
QDA Static Logic Artifact Defect defined by Inspect rules.
EXAn Explicit Risk expression or dangerous configuration classification defined in Risk Explicit.
ASA candidate Attack Surface defined by Relevance.
FMA candidate behavioral Failure Mode defined by Relevance.
FindingAn instantiated discovery record produced against a specific object, version, environment, and rule.
ControlA technical or administrative measure for preventing, limiting, detecting, intercepting, auditing, or recovering from risk behaviors.
BaselineA confirmed, reproducible, comparable result snapshot.
RegressionComparative validation of a changed object against an established Baseline or expected result.
GateAn independent adjudication mechanism for whether a specific rule domain or evaluation domain meets entry criteria.
ExceptionAn approved, traceable, time-limited Risk Acceptance or rule exception.
Compensating ControlAn alternative or supplementary control for reducing residual risk when the root cause cannot be immediately eliminated.
Candidate Root CauseA possible root cause with associated evidence, but not yet sufficiently confirmed.
Shadow SandboxA fully isolated, high-fidelity sandbox in which an Agent's behavior is observed under controlled conditions — attack execution in Risk Implicit, quality evaluation in Quality — without affecting real data or business.
Ratchet MechanismA cross-round constraint mechanism ensuring fixed defects cannot silently recur and narrowed permissions cannot silently expand, so quality and security posture can only rise or hold; any relaxation must be explicit and approved.

Appendix B: Status, Level, and Grading Model Quick Reference ​

B.1 Grading Models ​

CodeMeaningWhat It Does NOT Indicate
AC-LAgent structural and collaboration complexityBusiness impact or vulnerability severity
OR-LOperational permission, side effect, and data riskAgent technical complexity
BI-LBusiness consequences of task errorTool operational permission risk
CH-LChange impact and regression scopeCurrent inherent system risk

B.2 Conclusions and Disposition Levels ​

Status / LevelDomainMeaning
P0/P1/P2Inspect, Risk ExplicitStatic audit disposition level
A/B/C/DRisk ImplicitDynamic attack test Termination Status
PASSRelease SummaryAll applicable mandatory Gates passed
PASS_WITH_EXCEPTIONRelease SummaryValid, approved Exception exists
FAILRelease SummaryUnacceptable blocking conclusion exists or mandatory Gate failed
NEEDS_REVIEWRelease SummaryUnresolved conflict in evidence, applicability, or conclusions
ERRORRelease SummaryInspection or assessment conditions incomplete, unexecutable, or invalid

B.3 Finding Lifecycle Statuses ​

text
open
  → confirmed
  → mitigated
  → closed

open / confirmed
  → accepted

open / confirmed
  → invalid

An accepted Finding continues to exist; its risk is accepted only within the approved scope and duration.


Appendix C: Unified Record Templates ​

C.1 Finding Record ​

yaml
finding_id: F-ORG-20260719-0001
finding_type: inspect
status: open

source:
  standard: Inspect Tool
  standard_version: "v1.1"
  rule_id: QD-T-3.2
  rule_version: "2.4.0"

subject:
  agent_id: agent-payment-ops
  artifact_type: tool_schema
  artifact_id: refund.execute
  artifact_version: "sha256:..."
  dependency_versions:
    - prompt: "sha256:..."
    - skill: "sha256:..."

execution_context:
  environment: staging
  model_or_runtime_version: "..."
  evaluator_or_tool_version: "..."
  test_asset_or_baseline_version: "..."
  executed_at: "2026-07-19T10:00:00Z"

determination:
  summary: "Refund amount parameter lacks enforceable server-side boundary constraints."
  severity_or_result: P0
  applicability: applicable
  confidence: high
  review_status: reviewed

evidence:
  - evidence_id: EV-001
    evidence_type: static_artifact
    location_or_reference: "refund.execute.parameters.amount"
    excerpt_or_hash: "..."
    collected_at: "2026-07-19T10:00:00Z"

links:
  related_findings: []
  related_qd: ["QD-T-3.2"]
  related_ex: []
  related_as: ["AS-..."]
  related_fm: []
  related_exception: []

ownership:
  owner: "payment-platform-team"
  reviewer: "security-reviewer"
  due_date: "2026-07-26"

C.2 Evidence Record ​

yaml
evidence_id: EV-001
evidence_type: static_artifact | dynamic_trace | quality_result | control_configuration | human_review
subject_reference:
  agent_id: ""
  artifact_id: ""
  artifact_version: ""
environment: ""
collected_at: ""
collector: ""
integrity:
  source_uri: ""
  content_hash: ""
  retention_reference: ""
summary: ""

C.3 Exception Record ​

yaml
exception_id: EXC-ORG-20260719-0003
status: approved

scope:
  finding_ids: ["F-ORG-20260719-0001"]
  rule_ids: ["QD-T-3.2"]
  agent_id: "agent-payment-ops"
  artifact_versions: ["sha256:..."]
  environment: "production"

risk:
  original_determination: "P0: Insufficient parameter boundary constraints"
  residual_risk: "Excess refund risk under specific anomalous invocation paths"
  business_justification: "Emergency fix window constrained"

controls:
  compensating_controls:
    - "Server-side refund cap validation"
    - "Dual approval"
    - "Refund audit alerting"
  validation_evidence:
    - "EV-..."
  monitoring_requirements:
    - "Daily anomalous refund monitoring"

governance:
  requested_by: ""
  reviewed_by: ""
  approved_by: ""
  approved_at: ""
  effective_from: ""
  expires_at: ""
  review_trigger:
    - "Tool or approval policy change"
    - "Anomalous refund event"

closure:
  remediation_plan: "v1.8 unifies Schema and server-side contract"
  revalidation_requirement: "Re-run Inspect Tool and Risk Implicit"
  closure_evidence: []

Appendix D: Document Dependency and Compatibility Matrix ​

This table expresses the logical dependencies established by Core. It does not replace the precise version compatibility matrices maintained by each sub-specification going forward.

Sub-SpecificationPrimary Core Content DependencyKey Relationships with Other Specifications
Inspect PromptObjects, Findings, AC-L, versioning and evidenceOutputs QD-P-* to Relevance
Inspect SkillObjects, Findings, OR-L, versioning and evidenceOutputs QD-S-* to Relevance
Inspect ToolObjects, Findings, OR-L, versioning and evidenceOutputs QD-T-* to Relevance; strong association with Risk / Quality
Inspect CrossObject relationships, AC-L, versioning and evidenceHandles Prompt–Skill–Tool contract and boundary issues
Inspect PermissionObjects, Findings, OR-L, AC-L, versioning and evidenceOutputs QD-PM-* to Relevance; subject to Gate-0 preconditions
Risk ExplicitExplicit·P0/P1/P2, Findings, evidence boundariesMay reference Inspect / Relevance risk signals
Risk ImplicitA/B/C/D, dynamic evidence, control boundariesConsumes Inspect / Relevance validation recommendations; outputs dynamic validation evidence
Quality Tool-AgentBI-L, Baselines, quality evidenceMay associate with FM-* and Candidate Root Causes
Quality RAG-AgentCH-L, Baselines, quality evidenceDepends on knowledge snapshots, evaluation assets, and change traceability
RelevanceUnified numbering, association types, causal boundariesConnects Inspect, Risk, and Quality
Control Assurance Standard (planned; name and numbering not yet frozen)Control, Evidence, Exception, dynamic validationWill define CA-* enforceability levels

Appendix E: Existing Specification Consistency Migration Checklist ​

This appendix guides patch-version revisions of existing documents. Except where explicitly marked as "requires rule adjudication," none of these items alter the existing rule definitions themselves.

IDMigration ItemDocuments InvolvedRecommended HandlingType
MIG-01L1/L2/L3 carries multiple meaningsInspect, Quality documentsAdd AC-L, OR-L, BI-L, or CH-L prefix on first occurrence; retain legacy expression mapping notesTerminology unification
MIG-02Risk IM and Risk Implicit coexistRisk, Relevance, reference documentsUnify formal name as Risk Implicit; retain historical abbreviation notesNaming unification
MIG-03Risk EX and Risk Explicit coexistRisk, Relevance, reference documentsUnify formal name as Risk ExplicitNaming unification
MIG-04Inspect Cross capitalization and abbreviation not fully consistentInspect Cross, RelevanceUnify as Inspect CrossNaming unification
MIG-05Quality failures may be described as "caused by" a Static DefectQuality, RelevanceChange to "candidate root cause," "associated Failure Mode," or "recommended investigation direction"; use confirmed root cause only when sufficient evidence existsCausal boundary
MIG-06Tool-Agent and RAG-Agent quality models not fully consistentQuality Tool-Agent, Quality RAG-AgentCore retains two independent models; cross-documents MUST NOT imply a shared metric systemScope clarification
MIG-07Artifact, rule, evaluator, and Baseline version recording granularity inconsistentAll documentsReference Core Chapter 8; supplement minimum version field requirementsEvidence unification
MIG-08PASS/FAIL/ERROR may be used at different levelsAll documentsClarify the relationship between original Gate conclusions and release summary status; MUST NOT substitute one for anotherStatus unification
MIG-09External interception may be misread as Agent's own securityRisk Implicit, RelevanceAdd cross-reference that "B only proves external control intercepted in the current test case"Conclusion boundary
MIG-10QD, EX, AS, FM and Finding instances may be conflatedInspect, Risk, Quality, RelevanceSupplement "rule definition vs. Finding instance" explanationObject unification
MIG-11Exception, remediation, regression, and Risk Acceptance fields scatteredRelevance, Quality, RiskSubsequently reference Core's Exception minimum record model uniformlyGovernance unification
MIG-12Knowledge, Memory, Workflow, and other objects may have descriptions but no dedicated rulesAll documentsMark as extension objects; MUST NOT be represented as fully coveredCoverage boundary
MIG-13Historical framework quantity or tier expressions such as "Six Subsets" may be outdatedFramework descriptions, overview materialsUnify to current specification register; express by capability domain rather than simple countFramework narrative
MIG-14Evidence levels for Defect Chains, Attack Chains, and Quality Chains may not be unifiedRelevanceAdopt association types and candidate causal boundaries; do not automatically upgrade mappings to factsEvidence unification
MIG-15P0/P1/P2 shared across Inspect and Risk Explicit may be misread as the same risk scaleInspect, Risk, Relevance, report templatesAdd domain prefix Inspect·/Explicit·; identify full prefix in reportsRisk semantics
MIG-16N/A rationale and review conditions not unifiedAll documents and tool implementationsAdopt Core Section 5.7 minimum requirementsApplicability governance

Appendix F: Cross-Subset Release Report Example ​

yaml
sanityops_release_report:
  report_id: SRR-ORG-20260719-001
  generated_at: "2026-07-19T12:00:00Z"

  subject:
    agent_id: "customer-service-agent"
    release_version: "2026.07.19-rc1"
    environment: "staging"

  profile:
    agent_complexity: AC-L3
    operation_risk: OR-L2
    business_impact: BI-L2
    change_level: CH-L3

  artifacts:
    - artifact_type: system_prompt
      artifact_id: "customer-service-main"
      version: "sha256:..."
    - artifact_type: skill
      artifact_id: "order-refund"
      version: "sha256:..."
    - artifact_type: tool_schema
      artifact_id: "refund.query"
      version: "sha256:..."

  applicability:
    inspect_prompt: applicable
    inspect_skill: applicable
    inspect_tool: applicable
    inspect_cross: applicable
    risk_explicit: applicable
    risk_implicit: applicable
    quality_tool_agent: applicable
    quality_rag_agent: not_applicable
    quality_rag_agent_reason: "Current version does not use retrieval-augmented pipeline"

  source_results:
    inspect:
      status: PASS
      open_p0: 0
      open_p1: 1
    risk_explicit:
      status: PASS
      highest_risk: "Explicit·P1"
    risk_implicit:
      status: PASS
      results:
        A: 0
        B: 2
        C: 5
        D: 0
    quality_tool_agent:
      status: PASS
      baseline: "tool-quality-baseline-2026.07.01"
      regressions: 0

  open_findings:
    - finding_id: F-ORG-20260719-014
      rule_id: QD-S-...
      priority: P1
      disposition: "planned_fix"
      due_date: "2026-07-30"

  exceptions: []

  release_summary:
    status: PASS
    decision_by: "release-owner"
    decision_at: "2026-07-19T12:10:00Z"
    rationale: "All applicable mandatory Gates passed; no open P0, no active Exceptions."

Closing Statement ​

The purpose of SanityOps Core is not to mask the differences between various risk types under a uniform label. It is to ensure that:

text
Different objects can be accurately identified
Different conclusions can be correctly understood
Different evidence can be reliably traced
Different Gates can be independently executed
Different risks can be explicitly disposed
Different versions can be continuously compared

Only when the common objects, terminology, evidence, and decision boundaries are stable can Inspect, Risk, Quality, Relevance, and the specialized specifications to come together form an enterprise Agent governance framework that is realizable, auditable, and sustainably evolvable.


© 2026 Sanity AI Labs. Licensed under Creative Commons Attribution-ShareAlike 4.0 International. https://www.sanityops.org · hello@sanityops.org v1.1 · October 2026