SanityOps Framework
Inspect Skill Specification
Version: v1.1
Release Date: October 2026
Maintained by: Sanity AI Labs
License: CC BY-SA 4.0
Depends on: Core v1.1 (objects, Finding, OR-L, versioning and evidence)
Table of Contents
- Foreword
- Part 1: Quality Defect Classification System Overview
- Part 2: QD-S-1 Resource Runaway (Boundary Absence)
- Part 3: QD-S-2 Unbounded Directive
- Part 4: QD-S-3 Ambiguity-Induced Inference
- Part 5: QD-S-4 Failure Escalation
- Part 6: QD-S-5 Permission Overflow
- Part 7: Skill Tiered Inspection Mechanism
- Part 8: Review Schema — Detailed Review Dimensions
- Part 9: Usage Examples
- Part 10: Relationship to Other Sub-Specifications
- Part 11: Quantitative Scoring Mechanism
- Part 12: New Check Items (Candidates)
- Appendix A: QD-S Complete Numbering Index
- Appendix B: OR-L1/OR-L2/OR-L3 Minimum Rule Sets
- Appendix C: Glossary
- Part Summaries
Foreword
Document design principle: serving both inspection and audit needs.
To reduce the risk of missed findings and to preserve traceable audit evidence, this document allows the same object to be hit by multiple inspection items across different dimensions — this may look like "similarity / duplication / subsumption" on the surface, but it is intentional by design, not redundancy.
At the quantitative scoring stage, however, to avoid double deductions and to reflect the true quality of the inspected Logic Artifact objectively, same-source deduplication and merging must follow the "Multi-Defect Handling Rules" (§11.6) of this document: one root cause and one harm path are deducted only once.
If you question whether certain inspection items are "overly redundant", verify consistency with counter-examples and edge cases — confirm whether the trigger conditions truly overlap and whether the attribution reaches consistent boundary conclusions — rather than concluding from surface similarity alone.
0.1 Positioning and Scope
0.1.1 What This Specification Is
This specification is one of the core sub-specifications of the SanityOps Framework's Inspect Standard. It defines the quality defect inspection standards for AI Agent Skills.
This specification is designed to:
- Identify quality defects unintentionally introduced by developers in Agent Skill definitions
- Provide a unified inspection framework for Skill inspection, review, and governance
- Provide explicit rule definitions for automated inspection tooling (the Inspect Skill inspection Agent)
Note: Defect determination is based on observable artifact defects (incomplete, ambiguous, unbounded, or inconsistent definitions). Whether the developer acted intentionally cannot be determined statically and is not a condition for finding a defect (see framework/core.md §5.1, Finding model).
0.1.2 What This Specification Is Not
This specification is not:
- A Skill development guide: it does not prescribe how to design "good" Skills
- An attack detection specification: it does not cover EX (Explicit attack risk) or IM (Implicit logic risk)
- A runtime monitoring specification: it does not address behavioral monitoring during Skill execution
- A permission management specification: it does not replace system-level permission control mechanisms
0.1.3 Position Within the SanityOps Framework
SanityOps Framework
│
├─ Inspect (Defect Inspection)
│ ├─ Inspect Tool ← Tool Schema defect inspection
│ ├─ Inspect Prompt ← System Prompt defect inspection
│ ├─ Inspect Skill ← Skill defect inspection ← You are here
│ ├─ Inspect Cross ← Cross-Artifact defect inspection
│ └─ Inspect Permission ← Permission-responsibility proportionality inspection (QD-PM)
│
├─ Risk (Risk Scanning)
│ ├─ Risk Explicit ← Explicit Logic Artifact risk detection
│ └─ Risk Implicit ← Implicit Agent runtime vulnerability scanning
│
├─ Quality (Service Quality)
│ ├─ Quality RAG-Agent ← RAG-Agent service quality assessment
│ └─ Quality Tool-Agent ← Tool-Agent service quality assessment
│
└─ Relevance (Impact Mapping)
└─ Relevance ← defect → risk/quality diagnostic mapping0.1.4 Inspection Scope
This specification's inspection scope includes:
Inspection Objects:
- Skill natural-language descriptions
- Skill name, purpose, and capability boundaries
- Skill input, execution, and output constraints
- Skill trigger conditions and failure handling
- Skill permission declarations and tool invocations
Core Inspection Dimensions:
├─ QD-S-0: Baseline Compliance Check (entry gate)
│
├─ QD-S-1: Resource Runaway (Boundary Absence)
├─ QD-S-2: Unbounded Directive
├─ QD-S-3: Ambiguity-Induced Inference
├─ QD-S-4: Failure Escalation
└─ QD-S-5: Permission Overflow0.1.5 Out of Scope
This specification does not cover:
- Runtime parameter value validation for Skills
- Skill execution process monitoring and logging
- System-level permission control implementation
- Skill functional completeness testing
0.2 Terminology and Numbering System
0.2.1 Core Terminology
| Term | Definition |
|---|---|
| Skill | A functional unit of an AI Agent, defined by its name, description, and input/output constraints |
| Quality Defect (QD-S) | An incomplete, ambiguous, or unbounded definition unintentionally introduced by the Skill developer |
| Review Schema | An intermediate review framework (6 dimensions) used by Inspect Skill to decompose and analyze Skill definitions |
| Gap Analysis | Dimension-by-dimension inspection of Skill definition gaps against the Review Schema |
| LLM Default Gap-Filling Behavior | The LLM's tendency to fill behavioral gaps with its own judgment when a Skill leaves scenarios undeclared |
| Inferential Verification | Static reasoning to demonstrate defect risk and remediation effectiveness |
| EX | Explicit Risk — explicit attack risk |
| IM | Implicit Risk — implicit logic risk |
OR-L1/OR-L2/OR-L3 | Skill operational risk levels (Low/Medium/High) |
| P0/P1/P2 | Defect Levels (Blocking/Warning/Advisory) |
0.2.2 Numbering System
This specification uses the QD-S-x.y numbering system:
QD-S-x.y
├─ QD: Quality Defect
├─ S: Skill
├─ x: Defect category number
│ ├─ 0: Baseline Compliance Check
│ ├─ 1: Resource Runaway
│ ├─ 2: Unbounded Directive
│ ├─ 3: Ambiguity-Induced Inference
│ ├─ 4: Failure Escalation
│ └─ 5: Permission Overflow
└─ y: Specific defect item numberExamples:
QD-S-1.1: Resource Runaway → Input Boundary MissingQD-S-3.3: Ambiguity-Induced Inference → Missing Ambiguous Input Handling Strategy
0.2.3 Skill Operational Risk Levels (OR-L)
【Global notation convention · Mandatory】 Throughout this document's body text, tables, examples, and appendices,
L1/L2/L3are always shorthand forOR-L1/OR-L2/OR-L3(Skill operational risk levels) and are fully equivalent. Except in this definition section, theOR-prefix is not repeated at each occurrence. Any standaloneL1/L2/L3must not be read asAC-L(Agent complexity level) or any other grading model. This convention applies with full force to every chapter, appendix, example, and report field in this document.
| Risk Level | Designation | Definition | Typical Characteristics |
|---|---|---|---|
| Low Risk | OR-L1 | Text-processing only; no tool invocations | Weekly report compilation, text polishing, translation |
| Medium Risk | OR-L2 | Resource access; may involve sensitive data | Data queries, knowledge retrieval, email reading |
| High Risk | OR-L3 | Operation execution; irreversible operations | Data deletion, payments, permission changes |
0.2.4 Defect Levels (P)
| Defect Level | Symbol | Definition | Release Recommendation |
|---|---|---|---|
| P0 | 🔴 | Blocking: a directly realizable resource runaway or behavioral overflow path exists | MUST NOT release; MUST be fixed |
| P1 | 🟡 | Warning: defect impacts behavioral quality; predictable unintended behavior exists | SHOULD be fixed before release |
| P2 | 🟢 | Advisory: minor specification compliance issue | SHOULD be included in the improvement plan or risk register |
0.3 Version and Maintenance Information
0.3.1 Current Version
- Version: v1.1
- Release Status: Released
- Last Updated: October 2026
- Maintained by: Sanity AI Labs
0.3.2 Version History
| Version | Release Date | Major Changes |
|---|---|---|
| v1.0 | 2026-10 | Initial release |
| v1.1 | 2026-10 | Expanded Inspect and Permission defect coverage; Risk Explicit regraded to P0/P1/P2; multi-hit design principle clarified; terminology aligned to Logic Artifacts |
0.4 QD-S-0 Baseline Compliance Check (Entry Gate)
QD-S-0 is the entry gate for Skill inspection and runs before the semantic inspection of QD-S-1 through QD-S-5. It verifies whether the Skill definition satisfies the platform's minimum structural specification — that is, "can it be parsed and inspected" — and does not assess semantic quality. QD-S-0 results are only "pass / fail" and are not counted in the Part 11 quantitative score (see 11.3).
| ID | Inspection Item | Definition | Inspection Method |
|---|---|---|---|
| QD-S-0.1 | Required Field Presence | Whether platform-required fields such as name and description are present | Automated |
| QD-S-0.2 | Field Content Non-Empty | Whether required field content is non-empty and not whitespace-only | Automated |
| QD-S-0.3 | Syntax Parsability | Whether the definition format (YAML frontmatter, Markdown structure, etc.) is parsable | Automated |
| QD-S-0.4 | Field Type Validity | Whether field value types are valid (enum, boolean, numeric, list, etc.) | Automated |
| QD-S-0.5 | No Conflicting Declarations | Whether the Skill body, examples (few-shot), failure strategy, and permission declarations are mutually consistent and non-conflicting | Automated (candidate) → human review |
| QD-S-0.6 | Minimum Structural Completeness | Whether a locatable minimum structure exists (referenceable positions, inspectable units) | Automated |
| QD-S-0.7 | Dependency and Resource Presence | Whether referenced scripts, templates, and downstream Skills exist and are parsable (credentials/callback URLs/PII belong to Risk Explicit; see Note 1) | Automated |
| QD-S-0.8 | Metadata Completeness | Whether version, owner, and changelog are complete and traceable (configurable as a platform-level requirement) | Automated |
| QD-S-0.9 | Description Length Budget | Whether description exceeds the token budget ceiling (SHOULD; complementary to QD-S-3.6) | Automated |
Note 1: QD-S-0.7 checks reference existence only. Hardcoded credentials/keys, anomalous callback URLs, and hardcoded PII belong to Risk Explicit's NR-S-9 / NR-S-7 / NR-S-10 (see risk/explicit.md §2.2). This subset does not build its own secret-scanning rules; when such content is found, cross-reference the EX ID instead of recording it under QD-S-0.
Determination and Failure Handling (relaxed from the original v1.0 rule "any failed item stops all inspection"):
- Only when the Skill definition is completely unparsable (QD-S-0.3 fails) is QD-S-1 through QD-S-5 inspection stopped;
- Other QD-S-0 defects (missing fields, empty content, local conflicts, etc.) do not block subsequent semantic inspection, but the report MUST mark "baseline compliance not passed" and list the defective items;
- In that case the formal evaluation conclusion is "evaluation incomplete (NEEDS_REVIEW)" (see framework/core.md §6.2, release summary status); confirmed QD-S defects MUST still be reported item by item.
Note: QD-S-0.5 is the formal definition of the "Skill internal consistency" check. When the inspector finds a conflict between body and examples, or between failure strategy and permission declarations, it MUST report both sides of the conflict and must not silently select the broader or stricter side.
Part 1: Quality Defect Classification System Overview
1.1 Five Defect Classification Framework
Inspect Skill v1.1 classifies defects by risk principle rather than structural field, comprising five defect categories:
| Category | ID | Risk Principle | Typical Manifestation |
|---|---|---|---|
| Resource Runaway | QD-S-1 | Execution boundary constraints are missing; the LLM infers scope on its own, and the inference is typically broader | No input limit, no output size limit, no retry limit |
| Unbounded Directive | QD-S-2 | Unbounded language is actively included, directly driving the LLM toward unbounded execution | Uses words like all/every/keep trying without corresponding limits |
| Ambiguity-Induced Inference | QD-S-3 | Descriptions are vague; the LLM tends toward "reasonable assumptions" rather than stopping, producing overflow behavior | Overly broad trigger conditions, undeclared default behavior, inaccurate metadata |
| Failure Escalation | QD-S-4 | Failure handling is undefined; the LLM's "complete the task" objective drives it to self-expand strategies | Undefined failure behavior, forced completeness, missing prohibited behaviors |
| Permission Overflow | QD-S-5 | Permission boundaries are unclear; the LLM interprets broad capability descriptions as implicit authorization | Read/write/delete not distinguished, high-risk operations lack confirmation, environment boundaries missing |
Part 2: QD-S-1 Resource Runaway (Boundary Absence)
2.1 Classification Positioning
Risk Principle: The Skill does not declare critical constraint boundaries (input size, output size, execution resources, data sources). The LLM, in the absence of boundaries, infers the execution scope on its own, and the inferred result is typically broader than the developer intended, causing resource runaway.
Typical Harms: API cost explosion, Token Consumption runaway, execution timeout, excessive system load.
2.2 Inspection Items in Detail
Note: Numeric values in this section's examples (such as "50 items", "2,000 words", "60 seconds") are illustrative values that must be calibrated to the business scenario; they are not normative thresholds.
QD-S-1.1 Input Boundary Missing
Defect Description: The Skill does not declare an upper limit on the number of input items or a size constraint, allowing unbounded input.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Organize work notes submitted by the user and generate a weekly report." (no maximum item count declared) |
| Problem | User pastes 1,000 records; the LLM attempts to process all of them |
| Default Behavior | Token Consumption runs away; costs accumulate |
Remediation Example:
Organize work notes submitted by the user and generate a weekly report.
- Input limit: Process up to 50 work notes
- When exceeding 50: Process the first 50 and notify the userSeverity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🔴 P0
- L3 (High Risk): 🔴 P0
Determination criterion: LLMs cannot reliably count, and the context window and max_tokens are already hard limits. P0 is retained only for "cost-amplifying scenarios" (loop processing, batching, per-item tool calls); otherwise downgrade to P1, and N/A where platform budget evidence exists.
QD-S-1.2 Output Size Missing
Defect Description: The Skill does not declare an upper limit on output size, potentially causing output bloat or Context Window overflow.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Generate a detailed summary of user documents." (no summary length declared) |
| Problem | Ten 50KB documents → LLM generates a 500KB summary |
| Default Behavior | Response body explodes; Context Window overflows |
Remediation Example:
Generate summaries of user documents.
- Single document summary: 200–500 words
- Multiple documents: Generate a consolidated summary, total length not exceeding 2,000 words
- When exceeding: Generate a table-of-contents-style summary with one sentence per documentSeverity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🟡 P1
Determination criterion: output size is likewise bounded by the context window and max_tokens hard limits. P0 is retained only for "cost-amplifying" scenarios (looped output, batch generation); otherwise P1.
QD-S-1.3 Execution Resource Boundary Missing
Defect Description: The Skill involves tool invocations or retries, but does not declare retry count, tool invocation count, or execution timeout.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Diagnose database issues. Keep trying until the problem is resolved." |
| Problem | "Keep trying" has no termination condition |
| Default Behavior | Infinite retries, infinite tool invocations; costs run away |
Remediation Example:
Diagnose database issues.
- Maximum query count: 5
- Single query data volume: Not exceeding 1,000 records
- Execution timeout: 60 seconds
- On timeout: Return the diagnosed content so far; inform the user that manual handling is requiredSeverity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🔴 P0
- L3 (High Risk): 🔴 P0
Determination criterion: timeouts and call caps are the Harness's responsibility; LLMs have no clock perception, so prose is only a soft constraint. Revised to "declaring it in either the Skill text or a traceable in-artifact configuration suffices"; with no source at all, L2/L3 retain P0; with configuration evidence, downgrade to SHOULD/N/A (see the effective-constraint clause in 7.5).
QD-S-1.4 Permission Boundary Missing
Defect Description: The Skill does not explicitly distinguish between read/write/delete permission types, or does not declare the types of operations that are prohibited.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Assist with database issues." ("assist" may include read, write, delete) |
| Problem | The LLM interprets broad capability descriptions as implicit authorization |
| Default Behavior | The LLM executes delete operations (as in the Replit incident) |
Remediation Example:
Diagnose database issues. This Skill provides read-only diagnosis only.
- Allowed: View error messages, analyze table structures, inspect query performance
- Prohibited: UPDATE, DELETE, INSERT, DROP, ALTER, data migration, data cleanup
Irreversible operations MUST be confirmed by a human before execution.Severity:
- L1 (Low Risk): 🟢 P2
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🔴 P0
Determination criterion: same root as QD-S-2.4 and QD-S-5.1 ("operational permission boundary undeclared"); deduplicated into 5.1 and not scored independently (see 11.6).
QD-S-1.5 Data Source Boundary Missing
Defect Description: The Skill does not declare permitted or prohibited data sources; the LLM may collect data from conversation history or external systems on its own.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Query user information." (no permitted data sources declared) |
| Problem | The LLM may collect information from conversation history, public databases, or other systems |
| Default Behavior | Data leaks; unauthorized access |
Remediation Example:
Query user information.
- Use only content explicitly provided by the user in the current request
- Prohibited: Inferring from conversation history, pulling from external systems, collecting from default configurations
- If the needed data is not provided by the user, ask the user rather than inferringSeverity:
- L1 (Low Risk): 🟢 P2
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🔴 P0
Determination criterion: revised to "declare the authorized data sources and purposes", distinguishing authorized from unauthorized history and not treating use of conversation history itself as a defect. When inferred or completed data enters write operations, external calls, or decision output, L3 rises to P0; display-only remains P1.
2.3 QD-S-1 Inspection Checklist
| ID | Inspection Item | Trigger Keywords | L1 | L2 | L3 | Inspection Method |
|---|---|---|---|---|---|---|
| QD-S-1.1 | Input Boundary Missing | accept, process, organize | P1 | P0 | P0 | Automated |
| QD-S-1.2 | Output Size Missing | generate, output, report | P1 | P1 | P1 | Automated |
| QD-S-1.3 | Execution Resource Boundary Missing | keep trying, until success, ensure complete | P1 | P0 | P0 | Automated |
| QD-S-1.4 | Permission Boundary Missing | manage, handle, assist | P2 | P1 | P0 | Human |
| QD-S-1.5 | Data Source Boundary Missing | collect, infer, supplement | P2 | P1 | P0 | Semi-automated |
Part 3: QD-S-2 Unbounded Directive
3.1 Classification Positioning
Risk Principle: The Skill actively includes directives or enumerations that are semantically unbounded, directly driving the LLM toward unbounded execution or unbounded expansion. These defects are not about "what is missing" — they are about "what was written that shouldn't have been."
Note: QD-S-2.1/2.2/2.4 are automated trigger-word lists for the corresponding missing-constraint defects (QD-S-1.1/1.3/5.1); they produce candidate findings only, are deduplicated into the primary items, and are not scored independently. QD-S-2.3 is an independent defect item.
Typical Harms: Infinite retries, infinite output expansion, automatic expansion of execution scope.
3.2 Inspection Items in Detail
QD-S-2.1 Unbounded Quantity Declaration
Defect Description: Uses unbounded words such as all / every / entire / complete without a corresponding quantity limit.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Process all work notes and generate a complete weekly report." |
| Problem | "All" and "complete" have no quantity constraints |
| Default Behavior | The LLM pursues exhaustive processing; Token runs away |
Remediation Example:
Process work notes provided by the user (up to 50) and generate a weekly report.
- If more than 50, process the first 50 and notify the userSeverity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🔴 P0
- L3 (High Risk): 🔴 P0
Determination criterion: "list all fields in the table" is a finite set, not unbounded. Judge a defect only when "the scope is expanded by the LLM itself and there is no external budget"; keywords are downgraded to candidate findings. This item serves as the automated trigger-word list for QD-S-1.1 "Input Boundary Missing"; same root as 1.1, deduplicated into 1.1, and not scored independently (see 11.6).
QD-S-2.2 Unterminated Execution Declaration
Defect Description: Uses words such as keep trying / until success / ensure complete / persist until without defining a termination condition.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Try to fix the database issue until the problem is resolved." |
| Problem | "Until success" has no failure threshold and no retry limit |
| Default Behavior | The LLM enters an infinite retry loop; costs run away |
Remediation Example:
Diagnose database issues and recommend remediation plans (not auto-executed).
- Maximum 3 rounds of diagnosis
- If the issue is not resolved after 3 rounds, return the diagnosed content so far and recommend human handlingSeverity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🔴 P0
- L3 (High Risk): 🔴 P0
Determination criterion: this item serves as the automated trigger-word list (until success, etc.) for QD-S-1.3 "Execution Resource Boundary Missing"; same root as 1.3, deduplicated into 1.3, and not scored independently (see 11.6).
QD-S-2.3 Open-Ended Enumeration in Structure Declaration
Defect Description: Output or functional structures use open-ended enumerations such as "etc.", "and others", "such as", allowing the LLM to add items on its own.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "The weekly report should include: completed work, progress, problems, etc." |
| Problem | "etc." implies additional sections may be added |
| Default Behavior | The LLM adds unrequested sections such as "risk analysis," "personal growth," "team recommendations" |
Remediation Example:
The weekly report should include the following sections (and only these):
1. Completed Work
2. Progress
3. Problems or Blockers
4. Next Week's Plan
Do not add additional sections unless explicitly requested by the user.Severity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🟡 P1
Determination criterion: judge a defect only when the enumerated object is a closed contract (machine-parsed output structure, resource/operation whitelist); do not judge defects for analytical processes or illustrative examples. The severity ceiling is P1; keywords only produce candidate findings.
QD-S-2.4 Broad Capability Verbs
Defect Description: Uses broad verbs such as manage / handle / coordinate / automate / process / address without explicitly excluding application scenarios via non_goals.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Assist with employee payroll-related matters." ("handle" may include modification) |
| Problem | The LLM infers that "handling payroll matters" includes modifying payroll |
| Default Behavior | The LLM executes overflow operations |
Remediation Example:
Assist with analyzing employee payroll issues. This Skill is for data query and issue analysis only.
- Allowed: View payroll data, analyze discrepancies, diagnose issues
- Prohibited: Modify payroll data, execute payments, adjust configurations
All recommendations require human confirmation before execution.Severity:
- L1 (Low Risk): 🟢 P2
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🔴 P0
Same root as QD-S-1.4 and QD-S-5.1; broad verbs serve only as a trigger cue for "operational permission boundary undeclared". See 11.6 for deduplication.
3.3 QD-S-2 Inspection Checklist
| ID | Inspection Item | Trigger Keywords | L1 | L2 | L3 | Inspection Method |
|---|---|---|---|---|---|---|
| QD-S-2.1 | Unbounded Quantity Declaration | all, every, entire, complete | P1 | P0 | P0 | Automated |
| QD-S-2.2 | Unterminated Execution | keep trying, until success, ensure | P1 | P0 | P0 | Automated |
| QD-S-2.3 | Open-Ended Enumeration | etc., and others, such as | P1 | P1 | P1 | Automated |
| QD-S-2.4 | Broad Capability Verbs | manage, handle, coordinate | P2 | P1 | P0 | Human |
Part 4: QD-S-3 Ambiguity-Induced Inference
4.1 Classification Positioning
Risk Principle: The Skill description contains semantic ambiguity or leaves critical scenarios undefined. When facing uncertainty, the LLM tends toward "reasonable assumptions" rather than stopping to ask, producing behavior that exceeds expectations.
Typical Harms: Mis-triggering, auto-completion of missing content, task scope expansion.
4.2 Inspection Items in Detail
QD-S-3.1 Vague Trigger Conditions
Defect Description: Activation conditions are overly broad; multiple unrelated intents can trigger the Skill.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Use this Skill when the user needs help." |
| Problem | "Needs help" is too broad; almost any request can trigger it |
| Default Behavior | The Skill is mis-triggered in inappropriate scenarios |
Remediation Example:
Use only when the user explicitly requests "compile a weekly report" or "generate a weekly report."
- Trigger condition: The user provides work notes and requests a weekly report
- Do not trigger: The user only asks about the weekly report format, the user is writing a personal journal, the user is discussing work contentSeverity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🔴 P0
QD-S-3.2 Missing Deactivation Conditions
Defect Description: Does not declare when the Skill should not be triggered, causing the LLM to forcibly activate the Skill in edge scenarios.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Handle user data requests." (does not state when to refuse) |
| Problem | The LLM may forcibly activate for privacy data requests, system data requests, etc. |
| Default Behavior | Unauthorized access or privacy leaks |
Remediation Example:
Handle user data requests.
- Scenarios NOT to handle:
- Privacy data not explicitly authorized by the user
- Data involving other users
- System configuration or backend data
- Production environment sensitive data
- Such requests should be refused with an explanation of whySeverity:
- L1 (Low Risk): 🟢 P2
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🔴 P0
When the positive trigger conditions are already sufficiently bounded, a counter-example list is not mandatory; this item may be merged with QD-S-3.4 as a "boundary declaration", or applied only to Skills with automatic model routing. See 11.6 for deduplication.
QD-S-3.3 Missing Ambiguous Input Handling Strategy
Defect Description: Does not declare how to handle incomplete or malformed input; the LLM tends to auto-complete or infer.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Organize work notes." (does not state how to handle insufficient notes) |
| Problem | The LLM may supplement work content the user did not provide in order to complete the weekly report |
| Default Behavior | Fabricated content, false data |
Remediation Example:
Organize work notes and generate a weekly report.
- If notes are insufficient: Stop and inform the user that more information is needed rather than generating false content
- If information for a section is missing: Skip that section and explain why, rather than generating examples
- Prohibited: Fabricating facts, inferring missing content, adding example dataSeverity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🔴 P0
QD-S-3.4 Undefined Task Scope
Defect Description: Missing non_goals declaration; does not clearly state what the Skill does NOT do, causing the LLM to expand task boundaries on its own.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Help users improve their weekly reports." (does not state what it does NOT do) |
| Problem | The LLM may interpret this as permission to modify content, add analysis, improve expressions, etc. |
| Default Behavior | Task scope exceeds expectations |
Remediation Example:
Help users improve their weekly reports.
- What it does: Format, polish expression, organize structure
- What it does NOT do:
- Change facts or data
- Add content the user did not provide
- Modify the work content itself
- Conceal negative information or problemsSeverity:
- L1 (Low Risk): 🟢 P2
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🟡 P1
Escalation condition: only when the positive purpose is insufficient to bound the scope and the Skill has side-effect capability does L3 rise to P0. "Purpose sufficiently closed" counts as passing; a negative list (non_goals) is not mandatory.
QD-S-3.5 Undeclared Default Behavior
Defect Description: Critical branches (empty input, format error, insufficient data, etc.) have no defined handling.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Generate a weekly report." (does not state how to handle empty input) |
| Problem | When the user provides no work notes, the LLM may generate an example weekly report |
| Default Behavior | False data, fabricated content |
Remediation Example:
Generate a weekly report.
- On empty input: Ask the user rather than generating an example
- On format error: Point out the error and request correction
- On insufficient data: Explain what information is missing and suggest the user supplement itSeverity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🔴 P0
QD-S-3.6 Metadata Expression Defects
Defect Description: The name or description is overly broad, inaccurate, or inconsistent with the body, causing the LLM to misunderstand the Skill's capabilities at the trigger stage.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | name: "database-manager" description: "Read-only diagnosis" |
| Problem | name implies database management capability; description says read-only → contradiction |
| Default Behavior | The LLM infers capabilities from the name and executes write operations |
Remediation Example:
name: "readonly-database-diagnostic-reporter"
description: "Use this Skill for read-only database diagnosis"
(Name and description are consistent, avoiding capability inference confusion)Severity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🟡 P1
Determination criterion: only when the name implies higher privileges and the body contains no prohibitive statement does L3 rise to P0; if the name is broad but does not contradict the body and no write authority exists, keep P1.
QD-S-3.7 Output Structure Contract Missing
Defect Description: Does not declare the output format, required fields, or how empty results are represented; the LLM decides the output structure on its own.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Generate an analysis report." (does not declare format or fields) |
| Problem | The LLM chooses the format, omits key fields, and fills in examples when results are empty |
| Default Behavior | The output structure is inconsistent with the downstream parsing/consumption contract |
Remediation Example:
Generate an analysis report.
- Output format: Markdown
- Required fields: Conclusion, Evidence, Risk
- Empty result: Output "no matching data"; do not fabricate examplesSeverity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🟡 P1
Determination criterion: 1.2 covers size only and 2.3 covers open-ended enumeration only; this item covers the output structure contract (format/required fields/empty-result representation). Cross-artifact parameter contracts belong to Inspect Cross; for display-only Skills with no downstream consumer, downgrade to P2.
4.3 QD-S-3 Inspection Checklist
| ID | Inspection Item | Trigger Keywords | L1 | L2 | L3 | Inspection Method |
|---|---|---|---|---|---|---|
| QD-S-3.1 | Vague Trigger Conditions | when, need, appropriate | P1 | P1 | P0 | Human |
| QD-S-3.2 | Missing Deactivation Conditions | No "prohibited"/"do not handle" | P2 | P1 | P0 | Human |
| QD-S-3.3 | Missing Ambiguous Input Handling | incomplete, format error, insufficient | P1 | P1 | P0 | Human |
| QD-S-3.4 | Undefined Task Scope | No non_goals | P2 | P1 | P1 | Human |
| QD-S-3.5 | Undeclared Default Behavior | empty, error, exception | P1 | P1 | P0 | Human |
| QD-S-3.6 | Metadata Expression Defects | contradictory, broad, vague | P1 | P1 | P1 | Human |
| QD-S-3.7 | Output Structure Contract Missing | format, fields, empty result | P1 | P1 | P1 | Human |
Part 5: QD-S-4 Failure Escalation
5.1 Classification Positioning
Risk Principle: The Skill does not define handling strategies for failure or exception scenarios. When encountering failure, the LLM's training objective of "complete the task as best you can" drives it to proactively expand its strategy to "solve the problem," creating even greater risk.
Typical Harms: Auto-completing content, lowering standards after failure, infinite rewriting, fabricating data.
5.2 Inspection Items in Detail
QD-S-4.1 Undefined Failure Behavior
Defect Description: Does not declare how to handle execution failure; the LLM may explore alternative paths on its own.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Query the database." (does not state how to handle query failure) |
| Problem | After query failure, the LLM may attempt other queries, relax filter conditions, etc. |
| Default Behavior | Unpredictable behavior; may expand query scope, causing unauthorized access |
Remediation Example:
Query the database.
- On query failure: Stop immediately; report the error and cause to the user
- Do NOT: Retry with other query conditions, expand search scope, lower standardsSeverity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🔴 P0
- L3 (High Risk): 🔴 P0
QD-S-4.2 Undeclared Partial Completion Legitimacy
Defect Description: Does not declare whether "partial completion is acceptable"; the LLM may forcibly pursue completeness, entering infinite iterations.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Generate a complete weekly report." (does not state what to do when information for a section is insufficient) |
| Problem | The LLM may forcibly fill all sections |
| Default Behavior | Fabricating content or infinite rewriting until complete |
Remediation Example:
Generate a weekly report.
- When information for a section is insufficient: Skip that section rather than fabricating content
- Partial completion is allowed: If information is insufficient, output what is available and note the missing partsSeverity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🟡 P1
Determination criterion: read-only or single-step Skills have no concept of "partial completion"; keep P1. For batch processing or write operations, raise to P0 at the instance level.
QD-S-4.3 Missing Post-Failure Prohibited Behaviors
Defect Description: Does not explicitly state what behaviors are prohibited after failure; the LLM decides on its own to lower standards or expand scope after failure.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Fix database issues." (does not state what can be done after failure) |
| Problem | After diagnosis failure, the LLM may expand the operation scope "to solve the problem" |
| Default Behavior | Executing overflow operations or generating false data |
Remediation Example:
Diagnose database issues.
- Prohibited behaviors after diagnosis failure:
- Expanding search scope is prohibited
- Generating false data to "cover up" errors is prohibited
- Lowering diagnostic standards is prohibited
- Automatically executing remediation operations is prohibited
- After failure: Return the diagnosed content so far; recommend human handlingSeverity:
- L1 (Low Risk): 🟡 P1
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🔴 P0
When closed failure behavior (stop/report) is already defined, do not judge a separate defect (it duplicates QD-S-4.1). See 11.6 for deduplication.
QD-S-4.4 Undefined Error Recovery Path
Defect Description: Lacks clear handling directives when encountering exceptions; the LLM decides on its own whether to retry, degrade, or expand.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Handle exception conditions." (does not state handling for specific exception types) |
| Problem | Different exceptions may be handled by the LLM with different strategies |
| Default Behavior | Unpredictable behavior |
Remediation Example:
Handle exception conditions.
- API timeout: Stop the request; return the partial results obtained so far
- Data format error: Request the user to correct the format
- Insufficient permission: Stop the operation; inform the user that elevated permissions are needed
- Network error: Stop retrying (maximum 2 retries); return the error messageSeverity:
- L1 (Low Risk): 🟢 P2
- L2 (Medium Risk): 🟢 P2
- L3 (High Risk): 🟢 P2
Determination criterion (conditional N/A): N/A for L1 without tool invocations; "safe termination with reporting" counts as a legitimate path — not every Skill is required to have recovery capability.
5.3 QD-S-4 Inspection Checklist
| ID | Inspection Item | Trigger Keywords | L1 | L2 | L3 | Inspection Method |
|---|---|---|---|---|---|---|
| QD-S-4.1 | Undefined Failure Behavior | failure, error, exception | P1 | P0 | P0 | Human |
| QD-S-4.2 | Undeclared Partial Completion Legitimacy | complete, all, must | P1 | P1 | P1 | Human |
| QD-S-4.3 | Missing Post-Failure Prohibited Behaviors | No "prohibited" declaration | P1 | P1 | P0 | Human |
| QD-S-4.4 | Undefined Error Recovery Path | exception, recovery, degradation | P2 | P2 | P2 | Human |
Part 6: QD-S-5 Permission Overflow
6.1 Classification Positioning
Risk Principle: The Skill does not explicitly declare operational permission boundaries. The LLM interprets broad capability descriptions as implicit authorization, executing high-impact operations beyond what was intended. This category differs from QD-S-1.4 in that: QD-S-1.4 addresses the complete absence of permission boundaries, while QD-S-5 addresses cases where permission declarations exist but are insufficient, or contain loopholes that can be broadly interpreted.
Typical Harms: Irreversible operations without confirmation, privilege escalation, data deletion, production environment damage.
6.2 Inspection Items in Detail
QD-S-5.1 Read/Write/Delete Permissions Not Distinguished
Defect Description: The Skill declares it can operate on data, but does not distinguish between read, modify, and delete permission scopes.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Manage user data." ("manage" may include read, write, delete) |
| Problem | The LLM infers that deletion is also a reasonable way to "manage data" |
| Default Behavior | Irreversible operations are executed (as in the Replit database deletion incident) |
Remediation Example:
Manage user data. This Skill is for data query and reporting only.
- Allowed: View, query, generate reports
- Prohibited: Create, modify, delete, export, migrate
All modification operations MUST be handled by humans through other processes.Severity:
- L1 (Low Risk): 🟢 P2
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🔴 P0
Same root as QD-S-1.4 and QD-S-2.4; merged as "operational permission boundary undeclared" (covering read/write/delete/send/export/execute). See 11.6 for deduplication.
QD-S-5.2 Missing High-Risk Operation Confirmation Mechanism
Defect Description: Irreversible operations such as deletion, overwriting, data exfiltration, or payments do not have a confirmation mechanism.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Clean up expired data." (does not state that confirmation is needed before deletion) |
| Problem | The LLM may directly execute deletion |
| Default Behavior | Data is lost and irrecoverable |
Remediation Example:
Clean up expired data. This operation is irreversible.
- Before execution: Clearly list the data to be deleted to the user; require secondary confirmation
- Confirmation content: Data quantity, deletion scope, deletion consequences
- Execute deletion only after the user explicitly agrees
- After deletion: Provide an operation log and verification resultsSeverity:
- L1 (Low Risk): — (does not involve irreversible operations)
- L2 (Medium Risk): — (does not involve irreversible operations)
- L3 (High Risk): 🔴 P0
Determination criterion: revised to "effective authorization gating" — operations within the pre-authorized scope (subject, object, quota, validity period) may execute automatically, and only thresholds beyond that enter confirmation. Pre-authorization applies only to reversible, low-consequence operations (
OR-L1/L2); irreversible or severe-consequence operations (OR-L3) are not eligible for pre-authorization and MUST require human confirmation/approval each time. Pre-authorization must come from the user/human and must not be self-granted by the model. The "always mandatory" human secondary confirmation has been unified with Inspect Permission QD-PM-6.3 under the "human decision point" framing; see Part 10 of this document.
QD-S-5.3 Undeclared Environment Boundaries
Defect Description: Does not declare that operating on production environments is prohibited, or does not explicitly specify the permitted operational environments.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Diagnose database issues." (does not state permitted environments) |
| Problem | The LLM may treat the production database as a legitimate diagnostic target |
| Default Behavior | Diagnostic operations are executed in the production environment, affecting production services |
Remediation Example:
Diagnose database issues.
- Permitted operational environments: Test/development databases
- Prohibited operational environments: Production databases, backup databases, disaster recovery databases
If the user specifies a production database for diagnosis:
1. Secondary confirmation that it is indeed the production environment
2. Remind of risks and wait for explicit consent
3. Execute only read-only diagnosis; do not execute any modificationsSeverity:
- L1 (Low Risk): 🟢 P2
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🔴 P0
Determination criterion (conditional N/A): applies only when the Skill can reach multiple environments or has environment-selection authority; check "is the target environment explicit + is the production operation set restricted". "Production is not prohibited" is not itself a defect (a production operations Agent's job is in production); N/A when environments are not involved. The split between environment-boundary "declaration vs. overall proportionality" and Inspect Permission QD-PM-2.4 is covered in Part 10 of this document.
QD-S-5.4 Undeclared Chained Invocation Permissions
Defect Description: Allows invoking other Skills but does not declare the permitted/prohibited downstream Skill scope.
Defect Example:
| Dimension | Content |
|---|---|
| Incorrect | description: "Process data. May invoke other Skills." (does not state which Skills may be invoked) |
| Problem | A low-permission Skill may invoke a high-permission Skill, causing privilege escalation |
| Default Behavior | Indirect execution of high-permission operations that should not be executed |
Remediation Example:
Process data. May invoke other Skills.
- Permitted downstream Skills:
- data-query
- data-report-generation
- Prohibited downstream Skills:
- data-delete
- data-modify
- user-permission-managementSeverity:
- L1 (Low Risk): 🟢 P2
- L2 (Medium Risk): 🟡 P1
- L3 (High Risk): 🔴 P0
Determination criterion (conditional N/A): applies only when
allow_skill_chaining=true; otherwise N/A. Loop/depth convergence for chained invocation belongs to Inspect Cross (QD-CYC-1.1/1.2) and Inspect Permission (QD-PM-4.x); this item no longer sets its own "depth ≤ 1" threshold.
6.3 QD-S-5 Inspection Checklist
| ID | Inspection Item | Trigger Keywords | L1 | L2 | L3 | Inspection Method |
|---|---|---|---|---|---|---|
| QD-S-5.1 | Read/Write/Delete Not Distinguished | manage, handle, operate | P2 | P1 | P0 | Human |
| QD-S-5.2 | Missing High-Risk Confirmation | delete, overwrite, payment | — | — | P0 | Human |
| QD-S-5.3 | Undeclared Environment Boundaries | production, test, development | P2 | P1 | P0 | Human |
| QD-S-5.4 | Undeclared Chained Invocation Permissions | invoke, downstream, chain | P2 | P1 | P0 | Human |
Part 7: Skill Tiered Inspection Mechanism
7.1 Skill Risk Level Definitions
7.1.1 L1 (Low Risk): Text-Processing Only
Definition:
- Processes only text/data provided by the user
- No tool invocations; no external system access
- No sensitive data; no irreversible operations
- Execution scope is fully bounded by user input
Core Risk Characteristics:
- The LLM's autonomous gap-filling scope is limited (primarily at the text content level)
- Risk primarily comes from output quality issues, not resource runaway or behavioral overflow
Typical Examples:
- Weekly report compilation
- Text polishing / translation
- Document summarization
- Format conversion
7.1.2 L2 (Medium Risk): Resource Access
Definition:
- Queries external data sources (databases, knowledge bases, APIs)
- May involve sensitive data exposure risk
- No irreversible operations
- Execution scope may be expanded by the LLM
Core Risk Characteristics:
- The LLM may fill in query scope and data volume
- Sensitive data exposure risk exists
- Resource usage runaway risk exists
Typical Examples:
- Data queries
- Knowledge retrieval
- Email / message reading
- Log analysis
7.1.3 L3 (High Risk): Operation Execution
Definition:
- Executes write operations (create, update, delete)
- Invokes external systems to execute functions
- May produce irreversible consequences
- Execution scope may be expanded by the LLM
Core Risk Characteristics:
- The LLM may fill in operation scope and execution parameters
- Irreversible consequence risk exists
- Resource usage runaway risk exists
- Behavioral overflow risk exists
Typical Examples:
- Database operations (create / update / delete)
- File operations (create / modify / delete)
- Email sending
- API invocations
- Permission operations
7.2 Tiered Inspection Checklist
Design Principles (aligned with Inspect Prompt §3.3.1):
| Principle | Description |
|---|---|
| Upward compatibility | Higher-level checklists include lower-level items (L3 ⊃ L2 ⊃ L1) |
| Focus on the core | L1 focuses on P1/P2 defects (no P0); L2 progressively covers P0/P1; L3 covers everything |
| Ordered by ID | Inspection items are ordered QD-S-0 → QD-S-5 |
The severity levels in the following checklists are default levels; any adjustment confirmed by human review MUST record its rationale.
7.2.1 L1 (Low Risk) Inspection Checklist
Mandatory Inspection Items (QD-S-0 gate + 8 core checks):
| ID | Inspection Item | Level | Inspection Method |
|---|---|---|---|
| QD-S-0 | Baseline Compliance | MUST pass | Automated |
| QD-S-1.1 | Input Boundary Missing | P1 | Automated |
| QD-S-1.2 | Output Size Missing | P1 | Automated |
| QD-S-2.1 | Unbounded Quantity Declaration | P1 | Automated |
| QD-S-3.1 | Vague Trigger Conditions | P1 | Human |
| QD-S-3.3 | Missing Ambiguous Input Handling Strategy | P1 | Human |
| QD-S-3.4 | Undefined Task Scope | P2 | Human |
| QD-S-4.1 | Undefined Failure Behavior | P1 | Human |
| QD-S-3.6 | Metadata Expression Defects | P1 | Human |
Optional Inspection Items (recommended but not required):
- QD-S-3.2, QD-S-3.7
7.2.2 L2 (Medium Risk) Inspection Checklist
Must-Pass Items (P0 defects):
| ID | Defect Item | Inspection Method |
|---|---|---|
| QD-S-0 | Baseline Compliance | Automated |
| QD-S-1.1 | Input Boundary Missing | Automated |
| QD-S-1.3 | Execution Resource Boundary Missing | Automated |
| QD-S-2.1 | Unbounded Quantity Declaration | Automated |
| QD-S-2.2 | Unterminated Execution | Automated |
| QD-S-4.1 | Undefined Failure Behavior | Human |
Recommended Inspection Items (P1 defects):
- QD-S-1.2, QD-S-1.4, QD-S-1.5, QD-S-2.3, QD-S-2.4
- QD-S-3.1, QD-S-3.2, QD-S-3.3, QD-S-3.4, QD-S-3.5, QD-S-3.6, QD-S-3.7
- QD-S-4.2, QD-S-4.3, QD-S-5.1, QD-S-5.3, QD-S-5.4
7.2.3 L3 (High Risk) Inspection Checklist
Full Inspection (all QD-S-1 through QD-S-5):
| Category | P0 Defects | P1 Defects |
|---|---|---|
| QD-S-1 | QD-S-1.1/1.3/1.4/1.5 | QD-S-1.2 |
| QD-S-2 | QD-S-2.1/2.2/2.4 | QD-S-2.3 |
| QD-S-3 | QD-S-3.1/3.2/3.3/3.5 | QD-S-3.4/3.6/3.7 |
| QD-S-4 | QD-S-4.1/4.3 | QD-S-4.2 |
| QD-S-5 | QD-S-5.1/5.2/5.3/5.4 | — |
P2 (should be included in the improvement plan or risk register; does not block release): QD-S-4.4
Inspection Methods:
- Automated: QD-S-1.1-1.3, QD-S-2.1-2.3
- Semi-automated: QD-S-1.5
- Human: all other items (QD-S-1.4, 2.4, 3.x, 4.x, 5.x)
7.3 Standard Inspection Process (6 Steps)
┌──────────────────────────────────────────────────┐
│ Skill Quality Defect Inspection Process │
│ (Inspect Skill) │
└──────────────────────────────────────────────────┘
│
▼
┌───────────────────────┐
│ Step 1: Baseline │
│ Compliance Check │
│ Execute QD-S-0 │
└───────────────────────┘
│
┌─────────┴─────────┐
▼ ▼
┌─────────────┐ ┌──────────────┐
│ PASS ✓ │ │ FAIL ✗ │
└─────────────┘ └──────────────┘
│ │
▼ ▼
┌─────────────────┐ ┌──────────────────┐
│ Step 2: Skill │ │ Output remediation│
│ Risk Level │ │ guidance; stop │
│ Determination │ │ inspection │
└─────────────────┘ └──────────────────┘
│
┌─────────┴─────────┐
▼ ▼
┌──────────────────┐ ┌──────────────────┐
│ Classify as │ │ Classification │
│ OR-L1/L2/L3 │ │ pending │
└──────────────────┘ │ Output "pending" │
│ │ (NEEDS_REVIEW) │
▼ └──────────────────┘
┌─────────────────────┐
│ Step 3: Select │
│ Inspection Checklist│
│ Based on risk level │
└─────────────────────┘
│
▼
┌─────────────────────┐
│ Step 4: Automated │
│ Inspection │
│ Execute automated │
│ inspection items │
└─────────────────────┘
│
▼
┌─────────────────────┐
│ Step 5: Semi-Auto │
│ and Human Inspection│
│ Execute complex │
│ inspection items │
└─────────────────────┘
│
▼
┌─────────────────────┐
│ Step 6: Output │
│ Inspection Report │
│ Generate optimized │
│ version │
└─────────────────────┘7.3.1 Step 1: Baseline Compliance Check
Execution Content: QD-S-0 inspection (automated)
Determination Criteria:
- ✅ PASS: All QD-S-0.1 through 0.8 inspection items pass
- ❌ FAIL: Any item fails
- Note: QD-S-0.9 (description length budget) is SHOULD-level and is not part of the "MUST pass" determination; exceeding the budget produces an advisory note only and does not block subsequent semantic inspection
Failure Handling (relaxed: stop only when "completely unparsable"; see 0.4 for the definition):
Output: QD-S-0 inspection failed
Failed items:
- QD-S-0.1: name field missing
- QD-S-0.3: Missing quotes on YAML line 7
Handling recommendation:
Stop QD-S-1 through QD-S-5 inspection only when the Skill definition is
completely unparsable (QD-S-0.3 fails);
other QD-S-0 defects do not block subsequent semantic inspection, but the
report MUST mark "baseline compliance not passed", and the formal evaluation
conclusion is recorded as "evaluation incomplete (NEEDS_REVIEW)"
(see framework/core.md §6.2).7.3.2 Step 2: Skill Risk Level Determination
Determination Basis:
| Determination Question | Determination Logic |
|---|---|
| Does it involve delete or irreversible write operations? | Yes → L3 |
| Does it involve reversible / isolated write operations? | Yes → L2 |
| Does it involve external system access? | Yes → at least L2 |
| Does it involve sensitive data? | Yes → at least L2 |
| Does it involve permission changes? | Yes → L3 |
| None of the above | → L1 |
Risk grading aligns with framework/core.md §4.3 OR-L (operational risk level): reversible/isolated writes ≈ OR-L2, irreversible writes ≈ OR-L3. When the mounted tool/permission context is unavailable, output "classification pending" (see 7.4, 7.5); do not classify directly.
Example Determinations:
- "Compile weekly report" → L1 (text-processing only)
- "Query database" → L2 (resource access, sensitive data)
- "Delete user records" → L3 (irreversible operation)
7.3.3 Step 3: Select Inspection Checklist
Based on the determined risk level, select the corresponding inspection checklist:
L1 Inspection Checklist (QD-S-0 gate + 8 core checks):
| ID | Inspection Item | Method |
|---|---|---|
| QD-S-0 | Baseline Compliance | Automated |
| QD-S-1.1 | Input Boundary Missing | Automated |
| QD-S-1.2 | Output Size Missing | Automated |
| QD-S-2.1 | Unbounded Quantity Declaration | Automated |
| QD-S-3.1 | Vague Trigger Conditions | Human |
| QD-S-3.3 | Missing Ambiguous Input Handling Strategy | Human |
| QD-S-3.4 | Undefined Task Scope | Human |
| QD-S-4.1 | Undefined Failure Behavior | Human |
| QD-S-3.6 | Metadata Expression Defects | Human |
L2 Inspection Checklist (P0 mandatory + P1 recommended; see 7.2.2)
L3 Inspection Checklist (full inspection; all items)
7.3.4 Step 4: Automated Inspection
Three-stage inspection method: candidate finding (automated) → semantic determination (LLM or human) → review. Automated inspection produces candidate findings only (such as keyword hits or structural omissions) and does not assign levels directly; the final defect level is confirmed by semantic determination. "Human" in the table below means human review or LLM semantic determination (the implementer MUST specify which one in the CI gate configuration); the two must not be silently mixed.
Execution Content (tool-automated):
- QD-S-0.1 through 0.9: Structural inspection
- QD-S-1.1 through 1.3: Quantity boundary keyword scanning
- QD-S-2.1 through 2.3: Unbounded language detection (all/every/until success, etc.)
- QD-S-1.5, QD-S-3.6: Candidate findings (keyword hits mark candidates only; final determination requires human review)
Output:
Automated inspection complete
Detected automated inspection items:
✓ QD-S-0.1 name field present
✓ QD-S-1.1 Input items detected; no upper limit declaration → requires human verification
✗ QD-S-2.1 Keyword "all" detected without corresponding constraint → P1 defect
Proceeding to Step 5: Semi-automated and human inspection7.3.5 Step 5: Semi-Automated and Human Inspection
Execution Content:
- QD-S-3.x: Trigger conditions, deactivation conditions, ambiguity handling, output structure (requires human judgment)
- QD-S-4.x: Failure behavior, partial completion strategy (requires human judgment)
- QD-S-5.x: Permission boundaries, environment declarations, chained invocation (requires human judgment)
Inspection Method:
- Human review of the Skill description and related fields
- Inspect for semantic gaps from the six Review Schema dimensions
- Perform Inferential Verification
Output (the example excerpt corresponds to an L3 Skill: QD-S-5.1 is P0 at L3, and QD-S-3.4 is P1 at L3):
Human inspection results
Detected human review items:
⚠️ QD-S-3.4 Missing non_goals declaration → P1 defect
⚠️ QD-S-5.1 Broad capability description; read/write/delete not distinguished → P0 defect
✓ QD-S-4.1 Failure handling declaration present → PASS
Proceeding to Step 6: Generate inspection report7.3.6 Step 6: Output Inspection Report
Three Output Documents:
Document 1: Defect List (by priority)
| ID | Defect Item | Level | Location | Risk Description |
|---|---|---|---|---|
| QD-S-3.4 | Undefined Task Scope | P1 | description | Missing non_goals; task boundaries can be expanded |
| QD-S-5.1 | Read/Write/Delete Not Distinguished | P0 | description | Broad capability verbs; write permission can be inferred |
| QD-S-2.1 | Unbounded Quantity Declaration | P1 | input section | Uses "all" without corresponding constraint |
Document 2: Optimized Skill (retaining original platform format)
(See Usage Examples chapter for details)
Document 3: Comparison Table (Original vs. Optimized vs. Inferential Verification)
| Original | Optimization Suggestion | Defect ID | Inferential Verification |
|---|---|---|---|
| "Process all work notes" | "Process up to 50 work notes; truncate and notify user when exceeded" | QD-S-1.1 | LLM processes all without constraint; clear stop condition after remediation |
7.4 Inspection Item Judgment States (Five States)
Each inspection item's result takes one of the following five states. These five states are inspection item judgment states, distinct from the Finding lifecycle states in framework/core.md §5.3 (open/confirmed/mitigated/accepted/closed/invalid); "N/A" follows the minimum recording requirements of §5.7, and "insufficient evidence" must not be marked N/A under §5.7.
| State | Meaning | Counted in scoring/deduction? |
|---|---|---|
| Pass | The item applies and no defect was found | Yes |
| Defect | The item applies and a defect was found (P0/P1/P2) | Yes |
| Not Applicable (N/A) | The item does not apply to the current Skill (e.g., QD-S-5.4 is N/A for an L1 Skill with no tool invocations) | No |
| Insufficient Evidence | The item applies but there is not enough evidence to determine (e.g., the mounted tool context is unavailable) | No (output "pending") |
| Not Inspected | Inspection was required but was not executed for process reasons | No |
Rules:
- "Not inspected" and "insufficient evidence" must not be recorded as pass;
- "Not applicable" items are not counted in scoring/deduction, and the reason for non-applicability MUST be recorded (see the conditional N/A in each item's determination note, e.g., QD-S-5.3 multi-environment, QD-S-5.4
allow_skill_chaining=true); - "Insufficient evidence" maps to
uncertain/needs_more_evidencein core.md §5.7 and must not be classified directly; when the mounted tool/permission context is unavailable, the risk level determination outputs "classification pending" (see 7.3.2).
7.5 Effective Constraints and Compensating Controls (Severity Determination Principles)
Cross-cutting rule: The inspection object is the Skill definition text, but severity determination should distinguish three outcomes — "constraint missing / constraint conflicting / insufficient evidence (pending)" — and consider the constraints actually in effect on that Skill's deployment instance.
- Traceable in-artifact constraints (Tool Schema fields delivered with the artifact, scope/service accounts declared in the Agent manifest, the Skill's
allowed_operations, etc.; see Inspect Permission §0.1.4) may serve as the basis for severity determination; - External controls from the Harness / RBAC / platform configuration must not be used as "pass evidence" to close a static defect; they can only serve as an Exception's
compensating_controls(affecting disposition, not changing the defect's existence). See framework/core.md §6.2-6.4, §9.3; - When evidence is insufficient, output "pending" (
NEEDS_REVIEW; see 7.4 and core.md §6.2); do not classify directly.
This rule is consistent with framework/relevance.md §2.4.1: being blocked by an external control does not automatically prove that the artifact itself has no defect.
Three layers carrying frequency/batch limits (consistent with Inspect Cross §7.4): the artifact declaration layer (the Skill's max_tool_calls / max_retries / timeout_seconds, the Tool's limit.maximum / maxItems) provides the business-applicable limit and ships with the artifact; the runtime fallback layer (rate limiting, concurrency, timeouts at the Harness / API gateway) provides a global fallback value; the business quota layer (business-side quotas, approvals, billing gates) provides commercial and compliance constraints. The three layers cannot substitute for one another:
- Do not cancel or downgrade artifact-layer inspection on the grounds that "the Harness has a fallback" — local debugging, single-machine deployments, and third-party hosted Agents often have no gateway, and in that case the artifact declaration is the only source of constraint;
- Harness thresholds are not equal to business-applicable limits — fallback values take the greatest common divisor "so as not to harm any Agent"; judging business reasonableness with fallback values would rate a Skill that "should query once per order" as acceptable.
Part 8: Review Schema — Detailed Review Dimensions
8.1 Six-Dimension Overview and Positioning
The Inspect Skill Review Schema is the inspection Agent's intermediate review framework for systematically decomposing and analyzing Skill definitions. It is not a production format — it is a working view.
MUST/SHOULD/MAY grading note: The MUST/SHOULD/MAY levels of the review items in the table below are the L3 (High Risk) default requirements. For L1/L2 they should be downgraded according to the table (most L2 MUSTs become SHOULD; L1 becomes SHOULD/MAY), avoiding Skills bloated and instructions diluted by treating everything as MUST. The Schema's MUST must not be equated directly with "mandatory at every level"; the tiered mapping in Part 7 governs.
Six Dimensions and Corresponding Risks:
| Dimension | Review Question | Associated Defect Types | Primary Risk |
|---|---|---|---|
| IDENTITY | What does this Skill do, not do, and what can it access? | QD-S-3.4, QD-S-5.1 | Task scope expansion, resource overflow |
| TRIGGER | When should it trigger, and when should it not? | QD-S-3.1, QD-S-3.2 | Mis-triggering, excessive invocation |
| INPUT | Are input size, format, and overflow behavior clearly defined? | QD-S-1.1, QD-S-3.3 | Token runaway, input ambiguity |
| EXECUTION | Are execution boundaries, retries, tool invocations, and chained calls controlled? | QD-S-1.3, QD-S-5.4 | Infinite loops, tool abuse, permission diffusion |
| OUTPUT | Are output size, structure, and partial completion strategy clearly defined? | QD-S-1.2, QD-S-2.3, QD-S-3.7 | Output bloat, content fabrication |
| FAILURE | How are failures, empty input, insufficient input, and user dissatisfaction handled? | QD-S-4.x | Auto-completion, post-failure expansion |
8.2 IDENTITY Dimension in Detail
Review Question: What does this Skill do, not do, and what can it access?
Inspection Checklist:
| Review Item | Level | Description | LLM Default Behavior When Missing |
|---|---|---|---|
| purpose | MUST | Precise objective description of the Skill; should constrain rather than expand capability boundaries | LLM interprets Skill scope on its own, typically broader than intended |
| non_goals | MUST | Explicitly declare what the Skill does NOT do | LLM expands task boundaries to "reasonably related" matters |
| allowed_resources | MUST | Permitted data sources (user-provided, knowledge base, external API, etc.) | LLM uses all available context, including conversation history |
| forbidden_resources | MUST | Prohibited data sources (production databases, backups, other users' data, etc.) | No prohibition declaration equals no access restriction |
| allowed_operations | SHOULD | Permitted operation types (query, aggregate, render, etc.) | LLM interprets all verbs in the capability description as authorization |
| forbidden_operations | MUST | Explicitly prohibited operation types (delete, modify, export, etc.) | Broad capability declarations are treated as implicit authorization for all related operations |
Inspection Example:
| Skill Description | IDENTITY Dimension Check | Defect Determination |
|---|---|---|
| "Query user data" | purpose: query ✓ / non_goals: ❌ missing / allowed_resources: user-provided only ✓ / forbidden: ❌ others' data not prohibited | QD-S-3.4 P1 + QD-S-1.5 P1 |
| "Manage database issues" | purpose: vague / non_goals: ❌ missing / allowed_operations: ❌ read/write/delete not distinguished / forbidden_operations: ❌ missing | QD-S-5.1 P0 + QD-S-3.6 P1 |
8.3 TRIGGER Dimension in Detail
Review Question: When should it trigger, and when should it not?
Inspection Checklist:
| Review Item | Level | Description | LLM Default Behavior When Missing |
|---|---|---|---|
| allowed_when | MUST | Explicit conditions under which the Skill should be activated | LLM judges whether to trigger based on vague relevance |
| forbidden_when | MUST | Scenarios in which the Skill should NOT be activated | LLM forcibly activates in edge scenarios |
| ambiguity_policy | MUST | Handling strategy when content is missing (ask_user / stop / infer) | LLM infers or completes missing content |
| may_expand_scope | MUST | Whether the LLM is permitted to expand the task scope | Expansion is permitted by default |
Inspection Example:
| Skill | TRIGGER Dimension Check | Defect Determination |
|---|---|---|
| "When the user needs help" | allowed_when: vague / forbidden_when: ❌ missing | QD-S-3.1 P1 + QD-S-3.2 P1 |
| "User provides work notes and requests weekly report" | allowed_when: explicit ✓ / forbidden_when: ✓ "user has not provided notes" | PASS |
8.4 INPUT Dimension in Detail
Review Question: Are input size, format, and overflow behavior clearly defined?
Inspection Checklist:
| Review Item | Level | Description | LLM Default Behavior When Missing |
|---|---|---|---|
| max_items | MUST | Upper limit on the number of list-type input items | Processes all submitted content without truncation |
| max_chars | SHOULD | Character limit for a single input item | Accepts input of arbitrary length |
| format | SHOULD | Input format requirements (JSON, CSV, free text, etc.) | LLM accepts any format |
| if_exceeds | MUST | Handling strategy when limits are exceeded (process_first_n / reject / ask) | LLM decides on its own how to handle excess input |
| safe_defaults | SHOULD | Default values for optional fields | LLM assumes values for missing fields |
Inspection Example:
| Skill Input | INPUT Dimension Check | Defect Determination |
|---|---|---|
| "Accept a list of work notes" | max_items: ❌ missing / if_exceeds: ❌ missing | QD-S-1.1 P0 |
| "Process up to 50 items; truncate and notify when exceeded" | max_items: 50 ✓ / if_exceeds: explicit ✓ | PASS |
8.5 EXECUTION Dimension in Detail
Review Question: Are execution boundaries, retries, tool invocations, and chained calls controlled?
Inspection Checklist:
| Review Item | Level | Description | LLM Default Behavior When Missing |
|---|---|---|---|
| max_tool_calls | MUST | Maximum tool invocation count per execution | Invokes tools without limit until the task is complete |
| max_retries | MUST | Maximum retry count | Retries without limit |
| timeout_seconds | MUST | Execution timeout duration | Long-running tasks are not terminated |
| allow_skill_chaining | SHOULD | Whether invoking other Skills is permitted | Chained invocation is permitted by default |
| allowed_downstream_skills | MUST (when chaining=true) | Whitelist of permitted downstream Skills | Any Skill, including high-permission ones, can be triggered |
| require_user_confirmation_for | MUST (when involving irreversible operations) | Operation types requiring user confirmation | Irreversible operations are executed without confirmation |
| if_budget_exceeded | MUST | Handling when resource budget is exceeded | Execution continues after resource exhaustion |
Inspection Example:
| Skill | EXECUTION Dimension Check | Defect Determination |
|---|---|---|
| "Keep trying until success" | max_retries: ❌ missing / timeout: ❌ missing | QD-S-1.3 P0 + QD-S-2.2 P0 |
| "Maximum 3 retries, 60-second timeout" | max_retries: 3 ✓ / timeout_seconds: 60 ✓ | PASS |
8.6 OUTPUT Dimension in Detail
Review Question: Are output size, structure, and partial completion strategy clearly defined?
Inspection Checklist:
| Review Item | Level | Description | LLM Default Behavior When Missing |
|---|---|---|---|
| format | SHOULD | Output format declaration (Markdown, JSON, structured text, etc.) | LLM selects format on its own |
| max_total_chars | MUST | Total output character limit | Output volume is unbounded |
| max_items_per_section | SHOULD | Item limit per output section | Single-section items are unbounded |
| allow_additional_sections | MUST | Whether the LLM is permitted to add output sections on its own | Addition is permitted by default |
| partial_result_allowed | MUST | Whether partial completion is permitted | LLM forcibly pursues completeness; enters infinite iterations |
| if_input_insufficient_for_section | MUST | Handling when input is insufficient to populate a section | LLM auto-completes or fabricates content |
Inspection Example:
| Skill Output | OUTPUT Dimension Check | Defect Determination |
|---|---|---|
| "Generate a complete weekly report with all sections" | max_total_chars: ❌ missing / partial_result_allowed: ❌ missing | QD-S-1.2 P1 + QD-S-4.2 P1 |
| "Output not exceeding 1,500 words; skip sections with insufficient information" | max_total_chars: 1500 ✓ / partial_result_allowed: true ✓ | PASS |
8.7 FAILURE Dimension in Detail
Review Question: How are failures, empty input, insufficient input, and user dissatisfaction handled?
Inspection Checklist:
| Review Item | Level | Description | LLM Default Behavior When Missing |
|---|---|---|---|
| on_empty_input | MUST | Handling when input is empty (ask_user / stop / generate_template) | LLM generates example content or template fill |
| on_insufficient_input | MUST | Handling when input is insufficient (generate_partial / ask_user / stop) | LLM auto-completes |
| on_user_dissatisfied | MAY | Handling strategy for user dissatisfaction (max rewrite count, escalation confirmation, etc.) | Unlimited rewrites |
| forbidden_behaviors | MUST | Explicitly prohibited post-failure behaviors (no auto-completion, no inference, no scope expansion) | LLM selects expansion strategy on its own after failure |
Inspection Example:
| Skill | FAILURE Dimension Check | Defect Determination |
|---|---|---|
| "No failure handling defined" | on_empty_input: ❌ missing / forbidden_behaviors: ❌ missing | QD-S-4.1 P0 |
| "On empty input: ask user; generating examples is prohibited; inferring missing content is prohibited" | on_empty_input: ask_user ✓ / forbidden_behaviors: ✓ explicit | PASS |
Part 9: Usage Examples
9.1 L1 Example: WeeklyReportWriter (Personal Productivity Scenario)
9.1.1 Original Skill
---
name: weekly-report-writer
description: Help users turn work notes into a weekly report.
---
Use this skill when the user wants to write or polish a weekly report.
The skill should organize the user's work notes into a clear weekly report.
It can include sections such as completed work, progress, problems, and next week's plan.
Make the report professional, concise, and suitable for workplace communication.
Do not make negative comments about colleagues or managers.
Do not fabricate facts.9.1.2 Inspection Process
Step 1: QD-S-0 Baseline Compliance → ✅ PASS
Step 2: Risk Level Determination → L1 (text-processing only; no tool invocations; no external system access)
Step 3: Select Inspection Checklist → L1 checklist (QD-S-0 gate + 8 core checks)
Steps 4–5: Inspection Execution
| Inspection Item | Result | Level | Description |
|---|---|---|---|
| QD-S-1.1 Input Boundary Missing | ❌ FAIL | P1 | No upper limit declared for work note count |
| QD-S-1.2 Output Size Missing | ❌ FAIL | P1 | No total length limit declared for the weekly report |
| QD-S-2.1 Unbounded Quantity Declaration | ✓ PASS | — | No relevant unbounded language |
| QD-S-3.1 Vague Trigger Conditions | ✓ PASS | — | Trigger conditions point to "weekly report" and are fairly explicit |
| QD-S-3.3 Missing Ambiguous Input Handling | ❌ FAIL | P1 | No handling strategy declared for insufficient notes |
| QD-S-3.4 Undefined Task Scope | ❌ FAIL | P2 | Missing non_goals (P2 for L1) |
| QD-S-4.1 Undefined Failure Behavior | ❌ FAIL | P1 | No handling strategy for empty input |
| QD-S-3.6 Metadata Expression Defects | ✓ PASS | — | name/description are substantially consistent |
9.1.3 Inferential Verification
Defect QD-S-1.1: Input Boundary Missing
[Defect]QD-S-1.1 Input Boundary Missing
[LLM Default Behavior Inference]
If the Skill does not declare an upper limit on the number of input work notes,
the LLM, driven by its "complete the task as best you can" tendency,
will attempt to process all records provided by the user. When the user pastes
a large number of records (e.g., 500+), the LLM will expand and analyze each one,
causing Token Consumption to far exceed expectations, resulting in cost overruns
or response timeouts.
[Remediation Declaration]
Process up to 50 work notes.
When exceeding 50: Process the first 50 and notify the user:
"Work notes exceed the 50-item limit. The first 50 have been processed.
Please submit the remaining items separately."
[Remediation Effectiveness Demonstration]
The upper limit (50 items) and overflow behavior (truncate and notify) are both
explicitly defined. The LLM has a clear stop condition during execution and does
not need to determine boundaries on its own. This remediation effectively
eliminates the Token runaway risk.
[Verification Conclusion]Static defect remediated (Static Pass). Runtime effectiveness
still requires Risk Implicit / Quality dynamic validation; a static pass does not
mean runtime safety (see 0.1.2 and framework/core.md §1.4).9.1.4 Optimized Skill
---
name: weekly-report-writer
description: Turn user-provided work notes into a concise, professional weekly report. Only summarizes provided content; does not infer or fabricate work details.
---
## Purpose
Help users organize their work notes into a clear weekly report for workplace communication.
## When to use
Use this skill only when the user explicitly provides work notes or bullet points and asks for a weekly report.
## Input constraints
- Process up to 50 work notes
- If more than 50 notes are provided, process the first 50 and notify: "Work notes exceed the 50-item limit. I've processed the first 50. Please submit the remaining items separately."
- Each note should be under 200 characters
- Total input should not exceed 10,000 characters
## Output format
Organize into these sections (and only these unless explicitly requested):
- Completed work
- Progress
- Problems or blockers
- Next week's plan
Output limits:
- Total report: not exceed 1,500 Chinese characters or 1,000 English words
## Do not
- Invent facts, numbers, people, projects, dates, or outcomes
- Infer work content from chat history or context outside the current request
- Add report sections beyond those listed above
- Make negative comments about colleagues, managers, or teams
## Failure handling
- If user provides no work notes: Ask them to provide notes instead of generating a sample report
- If some sections lack information: Skip those sections and explain which information was missing
- If output is incomplete due to input limits: Return the partial report and note what was not included9.1.5 Governance Effect
| Dimension | Before Governance | After Governance |
|---|---|---|
| Input limit | Undeclared | Up to 50 items; truncate and notify when exceeded |
| Output limit | Undeclared | Not exceeding 1,500 words |
| Task boundary | Vague ("such as" open enumeration) | Explicit (four sections only) |
| Missing content handling | Undeclared | Skip missing sections and explain why |
| Failure handling | Undeclared | Empty input → ask user; generating examples is prohibited |
9.2 L2 Example: CustomerDataQuery (Enterprise Data Query Scenario)
9.2.1 Original Skill
---
name: customer-data-query
description: Helps users query customer information and generate reports.
---
Use this skill when the user needs customer data.
The skill can search customer records, filter by various criteria,
and summarize the results into a report.9.2.2 Inspection Process
Step 1: QD-S-0 Baseline Compliance → ✅ PASS
Step 2: Risk Level Determination → L2 (resource access: queries external data sources, involves sensitive data, no irreversible operations)
Step 3: Select Inspection Checklist → L2 checklist (P0 mandatory + P1 recommended)
Steps 4–5: Inspection Execution (excerpt of the defects found):
| Inspection Item | Result | Level | Description |
|---|---|---|---|
| QD-S-1.1 Input Boundary Missing | ❌ FAIL | P0 | No upper limit declared for query conditions or result count |
| QD-S-1.5 Data Source Boundary Missing | ❌ FAIL | P1 | "search customer records" does not declare permitted data sources |
| QD-S-5.1 Read/Write/Delete Not Distinguished | ❌ FAIL | P1 | "query and generate reports" does not declare read-only or prohibit modification |
| QD-S-4.1 Undefined Failure Behavior | ❌ FAIL | P0 | No handling strategy when the query fails |
Note: QD-S-1.4 and QD-S-5.1 share the same root (both are "operational permission boundary undeclared") and are deduplicated into 5.1 per 11.6.
9.2.3 Inferential Verification
Defect QD-S-5.1: Read/Write/Delete Permissions Not Distinguished
[Defect]QD-S-5.1 Read/Write/Delete Permissions Not Distinguished
[LLM Default Behavior Inference]
In "query and generate reports", "generate reports" may be understood by the LLM
as including exporting, modifying, or even deleting customer data. When the Skill
does not declare "read-only, modification prohibited", the LLM may interpret a
broad capability description as implicit authorization.
[Remediation Declaration]
Explicitly declare that this Skill is for read-only query and report generation only:
- Allowed: Query, filter, summarize
- Prohibited: Create, modify, delete, export, migrate customer data
[Remediation Effectiveness Demonstration]
The permission boundary moves from "vague" to "explicitly read-only"; the LLM no
longer has room for ambiguous inference.
[Verification Conclusion]Static defect remediated (Static Pass). Runtime effectiveness
still requires Risk Implicit / Quality dynamic validation (see 0.1.2 and
framework/core.md §1.4).9.2.4 Optimized Skill
---
name: customer-data-query
description: Read-only query and reporting over customer data. Does not modify, delete, or export customer records.
---
## When to use
Use this skill only when the user explicitly asks to query or summarize customer data.
## Data source
- Only query the customer database specified by the user's request
- Do not infer or pull from other systems, chat history, or default configs
## Allowed operations
- Query, filter, summarize customer records
- Generate read-only reports
## Prohibited operations
- Create, modify, delete, export, or migrate customer data
## Input limits
- Accept at most 50 filter conditions
- Return at most 500 records per query
## Failure handling
- Query fails: stop and report the error, do not retry with broader filters
- No matching records: report "no results" instead of fabricating data9.2.5 Governance Effect
| Dimension | Before Governance | After Governance |
|---|---|---|
| Data source | Undeclared | Only the customer database specified by the user; no inference |
| Permission boundary | Read/write/delete not distinguished | Explicit read-only + prohibition list |
| Input/result limits | Undeclared | 50 conditions / 500 records |
| Failure handling | Undeclared | Stop and report; do not broaden filters |
9.3 L3 Example: DatabaseDiagnosticReporter (Enterprise Operations Scenario)
9.3.1 Original Skill
---
name: database-maintenance-assistant
description: Helps users diagnose database issues and suggest fixes.
---
Use this skill when the user needs help with database problems.
The skill can review database error messages, table information, query results,
or user descriptions of database issues.
It should explain what may be wrong and suggest practical ways to fix the problem.
Before taking any action, explain the plan.
Do not access unrelated systems.
Be careful with sensitive data.9.3.2 Inspection Process
Step 1: QD-S-0 Baseline Compliance → ✅ PASS
Step 2: Risk Level Determination → L3 (involves database operations; may involve sensitive data; permission overflow risk exists)
Step 3: Select Inspection Checklist → L3 full inspection checklist
Steps 4–5: Inspection Execution
| Inspection Item | Result | Level | Description |
|---|---|---|---|
| QD-S-5.1 Read/Write/Delete Not Distinguished | ❌ FAIL | P0 | "maintenance"/"fixes" imply data modification capability (same root as 1.4 Permission Boundary Missing; deduplicated into this item) |
| QD-S-5.3 Undeclared Environment Boundaries | ❌ FAIL | P0 | Target environment is unclear; the production operation set is unrestricted |
| QD-S-3.6 Metadata Expression Defects | ❌ FAIL | P1 | name "maintenance" contradicts description "diagnosis" |
| QD-S-4.1 Undefined Failure Behavior | ❌ FAIL | P0 | No handling strategy when diagnosis fails |
Note: QD-S-1.4 and QD-S-5.1 share the same root and are deduplicated into 5.1 per 11.6; QD-S-5.2 (high-risk confirmation) — because the original Skill does not declare deletion capability — reports "capability boundary unclear" rather than "deletion executable", which is "pending" rather than a direct P0 defect.
9.3.3 Inferential Verification
Defect QD-S-5.1: Read/Write/Delete Permissions Not Distinguished
[Defect]QD-S-5.1 Read/Write/Delete Permissions Not Distinguished
[LLM Default Behavior Inference]
The Skill's name contains "maintenance" and its description contains "fixes."
In a database context, both words typically imply maintenance and repair
capabilities, which may include data modification. When the Skill does not
explicitly say "deletion and modification are prohibited," the LLM may
understand these operations as permitted.
If a user says "the data in this table is wrong," the LLM may infer that
"needs fixing" is equivalent to "needs deletion or cleanup." This is precisely
the typical cause of incidents where "an ambiguous permission boundary is
equivalent to implicit authorization" (such as publicly reported Agent data
deletion incidents).
[Remediation Declaration]
Rename the Skill to `readonly-database-diagnostic-reporter`
Explicitly declare: This Skill is for read-only diagnosis only.
The following operations are prohibited:
- UPDATE / DELETE / INSERT / DROP / TRUNCATE / ALTER
- Data migration, data cleanup, data backup
- Any operation that modifies database structure or data
All remediation recommendations are labeled as "Suggestions for human review"
and are not directly executed.
[Remediation Effectiveness Demonstration]
Through the explicit name and prohibition list, the LLM no longer has room
for ambiguous inference. The permission boundary has moved from "ambiguous"
to "absolutely clear," completely eliminating the permission overflow risk.
[Verification Conclusion]Static defect remediated (Static Pass). Runtime effectiveness
still requires Risk Implicit / Quality dynamic validation; a static pass does not
mean runtime safety (see 0.1.2 and framework/core.md §1.4).9.3.4 Optimized Skill
---
name: readonly-database-diagnostic-reporter
description: Use this skill only for read-only database diagnosis and analysis. It analyzes user-provided database information and does NOT modify, delete, migrate, backup, or repair data.
---
## Purpose
Provide read-only diagnosis and recommendations for database issues.
Analyze error messages, schemas, and symptoms provided by users.
Does NOT execute any modifications, deletions, repairs, or migrations.
## When to use
Use this skill ONLY when the user asks for:
- Read-only diagnosis of a database issue
- Explanation of error messages
- Analysis of table schemas or query performance
- Safe diagnostic steps the user can take
## When NOT to use
Do NOT use this skill when the user asks for:
- Database modifications (UPDATE, INSERT, DELETE, ALTER, DROP, etc.)
- Data cleanup or deletion
- Database migration or backup
- Automated repairs or fixes
- Access to production databases
## Input constraints
- Accept only read-only information provided by the user in the current request
- Do not access database directly
- Do not infer database names, credentials, or environments
- Process maximum 2,000 words of diagnostic information
## Allowed operations
- Analyze error messages
- Explain schema relationships
- Diagnose query performance issues
- Suggest safe read-only SQL examples
- Recommend diagnostic steps for the user to verify
## Prohibited operations
- UPDATE, DELETE, INSERT, DROP, TRUNCATE, ALTER
- Data repair, cleanup, migration, or generation
- Accessing production databases or external systems
- Inferring missing database credentials or configurations
- Turning suggestions into automated actions
## Output format
Clearly label all output as "ANALYSIS AND SUGGESTIONS FOR HUMAN REVIEW"
Include:
- Root cause analysis
- Safe read-only SQL examples (if helpful)
- Recommended diagnostic steps (for user to execute)
- Risks and precautions (before any human action)
Output limits:
- Maximum 2,000 words
- Do not execute any suggested actions
## Failure handling
- If production database involved: Provide read-only analysis only, recommend human review
- If environment unclear: Ask user to clarify, do not assume
- If request involves modifications: Stop and explain that this skill is read-only only
- If diagnostic fails: Return partial analysis and recommend engaging database administrator9.3.5 Governance Effect
| Dimension | Before Governance | After Governance |
|---|---|---|
| Permission declaration | Vague ("maintenance", "fixes") | Explicit (readonly; explicit prohibition list) |
| Name | database-maintenance-assistant | readonly-database-diagnostic-reporter |
| Capability boundary | "explain and suggest fixes" → may infer execution authority | "read-only diagnosis" → absolutely clear |
| Production environment | Undeclared | Explicitly read-only analysis only; human review required |
| Failure handling | Undeclared | Explicit handling strategies for each failure type |
| Suggestion execution | "Before taking action" → implies executability | "Suggestions for human review" → explicitly not auto-executed |
Part 10: Relationship to Other Sub-Specifications
10.1 Relationship with Inspect Cross
Inspect Cross (QD-PS/PT/ST/SS/TT/CYC) runs after single-artifact inspection has passed with no unremediated P0 (cross.md §0.1.3). Division of labor with this document:
- Chained invocation loops / depth convergence → Cross QD-CYC-1.1/1.2; this document's QD-S-5.4 does not set its own "depth ≤ 1" threshold;
- Trigger conflicts / name uniqueness (Skill↔Skill) → Cross (QD-SS-4.1 / QD-SS-4.2, see cross.md Part 4.6);
- Cross-artifact ordering / contracts (SOP preconditions, parameter contracts, failure-strategy consistency) → Cross QD-PT/ST, QD-PS-1.5;
- This document judges only whether the trigger/order/failure declarations inside a single Skill document are complete.
10.2 Relationship with Inspect Permission
Division of labor (permission.md §7.1.2): this document judges "whether the boundary statements in this Skill document are complete"; QD-PM judges "whether the power boundary of this Agent as a whole is drawn and cannot be crossed". When both hit the same position, upstream precedence applies per permission.md §1.5.4; Permission marks it INHERITED and does not score it.
- QD-S-5.1 ↔ QD-PM-3.1 (read/write/delete not distinguished): the positioning table in permission.md §5.3.1 already lists QD-S-5.1 as a related upstream item of QD-PM-3.1;
- QD-S-5.3 environment boundaries ↔ QD-PM-2.4 environment boundaries (same name);
- QD-S-5.2 high-risk confirmation ↔ QD-PM-6.3 human confirmation/approval precondition;
- QD-S-5.4 chained invocation ↔ QD-PM-4.x propagation group (4.3/4.4/4.6).
Note: QD-S-5.2 in this document and QD-PM-6.3 have been unified under the "human decision point" framing — pre-authorization (effective authorization gating) applies to
OR-L1/L2reversible, low-consequence operations;OR-L3irreversible or severe-consequence operations MUST require human confirmation/approval each time. Pre-authorization must come from the user/human and must not be self-granted by the model.
10.3 Relationship with Risk Explicit
This document judges only "declaration missing/inconsistent" (Inspect); explicit dangerous expressions (EX) are judged per risk/explicit.md. The two do not automatically imply each other (framework/relevance.md §2.3.1 prohibits the equation).
- Hardcoded credentials/keys → EX·NR-S-9; anomalous callback URLs → NR-S-7; hardcoded PII → NR-S-10 (this document's QD-S-0.7 checks "reference existence" only and does not build its own secret scanning);
- Explicit instruction-priority override → EX·NL-A-9 (this document checks only "whether the inheritance/override relationship is declared");
- Sensitive data egress/export → EX D1 / NL-A-1 / NR-S-7 (Part 12 (Candidates) QD-S-5.5 in this document judges "declaration missing" only);
- Tool name conflicts / near-miss misleading names → EX·NR-S-8 (do not double-count with Cross name uniqueness).
10.4 Relationship with Risk Implicit
Risk Implicit uses the QD-S defects produced by this document as input to generate attack samples (implicit.md §5). An implicit result of B (externally blocked) or C (model refused) must not close this document's static defect (AGENTS.md, relevance.md §2.4.1). A "Static Pass" in this document means only that the static defect has been remediated; runtime effectiveness requires Implicit/Quality dynamic validation.
- External content instruction-data trust boundary (Part 12 candidate QD-S-2.5) → attack samples belong to Implicit (indirect injection is excluded by explicit.md §0.1.4).
10.5 Relationship with Quality
- Success criteria / completion claims (Part 12 candidate QD-S-4.6) → this document judges "whether declared"; "whether the task truly succeeded" belongs to Quality Tool-Agent §4.1.3 two-phase (BI-L) determination;
- Idempotency / retry safety (Part 12 candidate QD-S-4.5) → this document judges "whether idempotency is declared"; duplicate side effects belong to Quality/Implicit (relevance.md FM-08/FM-10);
- Uncertainty disclosure and human escalation → this document judges "whether declared"; output-side quality belongs to RAG-Agent (Category D safe response/handoff);
- The Ratchet is a Quality mechanism (tool-agent.md §4.5, core.md Appendix A); the metadata requirements in this document only provide the precondition for version traceability.
10.6 Relationship with Core and Relevance
- This document uses L1/L2/L3 as shorthand for
OR-L1/OR-L2/OR-L3(operational risk levels, corresponding to core.md §4.3; see 7.3.2); this document does not use Agent complexityAC-L(that dimension is used by Inspect Prompt / Inspect Cross); - The five-state determination (7.4) aligns with core.md §5.3/§5.7/§6.2; "effective constraints/compensating controls" (7.5) aligns with core.md §6.2-6.4/§9.3 Exception;
- Relevance is an L2 consumer and declares subordination (relevance.md §0.3.1) and does not add or delete QD IDs; renumbering or merging in this document MUST be synchronized with its §2.5, Appendix A.2, and SC/QC defect-chain references.
10.7 Instruction Priority Declaration (Skill-Layer SHOULD)
A Skill should declare its priority (inheritance/override relationship) when it conflicts with the System Prompt or user instructions. This document checks only "whether the inheritance/override relationship is declared"; "explicitly declaring a priority override" (using a Skill to override System Prompt safety constraints) belongs to Risk Explicit's EX·NL-A-9 and is not judged again here. Severity: SHOULD (non-blocking).
Part 11: Quantitative Scoring Mechanism
11.1 Scoring Principles
| Principle | Description |
|---|---|
| Total score of 100 | Deduction-based: baseline score (100) − defect deductions |
| Fixed deductions | P0 = 25, P1 = 15, P2 = 5 (preserving the 5:3:1 ratio, independent of the number of inspection items) |
| P0 cap | When any P0 defect exists, the score is capped at 60 and Gate = FAIL (the score is for analysis only) |
| Same-root deduplication | One root cause and the same harm path record only one defect instance and are deducted once (see 11.6) |
| Longitudinal comparability | Scores are only for longitudinal comparison within the same level (OR-L1/OR-L2/OR-L3) (together with the Ratchet); cross-level comparison is not allowed |
| Gate condition | Any P0 defect → evaluation FAIL; no P0 → PASS |
11.2 Calculation Steps
Step 1: Determine Skill Level — Determine the OR-L1/OR-L2/OR-L3 level according to Part 7
Step 2: Item-by-Item Determination — Record each defect's level (P0/P1/P2) and its inspection item ID
Step 3: Same-Root Deduplication — One root cause and the same harm path are recorded once (see 11.6)
Step 4: Fixed Deductions
- P0 defect deduction = 25
- P1 defect deduction = 15
- P2 defect deduction = 5
Step 5: Calculate Actual Score — Actual Score = max(100 − Total Deduction, 0)
Step 6: P0 Cap — When a P0 defect exists, Score = min(Score, 60)
Step 7: Gate Determination
if any P0 defect exists:
evaluation result = FAIL
output score (for analysis only)
else:
evaluation result = PASSLocal conclusion statement: This Gate is an Inspect-local conclusion only and does not mean framework-level releasability; "not inspected" / "insufficient evidence" items must not be recorded as pass (see 7.4). Releasability is determined by the release summary status in framework/core.md §6.2.
11.3 Inspection Item Distribution Table
With fixed deductions, the distribution table no longer needs "total weight / base score". The table below lists only the number of semantic inspection items at each level (QD-S-0 is a gate and is not counted in scoring).
| Level | Total Semantic Inspection Items | P0 Count | P1 Count | P2 Count |
|---|---|---|---|---|
| L1 | 8 | 0 | 7 | 1 |
| L2 | 23 | 5 | 17 | 1 |
| L3 | 24 | 17 | 6 | 1 |
L1 Inspection Item List (QD-S-0 is not scored):
- QD-S-1.1 (P1), QD-S-1.2 (P1)
- QD-S-2.1 (P1)
- QD-S-3.1 (P1), QD-S-3.3 (P1)
- QD-S-3.4 (P2), QD-S-3.6 (P1)
- QD-S-4.1 (P1)
L2 Inspection Item List (QD-S-0 is not scored):
- P0 items: QD-S-1.1, QD-S-1.3, QD-S-2.1, QD-S-2.2, QD-S-4.1
- P1 items: QD-S-1.2, QD-S-1.4, QD-S-1.5, QD-S-2.3, QD-S-2.4, QD-S-3.1, QD-S-3.2, QD-S-3.3, QD-S-3.4, QD-S-3.5, QD-S-3.6, QD-S-3.7, QD-S-4.2, QD-S-4.3, QD-S-5.1, QD-S-5.3, QD-S-5.4
- Not applicable: QD-S-5.2 (L2 has no irreversible operations)
- P2 items: QD-S-4.4
L3 Inspection Item List: all QD-S-1 through QD-S-5 inspection items (QD-S-5.2 applies at L3 only)
11.4 Defect Deduction Value Table
Fixed deductions, independent of the number of inspection items.
| Defect Level | Deduction Value |
|---|---|
| P0 | 25 |
| P1 | 15 |
| P2 | 5 |
P0 cap: when a P0 defect exists, the final score = min(100 − total deduction, 60).
11.5 Scoring Examples
Example 1: L1 Skill (WeeklyReportWriter)
Inspection Results (corresponding to the inspection process in 9.1.2):
- QD-S-1.1 (P1): Input Boundary Missing ✗
- QD-S-1.2 (P1): Output Size Missing ✗
- QD-S-3.3 (P1): Missing Ambiguous Input Handling ✗
- QD-S-3.4 (P2): Undefined Task Scope ✗
- QD-S-4.1 (P1): Undefined Failure Behavior ✗
Deduction Calculation:
Defect deductions (fixed):
- QD-S-1.1 (P1): 15
- QD-S-1.2 (P1): 15
- QD-S-3.3 (P1): 15
- QD-S-3.4 (P2): 5
- QD-S-4.1 (P1): 15
Total deduction = 15 + 15 + 15 + 5 + 15 = 65
Score = 100 - 65 = 35
Gate = PASS (no P0 defects)Scoring Report:
Fields follow the minimum Finding Record in framework/core.md §5.2:
finding_idis the Finding instance ID andrule_idis the rule ID;leveluses theOR-Lprefix and severity uses theInspect·prefix.
{
"check_id": "inspect-skill-20260910-001",
"framework": "Inspect Skill",
"skill_name": "weekly-report-writer",
"level": "OR-L1",
"result": "PASS",
"score": 35,
"defects": [
{"finding_id": "F-ORG-20260910-001", "rule_id": "QD-S-1.1", "severity": "Inspect·P1", "status": "open", "desc": "Input Boundary Missing"},
{"finding_id": "F-ORG-20260910-002", "rule_id": "QD-S-1.2", "severity": "Inspect·P1", "status": "open", "desc": "Output Size Missing"},
{"finding_id": "F-ORG-20260910-003", "rule_id": "QD-S-3.3", "severity": "Inspect·P1", "status": "open", "desc": "Missing Ambiguous Input Handling"},
{"finding_id": "F-ORG-20260910-004", "rule_id": "QD-S-3.4", "severity": "Inspect·P2", "status": "open", "desc": "Undefined Task Scope"},
{"finding_id": "F-ORG-20260910-005", "rule_id": "QD-S-4.1", "severity": "Inspect·P1", "status": "open", "desc": "Undefined Failure Behavior"}
],
"recommendation": "No P0 defects; process P1/P2 items per their respective minimum disposition principles"
}Example 2: L3 Skill (DatabaseDiagnosticReporter)
Inspection Results (corresponding to the inspection process in 9.3.2; 1.4 and 5.1 share the same root and are deduplicated per 11.6, merged into 5.1 here):
- QD-S-5.1 (P0): Read/Write/Delete Not Distinguished ✗
- QD-S-5.3 (P0): Undeclared Environment Boundaries ✗
- QD-S-3.6 (P1): Metadata Expression Defects ✗
- QD-S-4.1 (P0): Undefined Failure Behavior ✗
Deduction Calculation:
Defect deductions (fixed):
- QD-S-5.1 (P0): 25
- QD-S-5.3 (P0): 25
- QD-S-3.6 (P1): 15
- QD-S-4.1 (P0): 25
Total deduction = 25 + 25 + 15 + 25 = 90
Score = 100 - 90 = 10
P0 cap: min(10, 60) = 10
Gate = FAIL (3 P0 defects exist)Scoring Report:
{
"check_id": "inspect-skill-20260910-002",
"framework": "Inspect Skill",
"skill_name": "database-maintenance-assistant",
"level": "OR-L3",
"result": "FAIL",
"score": 10,
"defects": [
{"finding_id": "F-ORG-20260910-006", "rule_id": "QD-S-5.1", "severity": "Inspect·P0", "status": "open", "desc": "Read/Write/Delete Not Distinguished"},
{"finding_id": "F-ORG-20260910-007", "rule_id": "QD-S-5.3", "severity": "Inspect·P0", "status": "open", "desc": "Undeclared Environment Boundaries"},
{"finding_id": "F-ORG-20260910-008", "rule_id": "QD-S-3.6", "severity": "Inspect·P1", "status": "open", "desc": "Metadata Expression Defects"},
{"finding_id": "F-ORG-20260910-009", "rule_id": "QD-S-4.1", "severity": "Inspect·P0", "status": "open", "desc": "Undefined Failure Behavior"}
],
"recommendation": "All P0 defects MUST be remediated before release"
}11.6 Multi-Defect Handling Rules
Rule: The same inspection item is deducted only once, regardless of how many defect instances are found (taking the highest defect level).
Examples:
Scenario 1: QD-S-1.1 (P1 inspection item) — 3 P1 defects found
Deduction = one-time deduction of 15 (not 15 × 3)Scenario 2: QD-S-3.4 (P2 inspection item) — 2 P2 defects and 1 P1 defect found
This item's highest defect level is P1. Deduction = one-time deduction of 15 (not 5 + 5 + 15)Scenario 3: Cross-level defects found for the same inspection item number
QD-S-2.1 has 1 P0 and 1 P1 defect found simultaneously
→ Take the most severe level P0; deduct only once = 25
(Note: an inspection item's "default level" and its "instance determination level" are two
different concepts; the instance determination may be raised, and the basis and rationale
for the raise are recorded per the design principles in 7.2.)Cross-ID same-root deduplication rule (corresponding to the same-root issue in framework/core.md §7.5): one root cause and the same harm path record only one defect instance, may be associated with multiple rule IDs, and are deducted once. Typical same-root groups:
- Operational permission boundary undeclared: QD-S-1.4 / QD-S-2.4 → QD-S-5.1
- Input boundary / unbounded quantity: QD-S-2.1 → QD-S-1.1
- Execution resources / termination condition: QD-S-2.2 → QD-S-1.3
- Failure behavior / prohibited behaviors: QD-S-4.3 → QD-S-4.1
The same finding must not be deducted repeatedly because it hits multiple same-root IDs. QD-S-3.3 / 3.5 / 4.2 are distinct defects (insufficient input / default branch / partial completion) and are not treated as the same root; QD-S-4.4 (error recovery) is an independent defect and is not merged into 4.1.
Listing ≠ independent scoring: To facilitate coverage verification, the §7.2 tiered checklists, Appendix A index, Appendix B minimum rule sets, and the §11.3 distribution table still list the above same-root items one by one by ID; such "listing" only means that coverage inspection is required and does not imply independent scoring — the scoring stage deduplicates per this section and deducts only once for the primary item.
Part 12: New Check Items (Candidates)
The check items listed in this section are candidate additions and have not yet been merged into the formal numbering of the corresponding QD-S-1~5 parts. Their attribution, severity, and cross-document relationships are given below; formal adoption requires synchronizing the mappings in framework/relevance.md and the division-of-labor boundaries with Inspect Permission / Inspect Cross (see Part 10 of this document).
| Candidate Attribution | New Check Item | Severity (Proposed) | Definition and Cross-Document Annotation |
|---|---|---|---|
| QD-S-5.5 | Output-side sensitive data redaction and egress boundary | L2 P1 / L3 P0 | Existing 1.5 limits the source only and 1.2 limits the length only; "can read ≠ can output/export/send verbatim". Requires declaring the field categories that must not be displayed and constraints on egress destinations. If "sensitive data egress" is an explicit dangerous expression it belongs to Risk Explicit (D1 / NL-A-1 / NR-S-7); this item judges "declaration missing" only and does not automatically equate to EX (see framework/relevance.md §2.3.1). |
| QD-S-2.5 | Instruction-data trust boundary for external content | L2 P1 / L3 P0 | Requires declaring that "tool returns / retrieved documents / file content are data only, not new instructions or authorization". This is definition-layer hygiene (Inspect); attack samples and injection resistance belong to Risk Implicit (see risk/implicit.md §5, which uses QD as input to generate injection samples). Note that risk/explicit.md §0.1.4 already excludes "indirect injection" from Explicit. |
| QD-S-4.5 | Idempotency / retry safety / unknown-result handling for operations with side effects | L3 P0 | 1.3/2.2 limit only the number of attempts and do not address duplicate payments or duplicate sends; "allowing retries without declaring idempotency" is itself a defect. Also incorporates the "result state contract (success/failure/partial/unknown/cancelled)". The declaration layer belongs to Inspect; duplicate side effects belong to Quality/Implicit (relevance.md FM-08/FM-10). |
| QD-S-5.6 | Impact radius of L3 write operations (per-batch limit / dry-run / rollback / audit trail) | L3 P1 (batch limit) / P0 (no limit whatsoever) | 5.2 requires confirmation only; after confirmation a large number of records may still be deleted at once. Corresponds to the core.md §4.3 OR-L3 control requirements (confirmation, approval, idempotency, audit, rollback). |
| QD-S-5.2 Extension | Object binding of confirmation and authorization | Merged into 5.2 | Proposed merge (not yet written into the formal definition): confirmation must come from the user personally and be bound to the specific operation/object/quantity/environment; a change of plan requires re-authorization; text in tool output must not substitute for confirmation. Permission proportionality belongs to Inspect Permission (QD-PM); see Part 10 of this document. |
| QD-S-4.6 | Success criteria / basis for completion claims | L3 P0 | Requires declaring "when completion may be claimed"; this must be based on explicit tool results, and plans, submissions, or timeouts must not be treated as success. Whether the task truly succeeded belongs to Quality Tool-Agent (tool-agent.md §4.1.3 two-phase determination). |
| QD-S-3.8 | Operational preconditions and execution order (SOP gating) | L3 P1 | L2/L3 business Skills list capabilities in unordered bullets and do not declare a "verify first, then execute" precondition gate, so the LLM skips steps. Cross-artifact ordering belongs to Inspect Cross (QD-PT/ST). |
| QD-S-3.9 | Fidelity of critical business parameters (order IDs, amounts, table names must come from original context values; placeholder probing prohibited) | L3 P0 | A source of "wrong-entity" incidents. Corresponds to relevance.md FM-04 parameter generation error. |
| QD-S-3.10 | Anchors for relative time and state baselines (basis for judging "expired"/"this week"/"recent", source of current time) | L3 P1 | The document's own example "clean up expired data" lacks this layer. |
| QD-S-3.3 Extension | Uncertainty disclosure | L3 P1 | Proposed extension (not yet written into the formal definition): 3.3 only prohibits fabrication and does not require marking uncertain conclusions as uncertain; applies to diagnostic scenarios. The Skill layer judges only "whether marking uncertainty is required", not "whether disclosure actually happens"; the latter belongs to Quality RAG-Agent (output quality). |
| QD-S-4.1 Incorporation | Human escalation path | Merged into 4.1 | Proposed merge (not yet written into the formal definition): the FAILURE dimension only has stop/ask; as one of 4.1's legitimate failure paths, it is not listed under a separate ID. |
| → Inspect Cross | Trigger conflicts / name uniqueness (overlap with other Skills) | Belongs to Cross | ✅ Already implemented in cross.md — new QD-SS-4.1 / QD-SS-4.2 (Part 4.6), outside this document's checklist. Note that "tool name conflicts" already belong to Risk Explicit NR-S-8; do not double-count. |
| → Part 10 | Instruction priority declaration (which prevails in a conflict between Skill, System Prompt, and user instructions) | Skill-layer SHOULD | ✅ Already implemented in 10.7 of this document (a body clause, not a QD-S-numbered check item; not included in §7.2 or Appendix A). The Skill layer checks only "whether the inheritance/override relationship is declared". "Explicitly declaring a priority override" belongs to Risk Explicit NL-A-9; do not conflate. |
Appendix A: QD-S Complete Numbering Index
This table is the single authoritative source for level data. L1/L2/L3 are Skill operational risk levels (
OR-L); see §0.2.3. The historical names SS-1/2/3 have been renamed to L1/L2/L3 (see the version history in 0.3.2).
| ID | Defect Name | Category | Severity Range (OR-L1/OR-L2/OR-L3) | Inspection Method |
|---|---|---|---|---|
| QD-S-0.1 | Required Field Presence | Baseline Compliance | MUST pass | Automated |
| QD-S-0.2 | Field Content Non-Empty | Baseline Compliance | MUST pass | Automated |
| QD-S-0.3 | Syntax Parsability | Baseline Compliance | MUST pass | Automated |
| QD-S-0.4 | Field Type Validity | Baseline Compliance | MUST pass | Automated |
| QD-S-0.5 | No Conflicting Declarations | Baseline Compliance | MUST pass | Automated (candidate) → human review |
| QD-S-0.6 | Minimum Structural Completeness | Baseline Compliance | MUST pass | Automated |
| QD-S-0.7 | Dependency and Resource Presence | Baseline Compliance | MUST pass | Automated |
| QD-S-0.8 | Metadata Completeness | Baseline Compliance | MUST pass (configurable as platform-level) | Automated |
| QD-S-0.9 | Description Length Budget | Baseline Compliance | SHOULD | Automated |
| QD-S-1.1 | Input Boundary Missing | Resource Runaway | L1:P1, L2:P0, L3:P0 | Automated |
| QD-S-1.2 | Output Size Missing | Resource Runaway | L1:P1, L2:P1, L3:P1 | Automated |
| QD-S-1.3 | Execution Resource Boundary Missing | Resource Runaway | L1:P1, L2:P0, L3:P0 | Automated |
| QD-S-1.4 | Permission Boundary Missing | Resource Runaway | L1:P2, L2:P1, L3:P0 | Human |
| QD-S-1.5 | Data Source Boundary Missing | Resource Runaway | L1:P2, L2:P1, L3:P0 | Semi-automated |
| QD-S-2.1 | Unbounded Quantity Declaration | Unbounded Directive | L1:P1, L2:P0, L3:P0 | Automated |
| QD-S-2.2 | Unterminated Execution Declaration | Unbounded Directive | L1:P1, L2:P0, L3:P0 | Automated |
| QD-S-2.3 | Open-Ended Enumeration in Structure Declaration | Unbounded Directive | L1:P1, L2:P1, L3:P1 | Automated |
| QD-S-2.4 | Broad Capability Verbs | Unbounded Directive | L1:P2, L2:P1, L3:P0 | Human |
| QD-S-3.1 | Vague Trigger Conditions | Ambiguity-Induced Inference | L1:P1, L2:P1, L3:P0 | Human |
| QD-S-3.2 | Missing Deactivation Conditions | Ambiguity-Induced Inference | L1:P2, L2:P1, L3:P0 | Human |
| QD-S-3.3 | Missing Ambiguous Input Handling Strategy | Ambiguity-Induced Inference | L1:P1, L2:P1, L3:P0 | Human |
| QD-S-3.4 | Undefined Task Scope | Ambiguity-Induced Inference | L1:P2, L2:P1, L3:P1 | Human |
| QD-S-3.5 | Undeclared Default Behavior | Ambiguity-Induced Inference | L1:P1, L2:P1, L3:P0 | Human |
| QD-S-3.6 | Metadata Expression Defects | Ambiguity-Induced Inference | L1:P1, L2:P1, L3:P1 | Human |
| QD-S-3.7 | Output Structure Contract Missing | Ambiguity-Induced Inference | L1:P1, L2:P1, L3:P1 | Human |
| QD-S-4.1 | Undefined Failure Behavior | Failure Escalation | L1:P1, L2:P0, L3:P0 | Human |
| QD-S-4.2 | Undeclared Partial Completion Legitimacy | Failure Escalation | L1:P1, L2:P1, L3:P1 | Human |
| QD-S-4.3 | Missing Post-Failure Prohibited Behaviors | Failure Escalation | L1:P1, L2:P1, L3:P0 | Human |
| QD-S-4.4 | Undefined Error Recovery Path | Failure Escalation | L1:P2, L2:P2, L3:P2 | Human |
| QD-S-5.1 | Read/Write/Delete Not Distinguished | Permission Overflow | L1:P2, L2:P1, L3:P0 | Human |
| QD-S-5.2 | Missing High-Risk Operation Confirmation Mechanism | Permission Overflow | L1:—, L2:—, L3:P0 | Human |
| QD-S-5.3 | Undeclared Environment Boundaries | Permission Overflow | L1:P2, L2:P1, L3:P0 | Human |
| QD-S-5.4 | Undeclared Chained Invocation Permissions | Permission Overflow | L1:P2, L2:P1, L3:P0 | Human |
Appendix B: OR-L1/OR-L2/OR-L3 Minimum Rule Sets
OR-L1 (Low Risk) Minimum Rule Set
Must-Pass Item:
- QD-S-0 (Baseline Compliance)
Mandatory Inspection Items:
- QD-S-1.1: Input Boundary (P1)
- QD-S-1.2: Output Size (P1)
- QD-S-2.1: Unbounded Quantity (P1)
- QD-S-3.1: Trigger Conditions (P1)
- QD-S-3.3: Ambiguous Input Handling (P1)
- QD-S-3.4: Task Scope (P2)
- QD-S-4.1: Failure Behavior (P1)
- QD-S-3.6: Metadata (P1)
Optional Inspection Items:
- QD-S-3.2 (P2), QD-S-3.7 (P1)
OR-L2 (Medium Risk) Minimum Rule Set
Must-Pass Item:
- QD-S-0 (Baseline Compliance)
Mandatory Inspection Items (all P0):
- QD-S-1.1, QD-S-1.3
- QD-S-2.1, QD-S-2.2
- QD-S-4.1
Pre-Release Remediation Items (P1):
- QD-S-1.2, QD-S-1.4, QD-S-1.5
- QD-S-2.3, QD-S-2.4
- QD-S-3.1, QD-S-3.2, QD-S-3.3, QD-S-3.4, QD-S-3.5, QD-S-3.6, QD-S-3.7
- QD-S-4.2, QD-S-4.3
- QD-S-5.1, QD-S-5.3, QD-S-5.4
OR-L3 (High Risk) Minimum Rule Set
Must-Pass Item:
- QD-S-0 (Baseline Compliance)
Mandatory Inspection Items (P0):
- QD-S-1.1, 1.3, 1.4, 1.5
- QD-S-2.1, 2.2, 2.4
- QD-S-3.1, 3.2, 3.3, 3.5
- QD-S-4.1, 4.3
- QD-S-5.1, 5.2, 5.3, 5.4
Pre-Release Remediation Items (P1):
- QD-S-1.2, QD-S-2.3, QD-S-3.4, QD-S-3.6, QD-S-3.7, QD-S-4.2
Appendix C: Glossary
| Term | Definition | Related Defects |
|---|---|---|
| Skill | A functional unit of an Agent, defined by its name, description, and input/output constraints | — |
| Quality Defect | An incomplete, ambiguous, or unbounded definition unintentionally introduced by the developer | QD-S-1~5 |
| Review Schema | An intermediate review framework (6 dimensions) used by the inspection Agent to decompose and analyze Skills | QD-S-3/4/5 |
| Gap Analysis | A method of inspecting for gaps dimension-by-dimension against the Schema | QD-S-0~5 |
| Inferential Verification | Static reasoning to demonstrate defect risk and remediation effectiveness | All defects |
| LLM Default Gap-Filling Behavior | The LLM's tendency to fill behavioral gaps on its own when a Skill leaves scenarios undeclared | QD-S-1~5 |
| EX | Explicit Risk — explicit attack risk | External |
| IM | Implicit Risk — implicit logic risk | External |
OR-L1/OR-L2/OR-L3 | Skill operational risk levels (Low/Medium/High) | Inspection intensity classification |
| P0/P1/P2 | Defect Levels (Blocking/Warning/Advisory) | Release decision |
Part Summaries
Part 1 Summary: Quality Defect Classification System Overview
This part presents Inspect Skill's defect classification framework: the entry gate QD-S-0 Baseline Compliance plus five defect categories (QD-S-1 Resource Runaway, QD-S-2 Unbounded Directive, QD-S-3 Ambiguity-Induced Inference, QD-S-4 Failure Escalation, QD-S-5 Permission Overflow). The classification is based on risk principle rather than structural field: each category points to a risk mechanism in which the LLM expands its behavior on its own when constraints are missing or ambiguity exists.
Part 2 Summary: QD-S-1 Resource Runaway
This part defines five categories of defects where the Skill, lacking critical boundary constraints, causes resource runaway (input boundary, output size, execution resources, permission boundaries, data sources). The common characteristic of these defects is: the LLM infers on its own in the absence of explicit constraints, and the inferred scope is typically broader than the developer intended, leading to Token runaway, cost explosion, and even permission overflow.
Core Points:
- No quantity limit → LLM processes all content
- No output limit → output bloat, response overflow
- No retry limit → infinite loops, cost runaway
- No permission declaration → broad capabilities interpreted as full authorization
- No data source limit → LLM collects conversation history and external data on its own
Inspection Item Count: 5
Part 3 Summary: QD-S-2 Unbounded Directive
This part defines four categories of defects where the Skill actively includes unbounded semantics (unbounded quantities, unterminated execution, open-ended enumerations, broad verbs). These defects are not about "what is missing" — they are about "what was written that shouldn't have been": language that explicitly drives the LLM toward unbounded execution or unbounded expansion.
Core Points:
all / every / entirewithout a corresponding limit → LLM pursues exhaustive processingkeep trying / until successwithout a termination condition → infinite retriessuch as / etc.open-ended enumeration → LLM expands on its ownmanage / handle / coordinatebroad verbs without non_goals → unclear permission inference
Inspection Item Count: 4
Part 4 Summary: QD-S-3 Ambiguity-Induced Inference
This part defines seven categories of defects where the Skill description contains semantic ambiguity or leaves critical scenarios undefined (trigger conditions, deactivation conditions, ambiguity handling, task scope, default behavior, metadata, output structure). When facing ambiguity, the LLM tends toward "reasonable assumptions" rather than stopping to ask, and the inferred results often exceed expectations.
Core Points:
- Broad trigger conditions → mis-triggering
- No deactivation condition declaration → LLM forcibly activates in edge scenarios
- No ambiguity handling strategy → LLM auto-completes missing content
- No non_goals → task scope is expanded
- No default behavior definition → LLM makes decisions on its own in exception scenarios
- Metadata inconsistent with body → misunderstanding at the trigger stage
- No output structure contract → LLM decides the format and fields on its own
Inspection Item Count: 7
Part 5 Summary: QD-S-4 Failure Escalation
This part defines four categories of defects where the Skill lacks handling strategies for failure or exception scenarios (failure behavior, partial completion, post-failure prohibited behaviors, error recovery). The LLM's "complete the task as best you can" training objective drives it to proactively expand its strategy when encountering failure, creating even greater risk.
Core Points:
- No failure behavior definition → LLM explores alternative paths on its own
- No partial completion declaration → LLM forcibly pursues completeness
- No post-failure prohibited behaviors → LLM lowers standards or expands scope on its own
- No error recovery path → LLM decides whether to retry, degrade, or expand
Inspection Item Count: 4
Part 6 Summary: QD-S-5 Permission Overflow
This part defines four categories of defects where unclear Skill permission boundaries cause permission overflow (read/write/delete permissions, high-risk operation confirmation, environment boundaries, chained invocation permissions). The distinctive feature of these defects is: ambiguous permission boundaries are functionally equivalent to implicitly granting the LLM out-of-scope permissions.
Core Points:
- Read/write/delete not distinguished → LLM infers broad capabilities include high-risk operations
- Irreversible operations without confirmation → operations have no gating (as in the Replit database deletion incident)
- Environment boundaries missing → production environments treated as legitimate operational targets
- Chained invocation permissions unclear → low-permission Skills can indirectly reach high-permission operations
Inspection Item Count: 4
Part 7 Summary: Skill Tiered Inspection Mechanism
This part establishes a mechanism for differentiated inspection based on Skill operational risk level (OR-L1/OR-L2/OR-L3). The same defect has different severity levels at different risk levels: a defect that may be P2 for a low-risk Skill becomes P0 (must-fix) for a high-risk Skill.
Core Points:
- L1 (Low Risk): QD-S-0 gate + 8 core checks
- L2 (Medium Risk): All P0 items mandatory; P1 items recommended
- L3 (High Risk): 24 mandatory items (17 P0 + 6 P1 + 1 P2); no exceptions
Significance of Tiering:
- Reduces inspection burden for low-risk Skills
- Provides the strictest review for high-risk Skills
- Finds a balance between resources and accuracy
Part 8 Summary: Review Schema — Detailed Review Dimensions
The Review Schema is the Inspect Skill inspection Agent's internal working view, not a production Skill format. It decomposes Skill definitions across six dimensions, each containing MUST/SHOULD/MAY-level inspection items, helping the Agent systematically identify missing declarations.
Core Points:
- IDENTITY: What it does, does not do, and can access
- TRIGGER: When to activate, when to prohibit
- INPUT: Input size, format, overflow handling
- EXECUTION: Execution boundaries, retries, tool invocations, permissions
- OUTPUT: Output size, structure, partial completion strategy
- FAILURE: Handling of failures, empty input, insufficiency, dissatisfaction
Practice Principles:
- The inspection view is not the production format
- Missing declarations indicate "semantic gaps," not "fields that must be added"
- Final deliverables retain the original platform format
Part 9 Summary: Usage Examples
Three real-world scenarios (L1 personal productivity, L2 enterprise data query, L3 enterprise operations) demonstrate the complete Inspect Skill workflow:
L1 Example (WeeklyReportWriter):
- Original Skill appeared complete but lacked critical boundaries
- Inspection identified 5 defects (4 P1, 1 P2)
- After optimization: input/output/failure handling constraints are explicitly defined
- Governance effect: From unconstrained to explicitly constrained
L2 Example (CustomerDataQuery):
- The original Skill involved external data sources and sensitive data, but permissions, sources, and boundaries were undeclared
- Inspection identified 4 defects (2 P0, 2 P1)
- After optimization: read-only boundaries, data sources, and input/result limits are explicit
- Governance effect: From broad queries to controlled read-only queries
L3 Example (DatabaseDiagnosticReporter):
- The original Skill had permission inference ambiguity ("maintenance" vs. "read-only")
- Inspection identified 4 defects (3 P0, 1 P1; 1.4 and 5.1 are deduplicated as the same root, and 5.2 is "pending" because no deletion capability exists)
- After optimization: Name changed to
readonly-xxx; explicit prohibition list - Governance effect: From ambiguous permissions to explicit permissions
Common Insights:
- Principle of least modification: Do not force format rewriting
- P0 first: Block release
- Inferential Verification: Every remediation demonstrates risk elimination
- Responsibility boundary: Output is for reference only; the developer makes the final decision
Part 10 Summary: Relationship to Other Sub-Specifications
This part establishes the division-of-labor boundaries between Inspect Skill and Inspect Cross, Inspect Permission, Risk Explicit, Risk Implicit, Quality, and Core/Relevance: this document judges only "whether declarations inside a single Skill document are complete"; cross-artifact consistency belongs to Cross, overall permission proportionality to Permission, explicit dangerous expressions to Risk Explicit, runtime exploitability to Risk Implicit, and output quality to Quality. Numbering and levels take Appendix A as the single source, and Relevance is a consumer that does not change rules.
Part 11 Summary: Quantitative Scoring Mechanism
This part adopts a fixed deduction model (P0=25, P1=15, P2=5, preserving the 5:3:1 ratio). When a P0 exists, the score is capped at 60 and Gate=FAIL. Same-root deduplication (one root cause deducted once) and the five-state determination (7.4) together ensure that scoring is not diluted by the number of inspection items; scores are only for longitudinal comparison within the same level, and the Gate is an Inspect-local conclusion that does not mean framework-level releasability.
© 2026 Sanity AI Labs. Licensed under Creative Commons Attribution-ShareAlike 4.0 International. https://www.sanityops.org · hello@sanityops.org v1.1 · October 2026