Skip to content

SanityOps Framework ​

Inspect Tool Specification ​


Version: v1.1

Release Date: October 2026

Maintained by: Sanity AI Labs

License: CC BY-SA 4.0

Dependent Specification: Core v1.1 (objects, Finding, OR-L, versioning and evidence)


Table of Contents ​


Foreword ​

Document design principle: serving both inspection and audit needs.

To reduce the risk of missed findings and to preserve traceable audit evidence, this document allows the same object to be hit by multiple inspection items across different dimensions — this may look like "similarity / duplication / subsumption" on the surface, but it is intentional by design, not redundancy.

At the quantitative scoring stage, however, to avoid double deductions and to reflect the true quality of the inspected Logic Artifact objectively, same-source deduplication and merging must follow the "Multi-Defect Handling Rules" (§5.5) of this document: one root cause and one harm path are deducted only once.

If you question whether certain inspection items are "overly redundant", verify consistency with counter-examples and edge cases — confirm whether the trigger conditions truly overlap and whether the attribution reaches consistent boundary conclusions — rather than concluding from surface similarity alone.

0.1 Positioning and Scope ​

0.1.1 What This Specification Is ​

This specification is one of the core sub-specifications of the SanityOps Framework. It defines the quality defect inspection standards for Tool Schemas in AI Agent systems.

This specification is designed to:

  • Identify defects in Tool Schemas across dimensions including structural validity, parameter constraints, and semantic clarity
  • Provide a unified inspection framework for Tool Schema design, review, and audit
  • Provide explicit rule definitions for automated inspection tooling

0.1.2 What This Specification Is Not ​

This specification is not:

  • A Tool development guide: it does not replace a Tool design guide
  • A runtime security specification: it does not cover security validation during Tool execution
  • A performance optimization specification: it does not address Tool execution efficiency
  • An API design specification: it does not replace RESTful or GraphQL API design best practices

0.1.3 Inspection Scope ​

This specification's inspection scope includes:

Inspection Objects:

  • Tool Schema definitions (including name, description, inputSchema)
  • Tool parameter definitions (including type, constraints, description)
  • Tool semantic definitions (including trigger conditions, side effects, relationships with other Tools)

Core Inspection Dimensions:

├─ Structural Validity (QD-T-1.x)
│   └─ Base field completeness, Schema legality
│
├─ Parameter Constraints (QD-T-2.x)
│   └─ Boundary constraints for String, Number, Array, Object
│
├─ High-Risk Operations (QD-T-3.x)
│   └─ Write operations, sensitive data, cross-Tool data flow
│
├─ Semantic Clarity (QD-T-4.x)
│   └─ Description quality, semantic competition
│
└─ Schema Changes (QD-T-5.x)
    └─ Compatibility, breaking change determination

0.1.4 Out of Scope ​

This specification does not cover:

  • Runtime parameter value validation for Tools
  • Tool execution process monitoring and logging
  • Tool performance and response time
  • Tool permission control implementation (only inspects Schema-level risk)

0.1.5 Position Within the SanityOps Framework ​

SanityOps Framework
│
├─ Inspect (Defect Inspection)
│   ├─ Inspect Tool ← Tool Schema defect inspection ← You are here
│   ├─ Inspect Prompt ← System Prompt defect inspection
│   ├─ Inspect Skill ← Skill defect inspection
│   ├─ Inspect Cross ← Cross-Artifact defect inspection
│   └─ Inspect Permission ← Permission-responsibility proportionality inspection (QD-PM)
│
├─ Risk (Risk Scanning)
│   ├─ Risk Explicit ← Explicit Logic Artifact risk detection
│   └─ Risk Implicit ← Implicit Agent runtime vulnerability scanning
│
├─ Quality (Service Quality)
│   ├─ Quality RAG-Agent ← RAG-Agent service quality assessment
│   └─ Quality Tool-Agent ← Tool-Agent service quality assessment
│
└─ Relevance (Impact Mapping)
    └─ Relevance ← defect → risk/quality diagnostic mapping

0.1.6 JSON Schema Dialect and Target Protocol ​

Rule determinations in this specification use the declared JSON Schema dialect (Draft 2020-12 by default) and the target platform's Schema subset as inspector input parameters. Rules such as QD-T-1.3 (enum legality), QD-T-1.4 (additionalProperties policy), and QD-T-2.2 (format contract) are all determined on this basis.

Different protocols name the Tool input parameter Schema field differently. This specification uniformly refers to the input parameter Schema, and the inspector adapts according to the table below:

Protocol/PlatformInput Parameter Schema FieldNote
MCPinputSchema
OpenAIparametersStrict Mode mandates additionalProperties: false
Anthropicinput_schema

For a Tool without parameters, the valid input parameter Schema form is {"type": "object", "properties": {}} (optionally with additionalProperties: false); in this case, Part 2 of this specification (QD-T-2.x parameter constraints) is recorded as "not applicable".

0.2 Terminology and Numbering System ​

0.2.1 Core Terminology ​

TermDefinition
ToolAn external functional unit invocable by an AI Agent, consisting of a name, description, and input parameter Schema
Tool SchemaThe structured definition of a Tool, including its name, description, and input parameter JSON Schema
inputSchemaThe JSON Schema definition of the Tool's input parameters
DefectA design issue in the Tool Schema that may cause LLM misuse, security risk, or quality problems
Operational Risk Level (OR-L)The tier assigned to a Tool based on operational risk (OR-L1/OR-L2/OR-L3); not the Agent Complexity Level (AC-L)
Defect LevelThe severity of a defect's impact on the system (P0/P1/P2)

0.2.2 Numbering System ​

This specification uses the QD-T-x.y numbering system:

QD-T-x.y

├─ QD: Quality Defect
├─ T: Tool Schema
├─ x: Defect category number
│   ├─ 1: Structural Validity Defects
│   ├─ 2: Parameter Constraint Defects
│   ├─ 3: High-Risk Operation Defects
│   ├─ 4: Semantic Clarity Defects
│   └─ 5: Schema Change Defects
└─ y: Specific defect item number

Examples:

  • QD-T-1.1: Structural Validity Defects → Base Field Missing
  • QD-T-2.1: Parameter Constraint Defects → String Without maxLength
  • QD-T-3.4: High-Risk Operation Defects → Authorization Context Parameter Exposure

0.2.3 Risk Grading Note ​

The risk grading in this specification (OR-L1/OR-L2/OR-L3) is the Operational Risk Level (OR-L), see §3.1; defect level symbols (P0/P1/P2) are defined in §3.2. No separate risk color scale is used.

0.2.4 Abbreviation Table ​

AbbreviationFull NameMeaning
QD-TQuality Defect - ToolTool Schema Quality Defect
SchemaJSON SchemaJSON data structure definition specification

0.2.5 Defect Level Definitions (P0/P1/P2) ​

Defect LevelSymbolDefinitionDisposition Meaning
P0⛔BlockingBlocks release; MUST be fixed
P1⚠️WarningSHOULD be fixed before release
P2ℹ️AdvisorySHOULD be included in the improvement plan or risk register

0.3 Version and Maintenance Information ​

0.3.1 Current Version ​

  • Version: v1.1
  • Release Status: In revision (includes Pending Cross-Spec Alignment items E-1 to E-21 in Appendix E)
  • Release Basis Note: This version contains the pending cross-spec alignment items listed in Appendix E (E-1 to E-21). Scoring results for inspection items marked PENDING are provisional and subject to review after alignment.
  • Last Updated: October 2026
  • Maintained by: Sanity AI Labs

0.3.2 Version History ​

VersionRelease DateMajor Changes
v1.02026-10Initial release
v1.12026-10Expanded Inspect and Permission defect coverage; Risk Explicit regraded to P0/P1/P2; multi-hit design principle clarified; terminology aligned to Logic Artifacts

0.3.3 Applicability Statement ​

This specification v1.1 applies to the Tool Schema quality inspection phase of the SanityOps Framework. When using this specification for Tool quality inspection, all requirements of the corresponding version SHALL be followed.


Part 1: Structural Validity Defects ​

1.1 Positioning of This Part ​

1.1.1 Inspection Objective ​

This part inspects the basic structural validity of Tool Schemas, ensuring that Tool definitions can be correctly parsed and understood.

1.1.2 Inspection Dimensions ​

Structural Validity Inspection
├─ Base Field Completeness
│   └─ Whether required name, description, inputSchema are present
│
├─ Name Validity and Uniqueness
│   └─ Naming format, uniqueness within the manifest
│
├─ Required Field Correctness
│   └─ Whether the required list references fields defined in properties
│
├─ Enum Constraint Validity
│   └─ Whether enum values conflict with type or are duplicated
│
├─ additionalProperties Policy
│   └─ Whether the policy is appropriate (checked layer by layer; nested objects drop one level)
│
├─ Constraint Self-Consistency and default Validity
│   └─ min/max boundaries; whether default violates constraints
│
└─ type Declaration and Schema Complexity
    └─ Whether type is missing, overly broad, or nesting/parameter scale is excessive

1.1.3 Relationship to Other Parts ​

Part 1: Structural Validity ← Foundation Layer
    ↓ Ensures the Tool can be correctly parsed
Part 2: Parameter Constraints, High-Risk Operations, and Semantic Clarity ← Completeness Layer
    ↓ Ensures parameters are adequately constrained
Part 3: Tiered Inspection Mechanism ← Inspection Layer
    ↓ Determines inspection intensity based on risk level
Part 4: Schema Changes ← Change Layer
    ↓ Inspects whether changes introduce risk
Part 5: Quantitative Scoring Mechanism ← Evaluation Layer
    ↓ Converts defects into comparable scores

1.2 Defect Classification ​

1.2.1 Base Field Missing (QD-T-1.1) ​

Definition: The Tool Schema is missing required base fields.

Defect Description:

Required fields:
├─ name: Tool name
├─ description: Tool description
└─ inputSchema: Input parameter Schema

Optional fields:
├─ outputSchema: Output parameter Schema
└─ metadata: Metadata

Defect Example:

❌ Incorrect:

json
{
  "name": "get_weather",
  "inputSchema": {
    "type": "object",
    "properties": {
      "city": {"type": "string"}
    }
  }
}

The description field is missing.

✅ Correct:

json
{
  "name": "get_weather",
  "description": "Get weather information for a specified city",
  "inputSchema": {
    "type": "object",
    "properties": {
      "city": {
        "type": "string",
        "description": "City name, e.g., Beijing, Shanghai"
      }
    },
    "required": ["city"]
  }
}

Impact:

  • The LLM cannot understand the Tool's purpose
  • May lead to incorrect Tool selection

Defect Level: P0

1.2.2 required References an Undefined Field (QD-T-1.2) ​

Definition: The required list references a field that is not defined in properties. Such a field reference is not illegal in general JSON Schema, but it is an explicit defect for an LLM Tool Schema.

Supplementary Check (manual): Field necessity is inconsistent with the business contract—a required field is not listed in required, or a non-required field is incorrectly marked as required. This sub-item requires business semantics and cannot be determined statically from the Schema alone.

Defect Example:

❌ Incorrect (required references an undefined field):

json
{
  "inputSchema": {
    "type": "object",
    "properties": {
      "city": {"type": "string"}
    },
    "required": ["city", "date"]
  }
}

required references "date," but "date" is not defined in properties.

✅ Correct:

json
{
  "inputSchema": {
    "type": "object",
    "properties": {
      "city": {"type": "string"},
      "date": {"type": "string"}
    },
    "required": ["city"]
  }
}

Impact:

  • A reference to an undefined field makes the required constraint impossible to validate correctly
  • May mislead the LLM into generating invalid invocations

Defect Level: P0 (undefined field reference, automated); P1 (business necessity inconsistency, manual)

1.2.3 enum–type Conflict / Duplicate Values (QD-T-1.3) ​

Definition: The enum field has defects, in two classes:

  1. Duplicate values: duplicate elements in enum (functionally harmless but redundant) — P2.
  2. enum value conflicts with type: an enum value does not conform to the declared type (unreachable value) — P1.

Caliber Notes:

  • A mixed-type enum or an enum containing null is legal in general JSON Schema and MUST NOT be judged illegal wholesale; whether it is allowed depends on the declared JSON Schema dialect (see §0.1.6).
  • "Empty value" means the empty string ""; null is treated as an optional legal value only when explicitly included in the enum, and is not a defect.

Defect Example:

❌ Incorrect (duplicate values):

json
{
  "type": "string",
  "enum": ["active", "inactive", "active"]
}

Duplicate values exist in the enum.

❌ Incorrect (type conflict):

json
{
  "type": "integer",
  "enum": [1, 2, "3"]
}

The enum value "3" conflicts with type integer and is unreachable.

✅ Correct:

json
{
  "type": "string",
  "enum": ["active", "inactive", "pending"]
}

Impact:

  • Duplicate values cause redundancy and may mislead the LLM
  • Enum values with type conflicts are unreachable, and parameter validation may fail

Defect Level: P1 (type conflict) / P2 (duplicate values)

1.2.4 Inappropriate additionalProperties Policy (QD-T-1.4) ​

Definition: The additionalProperties policy for an Object-type parameter is inappropriate—this is not "a defect whenever not explicitly declared." The policy is checked for reasonableness: closed-contract fields SHOULD declare additionalProperties: false; open dictionaries such as tags/metadata MAY omit it (default true, i.e., open).

Caliber Note: Nested objects are checked layer by layer; the defect level for nested objects is one level lower than for the root object.

Defect Example:

❌ Incorrect (closed-contract field does not disable additional properties):

json
{
  "type": "object",
  "properties": {
    "name": {"type": "string"}
  }
}

A closed-contract field does not declare additionalProperties; arbitrary extra fields are permitted by default.

✅ Correct (closed contract):

json
{
  "type": "object",
  "properties": {
    "name": {"type": "string"}
  },
  "additionalProperties": false
}

✅ Correct (open dictionary scenario permitted):

json
{
  "type": "object",
  "properties": {
    "tags": {
      "type": "object",
      "additionalProperties": {"type": "string"}
    }
  }
}

Impact:

  • An inappropriate policy may let the LLM pass unexpected fields
  • Incorrectly prohibiting open-dictionary scenarios restricts legitimate input

Defect Level: L1 P2 / L2 P1 / L3 P1 (nested objects one level lower)

1.2.5 Name Validity and Uniqueness (QD-T-1.5) ​

Definition: The Tool name is illegal or duplicated within the same Tool manifest. Legal requirements: non-empty, matching ^[a-zA-Z0-9_-]{1,64}$, unique within the same Tool manifest; the verb_noun structure is recommended (style suggestion only, P2).

Defect Example:

❌ Incorrect (illegal characters):

json
{
  "name": "get weather!!",
  "description": "Get weather"
}

❌ Incorrect (duplicate within manifest): two Tools in the same Tool manifest are both named get_user.

✅ Correct:

json
{
  "name": "get_weather",
  "description": "Get weather information for a specified city"
}

Impact:

  • An illegal name may cause platform parsing to fail
  • Duplicate names make it impossible for the LLM to distinguish Tools

Defect Level: Illegal/duplicate P0 (all levels); style P2

Inspection Method: Automated

1.2.6 Constraint Self-Consistency and default (QD-T-1.6) ​

Definition: Constraints within the Schema contradict each other or the default violates a constraint, including: minimum > maximum, minLength > maxLength, minItems > maxItems, default violating type/enum/range, required and default coexisting (semantically contradictory and requiring explanation), and unsatisfiable combined constraints.

Defect Example:

❌ Incorrect (contradictory boundaries):

json
{
  "type": "integer",
  "minimum": 10,
  "maximum": 5
}

❌ Incorrect (default violates enum):

json
{
  "type": "string",
  "enum": ["active", "inactive"],
  "default": "deleted"
}

✅ Correct:

json
{
  "type": "integer",
  "minimum": 5,
  "maximum": 10
}

Impact:

  • Unsatisfiable constraints mean parameters can never pass validation
  • A default violating constraints causes inconsistent behavior

Defect Level: P1 (all levels); P0 in L3 when it affects high-risk parameters

Inspection Method: Automated

1.2.7 Missing type / Overly Broad Type / Schema Complexity (QD-T-1.7) ​

Definition: A parameter lacks a type, has an overly broad type, or the Schema complexity exceeds limits, including: no type, multi-type arrays, unconstrained object, nesting >4 levels, >30 parameters, abuse of oneOf/anyOf/allOf/$ref, and nullable union types.

Defect Example:

❌ Incorrect (no type and no constraint):

json
{
  "type": "object",
  "properties": {
    "data": {}
  }
}

✅ Correct:

json
{
  "type": "object",
  "properties": {
    "data": {
      "type": "object",
      "additionalProperties": false,
      "properties": {}
    }
  }
}

Impact:

  • A missing type invalidates validation
  • Excessive complexity makes it difficult for the LLM to generate correct invocations

Defect Level: L1 P2 / L2 P1 / L3 P1

Inspection Method: Automated

1.3 Severity Classification ​

Defect IDDefect NameL1L2L3Inspection Method
QD-T-1.1Base Field MissingP0P0P0Automated
QD-T-1.2required References Undefined FieldP0P0P0Automated
QD-T-1.3enum–type Conflict / Duplicate ValuesP1 / P2P1 / P2P1 / P2Automated
QD-T-1.4Inappropriate additionalProperties PolicyP2P1P1Automated
QD-T-1.5Name Validity and UniquenessP0 (style P2)P0 (style P2)P0 (style P2)Automated
QD-T-1.6Constraint Self-Consistency and defaultP1P1P1 (high-risk parameters P0)Automated
QD-T-1.7Missing type / ComplexityP2P1P1Automated

1.4 Detection Methods ​

The defects in this part can be detected automatically through protocol meta-Schema validation + custom policy checker:

Detection Tools:
├─ JSON Schema meta-Schema validator
├─ OpenAPI Schema Validator
└─ Custom policy checker

Detection Rules:
├─ Check whether required fields exist
├─ Check whether required references fields defined in properties
├─ Check name validity and uniqueness within the manifest
├─ Check whether enum conflicts with type or contains duplicates
├─ Check whether the additionalProperties policy is appropriate
├─ Check constraint self-consistency and default validity
└─ Check type declarations and Schema complexity

1.5 Part Summary ​

This part defines the basic structural validity inspection for Tool Schemas, ensuring that Tool definitions can be correctly parsed and understood.

Core Points:

  • Base fields (name, description, inputSchema) MUST be complete
  • name MUST be legal and unique within the manifest
  • The required list MUST reference fields defined in properties
  • Enum values MUST NOT conflict with type and MUST NOT be duplicated
  • The additionalProperties policy MUST be appropriate
  • Constraints MUST be self-consistent, and default MUST NOT violate constraints
  • type MUST be declared and complexity MUST be controllable

Inspection Item Statistics:

  • Inspection item count: 7

Part 2: Parameter Constraints, High-Risk Operations, and Semantic Clarity Defects ​

2.1 Positioning of This Part ​

2.1.1 Inspection Objective ​

This part inspects the parameter constraint completeness, high-risk operation identification, and semantic clarity of Tool Schemas, ensuring that parameters are adequately constrained, high-risk operations are explicitly identified, and semantics are clear and unambiguous.

2.1.2 Inspection Dimensions ​

Parameter Constraints, High-Risk Operations, and Semantic Clarity Inspection
├─ Parameter Constraint Defects (QD-T-2.x)
│   ├─ String constraints (length, format)
│   ├─ Number/Integer constraints (boundaries, unit/precision)
│   ├─ Array constraints (maxItems)
│   ├─ Object constraints (type declaration, complexity)
│   ├─ Query pagination and result-set caps
│   └─ Inter-parameter dependencies and exclusivity
│
├─ High-Risk Operation Defects (QD-T-3.x)
│   ├─ State change and side effect declaration
│   ├─ Untrusted input entering an executable context
│   ├─ Batch/range write operation caps
│   ├─ Authorization context parameter exposure
│   ├─ Cross-Tool data flow
│   └─ Dangerous default values and operation scope
│
└─ Semantic Clarity Defects (QD-T-4.x)
    ├─ Description quality
    ├─ Parameter-level description
    ├─ Multi-Tool semantic competition
    ├─ description–Schema consistency
    └─ Sensitive input handling declaration

2.1.3 Relationship to Other Parts ​

Building on Part 1's assurance of structural validity, this part further inspects:

  • Whether parameters are adequately constrained
  • Whether high-risk operations are explicitly identified
  • Whether semantics are clear and unambiguous

2.2 Defect Classification ​

2.2.1 Parameter Constraint Defects (QD-T-2.x) ​

QD-T-2.1 String Type Without Length Constraint ​

Definition: A String-type parameter does not set maxLength (minLength is not required). Fields that already have enum / const / fixed-length pattern are automatically exempt.

Defect Example:

❌ Incorrect:

json
{
  "type": "object",
  "properties": {
    "content": {
      "type": "string",
      "description": "User input content"
    }
  }
}

✅ Correct:

json
{
  "type": "object",
  "properties": {
    "content": {
      "type": "string",
      "description": "User input content",
      "maxLength": 1000
    }
  }
}

Impact:

  • The LLM may pass excessively long strings
  • May cause excessive system load or security risks

Defect Level: L1 P2 / L2 P1 / L3 P1 (fields entering storage/commands/external interfaces remain P0 at L3)

Inspection Method: Automated

QD-T-2.2 String Type Without Format Constraint ​

Definition: A String parameter requiring a specific format does not set pattern. format is an annotation-only assertion by default in JSON Schema, so pattern SHOULD be preferred; if the validator's format-assertion mode is relied upon, this MUST be noted in the description.

Defect Example:

❌ Incorrect:

json
{
  "type": "object",
  "properties": {
    "email": {
      "type": "string",
      "description": "User email"
    }
  }
}

✅ Correct:

json
{
  "type": "object",
  "properties": {
    "email": {
      "type": "string",
      "description": "User email",
      "pattern": "^[^@\\s]+@[^@\\s]+\\.[^@\\s]+$"
    }
  }
}

Impact:

  • The LLM may pass incorrectly formatted values
  • May cause business logic failures

Defect Level: L1 P2 / L2 P1 / L3 P1

Inspection Method: Semi-automated

QD-T-2.3 Number/Integer Type Without Boundary Constraint ​

Definition: A Number or Integer parameter does not set minimum/maximum (or exclusiveMinimum/exclusiveMaximum).

Defect Example:

❌ Incorrect (amount as floating-point number):

json
{
  "type": "object",
  "properties": {
    "amount": {
      "type": "number",
      "description": "Payment amount"
    }
  }
}

✅ Correct (integer cents with unit and currency declared):

json
{
  "type": "object",
  "properties": {
    "amount_cents": {
      "type": "integer",
      "description": "Payment amount, unit: cents, currency: CNY",
      "minimum": 1,
      "maximum": 10000000
    }
  }
}

Impact:

  • The LLM may pass values outside the business range
  • Floating-point representation of amounts easily introduces precision errors

Defect Level: L1 P2 / L2 P1 / L3 P1 (amount/quantity/ratio fields remain P0 at L3)

Inspection Method: Automated

QD-T-2.4 Array Type Without Length Constraint ​

Definition: An Array-type parameter does not set maxItems (minItems is not required; an empty array is often a legitimate input).

Defect Example:

❌ Incorrect:

json
{
  "type": "object",
  "properties": {
    "ids": {
      "type": "array",
      "items": {"type": "string"},
      "description": "User ID list"
    }
  }
}

✅ Correct:

json
{
  "type": "object",
  "properties": {
    "ids": {
      "type": "array",
      "items": {"type": "string"},
      "description": "User ID list",
      "maxItems": 100
    }
  }
}

Impact:

  • The LLM may pass excessively long arrays
  • May cause performance issues or denial of service

Defect Level: L1 P2 / L2 P1 / L3 P1

Inspection Method: Automated

QD-T-2.5 Missing Unit/Precision/Time Zone/Currency Semantics ​

Definition: Numeric or temporal parameters lack declarations of unit, precision, time zone, or currency semantics, including: amount in yuan/cents, time in seconds/milliseconds and time zone, date format, ratio vs. percentage; monetary numeric values lack multipleOf.

Defect Example:

❌ Incorrect:

json
{
  "type": "object",
  "properties": {
    "timestamp": {
      "type": "integer",
      "description": "Timestamp"
    }
  }
}

The description does not state the timestamp unit (seconds/milliseconds) or time zone.

✅ Correct:

json
{
  "type": "object",
  "properties": {
    "timestamp": {
      "type": "integer",
      "description": "Unix timestamp, unit: seconds, time zone: UTC"
    }
  }
}

Impact:

  • The LLM may generate values with the wrong unit/time zone
  • Missing monetary precision may cause financial discrepancies

Defect Level: L1 P2 / L2 P1 / L3 P0 (monetary and time-critical parameters)

Inspection Method: Semi-automated

QD-T-2.6 Query Tool Without Pagination/Result-Set Cap ​

Definition: A list_* / search_* query Tool lacks a limit / page_size parameter, or the limit parameter has no maximum.

Defect Example:

❌ Incorrect:

json
{
  "name": "list_orders",
  "description": "List all orders",
  "inputSchema": {
    "type": "object",
    "properties": {
      "status": {"type": "string"}
    }
  }
}

No pagination/result-set cap parameter.

✅ Correct:

json
{
  "name": "list_orders",
  "description": "List orders with pagination",
  "inputSchema": {
    "type": "object",
    "properties": {
      "status": {"type": "string"},
      "limit": {
        "type": "integer",
        "description": "Maximum number of results to return",
        "minimum": 1,
        "maximum": 100
      }
    }
  }
}

Impact:

  • May return a massive amount of data at once
  • May cause performance issues or memory exhaustion

Defect Level: L1 P1 / L2 P1 / L3 P1

Inspection Method: Automated

QD-T-2.7 Unexpressed Inter-Parameter Dependencies/Exclusivity ​

Definition: Parameters have relationships such as pairwise-required, mutually exclusive, conditionally required, or at-least-one-of, but these are not expressed via keywords such as dependentRequired / oneOf. The target platform's support for these keywords MUST be noted.

Defect Example:

❌ Incorrect:

json
{
  "type": "object",
  "properties": {
    "email": {"type": "string"},
    "phone": {"type": "string"}
  }
}

The at-least-one-of relationship between email and phone is not expressed.

✅ Correct:

json
{
  "type": "object",
  "properties": {
    "email": {"type": "string"},
    "phone": {"type": "string"}
  },
  "anyOf": [
    {"required": ["email"]},
    {"required": ["phone"]}
  ]
}

Impact:

  • The LLM may generate invocations with missing required parameters or with mutually exclusive parameters coexisting

Defect Level: P1 (all levels)

Inspection Method: Semi-automated

Three-Layer Bearing of Frequency/Batch Caps (consistent with Inspect Cross §7.4): the maxItems (QD-T-2.4), limit.maximum (QD-T-2.6), etc. in this section's parameter constraints are artifact declaration layer business-applicable caps delivered with the Tool; runtime Harness/API gateway rate limiting, concurrency, and timeouts are the runtime fallback layer global fallback values; business-side quotas, approvals, and billing are the business quota layer. The three layers cannot substitute for one another: the inspection of maxItems/limit.maximum in this section MUST NOT be waived on the grounds that "the Harness has a fallback"—not every deployment environment has a Harness; Harness fallback thresholds are not equal to business-applicable caps, since fallback values usually take the greatest common divisor and are maximally redundant, and cannot replace business-level constraints. This specification only judges whether the artifact declares a cap; it does not judge whether the Harness configuration is correct or whether the threshold is optimal.

2.2.2 High-Risk Operation Defects (QD-T-3.x) ​

QD-T-3.1 State Change and Material Side Effect Declaration ​

Definition: The Tool has a state change or material side effect but does not declare it in the description. A defect is determined only when the name/description presents as read-only yet a hidden side effect exists (e.g., verify_order actually charges the customer), or when a side effect (irreversible, externally propagated, or with financial consequences) is not declared at all; when the name already clearly expresses write intent (e.g., update_user), fixed warning wording is no longer required.

Structured annotations (readOnlyHint / destructiveHint / idempotentHint) are recommended for expression, without binding to specific wording. The description marker is only a hint to the model; confirmation/approval/permission control is not borne by this specification (for the bearing relationship, see Appendix E, E-19).

Defect Example:

❌ Incorrect (name presents as read-only while a hidden side effect exists):

json
{
  "name": "verify_order",
  "description": "Verify order status"
}

It actually charges the customer, but the description does not declare the side effect.

✅ Correct:

json
{
  "name": "verify_order",
  "description": "Verify the order and complete payment. [This operation produces a financial side effect.]"
}

Impact:

  • The LLM may misjudge the operation type
  • May cause unintended data modifications or financial loss

Defect Level: L2 P1 / L3 P0

Inspection Method: Human

QD-T-3.2 Untrusted Input Entering an Executable Context ​

Definition: Parameters where untrusted input enters an executable interpretation context (SQL / shell / code / template expressions / file paths / URLs). Determination basis: executable semantic features in the parameter name or description (automatically identified as suspected) + manual confirmation of whether the backend has parameterization/sandboxing/allowlisting.

URL / file path parameters without constraints (domain allowlist, protocol restrictions, directory prefix) are an explicit sub-item of this rule.

Defect Example:

❌ Incorrect:

json
{
  "name": "execute_query",
  "description": "Execute user query",
  "inputSchema": {
    "type": "object",
    "properties": {
      "query": {
        "type": "string",
        "description": "User-entered query statement"
      }
    }
  }
}

Free-text query statements go directly into the SQL context.

✅ Correct (parameterized query fields + controlled filter structure):

json
{
  "name": "execute_query",
  "description": "Execute structured queries",
  "inputSchema": {
    "type": "object",
    "properties": {
      "table": {
        "type": "string",
        "enum": ["users", "orders", "products"]
      },
      "filter": {
        "type": "object",
        "description": "Controlled filter conditions, parameterized by the backend",
        "properties": {
          "field": {
            "type": "string",
            "enum": ["id", "name", "email"]
          },
          "value": {
            "type": "string",
            "maxLength": 200
          }
        },
        "required": ["field", "value"],
        "additionalProperties": false
      }
    },
    "required": ["table"],
    "additionalProperties": false
  }
}

Impact:

  • May cause SQL injection, command injection, and other security risks
  • May lead to system compromise

Defect Level: L1 P1 / L2 P0 / L3 P0 (triggered by parameter characteristics, not fixed per level)

Inspection Method: Semi-automated

QD-T-3.3 Batch/Range Write Operation Without Cap ​

Definition: A batch or range write operation does not set a cap, covering three forms: array parameters (e.g., ids), filter conditions (empty filter = full scope), and wildcards (e.g., *).

Defect Example:

❌ Incorrect:

json
{
  "name": "delete_users",
  "description": "Batch delete users",
  "inputSchema": {
    "type": "object",
    "properties": {
      "user_ids": {
        "type": "array",
        "items": {"type": "string"}
      }
    }
  }
}

✅ Correct:

json
{
  "name": "delete_users",
  "description": "Batch delete users (up to 10)",
  "inputSchema": {
    "type": "object",
    "properties": {
      "user_ids": {
        "type": "array",
        "items": {"type": "string"},
        "maxItems": 10
      }
    },
    "required": ["user_ids"],
    "additionalProperties": false
  }
}

Impact:

  • May cause large-scale accidental data deletion
  • May cause system performance issues

Defect Level: L2 P1 / L3 P0

Inspection Method: Semi-automated

QD-T-3.4 Authorization Context Parameter Exposure ​

Definition: The Tool exposes parameters that should be controlled internally by the system, such as tenantId, admin, token, etc.

Defect Example:

❌ Incorrect:

json
{
  "name": "create_user",
  "description": "Create user",
  "inputSchema": {
    "type": "object",
    "properties": {
      "name": {"type": "string"},
      "tenantId": {"type": "string"},
      "isAdmin": {"type": "boolean"}
    }
  }
}

Exposes tenantId and isAdmin parameters.

✅ Correct:

json
{
  "name": "create_user",
  "description": "Create user",
  "inputSchema": {
    "type": "object",
    "properties": {
      "name": {"type": "string"}
    }
  }
}

tenantId is obtained from context; isAdmin is prohibited from being set via the Tool.

Impact:

  • May cause privilege escalation
  • May cause tenant data leaks

Defect Level: L3 P0; L1/L2 triggered by parameter characteristics (see Appendix E, E-14)

Inspection Method: Human

QD-T-3.5 Cross-Tool Data Flow ​

Definition (retained as a P2 lead; rule ownership has moved to QD-TT-5.1): The output of one Tool may be used by the LLM as input to another Tool, causing unintended data flow.

Defect Example:

❌ Incorrect:

Tool 1: get_user_email(user_id) → returns {email: "user@example.com"}
Tool 2: send_notification(email, message) → sends notification

The LLM may automatically pass Tool 1's email to Tool 2.

✅ Correct:

Tool 1: get_user_email(user_id) → returns {email: "[PROTECTED]"}
(Sensitive fields are masked)

or

Tool 2: send_notification(user_id, message) → retrieves email internally
(Does not expose the email parameter)

Impact:

  • May cause data leaks
  • May cause unintended operations

Defect Level: Transferred to Inspect Cross (QD-TT-5.1); this specification retains a P2 lead

Inspection Method: Semi-automated

QD-T-3.6 Dangerous Default Values and Operation Scope Preconditions ​

Definition: A parameter has a dangerous default value or a precondition that expands the operation scope, including: force: default true, dry_run: default false, scope: default ALL, empty filter representing full scope, and wildcards expanding the scope.

Defect Example:

❌ Incorrect:

json
{
  "type": "object",
  "properties": {
    "force": {
      "type": "boolean",
      "description": "Whether to force deletion",
      "default": true
    }
  }
}

force defaults to true, expanding the destructive scope.

✅ Correct:

json
{
  "type": "object",
  "properties": {
    "force": {
      "type": "boolean",
      "description": "Whether to force deletion (even if associated data exists)",
      "default": false
    }
  }
}

Impact:

  • Dangerous default values may cause unexpected large-scale destruction

Defect Level: L2 P1 / L3 P0

Inspection Method: Automated + human

QD-T-3.7 Undeclared Operation Allowlist for Parameterized Tools ​

Definition: For parameterized/open-ended tools (sql_query / exec / http_request / eval, etc.), the operation target is determined by the actual argument ("execute arbitrary SQL," "run arbitrary commands," "call arbitrary URLs"), while the Tool Schema does not declare the permitted operation scope (table allowlist, command allowlist, domain allowlist, etc.). The permission set of such tools cannot be statically enumerated ("the set is not closed"); this is a pre-emptive blocking form, attributed from inspect/permission.md §2.2.5 (see QD-PM §2.8 item 4).

Defect Example:

❌ Incorrect:

json
{
  "name": "sql_query",
  "description": "Execute arbitrary SQL queries",
  "inputSchema": {
    "type": "object",
    "properties": {
      "sql": { "type": "string", "description": "Arbitrary SQL statement" }
    },
    "required": ["sql"]
  }
}

sql accepts arbitrary SQL, and the operation targets (tables/databases) cannot be enumerated.

✅ Correct:

json
{
  "name": "sql_query",
  "description": "Execute restricted queries, limited to the users and orders tables",
  "inputSchema": {
    "type": "object",
    "properties": {
      "table": { "type": "string", "enum": ["users", "orders"] },
      "filter": {
        "type": "object",
        "description": "Controlled filter conditions, parameterized by the backend",
        "properties": {
          "field": { "type": "string", "enum": ["id", "name", "email"] },
          "value": { "type": "string", "maxLength": 200 }
        },
        "required": ["field", "value"],
        "additionalProperties": false
      }
    },
    "required": ["table"]
  }
}

table is restricted by an enum allowlist, so the operation targets are enumerable.

Impact:

  • The permission set cannot be statically enumerated, and the proportionality assessment cannot converge
  • A single tool constitutes a data exfiltration/destruction surface (arbitrary tables, arbitrary commands, arbitrary URLs)

Defect Level: P0 (all levels; a non-closed set is a pre-emptive blocking form, and a hit means FAIL)

Inspection Method: Semi-automated

2.2.3 Semantic Clarity Defects (QD-T-4.x) ​

QD-T-4.1 Insufficient description Quality ​

Definition: The Tool's description lacks mandatory elements. Mandatory element checklist: action, object, scope/timeliness, key parameter semantics, return summary. A defect is determined only when ≥2 items are missing.

Defect Example:

❌ Incorrect:

json
{
  "name": "get_data",
  "description": "Get data"
}

Missing most elements beyond action and object.

✅ Correct:

json
{
  "name": "get_user_orders",
  "description": "Get the order list for a specified user. Returns orders from the last 30 days, sorted in reverse chronological order."
}

Impact:

  • The LLM may misunderstand the Tool's purpose
  • May lead to incorrect Tool selection

Defect Level: L1 P2 / L2 P1 / L3 P1

Inspection Method: Semi-automated

Cross-Subset Attribution: Instruction injection in the description field (tool description poisoning) belongs to Risk Explicit NR-S-5 and is not determined by this item; this item only checks "whether the description contains the mandatory elements."

QD-T-4.2 Undeclared Side Effects (Merged into QD-T-3.1) ​

This rule has been merged into QD-T-3.1 (State Change and Material Side Effect Declaration). The number is retained and is no longer used independently.

QD-T-4.3 Multi-Tool Semantic Competition ​

Definition (retained as a P2 lead; rule ownership has moved to QD-TT-5.2): Multiple Tools have semantically similar names or descriptions, making it difficult for the LLM to distinguish between them.

Defect Example:

❌ Incorrect:

json
{
  "tools": [
    {
      "name": "get_user",
      "description": "Get user information"
    },
    {
      "name": "get_user_info",
      "description": "Get detailed user information"
    }
  ]
}

The two Tools have similar functionality and are difficult to distinguish.

✅ Correct:

json
{
  "tools": [
    {
      "name": "get_user_basic",
      "description": "Get basic user information (ID, name)"
    },
    {
      "name": "get_user_profile",
      "description": "Get the user's complete profile (including contact information, preferences)"
    }
  ]
}

Impact:

  • The LLM may select the wrong Tool
  • May cause redundant function invocations

Defect Level: Transferred to Inspect Cross (QD-TT-5.2); this specification retains a P2 lead

Inspection Method: Semi-automated

QD-T-4.4 Missing Parameter-Level description ​

Definition: A property has no description.

Defect Example:

❌ Incorrect:

json
{
  "type": "object",
  "properties": {
    "user_id": {"type": "string"}
  }
}

user_id has no description.

✅ Correct:

json
{
  "type": "object",
  "properties": {
    "user_id": {
      "type": "string",
      "description": "Unique user identifier"
    }
  }
}

Impact:

  • The LLM may misunderstand the parameter's meaning

Defect Level: L1 P2 / L2 P1 / L3 P1

Inspection Method: Automated

QD-T-4.5 description–Schema Inconsistency ​

Definition: Within the same Tool, the description is inconsistent with the Schema, including: the description says "up to 10" but the Schema allows 100; the description mentions a nonexistent parameter; an example value is not in the enum; a Tool named get_* actually writes data.

Defect Example:

❌ Incorrect:

json
{
  "name": "get_report",
  "description": "Get report",
  "inputSchema": {
    "type": "object",
    "properties": {
      "limit": {
        "type": "integer",
        "description": "Return at most 10 records",
        "maximum": 100
      }
    }
  }
}

The description says "at most 10 records," but the Schema allows 100.

✅ Correct:

json
{
  "name": "get_report",
  "description": "Get report",
  "inputSchema": {
    "type": "object",
    "properties": {
      "limit": {
        "type": "integer",
        "description": "Return at most 10 records",
        "maximum": 10
      }
    }
  }
}

Impact:

  • The LLM may generate invocations that do not match the description

Defect Level: P1 (all levels)

Inspection Method: Semi-automated

QD-T-4.6 Sensitive Input Handling Declaration ​

Definition: When passwords, keys, ID numbers, etc. are legitimate input parameters, writeOnly / secret handling (not logged, not echoed) SHOULD be declared.

Defect Example:

❌ Incorrect:

json
{
  "type": "object",
  "properties": {
    "password": {
      "type": "string",
      "description": "User password"
    }
  }
}

Sensitive handling is not declared.

✅ Correct:

json
{
  "type": "object",
  "properties": {
    "password": {
      "type": "string",
      "description": "User password (sensitive field; not logged, not echoed)",
      "writeOnly": true
    }
  }
}

Impact:

  • Sensitive information may be written to logs or echoed

Defect Level: L2 P1 / L3 P1

Inspection Method: Semi-automated

QD-T-4.7 Missing Output Contract (outputSchema) ​

Definition: The Tool does not declare outputSchema (the structured definition of output parameters), leaving downstream unable to determine its output structure.

Defect Example:

❌ Incorrect:

json
{
  "name": "get_user",
  "description": "Get user information",
  "inputSchema": { "type": "object", "properties": { "user_id": {"type": "string"} } }
}

outputSchema is not declared.

✅ Correct:

json
{
  "name": "get_user",
  "description": "Get user information",
  "inputSchema": { "type": "object", "properties": { "user_id": {"type": "string"} } },
  "outputSchema": {
    "type": "object",
    "properties": {
      "email": {"type": "string", "description": "User email"},
      "name": {"type": "string", "description": "User name"}
    }
  }
}

Impact:

  • Downstream Tools / callers cannot determine the output structure, which may lead to misuse

Defect Level: Default L2/L3 P1 (advisory); if a cross-Tool data flow exists (Inspect Cross QD-TT-5.1 hit) → upgraded to P0

Inspection Method: Semi-automated

2.3 Severity Classification ​

Defect CategoryIDsItem Count
Parameter Constraint DefectsQD-T-2.1 to QD-T-2.77
High-Risk Operation DefectsQD-T-3.1 to QD-T-3.77
Semantic Clarity DefectsQD-T-4.1 to QD-T-4.77 (of which QD-T-4.2 has been merged into QD-T-3.1)
Total21

2.4 Detection Method: Checklist Mechanism ​

The defects in this part require Checklist inspection combined with human review:

Detection Process:
├─ Automated Inspection
│   ├─ JSON Schema meta-Schema validation
│   ├─ Parameter constraint completeness check (length/boundary/format/pagination)
│   ├─ Constraint self-consistency and default validity check
│   └─ Dangerous default values and high-risk keyword scanning
│
├─ Semi-Automated Inspection
│   ├─ description quality scoring and parameter-level description check
│   ├─ Unit/precision/time zone/currency semantics check
│   ├─ description–Schema consistency check
│   ├─ Sensitive input handling declaration check
│   ├─ Multi-Tool semantic similarity calculation
│   └─ Cross-Tool data flow analysis
│
└─ Human Review
    ├─ State change and side effect identification
    ├─ Untrusted input executable context confirmation
    ├─ Authorization context parameter exposure review
    └─ Semantic competition confirmation

2.5 Part Summary ​

This part defines the parameter constraint, high-risk operation, and semantic clarity inspection for Tool Schemas, ensuring that parameters are adequately constrained, high-risk operations are explicitly identified, and semantics are clear and unambiguous.

Core Points:

  • Parameters MUST have explicit constraints (length, boundary, format, unit/precision, pagination, dependency/exclusivity)
  • State changes and material side effects MUST be declared
  • Untrusted input MUST NOT enter an executable context
  • Batch/range write operations MUST have caps
  • Internal parameters MUST be isolated
  • Dangerous default values MUST be avoided
  • Cross-Tool data flows MUST be assessed
  • Semantics MUST be clear and unambiguous

Inspection Item Statistics:

  • Inspection item count: 21 numbered items (of which QD-T-4.2 has been merged into QD-T-3.1; QD-T-3.5 and QD-T-4.3 have been transferred to Inspect Cross QD-TT; QD-T-4.7 is newly added)

Part 3: Tiered Inspection Mechanism ​

3.1 Tool Operational Risk Level Definitions (OR-L) ​

【Global Symbol Convention · Mandatory】 In the body text, tables, examples, and appendices of this document, L1 / L2 / L3 are always shorthand for OR-L1 / OR-L2 / OR-L3 (Tool Operational Risk Level); the two are completely equivalent. Except in this definition section, the OR- prefix is not repeated everywhere in the document. Any isolated L1 / L2 / L3 MUST NOT be understood as AC-L (Agent Complexity Level) or any other grading model. This convention is mandatory for all chapters, appendices, examples, and report fields in this document.

3.1.1 Operational Risk Level Classification ​

Operational Risk LevelDesignationDefinitionTypical Characteristics
Low RiskOR-L1Read-only query; no state changesQuery, search, retrieve information
Medium RiskOR-L2State-changing; recoverableCreate, update, send notifications
High RiskOR-L3Irreversible / sensitive operationsDelete, payment, permission changes

3.1.2 Level Determination Criteria ​

OR-L1 (Low Risk) Determination Criteria:

  • ✅ Performs data-read operations only
  • ✅ Does not involve sensitive data
  • ✅ Operations are repeatable with no side effects
  • ✅ Failure consequence: query failure only; no other impact

OR-L2 (Medium Risk) Determination Criteria:

  • ✅ Performs data creation or update operations
  • ✅ Changes are recoverable or overwritable
  • ⚠️ May involve some sensitive data
  • ⚠️ Failure consequence: data quality degradation; human intervention required

OR-L3 (High Risk) Determination Criteria:

  • ✅ Performs data deletion, payment, permission change, or similar operations
  • ✅ Involves highly sensitive data
  • ✅ Operations are irreversible or difficult to recover
  • ✅ Failure consequence: data loss, financial loss, permission leaks

3.1.3 Typical Examples ​

Operational Risk LevelTypical Tool Examples
OR-L1get_weather, get_current_time, search_products, list_orders
OR-L2create_user, update_profile, send_notification, add_to_cart
OR-L3delete_user, process_payment, grant_permission, transfer_data

3.2 Defect Level Definitions (P0/P1/P2) ​

3.2.1 Defect Level Classification ​

Defect LevelSymbolDefinitionTypical Defects
P0⛔Blocks release; MUST be fixedBase field missing, Authorization Context Parameter Exposure, Untrusted Input Entering an Executable Context
P1⚠️SHOULD be fixed before releaseInsufficient parameter constraints, incomplete description
P2ℹ️SHOULD be included in the improvement plan or risk registerMinor description omissions, non-critical field omissions

3.2.2 Defect Level Upgrade Rules ​

The same defect may have its defect level upgraded based on the Tool's Operational Risk Level:

Defect ExampleOR-L1 (Low Risk)OR-L2 (Medium Risk)OR-L3 (High Risk)
String without maxLengthP2 (Advisory)P1 (Warning)P1 (Blocking; executable fields P0)
Number without boundaryP2 (Advisory)P1 (Warning)P1 (Blocking; amount/quantity P0)
State change not declared-P1 (Warning)P0 (Blocking)
Authorization Context Parameter ExposureTriggered by parameter characteristics (see E-14)Triggered by parameter characteristics (see E-14)P0 (Blocking)

3.3 Tiered Inspection Checklist ​

3.3.1 OR-L1 (Low Risk) Inspection Checklist ​

IDInspection ItemDefect LevelInspection Method
QD-T-1.1Base Field MissingP0Automated
QD-T-1.2required References Undefined FieldP0Automated
QD-T-1.3enum–type Conflict / Duplicate ValuesP1 / P2Automated
QD-T-1.4Inappropriate additionalProperties PolicyP2Automated
QD-T-1.5Name Validity and UniquenessP0 (style P2)Automated
QD-T-1.6Constraint Self-Consistency and defaultP1Automated
QD-T-1.7Missing type / ComplexityP2Automated
QD-T-2.1String Without maxLengthP2Automated
QD-T-2.2Missing Format ContractP2Semi-automated
QD-T-2.3Number/Integer Without BoundaryP2Automated
QD-T-2.4Array Without maxItemsP2Automated
QD-T-2.5Unit/Precision/Time Zone/CurrencyP2Semi-automated
QD-T-2.6Query Without Pagination/CapP1Automated
QD-T-2.7Parameter Dependency/ExclusivityP1Semi-automated
QD-T-3.2Untrusted Input Executable ContextP1 (by characteristic)Semi-automated
QD-T-3.4Authorization Context Parameter ExposureTriggered by parameter characteristics (provisional P0, see E-14)Human
QD-T-3.5Cross-Tool Data FlowTransferred to cross QD-TT-5.1Semi-automated
QD-T-3.7Undeclared Operation Allowlist for Parameterized ToolsP0Semi-automated
QD-T-4.1description QualityP2Semi-automated
QD-T-4.3Multi-Tool Semantic CompetitionTransferred to cross QD-TT-5.2Semi-automated
QD-T-4.4Missing Parameter-Level descriptionP2Automated
QD-T-4.5description–Schema InconsistencyP1Semi-automated

Inspection Item Count: 22 numbered items (P0: 5, P1: 6, P2: 9, plus 2 transferred to cross QD-TT)

3.3.2 OR-L2 (Medium Risk) Inspection Checklist ​

IDInspection ItemDefect LevelInspection Method
QD-T-1.1Base Field MissingP0Automated
QD-T-1.2required References Undefined FieldP0Automated
QD-T-1.3enum–type Conflict / Duplicate ValuesP1 / P2Automated
QD-T-1.4Inappropriate additionalProperties PolicyP1Automated
QD-T-1.5Name Validity and UniquenessP0 (style P2)Automated
QD-T-1.6Constraint Self-Consistency and defaultP1Automated
QD-T-1.7Missing type / ComplexityP1Automated
QD-T-2.1String Without maxLengthP1Automated
QD-T-2.2Missing Format ContractP1Semi-automated
QD-T-2.3Number/Integer Without BoundaryP1Automated
QD-T-2.4Array Without maxItemsP1Automated
QD-T-2.5Unit/Precision/Time Zone/CurrencyP1Semi-automated
QD-T-2.6Query Without Pagination/CapP1Automated
QD-T-2.7Parameter Dependency/ExclusivityP1Semi-automated
QD-T-3.1State Change and Material Side Effect DeclarationP1Human
QD-T-3.2Untrusted Input Executable ContextP0 (by characteristic)Semi-automated
QD-T-3.3Batch/Range Write Operation Without CapP1Semi-automated
QD-T-3.4Authorization Context Parameter ExposureTriggered by parameter characteristics (provisional P0, see E-14)Human
QD-T-3.5Cross-Tool Data FlowTransferred to cross QD-TT-5.1Semi-automated
QD-T-3.6Dangerous Default/Operation ScopeP1Automated + human
QD-T-3.7Undeclared Operation Allowlist for Parameterized ToolsP0Semi-automated
QD-T-4.1description QualityP1Semi-automated
QD-T-4.3Multi-Tool Semantic CompetitionTransferred to cross QD-TT-5.2Semi-automated
QD-T-4.4Missing Parameter-Level descriptionP1Automated
QD-T-4.5description–Schema InconsistencyP1Semi-automated
QD-T-4.6Sensitive Input Handling DeclarationP1Semi-automated
QD-T-4.7Missing Output Contract (outputSchema)P1Semi-automated

Inspection Item Count: 27 numbered items (P0: 6, P1: 19, P2: 0, plus 2 transferred to cross QD-TT)

3.3.3 OR-L3 (High Risk) Inspection Checklist ​

IDInspection ItemDefect LevelInspection Method
QD-T-1.1Base Field MissingP0Automated
QD-T-1.2required References Undefined FieldP0Automated
QD-T-1.3enum–type Conflict / Duplicate ValuesP1 / P2Automated
QD-T-1.4Inappropriate additionalProperties PolicyP1Automated
QD-T-1.5Name Validity and UniquenessP0 (style P2)Automated
QD-T-1.6Constraint Self-Consistency and defaultP1 (high-risk parameters P0)Automated
QD-T-1.7Missing type / ComplexityP1Automated
QD-T-2.1String Without maxLengthP1 (executable/external interface fields P0)Automated
QD-T-2.2Missing Format ContractP1Semi-automated
QD-T-2.3Number/Integer Without BoundaryP1 (amount/quantity P0)Automated
QD-T-2.4Array Without maxItemsP1Automated
QD-T-2.5Unit/Precision/Time Zone/CurrencyP0 (critical parameters)Semi-automated
QD-T-2.6Query Without Pagination/CapP1Automated
QD-T-2.7Parameter Dependency/ExclusivityP1Semi-automated
QD-T-3.1State Change and Material Side Effect DeclarationP0Human
QD-T-3.2Untrusted Input Executable ContextP0 (by characteristic)Semi-automated
QD-T-3.3Batch/Range Write Operation Without CapP0Semi-automated
QD-T-3.4Authorization Context Parameter ExposureP0Human
QD-T-3.5Cross-Tool Data FlowTransferred to cross QD-TT-5.1Semi-automated
QD-T-3.6Dangerous Default/Operation ScopeP0Automated + human
QD-T-3.7Undeclared Operation Allowlist for Parameterized ToolsP0Semi-automated
QD-T-4.1description QualityP1Semi-automated
QD-T-4.3Multi-Tool Semantic CompetitionTransferred to cross QD-TT-5.2Semi-automated
QD-T-4.4Missing Parameter-Level descriptionP1Automated
QD-T-4.5description–Schema InconsistencyP1Semi-automated
QD-T-4.6Sensitive Input Handling DeclarationP1Semi-automated
QD-T-4.7Missing Output Contract (outputSchema)P1 (cross-Tool data flow P0)Semi-automated

Inspection Item Count: 27 numbered items (P0: 10, P1: 15, P2: 0, plus 2 transferred to cross QD-TT)

3.4 Standard Inspection Process ​

3.4.1 Inspection Process Flow ​

┌─────────────────────────────────────────────────────────────┐
│                  Tool Schema Inspection Process              │
└─────────────────────────────────────────────────────────────┘
                                   │
                                   ▼
                       ┌───────────────────────┐
                       │ Step 1: Determine     │
                       │ Operational Risk Level│
                       │ Determine OR-L1/L2/L3 │
                       └───────────────────────┘
                                   │
                                   ▼
                       ┌───────────────────────┐
                       │ Step 2: Select        │
                       │ Inspection Checklist  │
                       │ Based on Operational  │
                       │ Risk Level            │
                       └───────────────────────┘
                                   │
                                   ▼
                       ┌───────────────────────┐
                       │ Step 3: Automated     │
                       │ Inspection            │
                       │ Execute automated     │
                       │ inspection items      │
                       └───────────────────────┘
                                   │
                          ┌────────────┴────────────┐
                          │                         │
                          ▼                         ▼
                ┌─────────────────┐      ┌─────────────────┐
                │ No P0 defects   │      │ P0 defects found │
                └─────────────────┘      └─────────────────┘
                          │                         │
                          │                         ▼
                          │              ┌─────────────────────┐
                          │              │ Output defect report │
                          │              │ Flag P0 defects      │
                          │              └─────────────────────┘
                          │                         │
                          │                         ▼
                          │              ┌─────────────────────┐
                          │              │ Re-inspect after     │
                          │              │ remediation          │
                          │              └─────────────────────┘
                          │
                          ▼
                           ┌───────────────────────┐
                           │ Step 4: Semi-Automated│
                           │ Inspection            │
                           │ Execute semi-automated│
                           │ inspection items      │
                           └───────────────────────┘
                                   │
                                   ▼
                           ┌───────────────────────┐
                           │ Step 5: Human Review   │
                           │ Execute human          │
                           │ inspection items       │
                           └───────────────────────┘
                                   │
                                   ▼
                           ┌───────────────────────┐
                           │ Step 6: Output         │
                           │ Inspection Report      │
                           │ Flag all defects and   │
                           │ recommendations        │
                           └───────────────────────┘

3.4.2 Inspection Process Description ​

Step 1: Determine Operational Risk Level

Determine the Operational Risk Level (OR-L1/OR-L2/OR-L3) based on the Tool's operation type.

Determination Basis:

  • Involves write operations → L2 minimum
  • Involves irreversible operations → L3
  • Involves sensitive data → L3

Step 2: Select Inspection Checklist

Select the corresponding inspection checklist based on the Operational Risk Level:

  • L1: 22 inspection items (including 2 transferred to cross; 20 scored)
  • L2: 27 inspection items (including 2 transferred to cross; 25 scored)
  • L3: 27 inspection items (including 2 transferred to cross; 25 scored)

Step 3: Automated Inspection

Execute automatable inspection items; output a defect report.

Step 4: Semi-Automated Inspection

Execute inspection items that require tool assistance, such as:

  • description quality scoring
  • Cross-Tool data flow analysis
  • Multi-Tool semantic similarity calculation

Step 5: Human Review

Execute inspection items that require human judgment, such as:

  • Side effect identification
  • Authorization context parameter exposure review
  • Semantic competition confirmation

Step 6: Output Inspection Report

Output a complete inspection report, including:

  • Operational Risk Level
  • Inspection item checklist
  • Defect list (sorted by defect level)
  • Remediation recommendations

3.5 Usage Examples ​

3.5.1 L1 Example: Weather Query Tool ​

Tool Definition:

json
{
  "name": "get_weather",
  "description": "Get weather information for a specified city",
  "inputSchema": {
    "type": "object",
    "properties": {
      "city": {
        "type": "string",
        "description": "City name"
      }
    },
    "required": ["city"]
  }
}

Operational Risk Level Determination: L1 (read-only query)

Inspection Results:

Inspection ItemResultDescription
QD-T-1.1✅ PASSBase fields complete
QD-T-1.2✅ PASSrequired references defined fields
QD-T-1.4ℹ️ P2additionalProperties not declared
QD-T-1.5✅ PASSname is legal and unique
QD-T-2.1ℹ️ P2city has no maxLength
QD-T-4.1ℹ️ P2description lacks scope/return summary
QD-T-4.4✅ PASScity has a description
Other itemsNot applicableNo enum/number/array/time/pagination/dependency, etc.

Remediation Recommendation:

json
{
  "name": "get_weather",
  "description": "Get weather information for a specified city. Returns current temperature, humidity, and weather conditions.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "city": {
        "type": "string",
        "description": "City name, e.g., Beijing, Shanghai, Guangzhou",
        "maxLength": 50
      }
    },
    "required": ["city"],
    "additionalProperties": false
  }
}

3.5.2 L3 Example: User Deletion Tool ​

Tool Definition:

json
{
  "name": "delete_user",
  "description": "Delete user",
  "inputSchema": {
    "type": "object",
    "properties": {
      "user_id": {
        "type": "string",
        "description": "User ID"
      },
      "tenantId": {
        "type": "string",
        "description": "Tenant ID"
      },
      "force": {
        "type": "boolean",
        "description": "Force delete"
      }
    },
    "required": ["user_id"]
  }
}

Operational Risk Level Determination: L3 (irreversible operation)

Inspection Results:

Inspection ItemResultDescription
QD-T-1.1✅ PASSBase fields complete
QD-T-1.2✅ PASSrequired references defined fields
QD-T-1.4⚠️ P1additionalProperties not declared
QD-T-2.1⚠️ P1user_id has no maxLength
QD-T-3.1⛔ P0Deletion is an irreversible side effect and is not declared
QD-T-3.4⛔ P0Exposes internal parameter tenantId
QD-T-4.1⚠️ P1description is too simple

Remediation Recommendation:

json
{
  "name": "delete_user",
  "description": "Delete the specified user. This operation is irreversible and permanently deletes user data; proceed with caution.",
  "inputSchema": {
    "type": "object",
    "properties": {
      "user_id": {
        "type": "string",
        "description": "Unique user identifier",
        "maxLength": 50
      },
      "force": {
        "type": "boolean",
        "description": "Whether to force deletion (even if associated data exists). Force deletion requires backend verification that the caller has administrator privileges.",
        "default": false
      }
    },
    "required": ["user_id"],
    "additionalProperties": false
  }
}

(tenantId should be obtained from context and should not be exposed in the Schema)

3.6 Part Summary ​

This part defines the tiered inspection mechanism for Tool Schemas, providing differentiated inspection intensity based on the Tool's Operational Risk Level.

Core Points:

  • Determine the Operational Risk Level (OR-L1/OR-L2/OR-L3) based on operation type
  • Different Operational Risk Levels correspond to different inspection intensities
  • Defect levels may be upgraded based on the Operational Risk Level
  • The standardized inspection process ensures inspection consistency

Inspection Item Statistics:

  • L1: 22 inspection items (including 2 transferred to cross; 20 scored)
  • L2: 27 inspection items (including 2 transferred to cross; 25 scored)
  • L3: 27 inspection items (including 2 transferred to cross; 25 scored)

Part 4: Schema Change and Compatibility Inspection ​

4.1 Positioning of This Part ​

4.1.1 Inspection Objective ​

This part inspects whether Tool Schema changes introduce new risks or compatibility issues. It does not address change management processes.

4.1.2 Inspection Scope ​

Schema Change Inspection
├─ Breaking Change Identification
│   ├─ Field deletion
│   ├─ Field type modification
│   ├─ New required field addition
│   └─ Enum value modification
│
├─ Compatibility Assessment
│   ├─ New optional field (compatible)
│   ├─ Relaxed constraint (compatible)
│   └─ Tightened constraint (potentially incompatible)
│
└─ Impact Scope Assessment
    ├─ Single-Tool impact
    ├─ Multi-Tool impact
    └─ System-level impact

4.2 Compatibility Inspection ​

QD-T-5.1 Breaking Change Determination ​

Definition: Identify Schema changes that may cause existing invocations to fail. Severity is given by the §4.3.1 matrix per level; this section provides only compatibility facts and no longer directly assigns P levels.

An uncontrolled breaking change = P0. "Controlled" is defined as: version isolation/parallel versions or migration assurance, declared in the change notes.

Change Compatibility Classification:

Change TypeCompatibilityDescription
Delete fieldBreakingExisting invocations may pass this field
Modify field typeBreakingExisting invocations may pass an incorrect type
Add required fieldBreakingExisting invocations lack this field
Delete enum valueBreakingExisting invocations may use this value
Tighten constraintConditionally compatibleExisting invocations may not satisfy the new constraint
Add optional fieldCompatibleFully compatible
Add enum value (input Schema)CompatibleExpanding the enum set is backward compatible; only additions to output Schema enums require attention
Relax constraint (including removal from required)CompatibleConstraint relaxed; generally compatible
Semantic change (see QD-T-5.3)Conditionally compatibleStructure unchanged but semantics changed

Inspection Rules:

  • Compare old and new Schemas
  • Identify change types
  • Assess destructiveness and compatibility
  • Output a change impact report

QD-T-5.2 Change Impact Assessment Omission or Error ​

Definition: The Schema change notes do not declare the impact scope, or the declared impact scope clearly does not match the facts.

Check Content:

  • Whether the change notes declare caller impact (Agents using this Tool, invocation frequency, urgent invocation scenarios)
  • Whether the change notes declare data impact (stored data, data migration, data consistency)
  • Whether the change notes declare system impact (other Tools, system configuration, monitoring and alerting)

Defect Level: OR-L1 P1 / OR-L2 P1 / OR-L3 P0

Inspection Method: Semi-automated

QD-T-5.3 Semantic Change ​

Definition: The field structure is unchanged, but semantics such as description, Tool name, default, unit, query scope, or side effects have changed. Deprecated fields SHOULD be marked with deprecated and retain old behavior for transition.

Check Points:

  • Whether description semantic changes affect the LLM's invocation decisions
  • Whether default changes alter default behavior
  • Whether unit/time zone/currency changes affect numeric interpretation
  • Whether query scope/side effect changes expand the impact surface
  • Whether deprecated is used to declare deprecation and retain a transition

Defect Level: Per the §4.3.1 matrix (L1 P1 / L2 P1 / L3 P0)

Inspection Method: Automated + human

4.3 Change Risk Assessment ​

4.3.1 Risk Assessment Matrix ​

Change TypeL1 ToolL2 ToolL3 Tool
Delete fieldP1P0P0
Modify field typeP1P0P0
Add required fieldP1P0P0
Delete enum valueP1P0P0
Tighten constraintP2P1P0
Add optional fieldP2P2P2
Add enum value (input Schema)P2P2P2
Relax constraint (including removal from required)P2P2P2
Semantic changeP1P1P0

4.3.2 Change Inspection Process ​

Step 1: Identify change types
    ├─ Compare old and new Schemas
    └─ Flag all change points

Step 2: Assess destructiveness
    ├─ Determine the destructiveness of each change
    └─ Flag high-risk changes

Step 3: Assess impact scope
    ├─ Query caller information
    └─ Assess impact level

Step 4: Composite risk assessment
    ├─ Combine with Tool Operational Risk Level
    ├─ Combine with change destructiveness
    └─ Combine with impact scope
    └─ Output risk assessment report

4.4 Part Summary ​

This part defines the compatibility inspection and risk assessment for Tool Schema changes, helping to identify and assess the risks changes may introduce.

Core Points:

  • Breaking changes require focused attention
  • Tools at different Operational Risk Levels have different sensitivity to changes
  • The impact scope of changes must be comprehensively assessed

Inspection Item Statistics:

  • Change type identification: 9 types
  • Impact scope assessment: 3 dimensions
  • Risk assessment matrix: 27 combinations

Part 5: Quantitative Scoring Mechanism ​

5.1 Scoring Principles ​

PrincipleDescription
Total score of 100Deduction-based: baseline score (100) − defect deductions
Weight ratioP0 : P1 : P2 = 5 : 3 : 1
Level differentiationDifferent Agents have different inspection item counts; per-item weight adjusts automatically
Mathematical self-consistencyAfter all defect deductions, minimum score ≥ 0; maximum score = 100
Simple ruleFor the same inspection item number, the weight is deducted only once regardless of how many defect instances are found
Gate conditionAny P0 defect → evaluation FAIL (score is still output)

Four-State Inspection Result: Each inspection item has one of the following four states:

  • Applicable: normal determination (P0/P1/P2 or pass).
  • Not Applicable: the inspection item does not apply to this Tool (e.g., QD-T-2.4 is not applicable if there is no array parameter); not-applicable items are removed from the weight denominator in §5.2.1.
  • Undeterminable: cannot be graded due to PENDING alignment or lack of evidence; it is counted at the highest possible level and flagged in the report.
  • Exempted: exemption approved through the Appendix D process; removed from the denominator and flagged in the report.

Score Comparability Scope: scores are comparable only within the same specification version, the same risk level, and the same inspection scope.

Instance Counting: scoring retains the "deduct only once per number" rule, but reports MUST also output the number of affected Tools, the number of instances, and the ratio.

5.2 Calculation Steps ​

Step 1: Determine the Operational Risk Level
Determine according to the OR-L definitions in Part 3, Section 3.1 of this document.

Step 2: Calculate Total Weight
Total Weight = P0 inspection item count × 5 + P1 inspection item count × 3 + P2 inspection item count × 1

Step 3: Calculate Base Score
Base Score = 100 / Total Weight

Step 4: Determine Per-Defect Deduction Values

  • P0 defect deduction = Base Score × 5
  • P1 defect deduction = Base Score × 3
  • P2 defect deduction = Base Score × 1

Step 5: Calculate Total Deduction
Total Deduction = Σ(deduction for each inspection item number, using the highest severity level)
Note: The same inspection item is deducted only once, regardless of how many defect instances are found.

Step 6: Calculate Actual Score
Actual Score = max(100 − Total Deduction, 0)

Step 7: Gate Determination

  • If any P0 defect exists: evaluation result = FAIL (score is still output)
  • Otherwise: evaluation result = PASS

5.2.1 Inspect Tool Inspection Item Distribution Table ​

Note: P0/P1/P2 counts come from the "Tiered Inspection Checklist (3.3)" for the corresponding Operational Risk Level in this document. PENDING items to be aligned are counted at the level marked in the checklist (provisionally at the highest possible level, see 5.1); conditionally upgraded items are counted at their base level.

Operational Risk LevelScored ItemsP0 CountP1 CountP2 CountTotal WeightBase Score
OR-L120569$5×5+6×3+9×1=52$$100/52≈1.9231$
OR-L2256190$6×5+19×3+0×1=87$$100/87≈1.1494$
OR-L32510150$10×5+15×3+0×1=95$$100/95≈1.0526$

5.3 Defect Deduction Value Table (Example) ​

Using Inspect Tool L1 as an example:

  • Base Score = $100/52≈1.9231$
Defect LevelDeduction CalculationDeduction Value
P0Base Score × 5$1.9231×5≈9.6155$
P1Base Score × 3$1.9231×3≈5.7693$
P2Base Score × 1$1.9231×1≈1.9231$

5.4 Scoring Examples ​

Example: OR-L2 (Medium Risk) Tool, N=5 defects found (involving 4 inspection item numbers)

  • Deducted inspection items (deduplicated by inspection item number; only the highest defect level is used):
    • QD-T-1.1 (P0): 1 found (deducted only once regardless of count)
    • QD-T-3.2 (P0): 1 found
    • QD-T-2.1 (P1): 2 found (deducted once at P1 weight)
    • QD-T-4.1 (P1): 1 found

Deduction Values (OR-L2):

  • Total Weight = 87; Base Score = $100/87≈1.1494$
  • P0 deduction = Base Score × 5 ≈ $1.1494×5≈5.7471$
  • P1 deduction = Base Score × 3 ≈ $1.1494×3≈3.4483$

Deduction Calculation:

  • Total Deduction = 2 × P0 deduction + 2 × P1 deduction
  • Total Deduction ≈ $2×5.7471 + 2×3.4483 = 11.4943 + 6.8966 = 18.3909$
  • Score = 100 − Total Deduction ≈ $100 - 18.3909 = 81.6091$

Gate = FAIL (P0 defects exist)

Scoring Report:

Fields follow the framework/core.md §5.2 Minimum Finding Record: finding_id is the Finding instance ID, rule_id is the rule ID; level uses the OR-L prefix, and the defect severity uses the Inspect· prefix in a separate column from level.

json
{
  "check_id": "inspect-20260710-001",
  "framework": "SanityOps/Inspect Tool",
  "level": "OR-L2",
  "result": "FAIL",
  "score": 81.6091,
  "defects": [
    {"finding_id": "F-ORG-20260710-001", "rule_id": "QD-T-1.1", "severity": "Inspect·P0", "status": "open", "found_count": 1},
    {"finding_id": "F-ORG-20260710-002", "rule_id": "QD-T-3.2", "severity": "Inspect·P0", "status": "open", "found_count": 1},
    {"finding_id": "F-ORG-20260710-003", "rule_id": "QD-T-2.1", "severity": "Inspect·P1", "status": "open", "found_count": 2},
    {"finding_id": "F-ORG-20260710-004", "rule_id": "QD-T-4.1", "severity": "Inspect·P1", "status": "open", "found_count": 1}
  ],
  "affected_tools": 2,
  "defect_instances": 5,
  "instance_ratio": "4/5 inspection items"
}

5.5 Multi-Defect Handling Rules ​

Rule: The same inspection item is deducted only once, regardless of how many defect instances are found.

Examples:

  • QD-T-3.2 (P0 inspection item): 3 P0 defects found → deduct the P0 deduction value (once only)
  • QD-T-4.1 (P1 inspection item): 2 P1 defects found → deduct the P1 deduction value (once only)

Appendix A: Minimum Rule Sets ​

A.1 L1 Minimum Rule Set (20 Items) ​

Must-Pass Items (P0):

  • QD-T-1.1 Base Field Missing (P0)
  • QD-T-1.2 required References Undefined Field (P0)
  • QD-T-1.5 Name Validity and Uniqueness (P0)
  • QD-T-3.4 Authorization Context Parameter Exposure (triggered by parameter characteristics; provisional P0, see E-14)
  • QD-T-3.7 Undeclared Operation Allowlist for Parameterized Tools (P0)

P1 Recommended-Pass Items:

  • QD-T-1.3 enum–type Conflict (duplicate values P2)
  • QD-T-1.6 Constraint Self-Consistency and default
  • QD-T-2.6 Query Without Pagination/Cap
  • QD-T-2.7 Parameter Dependency/Exclusivity
  • QD-T-3.2 Untrusted Input Executable Context (by characteristic)
  • QD-T-4.5 description–Schema Inconsistency

P2 Recommended-Pass Items:

  • QD-T-1.4 Inappropriate additionalProperties Policy
  • QD-T-1.7 Missing type/Complexity
  • QD-T-2.1 String Without maxLength
  • QD-T-2.2 Missing Format Contract
  • QD-T-2.3 Number/Integer Without Boundary
  • QD-T-2.4 Array Without maxItems
  • QD-T-2.5 Unit/Precision/Time Zone/Currency
  • QD-T-4.1 description Quality
  • QD-T-4.4 Missing Parameter-Level description

A.2 L2 Minimum Rule Set (25 Items) ​

Must-Pass Items (P0):

  • QD-T-1.1 Base Field Missing (P0)
  • QD-T-1.2 required References Undefined Field (P0)
  • QD-T-1.5 Name Validity and Uniqueness (P0)
  • QD-T-3.2 Untrusted Input Executable Context (P0, by characteristic)
  • QD-T-3.4 Authorization Context Parameter Exposure (triggered by parameter characteristics; provisional P0, see E-14)
  • QD-T-3.7 Undeclared Operation Allowlist for Parameterized Tools (P0)

P1 Recommended-Pass Items:

  • QD-T-1.3 enum–type Conflict (duplicate values P2)
  • QD-T-1.4 Inappropriate additionalProperties Policy
  • QD-T-1.6 Constraint Self-Consistency and default
  • QD-T-1.7 Missing type/Complexity
  • QD-T-2.1 to QD-T-2.7 (parameter constraints)
  • QD-T-3.1 State Change and Material Side Effect Declaration
  • QD-T-3.3 Batch/Range Write Operation Without Cap
  • QD-T-3.6 Dangerous Default/Operation Scope
  • QD-T-4.1 description Quality
  • QD-T-4.4 Missing Parameter-Level description
  • QD-T-4.5 description–Schema Inconsistency
  • QD-T-4.6 Sensitive Input Handling Declaration
  • QD-T-4.7 Missing Output Contract (outputSchema; cross-Tool data flow P0)

A.3 L3 Minimum Rule Set (25 Items) ​

Must-Pass Items (P0):

  • QD-T-1.1 Base Field Missing (P0)
  • QD-T-1.2 required References Undefined Field (P0)
  • QD-T-1.5 Name Validity and Uniqueness (P0)
  • QD-T-2.5 Unit/Precision/Time Zone/Currency (P0, critical parameters)
  • QD-T-3.1 State Change and Material Side Effect Declaration (P0)
  • QD-T-3.2 Untrusted Input Executable Context (P0, by characteristic)
  • QD-T-3.3 Batch/Range Write Operation Without Cap (P0)
  • QD-T-3.4 Authorization Context Parameter Exposure (P0)
  • QD-T-3.6 Dangerous Default/Operation Scope (P0)
  • QD-T-3.7 Undeclared Operation Allowlist for Parameterized Tools (P0)

P1 Recommended-Pass Items:

  • QD-T-1.3 enum–type Conflict (duplicate values P2)
  • QD-T-1.4 Inappropriate additionalProperties Policy
  • QD-T-1.6 Constraint Self-Consistency and default (high-risk parameters P0)
  • QD-T-1.7 Missing type/Complexity
  • QD-T-2.1 String Without maxLength (executable/external interface fields P0)
  • QD-T-2.2 Missing Format Contract
  • QD-T-2.3 Number/Integer Without Boundary (amount/quantity P0)
  • QD-T-2.4 Array Without maxItems
  • QD-T-2.6 Query Without Pagination/Cap
  • QD-T-2.7 Parameter Dependency/Exclusivity
  • QD-T-4.1 description Quality
  • QD-T-4.4 Missing Parameter-Level description
  • QD-T-4.5 description–Schema Inconsistency
  • QD-T-4.6 Sensitive Input Handling Declaration
  • QD-T-4.7 Missing Output Contract (outputSchema; cross-Tool data flow P0)

Appendix B: QD-T Complete Numbering Index ​

Scoring Caliber Note: This table is a complete index. Among these, QD-T-5.x (Schema Changes) are change management rules that are triggered only when an artifact version changes and an old/new Schema comparison object exists; they are not counted in the static inspection denominator of 5.2.1 (20/25/25).

IDDefect NameCategoryDefect Level (OR-L1/OR-L2/OR-L3)Inspection Method
QD-T-1.1Base Field MissingStructural ValidityP0/P0/P0Automated
QD-T-1.2required References Undefined FieldStructural ValidityP0/P0/P0Automated
QD-T-1.3enum–type Conflict / Duplicate ValuesStructural ValidityP1 / P2Automated
QD-T-1.4Inappropriate additionalProperties PolicyStructural ValidityP2/P1/P1Automated
QD-T-1.5Name Validity and UniquenessStructural ValidityP0 (style P2)Automated
QD-T-1.6Constraint Self-Consistency and defaultStructural ValidityP1/P1/P1 (L3 high-risk parameters P0)Automated
QD-T-1.7Missing type / ComplexityStructural ValidityP2/P1/P1Automated
QD-T-2.1String Without maxLengthParameter ConstraintsP2/P1/P1 (L3 executable/external interface fields P0)Automated
QD-T-2.2Missing Format ContractParameter ConstraintsP2/P1/P1Semi-automated
QD-T-2.3Number/Integer Without BoundaryParameter ConstraintsP2/P1/P1 (L3 amount/quantity P0)Automated
QD-T-2.4Array Without maxItemsParameter ConstraintsP2/P1/P1Automated
QD-T-2.5Unit/Precision/Time Zone/CurrencyParameter ConstraintsP2/P1/P0 (critical parameters)Semi-automated
QD-T-2.6Query Without Pagination/CapParameter ConstraintsP1/P1/P1Automated
QD-T-2.7Parameter Dependency/ExclusivityParameter ConstraintsP1/P1/P1Semi-automated
QD-T-3.1State Change and Material Side Effect DeclarationHigh-Risk Operations–/P1/P0Human
QD-T-3.2Untrusted Input Executable ContextHigh-Risk OperationsP1/P0/P0 (by characteristic)Semi-automated
QD-T-3.3Batch/Range Write Operation Without CapHigh-Risk Operations–/P1/P0Semi-automated
QD-T-3.4Authorization Context Parameter ExposureHigh-Risk OperationsTriggered by parameter characteristics / Triggered by parameter characteristics / P0Human
QD-T-3.5Cross-Tool Data FlowHigh-Risk OperationsTransferred to cross QD-TT-5.1Semi-automated
QD-T-3.6Dangerous Default/Operation ScopeHigh-Risk Operations–/P1/P0Automated + human
QD-T-3.7Undeclared Operation Allowlist for Parameterized ToolsHigh-Risk OperationsP0/P0/P0Semi-automated
QD-T-4.1description QualitySemantic ClarityP2/P1/P1Semi-automated
QD-T-4.2Undeclared Side Effects (merged into QD-T-3.1)Semantic ClarityMerged–
QD-T-4.3Multi-Tool Semantic CompetitionSemantic ClarityTransferred to cross QD-TT-5.2Semi-automated
QD-T-4.4Missing Parameter-Level descriptionSemantic ClarityP2/P1/P1Automated
QD-T-4.5description–Schema InconsistencySemantic ClarityP1/P1/P1Semi-automated
QD-T-4.6Sensitive Input Handling DeclarationSemantic Clarity–/P1/P1Semi-automated
QD-T-4.7Missing Output Contract (outputSchema)Semantic Clarity–/P1/P1 (cross-Tool data flow P0)Semi-automated
QD-T-5.1Breaking Change DeterminationSchema ChangesPer §4.3.1 matrixAutomated
QD-T-5.2Change Impact Assessment Omission or ErrorSchema ChangesP1/P1/P0Semi-automated
QD-T-5.3Semantic ChangeSchema ChangesP1/P1/P0Automated + human

Appendix C: Tool Overview ​

The inspection tooling for this specification provides automated detection, assisted review, and report generation; rules requiring business semantics or implementation evidence are confirmed by the responsible party.

C.1 Tool Formats ​

  • Integration Options: CI/CD pipeline, IDE plugin, Schema Registry integration

C.2 Quick Start ​

Visit the SanityOps Tools Website for detailed information.


Appendix D: Exemptions and Risk Acceptance ​

D.1 Exemption Application Conditions ​

Exemptions may be applied for under the following circumstances:

  • Legacy Tools that cannot be remediated in the short term
  • Third-party Tools whose Schemas are beyond the organization's control
  • Business urgency requiring temporary risk acceptance

D.2 Risk Acceptance Statement ​

For defects that cannot be remediated, a Risk Acceptance statement must be signed:

  • Risk Description: Clearly state the defect content and impact
  • Impact Scope: Clearly identify the systems that may be affected
  • Responsibility Confirmation: Clearly identify the responsible party and validity period

Appendix E: Pending Cross-Spec Alignment ​

The items below involve responsibility boundaries with other subsets (Core / Inspect Cross / Inspect Permission / Risk / Quality / Relevance). This specification does not decide them unilaterally without consulting the corresponding subset documents. The corresponding positions in the body text are marked with <!-- PENDING: E-x --> (proposed new-rule items E-16 / E-18 and cross-comparison items E-20 / E-21 have no body-text anchor yet and are registered only in this table). Each item lists the rules involved, the issue, the dependent subset (see the corresponding document for details), and the disposition recommendations for both cases.

IDStatusRules/Sections InvolvedIssueDependent Subset (see details)If covered →If not covered →
E-1✅ ResolvedGlossary, §3.1, §3.5 report level fieldL1/L2/L3 assigned per Tool or per Agent; mixed usageCore (framework/core.md Appendix A)Cite the Core definition; add only mixed-level rulesDefine both Tool risk level and Agent risk level
E-2⏳ Pending alignment§5.2, §5.3, "deduct once per number"Insufficient scoring discrimination; disconnected from Ratchet/BaselineCore (framework/core.md Baseline/Ratchet)Declare that scoring is not used as a Ratchet basisSwitch to fixed deductions or instance-level scoring
E-3⏳ Pending alignmentQD-T-5.x, §4, Appendix BChange inspection not wired into checklist/scoring/GateCore (framework/core.md Gate)Cross-reference the version Diff gateDefine an independent Diff gate
E-4⏳ Pending alignment§3.4 flow diagramOnly the automated stage blocks on P0; no loop-back; fail-fast report incompleteCore / Inspect overview (inspect/index.md)Delete the self-built flow and cite insteadSwitch to full inspection, unified Gate, and add loop-back
E-5⏳ Pending alignmentAppendix DExemptions lack approver/validity cap/review; whether P0 can be exemptedCore (framework/core.md exception management)Keep only Tool-specific exemption conditionsComplete the closed loop; P0 exemption requires compensating controls
E-6⏳ Pending alignment§3.5 report formatInspect PASS and release PASS are not distinguishedCore (framework/core.md report/Gate)Cite Core fieldsAdd inspect_result and release_gate_result
E-7✅ ResolvedGlobal L→P upgrade rules; L3 almost entirely P0Severity mechanically determined by L levelCore / Inspect overviewRe-determine after horizontal alignment with Prompt/SkillSeverity determined by consequence/reachability/scope/existing controls
E-8✅ Resolved§0.2.3 🔴🟡🟢, §1.5/§2.3, §5.2 impact levelsRisk symbols unused; high/medium/low ambiguous; impact levels reuse P0/P1/P2Core / Inspect overviewAdd unified symbols and impact classificationDelete the parallel symbol system; change impact levels to I1/I2/I3
E-9✅ Resolved§0.2.5 vs §3.2.1P1 "requires fix" vs "recommended fix"Core (framework/core.md)Unify according to CoreUnify as "requires fix"
E-10✅ ResolvedQD-T-5.2Assessment activity was coded as a defect; depends on runtime invocation dataCore / Quality (quality/tool-agent.md)Renamed "Change Impact Assessment Omission or Error"; levels P1/P1/P0 added—
E-11✅ ResolvedQD-T-3.5 Cross-Tool Data FlowCurrent definition makes every multi-tool Agent non-compliantInspect Cross (inspect/cross.md)Landed as QD-TT-5.1—
E-12✅ ResolvedQD-T-4.3 Multi-Tool Semantic CompetitionInspection object is a Tool set; the single-Tool process cannot execute itInspect Cross (inspect/cross.md)Landed as QD-TT-5.2—
E-13✅ ResolvedNew outputSchema / output contractWhether it is mandatory depends on CrossInspect Cross (inspect/cross.md)Landed as QD-T-4.7—
E-14⏳ Pending alignmentQD-T-3.4 Authorization Context Parameter ExposureJudging defects by field name is too broadInspect Permission (inspect/permission.md)Narrow to pure Schema signals and cite QD-PMNarrow the definition; trigger P0 at all three levels by parameter characteristics
E-15⏳ Pending alignment§3.1.2 risk determination criteriaOver-reliance on operation namesInspect Permission (inspect/permission.md)Cite its gradingAdd dimensions such as data sensitivity/impact scope/financial consequence
E-16✅ ResolvedNew description poisoning checkManipulative textRisk Explicit (risk/explicit.md)Do not add; only add a cross-referenceAdd QD-T-3.9, P0 all levels (3.7 already used for parameterized tool operation allowlist)
E-17⏳ Pending alignmentQD-T-3.2 narrowed evidence sourceDetermination conclusion depends on implementation evidenceRisk Explicit / Risk Implicit (risk/explicit.md, risk/implicit.md)tool.md retains only Schema signalsRetain the manual confirmation step
E-18⏳ Not landedNew idempotency/retry semantics declarationLoop retries cause duplicate orders/transfersQuality Tool-Agent / Relevance (quality/tool-agent.md, framework/relevance.md)Retain the declaration-layer check but the P level may be loweredAdd QD-T-3.8
E-19✅ ResolvedQD-T-3.1 wordingConfirmation/approval bearing relationshipInspect Permission / Relevance (inspect/permission.md, framework/relevance.md)State the bearing subsetKeep neutral wording
E-20⏳ Pending alignmentNumbering categories 1–5 vs chapter correspondenceWhether it is a unified template for InspectInspect Prompt/Skill/overview (inspect/prompt.md, inspect/skill.md, inspect/index.md)Do not change the structureAllow A-12 chapter adjustments
E-21⏳ Pending alignmentPart 5 unified rule tableNeeds horizontal comparison with Prompt/Skill rules at the same levelInspect Prompt/Skill (inspect/prompt.md, inspect/skill.md)Fine-tune after alignmentKeep this table

Resolved Items (2026-10):

  • E-1 ✅ Resolved—OR-L placement (see §3.1 heading, glossary, and report level field prefix);
  • E-7 ✅ Resolved—severity decoupled from mechanical L-level mapping (core.md §4.1 rules that "severity is determined by OR-L and applicability by AC-L");
  • E-8 ✅ Resolved—deleted the §0.2.3 🔴🟡🟢 risk symbol table (core.md §4.1 adds the exclusivity constraint for P symbols);
  • E-9 ✅ Resolved—the P0/P1/P2 wording in §3.2.1 aligned with §0.2.5 (P1 unified as "requires fix, recommended before release");
  • E-11, E-12 ✅ Transferred to Inspect Cross, landed as QD-TT-5.1 / QD-TT-5.2 (cross.md Part 4.7); this specification retains a P2 lead;
  • E-13 ✅ Landed as QD-T-4.7 (this specification §2.2.3), default L2/L3 P1, upgraded to P0 when a cross-Tool data flow (QD-TT-5.1) is hit;
  • E-16 ✅ Handled via the "add cross-reference only" path—instruction injection in the tool description field belongs to Risk Explicit NR-S-5, and QD-T-4.1 has a cross-subset attribution note; QD-T-3.9 is not added;
  • E-19 ✅ Closed—QD-T-3.1 wording bears to Inspect Permission QD-PM-6.3, and permission has been unified to the "human decision point (pre-authorization/confirmation/approval all acceptable)" caliber;

Items Pending Alignment / Not Landed: E-2/3/4/5/6/10 → Core; E-14/15 → Permission or narrowing within tool; E-17 → Risk; E-18 → Quality/Tool-Agent (QD-T-3.8 remains a candidate number and is not landed); E-20/21 → Prompt/Skill/Tool template cross-comparison. The above are deferred to the respective subset version updates and do not block this round of cross alignment.

End of Document


© 2026 Sanity AI Labs. Licensed under Creative Commons Attribution-ShareAlike 4.0 International. https://www.sanityops.org · hello@sanityops.org v1.1 · October 2026