Skip to content

Risk — Agent Security Risk Detection ​

Proactive security validation for AI Agents. Audits explicit risks embedded in Logic Artifacts and dynamically validates implicit vulnerabilities through attack execution in shadow sandboxes.


Why Risk? ​

Inspect verifies that Logic Artifacts are well-formed — but even well-formed artifacts can harbor security risks that static inspection alone cannot surface:

  • Explicit risks: Dangerous instructions, privilege escalation paths, or hijacking triggers deliberately or inadvertently expressed in System Prompts, Skills, or Tool Schemas — readable yet easily overlooked in manual review
  • Implicit risks: Runtime vulnerabilities that only emerge under multi-turn logic attacks, where imprecise definitions and boundary gaps become exploitable attack surfaces

Traditional runtime Guardrails face an impossible triangle of real-time detection, complex attack logic, and massive context — making them structurally incapable of catching sophisticated logic attacks. Meanwhile, generic Red Teaming tools rely on blind testing with no awareness of artifact-specific weaknesses.

Risk bridges this gap: audit what can be seen statically, then validate what can only be found dynamically.


Two Sub-Specifications ​

  • Risk Explicit ​

    Static audit of explicit risks expressed in Logic Artifacts.

    Scans System Prompts, Skills, and Tool Schemas for dangerous content across 32 subcategories in 4 families: Continuous NL (NL-A), Non-Continuous NL (NL-B), Obfuscated Coding (NR-O), and Structured Configuration (NR-S). Assigns impact type labels — confidentiality (D1), integrity (D2), and authorization (D3) — recorded for traceability only, and grades severity as Explicit·P0/P1/P2.

    Numbering system: EX-x.y — NL-A, NL-B, NR-O, NR-S

  • Risk Implicit ​

    Dynamic validation of implicit runtime vulnerabilities.

    Generates artifact-driven, targeted attack test cases from Inspect defect data, executes them against the Agent in a Shadow Sandbox, and classifies outcomes into four termination statuses: A (Attack Successful), B (Blocked by Third Party), C (LLM Refused), and D (Test Conditions Not Met). Produces a Signal Score and Gate decision for release admission.

    Four core design principles: Relevance, Safety, Measurability, Integrability


Explicit Risk: Audit Dimensions ​

FamilyTypeCountDescription
NL-AContinuous Natural Language9Risk semantics complete within a single sentence or adjacent statements
NL-BNon-Continuous Natural Language10Risk fragments dispersed across multiple locations, requiring association analysis
NR-OObfuscated Coding5Hidden instruction content identifiable after decoding
NR-SStructured Configuration8Implicit dangerous configurations identifiable through field validation

Harm Assessment Model (three independent dimensions):

DimensionFocus
D1Confidentiality — data exposure, information leakage
D2Integrity — unauthorized modification, logic tampering
D3Authorization — privilege escalation, permission bypass

Implicit Risk: Validation Flow ​

Inspect Defect Data
    │
    ▼
Attack Strategy Derivation ── artifact-driven, not blind
    │
    ▼
Test Case Generation ── parameterized, targeted
    │
    ▼
Shadow Sandbox Execution ── safe isolation
    │
    ▼
Termination Status Classification ── A / B / C / D
    │
    ▼
Signal Score → Gate Decision

Severity Levels ​

LevelMeaningDisposition
P0Data/conversation leakage, irreversible consequences, core functionality bypassed, or security restrictions completely bypassedBlock release
P1Recoverable tampering or limited privilege escalationFix before release, or approved exemption
P2Read-only, or within existing permission scopeRecord in risk ledger

Position Within the SanityOps Framework ​

Inspect (Defect Inspection) ── finds the defects
    │
    ▼
Risk (Risk Scanning) ── You are here
    │   ├─ Explicit: audits dangerous content in artifacts
    │   └─ Implicit: validates exploitable attack surfaces at runtime
    │
    ▼
Quality (Service Quality) ── RAG Agent + Tool Agent
    │
    ▼
Relevance (Relevance Assessment)

Risk answers the question Inspect cannot: given the defects found, is the Agent actually vulnerable to attack? It provides independent, auditable security proofs for compliance, investor, and customer requirements.


Quick Start ​

  1. Complete Inspect first — Risk Explicit and Implicit both depend on defect data from Inspect
  2. Run Risk Explicit to audit artifacts for dangerous expressions, scripts, and dangerous authorizations
  3. Run Risk Implicit in a Shadow Sandbox to validate whether identified defects form real attack surfaces
  4. Feed results to Relevance for defect → risk diagnostic mapping