Architecture & Design Overview: RAMPART & Clarity

As enterprise AI transitions from simple retrieval-augmented generation (RAG) text assistants to autonomous agentic systems—which can execute code, modify CRM data, read employee emails, and orchestrate third-party APIs—the safety attack surface expands exponentially. Traditional static or black-box LLM safety scanning is no longer sufficient.

Microsoft’s release introduces two open-source frameworks to bridge the gap between design-phase risk evaluation and continuous integration (CI) safety testing.

1. RAMPART: Continuous Safety Testing for Agentic AI

RAMPART is an open-source test framework built to run automated adversarial and functional safety test suites natively within modern CI/CD pipelines.

Underlying Engine & Developer Experience

• Core Foundation: RAMPART is built directly on top of PyRIT (Python Risk Identification Tool), Microsoft’s established red-teaming framework. While PyRIT is designed for security researchers conducting black-box discovery, RAMPART is built for developers to use during the active build phase.
• Native Pytest Integration: Tests are written as standard pytest integration suites. This allows developers to gate PRs based on security assertions.
• Loose Coupling (Adapters): RAMPART interfaces with your agentic infrastructure via thin, custom-built adapters. Your test code orchestrates the conversation, interacts with the agent, and hooks into output streams to validate results.

Key Architectural Technicalities

• Cross-Prompt Injection Defense: RAMPART’s primary production-ready layer focuses heavily on indirect prompt injection. This targets scenarios where an agent ingests untrusted or third-party data (e.g., parsing an external email or a ticket payload containing hidden instructions) that hijacks the LLM’s system prompt to force unauthorized tool execution.
• Probabilistic / Statistical Asserts: Because LLMs are inherently non-deterministic, a standard single-pass test is insufficient. RAMPART introduces statistical trial configurations. You can run a test multiple times and configure threshold assertions (e.g., “This data-exfiltration boundary must not be breached in over 80% of 50 concurrent test iterations”).
• Protocol-Driven Extensibility: Extension points, evaluators, and payload injectors are defined using native Python Protocols, making it straightforward to build custom plugins for non-standard agent architectures.
• Multi-Signal Composable Evaluators: Rather than evaluating text outputs with a simple binary string match, RAMPART utilizes composable evaluators linked by boolean logic (AND, OR, NOT). These evaluators monitor:
• • LLM-generated string variations.
  • System side-effects (e.g., verifying if specific database wrappers or APIs were called under injection conditions).
  • Unauthorized down-stream tool execution.

2. Clarity: Specification and Assumption Verification

Clarity is an engineering-native framework designed to eliminate architectural and semantic flaws before implementation begins. It turns architectural design decisions and failure analysis into source-controlled, reviewable code artifacts.

Operational Environment

Clarity runs as a cross-platform desktop application, a web-based user interface, or integrated directly into specialized engineering agents.

The .clarity-protocol/ Pipeline

As teams interact with Clarity to refine a feature or component, the tool writes its state down to a .clarity-protocol/ directory inside your git repository.

• Format: Every architectural constraint, user flow requirement, and threat mitigation plan is stored as raw, human-readable Markdown files.
• Git Native: Because these specs are raw text, they are tracked, diffed, and reviewed via standard Pull Requests, making system intent explicit to security reviewers.

Technical Capabilities

• System-Level Disambiguation: Clarity focuses on forcing explicit engineering scope. It prompts engineers to break ambiguous requests down into architectural boundaries (e.g., evaluating if a real-time collaborative tool truly needs an overly complex web-socket operational transformation layer, or if simpler transactional locks satisfy the core user requirement).
• Automated Parallel Threat Modeling: Clarity utilizes multiple specialized LLM “thinkers” that independently critique design documentation across four distinct vectors simultaneously:

1. 1. Security & Cryptography / Authorization
   2. Human Factors & UX Failure Modes
   3. Adversarial Exploitation Vectors
   4. Operational & Scale Bottlenecks

• Dependency Graph & Staleness Tracking: The markdown files in .clarity-protocol/ form an explicit dependency graph. If a product engineer updates a high-level system requirement or problem statement file, Clarity flags down-stream implementation notes and failure analyses as “stale,” forcing the team to systematically re-evaluate their assumptions during code reviews.
• Decision Tracking (ADRs): Clarity structures and logs Architectural Decision Records natively, tracking options considered, criteria used, and reasons for rejection, generating a coherent historical review packet for external compliance or security auditors.

3. Workflow Synthesis: The End-to-End Safety Lifecycle

When deployed in tandem, RAMPART and Clarity create a closed-loop engineering feedback system:

1. Capture Intent (Clarity): The product team and safety architects map out the agent tools and access boundaries. The automated failure analysis surfaces early design flaws.
2. Commit Spec: The verified design is checked into the repository under .clarity-protocol/.
3. Write Regressions (RAMPART): When engineering implements the agent, they simultaneously build the corresponding RAMPART testing suite to validate those design boundaries in code.
4. Automated Enforcement: RAMPART runs inside the CI/CD environment. If an LLM optimization or prompt change causes an agent to accidentally bypass a data safety boundary, the build fails before deployment.
5. Incident Handling: If a new exploit bypasses defenses in production, the incident response team can replicate the vector, write a RAMPART test replicating the failure, fix the prompt or code, and permanently add that test to the CI pipeline to prevent future regression.

Getting in Touch / Contributing

Both toolsets are open-source and managed by Microsoft’s AI Red Team. For deployment feedback or enterprise partnerships, Microsoft has opened a dedicated channel at aisafetytools@microsoft.com.