Skip to main content
Reply IntelligenceGroup

DriftGuard · v0.0.1

Regression locks + honest failure diagnosis for AI/LLM features.

Define behavioural invariants (locks) for your AI features. When a code or prompt change makes one fail, DriftGuard tells you what likely broke it, how confident it is — and refuses to fake confidence it does not have — then hands your coding agent a ready-to-run repair prompt.

Request early accessAll products

DriftGuard pipeline

Behavioural invariants run against fixtures

Why it's different

Diagnosis + repair, with earned confidence — not dashboards alone.

  • Most AI-eval tools focus on detection. DriftGuard's distinctive half is diagnosis + repair, deliberately honest about certainty.
  • Confidence is earned, not guessed — a three-tier honesty cap tied to available evidence.
  • Repair prompts are mode-gated: suggest_patch (high), minimal_fix (medium), investigate_first (low).
  • Anti-gaming: every prompt forbids editing the lock or fixtures to force a pass.
Evidence availableConfidence ceilingMeaning
Signature/pattern only0.40 (low/medium)A heuristic, not proof — investigate_first
+ execution trace0.69 (medium)We know the mechanism, not the commit
+ git diff or bisected commituncapped (up to high)Causal evidence links it to a specific change

How it works

Locks → engine → repair prompt

  1. 01

    Locks

    Your behavioural invariants — run() returns pass, drift, or error with failed assertions.

  2. 02

    Engine

    Signatures match failures to drift patterns; scorer ranks changed files; confidence applies honesty caps.

  3. 03

    Repair plan

    Mode-gated, coding-agent-ready repair prompt emitted per failing lock.

  4. 04

    Report

    DriftAnalysisReport — cause, ranked files, confidence, severity, repair prompt.

Quick start

Run the demo-shop pack to watch DriftGuard catch a planted regression.

shell
npm install
npm run build
npm test
npm run demo
driftguard output
DriftGuard — pack demo-shop
  1 passed · 1 drifted · 0 errored (of 2)
  ✗ shop.latest_intent_wins — drift
      cause: Likely: stale intent carryover …
      confidence: high (86%) · severity: high · mode: suggest_patch

CLI

driftguard
driftguard <pack.js> [--git <git-context.json>] [--prompts] [--emit <dir>]

Bisection

git-bisect for AI behaviour
driftguard bisect --range "main~30..main" \
  --command "npm run build --silent && node dist/src/cli.js dist/packs/demo-shop/pack.js"

Authoring — JSON pack

drift.pack.json
{
  "id": "my-product",
  "locks": [{
    "key": "support.latest_intent_wins",
    "scope": "ai",
    "invariants": ["When the user pivots topic, the reply addresses the latest intent."],
    "failureSignatures": ["stale_intent_carryover"],
    "watchedSymbols": ["composeReply"],
    "target": { "command": "node assistant.mjs", "inputFormat": "stdin-json" },
    "fixture": { "turns": [] },
    "assertions": [
      { "name": "focuses_on_latest_intent", "type": "matches", "value": "track|delivery" }
    ]
  }]
}

Coding-agent integration

Hand your agent a ready-to-run repair prompt.

headless repair
claude -p "$(cat .driftguard/shop.latest_intent_wins.repair.md)"

Roadmap

  • ✓Earned causality — execution traces, bisection, watched-symbol analysis
  • ✓Coding-agent integration — --emit repair prompts + AGENT_RULES
  • ✓Bisection host adapter — real git checkout + probe + restore
  • ✓Self-serve authoring — JSON packs with declarative assertions
  • ○Hosted cloud — run history, shared lock catalogues, CI dashboards, RBAC

FAQ

What is a DriftGuard lock?

A lock is a behavioural invariant you define for an AI feature. It runs against fixtures or live targets and returns pass, drift, or error. When a lock drifts after a code or prompt change, DriftGuard diagnoses the likely cause and generates a repair prompt appropriate to the confidence level.

What is a DriftGuard pack?

A pack is the entire domain-specific configuration DriftGuard needs: signatures, system map, constraints, and locks. The engine imports nothing domain-specific — swap the pack and point it at a different product with zero engine changes.

Can I define locks without writing TypeScript?

Yes. DriftGuard supports config-only JSON packs with target commands and declarative assertions on stdout. Assertion types include contains, not_contains, matches, not_matches, equals, and not_equals. See the demo-shop-declarative pack for a complete example.

How does DriftGuard integrate with coding agents?

Run driftguard with --emit <dir> to write per-lock repair markdown files, an index summary, and AGENT_RULES.md for safe agent behaviour. You can pipe a repair prompt directly to Claude Code or Cursor: claude -p "$(cat .driftguard/lock-key.repair.md)".