Stopping Ungoverned AI Agents with Compass

A coding agent wiped a software company’s production database, then told the team exactly what it had done: “I destroyed months of work in seconds.” The incident, covered by Fortune in July 2025, became one of the canonical stories of the agentic era because it was so concrete. Nobody debated whether the agent was capable. Nobody debated whether the company wanted to use agents. The question that actually mattered was why the agent was allowed to touch a production database at all.

That question has a name in 2026, and it is governance. Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027, with inadequate risk controls cited alongside escalating costs and unclear business value. That is not a forecast about technology. It is a forecast about control. Organizations are discovering that they can give agents capability faster than they can give them guardrails, and the projects that die are dying on the control side, not the capability side.

At Shakudo, we watched that gap widen for a year. We built the agentic framework. We built the gateway that routes and protects LLM traffic. And then we built the piece that was missing between them: a policy gate that evaluates what an agent is about to do before it does it. That product is Compass, and this post is our first deep dive into how it works, why it exists, and where it fits in a market that is only now forming around AI agent governance. More of the thinking behind our agent stack is on the Shakudo blog.

The cost of ungoverned agents

Before we explain what Compass does, it is worth being precise about what “ungoverned” means in practice, because the failure mode is not one thing. In the customer conversations we have had this year, the same handful of problems keep surfacing, and they cluster into a short list:

1. No proof that guardrails hold at scale. A guardrail that works on ten prompts a day is a toy. The moment an agent is making hundreds of tool calls a minute, the question changes from “does this work” to “how do we know it keeps working, and who demonstrated it?” Most teams have no answer to that question.
2. Shadow AI. Teams are using agents with real credentials and real tools without anyone in security knowing. The agent estate is larger and riskier than the organization thinks, and discovery becomes a permanent arms race.
3. Human-in-the-loop is a developer pattern, not an org workflow. Most frameworks ship “ask a human” as an SDK callback. It has no roles, no escalation path, no audit semantics, and no answer to “who was supposed to see this, and did they?”
4. No who-did-what. When something goes wrong, the team cannot answer which agent did what, on whose behalf, under which policy, and who approved it. The logs, if they exist, are scattered across several tools and cannot be joined.
5. Nothing a regulator can consume. Compliance teams do not ask for “agent logs” in the abstract. They ask for a record that maps to a specific requirement, with retention and integrity guarantees. Most agent platforms cannot produce that record.
6. The silo objection. Security teams do not want a fourteenth console. If governance lives somewhere the agents do not, it gets bypassed the first time a deadline lands.
7. Cost opacity. An ungoverned agent is also an unpriced agent. When you cannot see what the agent is doing, you cannot see what it is spending either, and the invoice arrives before the incident report does.

None of these requires a malicious actor. They are what happens when capability outruns control, which is the default state in most organizations right now. Here is what that state looks like when an agent runs in production with no gate in front of it:

The adoption data confirms the control gap. In McKinsey’s State of AI survey, only a minority of organizations had scaled agentic AI, while most were still experimenting or had not started at all, and the gap McKinsey identifies between pilot and production is operational rather than technical. Translation: the models work. What is missing is the operational layer, the part that answers who may do what, under whose approval, with what record. The gap shows up the same way in every organization we have talked to:

• Nobody owns the policy. Guardrails live in prompts, in SDK code, and in the heads of the people who wrote the agent. There is no version of them to review and no owner to hold accountable.
• Nobody is notified when an agent does something unusual. The signal arrives in a log, and by the time a human reads it, the database is already gone.
• Nobody can answer the post-incident question. When something happens, the team has to reconstruct what the agent did from logs scattered across four or five tools, most of them mutable.
• Nobody is accountable for what the agent did. When an agent acts on a service account, “the agent did it” is not a valid answer in an audit.

Why current approaches fail

The good news is that the industry has named the problem with a precision that only appears when buyers are hurting. The bad news is that most of the tools on sale today do not actually address the core failure.

The most useful taxonomy we have found is APORT’s 2026 guide to agent guardrails. It describes four layers of defense: content filtering, evaluation and monitoring, sandboxing, and pre-action authorization. Their summary of the difference between the last two layers is exactly right. “A useful way to think about the difference between sandbox and action: a sandbox stops rm -rf /. The action layer stops transfer_funds(amount=50000, to=attacker_account).” A sandbox confines where an agent can break things. It says nothing about which of the allowed things the agent is permitted to do, which is where most of the real business risk lives.

Forrester’s AEGIS framework reaches the same conclusion from the risk side. The agentic risks it names include emergent behavior that can bypass entitlements and escalate privileges, obscured causal provenance making post-incident forensics nearly impossible, and decision fatigue for the humans who are supposed to be in the loop. Forrester’s one-line summary is “secure intent, not just infrastructure.” Intent is the part of the problem that most current products do not touch.

Kosmoy’s 2026 survey of governance platforms puts it most bluntly: almost everyone can discover and monitor agents now. Almost no one can contain one. The vendor categories in that survey are a good map of the whole market:

• Observability and monitoring (Galileo, Arize AX, Weave, Langfuse). Excellent at telling you what an agent did, after the fact. Post-hoc by definition.
• LLM firewalls and content gateways (Palo Alto Prisma AIRS, Lakera, Prompt Security, ZNYX, LlamaFirewall, NeMo Guardrails). Score and filter content: prompts, responses, PII, jailbreaks. They do not evaluate the business action, such as “write to this production table” or “send this email as this user.”
• GRC and governance platforms (Credo AI, SAP AI Agent Hub). Program of record, risk registers, vendor inventories. Strong on paperwork, no runtime enforcement.
• Kill switches and control towers (ServiceNow AI Control Tower, Zenity). Stop the bleeding after something observable has already happened.

The Replit incident is the canonical failure of the post-hoc world. If your primary defense is a camera, a wiped database is a photo you look at during the post-mortem. The damage is done before the camera is relevant. Every category above has a real job to do in a mature stack, but none of them, alone, answers the question the Replit team should have been able to answer in advance: is this agent allowed to touch production?

Introducing Compass

Compass is policy-as-code governance for AI agents. It is the third pillar of Shakudo’s agent stack, sitting alongside the AI Gateway, which routes and protects LLM traffic, and Kaji, the agentic framework. Compass is the enforcement point: it evaluates each prompt and each planned action before the model executes, and it returns one of three decisions.

The policies themselves are YAML, owned by the security team, versioned in git, and deployed like any other infrastructure policy. A real policy from our own environment:

kind: Policy
apiVersion: compass.kaji.shakudo.io/v1
metadata:
  name: no-prod-writes
spec:
  match:
    action: database.write
    environment: production
  decision: REQUIRE_APPROVAL
  approver: data-platform-oncall

Three things about that file matter. First, it is a document a security engineer can review in a pull request, not a config buried in a model team’s notebook. Second, it matches on the dimensions that actually distinguish risk. Compass policies can match on:

• Prompt patterns, for example a prompt that looks like it is attempting to exfiltrate data
• Data sensitivity, such as PII or customer identifiers present in the context
• Action types, the difference between read, write, delete, send, and pay
• Agent roles, a billing agent is not a research agent
• Environments, the same agent in staging is a different risk than in production
• Time windows, for instance no unattended agent runs on weekends

When multiple policies match the same prompt, Compass applies strict precedence. There is no fuzzy scoring and no “highest score wins”:

• BLOCK means the action is denied, the run stops, and the event is logged.
• REQUIRE_APPROVAL means the run suspends and an approval card is raised for a human.
• ALLOW means the action proceeds to the LLM.

BLOCK always wins. If any policy says block, the answer is block. If nothing blocks and something requires approval, the answer is approval. Only when no policy objects does the action pass. This is the same shape as a network firewall, and it is deliberate: governance that requires a human to interpret scores is governance that gets bypassed under pressure.

Enforcement happens in the agent runtime rather than in a sidecar you have to remember to query. The gate is a hook in the kaji-core agent loop, so an agent running on Kaji is governed by default, with no wrapper and no separate service in the call path. The policy engine compiles YAML into a match plan and evaluates it at line rate with caching, so the gate does not become a throughput bottleneck on the way to the model.

How it works

Here is what happens when a governed agent runs in production.

The gate. Every prompt and planned action passes through the policy gate before the LLM. The gate evaluates the compiled policy set and returns a decision, and the decision itself, including denials and allows, becomes an audit event. Nothing about the evaluation is implicit.

Approval is a state, not a message. When the decision is REQUIRE_APPROVAL, the run does not block a thread. It suspends. Compass raises an approval card in KajiChat carrying the full context of the decision. The suspended run has a timeout, 600 seconds by default. If no human decides in time, the safe default is block, not allow. The card carries everything an approver needs to judge the decision:

• The prompt that triggered the policy match, so the approver sees exactly what the agent intended to do.
• The policy that matched, including which rule fired, so the decision can be traced to policy text instead of a human’s memory.
• The action type and the environment, so the approver can judge scope, not just intent.
• The state of the run itself, including the countdown, so that silence is an explicit outcome rather than an ambiguity.

The resolution comes back over a signed callback: KajiChat calls Compass with a shared key, and Compass either resumes the run or blocks it. The decision, approve, reject, or timeout, is the next audit event in the chain, so the human’s choice has the same integrity as the machine’s.

The audit trail is tamper-evident. Every decision event is appended to a hash-chained log: each event records the hash of the previous event, and its own hash is computed from its content plus that previous hash. The chain is stored through NATS JetStream and PostgreSQL. The practical property is that if someone alters or deletes event 3, event 4’s hash no longer verifies, and the break is visible to anyone who walks the chain.

A log file that “the ops team says is fine” is one thing. A chain that proves its own integrity is another, and it matters more than it sounds because of where regulation is heading. If your agent logs can be silently altered and you cannot show otherwise, their evidentiary value is zero. The hash chain is the cheapest, most boring way to turn “we did not alter the logs” from a promise into a demonstration.

How Compass compares

Every row in that table is doing real work, and a mature agent estate will have all of them. The difference is one word: gate versus monitor. A monitor tells you what happened. A gate decides what may happen. The Replit incident was not a monitoring problem. The company was probably logging everything. The database was gone anyway.

KajiChat integration: approval is the workflow

Approval is where most “governance” products give up, because it means building a human workflow: notifications, context, a decision surface, a timeout, a callback, and an audit record. We built it into KajiChat, which is where our customers already talk to their agents. The workflow is five steps, and every step leaves a record:

1. The agent’s run hits a REQUIRE_APPROVAL policy match and suspends. Nothing executes while it waits.
2. Compass raises an approval card in KajiChat with the full decision context attached.
3. A human approves or rejects from the card, optionally in a forked chat where they can question the agent first.
4. The decision flows back to Compass through a signed callback, or the run times out and the safe default, block, applies.
5. The decision, with the reason if one was given, is appended as a tamper-evident audit event.

When a run suspends on REQUIRE_APPROVAL, the approver gets an approval card in KajiChat. The card shows the prompt that triggered it, the policy that matched, the action type, and the environment, and the approver can approve or reject directly from the card. Approvals can be handled in a forked chat, so the approver can ask the agent follow-up questions, “why are you writing to that table?” without touching the suspended run. The agent can answer, the approver decides, and the decision flows back through the signed callback and becomes an audit event. Nobody has to leave the workflow they are already in, and nobody has to trust that the right person saw the right card.

The reason this matters beyond ergonomics: the OECD AI Incident Database now tracks incidents in which autonomous agents leaked sensitive commercial information. In November 2025, one such incident was registered in which an agent exposed confidential business information to an unintended recipient. Per the OECD registry, the incident class is the same shape as the ones that will define this category: not a model failure, but a permission failure. An agent that can see the information and send it will, eventually, send it. The only reliable fix is to make the send itself the thing that requires approval.

Compliance and regulatory alignment

For the compliance team, Compass is best described as the part of the agent stack that produces evidence. The relevant requirements, and where they land:

• EU AI Act, Article 12: high-risk AI systems “shall technically allow for the automatic recording of events (logs) over the lifetime of the system.” Compass records every decision, allow, deny, approve, reject, and timeout, at the enforcement point, automatically, for the lifetime of the deployment. See Article 12.
• EU AI Act, Article 13: providers must document how logs are collected and can be interpreted. Compass events have a fixed schema, what matched, what was decided, who was involved, and the chain position, so the documentation is a page, not an excavation. See Article 13.
• EU AI Act, Articles 19 and 26: logs for high-risk systems must be retained for at least six months. Compass retention is configurable per deployment, on JetStream and PostgreSQL. See Article 19 and Article 26.
• Annex III high-risk uses (credit scoring, resume filtering, insurance pricing, and more) are precisely the cases where a wrong agent action is a regulatory event, not just an operational one. The categories are listed in the EU AI Act overview.

Beyond the AI Act, SOC 2 and GDPR engagements are beginning to name agent decision records as an explicit audit topic, and the direction is clear even where the language is still settling.

The standards bodies are converging on the same shape. The IETF has a working draft, “Agent Audit Trail: A Standard Logging Format for Autonomous AI Systems”, that standardizes the logging format for autonomous AI agents. That is the exact artifact Compass already produces, which tells you where the category is heading. Compass records an event for every decision the gate makes, not just the interesting ones:

• Every ALLOW, so you can prove the agent was checked, not just that the blocked things were checked.
• Every BLOCK, with the policy that fired and the prompt that matched it.
• Every APPROVED and every REJECTED, with the approver’s identity and the reason when one is provided.
• Every TIMEOUT, recorded as a decision rather than a gap, because a run that expired unanswered is a decision the organization made by design.

Honest about maturity

Two honest caveats. First, the end-to-end matrix of approval, reject, block, and timeout paths has been verified green in our UAT environment, and we treat that as production-ready for the core governance loop. We will keep publishing the matrix as we add cells. Second, role-based access control is on the roadmap, not in the release today. The role header that policies can match against is a stub in the current build, and we are not going to oversell it as if it were not. If your use case depends on fine-grained per-role policy today, that is a conversation to have before you commit.

The deployment story matters too. Compass is built to run on your own infrastructure, including air-gapped environments, and the dependency footprint is small enough to operate:

• An embedded PostgreSQL for policy and audit state, with no managed database to hand over.
• NATS JetStream for the asynchronous audit event stream that feeds the hash chain.
• Keycloak for OIDC, so the governance plane authenticates against your existing identity providers.

Few governance platforms in the current set document a self-hosted story at all. For organizations where “own your AI” means the data plane never leaves the network, the governance plane has to live there too. That is the Shakudo Platform story, and Compass is the governance pillar of it.

One positioning note: most agent governance is being sold to security teams. We believe a large part of the buyer pool is the IT leader whose organization is vibe-coding, and who wants teams to keep shipping, with guardrails and an audit trail, without handing agents admin keys to the cloud. That is the buyer Compass is built for.

FAQ

Q: What does “ungoverned agent risk” actually mean?

It means an agent with real capabilities and real credentials, operating under rules that nobody can see, prove, or enforce. The agent can read, write, send, and pay. The organization’s protection is a log file that someone will look at after the fact. The risk is not that the model is malicious. The risk is that the model is competent, and competence without a gate is exactly what produced the Replit database wipe.

Q: How does Compass stop an agent before it acts?

By sitting between the agent and the model, and between the agent and its tools, as a pre-action policy gate. Every prompt and planned action is evaluated against the compiled policy set before execution. The decision is one of three, BLOCK, REQUIRE_APPROVAL, or ALLOW, with BLOCK taking precedence. A blocked action never reaches the model or the tool. An approval-required action suspends the run until a human decides, with a timeout that defaults to block.

Q: Is Compass an observability tool or a governance tool?

A governance tool, deliberately. Observability tells you what happened. Compass decides what may happen, before it happens. In practice the two pair well: Compass produces a clean, structured, per-decision event stream that an observability platform can ingest, and the observability layer can feed anomaly signals back into policy. But the categories are different, and a camera is not a gate.

Q: Can Compass run on-premises or air-gapped?

Yes. Compass is built to deploy on your own infrastructure. It runs on embedded PostgreSQL, NATS JetStream, and Keycloak OIDC, with no dependency on external SaaS services in the enforcement path. For air-gapped environments, the same image runs, and policy updates flow through whatever change management process your infrastructure already uses. This matters because a governance system that phones home is not a governance system for the organizations that need it most.

Q: How does Compass handle human approval workflows?

As a first-class product feature, not an SDK pattern. A REQUIRE_APPROVAL decision suspends the run and raises an approval card in KajiChat with the full decision context. The approver approves or rejects from the card, optionally after a forked chat with the agent. The decision returns to Compass over a signed callback and is recorded as an audit event. The run has a timeout, and the timeout behavior is block. Approval maps to your organization, not to your codebase.

Q: Does Compass integrate with existing agent frameworks?

The gate is a hook in the kaji-core agent loop, so Kaji agents are governed by default. For other frameworks, the enforcement point is the same architectural position: a pre-action check on prompts and tool calls. We integrate where the agent is already running, because a governance tool that requires re-architecture gets bypassed the first time a deadline lands.

Q: What is a tamper-evident audit trail, and why does it matter?

A tamper-evident audit trail is a log that can prove its own integrity. Compass appends every decision event to a hash chain: each event embeds the hash of the previous event, so altering any single event breaks the chain from that point forward. Verification is a walk of the chain, and the break is visible to anyone performing it. It matters because regulation is converging on exactly this artifact. The EU AI Act requires automatic, lifetime event recording for high-risk systems, and an IETF draft is standardizing the format. A log you can prove you did not alter is a different legal object than a log you can only promise you did not alter.

Conclusion

The market is telling you what it thinks. Gartner published its first Magic Quadrant for AI Governance Platforms in 2026, signaling a market it sizes in the billions by 2030. Forrester is telling CISOs to secure intent, not just infrastructure. The IETF is writing the log format for the exact artifact Compass already produces. The direction is unambiguous.

The Replit agent’s database is not coming back. What can come back is the version of your agent estate where every action is checked before it happens, every decision is provable, and the humans in the loop have a real workflow instead of a callback. That is what governance means, and that is what Compass is.

Monitoring is not enough. You need a gate. If you are running agents in production today, there are four things worth checking this week:

• Whether any agent can reach production data without a policy match, and if so, what that first match should block.
• Whether your current logs would hold up as evidence: can you show they were not altered after the fact?
• Whether your human approval is a product or a pattern, meaning whether it has an owner, a timeout, and a record.
• Whether your approvers actually see the cards, and how long the median decision takes.

Own your AI. Govern your AI.

Want to see Compass on your own stack? Contact us and we will walk through the policy model, the approval workflow, and the audit chain against your actual workloads.