An AI agent that can read production data and propose changes is only useful if the team running it trusts it. Trust here does not mean "the model is accurate." It means the team always knows three things: what the agent can see, what it is about to do, and who decided.
Most conversations about AI agent security start with the model: how often it hallucinates, how well it reasons, which benchmark it tops. Those questions matter, but they are not what keeps an on-call engineer up at night. What keeps them up is an agent restarting a service at 3am with nobody watching, or a change nobody can explain the following week.
When we designed Toby AI Agents, we started from that question instead of from the model. We asked what an SRE lead, a security reviewer and an auditor would each need to see before letting an agent near production. The answer became five guardrails. This post explains each one, why we built it, and the things we chose not to build.
Why AI Agent Guardrails Matter More Than Model Accuracy
A chatbot that gets an answer wrong wastes a few minutes. An agent that acts on a wrong answer can take down checkout, rotate the wrong credential or scale a cluster past budget. The cost of a mistake moves from "annoying" to "incident."
That shift is why agentic AI guardrails are a design problem, not a prompt problem. No model is right every time, so the system around it has to make mistakes visible, small and reversible. In practice that means four properties:
- Bounded: the agent can only reach the tools and data its job needs.
- Supervised: a person decides before anything in production changes.
- Traceable: every step can be reconstructed after the fact.
- Consistent: the same rules hold whether you approve from a laptop, a phone or a schedule.
The five decisions below are how we turned those properties into product behaviour.
[Insert image here]
1. Every Agent Holds One Role, With a Defined Scope
We did not build one general agent with access to everything. Each agent takes a role, and Toby AI Agents ships with four of them: SRE, Application, FinOps and Security. Each role comes with three limits.
- The tools it may call. An SRE Agent can search logs, query metrics, look up traces and read deploy events. It cannot file a security escalation.
- The data it may read. A Security Agent reads access and audit streams, not application payloads or customer records.
- What it hands back. Each role returns a defined result, such as a root cause with evidence, or a cost change with its expected saving.
A Custom Agent gets only the skills and tools your team registers for it. Nothing wider.
Think of it the way you already manage people. A new SRE does not get production admin rights on day one, and a finance analyst does not get write access to your Kubernetes clusters. AI agent permissions should follow the same principle of least privilege, and they should be visible to the team rather than buried in a config file.
Why: a narrow scope makes an agent's behaviour predictable, and it limits the damage if something goes wrong. When an AI SRE agent can only touch what an SRE workflow needs, a bad conclusion stays a bad conclusion. It cannot quietly become a bad change somewhere else.
2. Nothing Runs Until a Person Approves It
The agent investigates on its own, but it stops before any change. At that point it shows:
- what it proposes to do,
- the scope the change would touch,
- whether the change can be undone.
Production does not change until someone with the right role presses Approve. If they decline, nothing runs, and the proposal stays on the case for the next reviewer.
Here is what that looks like during an incident. Latency on a payment API spikes. The SRE Agent pulls the traces, correlates the spike with a deploy 12 minutes earlier, and finds the new build exhausting a connection pool. It proposes a rollback to the previous version, notes that the rollback affects one service in one region, and marks it as reversible. The on-call engineer reads the evidence, agrees, and approves. The agent did the digging; the engineer made the call.
Why: investigation is where agents save the most time. Execution is where mistakes cost the most. We put the gate between the two. This is human in the loop where it counts, not a prompt on every keystroke.
[Insert image here]
3. Every Step Is Written to the Run Record
Every tool call, its parameters and its result are saved to the run record, along with the decision and the person who made it. When someone asks a month later why a rollback happened, the evidence and the approver are already there.
A run record answers the questions that come up in every post-incident review:
- What did the agent look at, and in what order?
- What did it conclude, and on what evidence?
- What did it propose, and who approved or declined it?
- What actually changed, and when?
This is the AI agent audit trail that security and compliance teams ask for. It also helps engineers. A run record is a ready-made timeline for the incident write-up, so nobody has to reconstruct events from memory and chat scrollback.
Why: an agent you cannot audit is an agent you cannot defend to a security team or an auditor. Good AI agent governance starts with being able to show your work.
4. Approvals Work the Same Everywhere
Teams do not always sit in a console. An agent can hand over its finding in Slack or on mobile, and the approval there is recorded exactly as it would be in the console, with the same author and time.
The same role checks apply in every channel. If an engineer cannot approve a change in the console, they cannot approve it from their phone either. The proposal, the evidence and the reversibility note travel with the request, so the person approving on the go sees what the person at a desk would see.
Why: if approving from a phone were less controlled than approving from a desk, people would route around the controls. Guardrails only work if the easiest path is also the governed one.
5. Routines Keep the Same Gates
Once a run works, a team can save it as a routine that repeats on a schedule or a trigger. A routine keeps the scope and approval gates of the run it came from. Automating a task does not quietly widen what the agent is allowed to do.
Take a weekly FinOps review. The first time, a FinOps Agent finds idle resources, estimates the saving, and proposes clean-up steps that an engineer approves. Saved as a routine, it runs every Monday with the same tool access and the same approval step. If the team later decides certain low-risk clean-ups can run on their own, that is a deliberate, recorded change, not something the routine drifts into.
Why: automation is where scope creep usually hides. A routine that inherits its gates means the controls you reviewed once stay in force every time it runs.
What We Decided Not to Do
Guardrails are also about the features you leave out. Three choices shaped Toby AI Agents as much as the five above.
- No silent autonomy. By default, every action is held for approval. Teams can allow specific actions to run automatically, but that is an explicit choice per action, not a global switch.
- No separate copy of your data. Agents query the same engine as your dashboards, in the region your workspace runs in. There is no second data store to secure, and no question about where agent data lives.
- No claims of replacing the team. Agents take on repetitive, well-defined work. Decisions stay with people.
What We Are Still Learning
Approval gates add a step. For low-risk, reversible actions, some teams want fewer prompts; for high-risk ones, they want two approvers. We are working with customers on where those lines should sit, and we will share what we learn.
Building Agents Your Team Can Trust
AI agent guardrails are not a layer you add after launch. They decide whether a team uses an agent for real work or leaves it in a sandbox. With Toby AI Agents, we chose to make every agent scoped, supervised, recorded and consistent, so the speed of automation never comes at the cost of control.
If you are evaluating agents for your own operations, start with the same questions we did. What can it see? What is it about to do? Who decided? If a platform cannot answer all three on every run, it is not ready for production.
FAQ
What are AI agent guardrails?
AI agent guardrails are the controls that limit what an agent can access, what it can change and how its actions are reviewed. In Toby AI Agents they include role scopes, approval gates, run records, consistent approvals across channels and routines that keep their original gates.
Can Toby AI Agents make changes to production on their own?
Not by default. Agents investigate on their own but hold every action for approval. Teams can allow specific actions to run automatically, as an explicit, per-action choice.
How does Toby AI Agents support AI agent governance and audits?
Every tool call, parameter, result, decision and approver is written to the run record. Security and compliance teams can trace any change back to the evidence and the person who approved it.
What permissions does each agent have?
Each agent holds one role, such as SRE, Application, FinOps or Security, with defined tools, readable data and outputs. A Custom Agent gets only the skills and tools your team registers for it.
Can I approve agent actions from Slack or mobile?
Yes. Approvals in Slack or on mobile follow the same role checks and are recorded with the same author and time as approvals in the console.
Do AI agents copy my observability data somewhere else?
No. Agents query the same engine as your dashboards, in the region your workspace runs in.
See how scoped, supervised agents fit into your operations. Explore Toby AI Agents or book a demo with our team.
