Self-Healing, Not Self-Governing: Where an Infrastructure Agent's Authority Should End

This one's speculative — a thought experiment about design principles, not a report on something I've built.

An agent with real write access to its own infrastructure — one that can read logs, form a diagnosis, and push a fix — is one of the most genuinely useful applications of an AI agent I can picture. It's also one of the easiest to get wrong in a way that looks completely fine right up until the day it isn't.

Diagnosis and remediation are different trust levels

Reading logs and forming a hypothesis about what's wrong is low-risk even when the hypothesis is wrong — worst case, a person reads a bad diagnosis and ignores it, and nothing changed. Executing a remediation is a different trust level entirely, because a wrong automated action doesn't just fail to fix the problem, it compounds the original incident with a second one. Those two capabilities look like one feature from the outside. They should never be governed by the same trust boundary.

Why "usually right" isn't the bar

The genuinely dangerous version of this system isn't the one that's frequently wrong — that failure mode is obvious and gets caught fast. It's the one that's right often enough that a human stops actually reading what it proposes before rubber-stamping it, which is the exact moment the approval step stops functioning as a safety check and becomes theater. The design has to account for that failure mode explicitly: what gets full manual review versus what gets auto-approved should be a function of the blast radius of the action, not of how confident the agent sounds while proposing it.

A whitelist of remediations, not a blank check

The practical version of this: define a fixed, human-reviewed set of remediation actions the agent is allowed to execute autonomously — restart this specific known-safe service, roll back to this specific known-good deployment, scale this specific resource within these specific bounds — and require a human for anything outside that list, no matter how well the agent believes it understands the problem. The agent's judgment gets exercised in two places: forming the diagnosis, which is cheap to be wrong about, and choosing among pre-approved actions, which is safe even when the choice turns out wrong. It never gets exercised in inventing a brand-new remediation on the fly, because that's exactly the case where "usually right" stops being good enough.

An audit trail isn't an afterthought

Every autonomous action needs a complete, human-readable record of what was observed, what was concluded, and what was actually done. Not for compliance box-checking — because the day something does go sideways, that record is the only way to tell whether the agent's judgment was the actual problem or whether the whitelist itself needs to shrink. Without it, you're debugging a black box with an opinion.

The real ceiling isn't a capability question

How much autonomy an infrastructure agent should have isn't primarily a question of how capable the model is. It's "how bad is the worst thing this specific action could do," asked action by action, not agent by agent. A system that's extremely good at diagnosis can still be scoped to a very narrow whitelist of executable remediations, and that's not underselling the agent's intelligence — it's correctly separating what it's good at from what it should be trusted to do unsupervised.

I'm Jesse Myers — Marine veteran, 32 years in enterprise IT, now building production AI systems. This site is where I write about what I've actually built, and occasionally about ideas I haven't built yet but think are worth taking seriously.