A Long-Running AI Agent Is a Protocol Stack, Not a Superprompt
I thought verification was the unlock. Then I audited the 33 skills behind my longest AI coding mission and realized that verification was only one part of a small synthetic engineering organization.
A few hours after I published I Used to Run AI Agents for an Hour. Now They Run for Days, I started to dislike my own explanation.
The article was true. My longest recorded Codex turn had reached 33.7 hours. A Self-Degree mission had stayed alive for more than seven days across interruptions. Stricter BetaBots had found real failures and given the main agent reasons to continue.
But I had made the story sound too simple:
Better model + independent verification = long-running agent.
That equation leaves out most of the work.
A BetaBot can discover that a student cannot finish a lesson. It cannot decide whether the defect lives in the mobile app, backend, deployment, test harness, or evaluator. It cannot repair the right repository, review the change, prove that the fix works, deploy the exact revision, and send a new user through the same journey.
Something has to connect all of those actions.
I went back to my local Codex history and looked at the skill files the agents loaded during the mission. I expected to find BetaBots at the center. Instead, I found a protocol stack.
The week-long Self-Degree mission loaded 33 distinct skills 1,664 times across its root and descendant threads. The six most frequently loaded were:
- Test-driven development: 297 loads
- Systematic debugging: 240 loads
- Using Superpowers: 214 loads
- Code review excellence: 209 loads
- Verification before completion: 182 loads
- BetaBots: 131 loads
BetaBots was important. It was not the brain, manager, builder, or release engineer. It was one independent judge inside a system that knew what to do with a failed judgment.
This is the next chapter of vibe coding
In January, I wrote Vibe Coding Is Team Leading with AI Agents. My argument was that AI coding is mostly a management skill. The model behaves like a fast and enthusiastic junior developer. The senior engineer moves from typing code toward writing specifications, providing context, reviewing assumptions, and rejecting superficial work.
That article described my first transition:
from programmer to team lead.
It was still a human-centered workflow. I wrote the brief. The agent implemented it. I reviewed the output. When something failed, the task came back to me. I decided what should happen next.
In April, I published Designing the AI Startup Factory. I described a different organization: expert agents that listen, decide, delegate, validate, persist, and reflect. Workers are disposable. Artifacts survive them. Failed tasks retry. Progress lives outside any single context window.
At the time, much of that was architecture and aspiration. I was describing the organization I wanted before I could reliably run it.
The week-long Self-Degree mission was the first time I could see the shape of that organization in ordinary Codex telemetry. There was no Temporal workflow coordinating it. No beautiful dashboard showed a CEO agent handing work to a CTO agent. It emerged from a supervisor, spawned workers, skill files, plans, worktrees, tests, reviews, deployments, browsers, and evidence.
My previous article measured how long that organization stayed alive. This article is about what it actually did while it was alive.
The skill that made the other skills matter
The third most loaded skill in the mission was Using Superpowers.
It does not implement features. It imposes a simple operating rule: before acting, check whether a relevant skill exists and load it. Process skills come before implementation skills.
That sounds like administrative overhead. It is closer to a constitution.
Without it, a capable model sees a bug and improvises. Sometimes it chooses a good process. Sometimes it edits the first plausible file. Sometimes it starts implementing before it understands the request. The result depends on what occurred to the model in that moment.
Using Superpowers turns procedures into defaults. A feature request routes toward brainstorming, planning, and TDD. A test failure routes toward systematic debugging. A completion claim routes toward fresh verification. A multi-part plan can route toward isolated workers and review.
The skill was loaded 214 times because the mission did not rely on the supervisor remembering every rule after every context compaction. The operating system was reloaded as needed.
This changes the role of a prompt. The prompt describes the goal. The skill stack describes how the organization behaves when reality disagrees with the prompt.
The builders were forced to produce evidence
The two most loaded skills were test-driven development and systematic debugging.
Together they accounted for 537 loads.
TDD gave every repair a contract:
- Write a test that expresses the missing behavior.
- Watch it fail for the expected reason.
- Make the smallest implementation change.
- Watch the test pass.
- Refactor without losing the proof.
Systematic debugging prevented the agent from treating every red signal as permission to edit code. It required reproduction and root-cause investigation first. In a system with a mobile client, web frontend, backend, PostgreSQL, browser sessions, deployment revisions, AI providers, and an evaluator, the first visible symptom is often far away from the defect.
This mattered because a week-long mission accumulates strange states.
A browser can be stale. A worker can edit the wrong checkout. A test can pass against a mock and fail against the real backend. A deployment can report success while production serves an older commit. A BetaBot can fail because the product is broken, or because its session expired, or because the evaluator misunderstood the evidence.
More time does not resolve these ambiguities. More time lets them compound.
Systematic debugging converted confusion into a reproducible failure. TDD converted that failure into an executable contract. This pair formed the repair engine of the mission.
That is also a direct answer to a mistake I documented in my 2025 personal review. I had let AI put roughly fifteen supposedly "90% done" features onto the main branch while barely inspecting them. None worked properly. The code was a mess.
That was not autonomy. It was unattended generation.
The difference was not that the 2026 models typed better code. The difference was that the newer system repeatedly forced failure to become evidence before it became another patch.
The judge was not allowed to be the builder
The next cluster was about judgment:
- Code review excellence: 209 loads
- Verification before completion: 182 loads
- BetaBots: 131 loads
- Playwright: 35 loads
- Agent Browser: 26 loads
- Requesting code review: 16 loads
These skills did not all answer the same question.
Code review asked whether a change was correct, maintainable, secure, and consistent with the architecture.
Verification before completion asked a narrower and surprisingly powerful question: what fresh command proves the statement you are about to make? A test result could prove that the tests passed. It could not prove that the production deployment served the same revision. A linter could not prove that the application built. An agent report could not prove that the agent's work was correct.
Playwright and Agent Browser operated the visible product. They could catch failures that unit tests never see: navigation, authentication, responsive layout, browser state, and broken interaction flows.
BetaBots added personality and intent. They did not follow a QA checklist. They tried to use the product as people with different goals, patience, trust, and prior experience. Their job was not to admire the interface. Their job was to produce a journey that either achieved something or failed with preserved evidence.
The layers disagreed usefully.
A unit test could pass while the browser journey failed. A browser journey could succeed while code review found a security problem. A BetaBot could report failure while the underlying product was correct and the evaluator was broken. A deployment could be healthy while serving the wrong revision.
I used to think verification meant adding one stronger test at the end. In practice, verification was a set of judges with different blind spots.
What happened after a BetaBot failed
The full handoff looked roughly like this:
- A BetaBot attempted a real Self-Degree journey and preserved screenshots, actions, and first-person observations.
- The evaluator marked the journey failed or unsupported.
- The supervisor classified the result: product defect, evaluator defect, browser/session failure, stale deployment, or unclear requirement.
- A specialist reproduced one narrow failure using systematic debugging.
- TDD turned the reproduction into a failing test.
- A worker implemented the smallest relevant repair.
- A separate review skill examined the diff.
- Deterministic suites ran again.
- The exact revision was deployed.
- Browser checks and a fresh BetaBot cohort judged the result again.
- Verification before completion blocked any success claim that lacked current evidence.
No single step is surprising. The important part is that the output of one role became the input of the next.
The BetaBot produced a journey. Debugging produced a root-cause hypothesis and reproduction. TDD produced a failing then passing contract. The worker produced a revision. Review produced findings against that revision. Deployment produced an addressable release. Verification produced a pass, fail, or blocked state.
The supervisor moved those artifacts through the loop.
This is what I was trying to describe in the AI Startup Factory when I wrote that experts should listen, decide, delegate, validate, persist, and reflect. The week-long mission did not implement that architecture literally, but the operating pattern was the same.
A supervisor and verifier need different authority
I initially framed this as a choice: who has final authority, the supervisor or the independent verifier?
That was wrong. They govern different things.
The verifier should be able to block a release. If the agreed evidence is missing or failed, the implementation does not become verified because the supervisor is tired or because the agent worked for thirty hours.
The supervisor owns the mission. When verification fails, it decides what the failure becomes next. It can assign a product repair, investigate the evaluator, resolve an infrastructure problem, rerun a deterministic suite, or escalate a genuine product decision to me.
A failed verification normally stops the release. It should not stop the work.
The loop is:
Supervisor delegates → workers build → independent verifier judges → supervisor interprets the failure → workers repair → verifier judges again.
I remain outside that loop at the constitutional layer. I own the goal, permissions, irreversible actions, major product boundaries, and any explicit exception to a gate.
This is a more precise version of the management idea from my vibe-coding article. Leading one AI implementer requires good briefs and review. Leading a synthetic organization requires separation of powers.
Planning and delegation kept the mission coherent
The coordination skills were loaded less often than TDD or debugging, but they shaped the work:
- Brainstorming: 57 loads
- Subagent-driven development: 38 loads
- Executing plans: 8 loads
- Writing plans: 7 loads
- Using Git worktrees: 2 loads
The low counts do not mean planning was unimportant. A plan can be loaded once and create dozens of implementation and verification tasks.
Brainstorming clarified what the product should become before implementation began. Writing Plans converted that direction into file-level, testable tasks. Subagent-driven development gave each bounded task to a fresh worker and required review before integration. Worktrees reduced the chance that parallel workers would overwrite one another.
The supervisor kept the durable goal. Workers received enough context to complete one assignment without inheriting the entire week of conversation.
That division is close to the expert-driven model I proposed in the AI Startup Factory: a few stable roles, many disposable workers, and artifacts that outlive both.
The mission still produced coordination failures. Workers saw stale state. Some duplicated investigations. Some needed follow-up. More agents created more ways to disagree. The answer was not an even larger swarm. It was tighter task boundaries and clearer handoff artifacts.
A fix was not real until it reached production
The release skills were another distinct layer:
- Coolify Deploy: 56 loads
- Coolify: 36 loads
- Coolify Manager: 32 loads
- Autodeploy: 27 loads
A change in a worktree is not a product improvement. A green test suite against commit A says nothing about production if production still serves commit B.
The release layer connected implementation evidence to the system users could reach. It handled builds, environment state, deployment health, logs, and exact revisions. Then the browser and BetaBots evaluated that deployed state.
This is where boring infrastructure became part of agent intelligence.
In the previous article, I described moving heavy work from my laptop to Hetzner. The skill audit shows the procedural side of that move. The system needed repeatable deployment and inspection, not merely SSH access. A durable server lets a mission survive; a release protocol tells the mission what is actually running there.
Skills told the agent how to work. Tools let it work
The same mission produced 65,866 top-level tool dispatches across the root and descendant threads.
Most were command and process operations: inspecting repositories, running tests and builds, operating Git, starting services, querying infrastructure, and waiting for long jobs. Another large group coordinated agents by spawning workers, sending assignments, requesting follow-ups, and waiting for results. The rest included explicit code patches, durable plan updates, screenshot inspection, and direct browser control.
The distinction matters:
Skills were operating procedures. Tools were interfaces to files, processes, browsers, agents, and infrastructure.
A systematic-debugging skill without shell access is advice. A BetaBots skill without a browser is a persona document. A deployment skill without access to the release system is a checklist.
The agents could continue because the procedures were attached to interfaces that changed real systems and returned evidence. The state that survived lived in Git history, plans, commits, tests, logs, deployment revisions, screenshots, BetaBot journeys, thread state, and context compaction.
Do not install 33 skills and expect autonomy
The obvious bad lesson from this audit is: install the same 33 skills and your agent will work for a week.
The data contradicts that.
A 4.6-hour mission in the same period loaded 38 distinct skills, more than the Self-Degree mission. The first completed 22.4-hour mission loaded 35. Skill count did not determine duration.
A folder full of procedures is not an organization. Skills can conflict, be ignored, or produce reports that nobody consumes. Ten reviewers can create ten documents that wait for a human to choose the next prompt.
The useful unit is a closed handoff:
- Which role creates the artifact?
- What does the artifact prove?
- Who consumes it next?
- Who can reject it?
- What happens after rejection?
The protocol becomes real when every output has an owner.
The next step is a skill runtime
There is an uncomfortable consequence if this thesis is right.
If skills improve agent behavior, I will keep adding skills. If I keep adding skills, the catalog itself starts consuming the context the agent needs to do the work. I already get a warning that my installed skills consume more than 2% of the available context. That is not a crisis, but it is a tax paid by every session before the work begins.
Loading every skill is the prompt equivalent of asking a new employee to reread the entire company handbook, every runbook, every engineering standard, and every incident report before answering a Slack message.
The obvious next step is a library. I think it needs to be more than search over a folder of Markdown files.
It needs a topology.
A flat catalog tells the agent that
systematic-debuggingtest-driven-developmentfailed journey → reproduce with systematic debugging → express the defect with TDD → request review → deploy the exact revision → verify the journey again
The edges matter as much as the nodes. One skill produces evidence that another skill consumes. Some skills are prerequisites. Some are alternatives. Some conflict. Some should only run after a particular artifact exists. Some require production access and should never be selected casually.
Each skill would need a compact manifest, not just a name and description. I would want it to declare:
- when it should and should not be selected;
- what inputs or artifacts it expects;
- what it produces;
- which skills commonly precede and follow it;
- permissions and tools it may use;
- cheaper alternatives and known conflicts;
- its version, context cost, and evaluation history.
The full procedure should stay out of context until the agent reaches that part of the work.
I imagine four levels of disclosure. The base prompt contains only a small map of domains: planning, implementation, debugging, review, product evaluation, deployment, research, and so on. After classifying the task, the runtime retrieves compact manifests from the relevant neighborhood. It chooses the smallest useful set under a context budget. Only then does it load the complete skill and any references or scripts. When that phase ends, the runtime preserves the resulting artifact and compacts the procedure out again.
The agent would carry the topology, not the entire library.
This also gives project context a proper role. A frontend repository might preload its design, browser, accessibility, and release neighborhood. A Python library might preload packaging, TDD, API compatibility, and benchmark paths. A production incident would route directly into diagnostics and rollback, not brainstorming.
The router should be measurable. It can combine explicit rules, semantic retrieval, repository context, and prior usage, but it should log why it selected a skill. We should be able to evaluate missed skills, unnecessary loads, context spent, reroutes, conflicts, and whether the chosen path reached a verified outcome.
Otherwise we will replace one large prompt with an opaque agent that guesses which smaller prompts to use.
Skills should improve, but they should not freely rewrite themselves
A topology solves selection. It does not solve quality.
My current skills are living documents, but their evolution is mostly manual. I notice a repeated failure, edit a procedure, and see whether the next agent behaves better. A mature system should learn from every run without turning every local accident into a global rule.
The tempting design is to let an agent modify a skill whenever the skill fails. I think that is too dangerous.
A failed task does not prove that the skill was wrong. The router may have selected the wrong skill. The implementation may have ignored it. The environment may have changed. The evaluator may be broken. A malicious page can even inject instructions into the evidence an agent sees. If the same agent can interpret the failure, rewrite its own operating rules, and declare the new version better, it can optimize away the controls meant to constrain it.
Skills should improve like production code:
- Capture where a skill was loaded, what state preceded it, which artifacts it produced, what happened next, and how much context and tool use it cost.
- Cluster repeated failures and human corrections instead of reacting to one run.
- Generate a candidate patch with a stated hypothesis: what behavior should change, and which observed failures support it?
- Replay old missions and run held-out evaluations against both the current and candidate versions.
- Let an independent reviewer inspect regressions, permission changes, conflicts, and prompt-injection risk.
- Promote the candidate gradually, with versioning, canaries, and an immediate rollback path.
The skill author, evaluator, and promoter should be separate roles. A skill cannot rewrite itself and call that evidence of improvement.
Routing can learn through the same process. Perhaps TDD was loaded after the implementation already existed. Perhaps BetaBots was repeatedly selected for a backend migration where deterministic tests were enough. Perhaps two review skills produced the same artifact. Those observations can improve the graph without changing the skill text.
I would track at least four states: experimental, candidate, stable, and deprecated. Stable skills would have passing evaluations and known permissions. Candidate skills could run on a small fraction of suitable missions. Experimental skills would require explicit selection. Deprecated skills would remain resolvable for old projects while the topology redirected new work.
This begins to look like a package manager, search engine, workflow graph, and CI system built around operating procedures. That sounds heavier than a directory of Markdown files because it is. Once skills influence production decisions, deployments, and autonomous repairs, they are part of the software supply chain.
The future I want is not a model with one perfect prompt. It is a runtime that assembles a temporary operating system for the current problem, observes whether that system worked, and proposes improvements without being allowed to approve its own changes.
Today I manually curate the protocol stack. The next generation should retrieve it just in time, trace every handoff, and improve it under verification.
The audit should also find missing skills
A router can only choose from procedures that exist. The same topology should reveal where the stack has no owner at all.
I extended the audit from the 33 skills in the Self-Degree mission to my current user and cached plugin inventory. After deduplicating by skill name, I found 124 current skills across 133 physical
SKILL.mdTheir descriptions alone contain 34,919 characters. More than half of that text belongs to skills with no detected load in the window. Eight names have multiple installed copies, and four of those names point to divergent content versions.
This does not prove that the unused skills are bad. I may simply not have needed them. It does prove that adding another skill has a carrying cost, and that my catalog already contains both redundancy and ambiguity.
Deployment is the clearest example. The Self-Degree mission loaded
coolify-deploycoolifycoolify-managerautodeployThe inventory also changed how I think about missing skills. My stack does not need another generic planner, debugger, code reviewer, browser driver, deployment guide, LLM evaluator, or skill-authoring tutorial. Those already exist.
The likely gaps are in the control plane:
- A skill topology router that retrieves the smallest relevant set under a context budget and records why it selected each procedure.
- Verifier-failure triage that classifies failed evidence as a product defect, evaluator defect, browser problem, stale release, infrastructure failure, or unclear requirement, then assigns the next owner without weakening the gate.
- A skill regression lab that turns repeated failures into candidate patches, replays historical and held-out missions, and separates the skill author from the evaluator and promoter.
- A mission-wide ledger that extends the progress file already used by subagent-driven development across deployment, external evaluation, interruption, and resume.
- Release artifact attestation that proves the reviewed code, tested code, build artifact, deployed revision, browser target, and BetaBot result all refer to the same candidate.
- An autonomous security and privacy gate for secrets, third-party data, consent, side effects, external communication, retention, rollback, and permission escalation.
Some of these should be new skills. Others should be extensions or orchestrators around what already works. Release attestation belongs inside a consolidated deployment path. The mission ledger should generalize the durable ledger already present in subagent-driven development. The security layer may begin by adapting an existing, reviewed threat-model skill rather than inventing another one.
The topology gives me a mechanical way to detect future gaps.
If one skill produces an artifact that nobody consumes, the next owner is missing. If an agent repeatedly asks me to classify the same failure, a decision procedure is missing. If two skills claim the same trigger and output, I probably have duplication rather than a gap. If a skill is loaded but leaves no artifact, decision, state transition, or measurable effect, it may be context overhead. If an irreversible action has no independent gate, the safety layer is missing.
This turns skill discovery into something more useful than browsing a marketplace. I can inspect the graph of real missions and ask where work repeatedly falls back to me.
Those human-owned transitions are the best candidates for the next skill.
The smallest stack I would start with
I would not reproduce the entire Self-Degree setup on day one. I would begin with six capabilities:
- Using Superpowers to route work into explicit procedures instead of improvisation.
- Writing Plans to turn a goal into bounded tasks with files, tests, and acceptance criteria.
- Test-driven development to make implementation answer to executable behavior.
- Systematic debugging to classify failures before editing code.
- Verification before completion plus independent code review to prevent unsupported success claims.
- One external product judge: Playwright for deterministic browser journeys, or BetaBots when user intent, comprehension, trust, and multi-session behavior matter.
Then I would define one failure route end to end:
browser journey fails → supervisor classifies it → worker reproduces it → failing test is written → repair is reviewed → exact revision is deployed → journey runs again.
If that loop works, add parallel workers, production canaries, infrastructure skills, accessibility review, security review, and more ambitious synthetic users.
Do not add them because the catalog looks impressive. Add them when the current loop produces an artifact that needs a new owner.
The protocol is the product
My three articles now describe three stages of the same transition.
Vibe Coding Is Team Leading with AI Agents was about learning to manage an implementer instead of typing everything myself.
Designing the AI Startup Factory was about designing a durable organization of expert supervisors and disposable workers.
I Used to Run AI Agents for an Hour. Now They Run for Days showed that some of those missions can now keep ownership across a day or a week.
This audit fills in the missing layer. The organization runs on protocols: plan before building, reproduce before fixing, test before claiming, review before merging, verify the deployed revision, preserve failed evidence, and give that evidence to someone who can act on it.
In my own taxonomy, 1,161 of the mission's 1,664 detected skill loads belonged to build/recovery or verification/judgment. Almost 70% of the procedural activity was about making, breaking, fixing, and proving the product.
That is not a superprompt. It is engineering culture written into files that agents can reload.
I am still not trying to build an agent that runs forever. A bad loop can waste compute forever.
I want a system in which the supervisor owns the mission, the verifier can reject the release, workers can be replaced, evidence survives them all, and a failure already has somewhere to go.
Sometimes that system will finish in twenty minutes. Sometimes it will work through the night.
Runtime is secondary.
The test is whether, after something fails, the system still knows who acts next.
A note on the audit
I analyzed a frozen window of local Codex telemetry from June 20 through July 26, 2026. I counted a skill load only when a captured tool command directly referenced a
SKILL.mdA detected load does not prove that the file was read successfully, followed correctly, or responsible for the mission's duration. The figures describe the procedures present in one operating system, not a controlled causal experiment.