I Spent $40k in AI Credits. Here Is What I Learned.

The hard part was not making agents smarter. It was making the whole system cheap enough, durable enough, and honest enough to keep working.

My AI usage dashboard currently says $39,594.23.

That is the raw token cost for May 26 through August 23 if everything had been billed at the full API rate. It is not the same thing as money that left my bank account. I use subscriptions, credits, caching, and different products. The dashboard itself labels the number correctly: if billed at full API rate.

Still, the number is useful. It shows the scale of the experiment.

Across that period, the dashboard recorded:

This is the original dashboard evidence. The amount is raw token cost at full API rates, not a claim about cash paid.

The statistical slide restates the dashboard's exact values in a more legible article format.

Codex made up $38,199.39 of the raw cost, or 96.5 percent. Claude Code made up $1,394.84, or 3.5 percent. The largest individual model was GPT-5.6 Sol at $34,507.63 and 52.6B tokens.

This is not a story about spending money to make a graph go up. It is a story about what I had to learn once agents became powerful enough to consume serious amounts of context, compute, and my attention.

1. The one-hour ceiling is gone

For a long time, my frustration with AI was not that it could not produce impressive answers. It was that I could not give it a serious mission and expect it to keep owning the problem for long.

The practical ceiling was about an hour.

I eventually checked this against my Codex history instead of trusting my memory. Between June 20 and July 9, I had 820 measured top-level turns. 799 of them, or 97.4 percent, ended within one hour. None exceeded twelve hours. The longest was 4.81 hours, and the median was only a few minutes.

Then the shape changed.

The operational shift happened around the release of Fable and the arrival of GPT-5.6 Sol in my workflow. My history can show the chronology, but it cannot isolate which component caused the improvement. GPT-5.6 Sol appeared in the measured history on July 10. Four days later, a completed turn ran for 22.44 hours. Later, one recorded 33.69 hours of active work across 36.08 hours on the clock. A Self-Degree mission stayed alive for seven days, fourteen hours, and twenty-six minutes across interruptions and continuations.

The mission spawned 527 descendant Codex threads over its lifetime. That does not mean 527 agents worked simultaneously, and it certainly does not mean all of them produced useful work. It is a count of thread records. The important change was that the mission could preserve ownership across turns, workers, failures, context compaction, and restarts.

I wrote the detailed chronology in I Used to Run AI Agents for an Hour. Now They Run for Days.

The most important correction is this: most of my agents did not suddenly become day-long agents. The long tail appeared. Some missions can now keep working for days when the problem deserves it.

The runtime slide shows the historical shift without claiming that GPT-5.6 Sol alone caused it.

The prompt that started my longest recorded turn was one word:

Continue

That word was not magic. It worked because plans, tests, worktrees, deployment state, evidence, and unresolved failures existed outside the context window. The agent could reconstruct the mission instead of starting from zero.

2. Credits are finite, so token optimization becomes part of the job

The dashboard made another thing impossible to ignore. Credits are not infinite.

For a while I used Sol for almost everything. My primary subscription was ChatGPT x20, and Sol was good enough that using a cheaper model felt like an unnecessary compromise. OpenAI was also resetting access generously, so I could spend heavily and assume another refresh would arrive soon.

Then the reset behavior changed. I could burn through a weekly allowance in two days.

This forced me to separate work by value. I started using GPT-5.6 Luna for casual Hermes tasks and lighter sub-agents, while reserving Sol for the work where deep reasoning, long context, supervision, and difficult judgment actually mattered. With that split, I can currently keep roughly two full-time agents running continuously for a week without destroying the rest of my weekly capacity.

That is my operating experience, not a controlled pricing experiment. But the lesson is clear enough: token optimization is not just about finding the cheapest model. It is about deciding which work deserves expensive attention.

The dashboard also shows why caching matters. 59.5B of the 61.2B processed tokens were cached input. The estimated cache savings are $257,337.63, or 6.5 times the raw token cost. Persistent artifacts, stable context, good task boundaries, and reusable instructions are not merely conveniences. They change the economics of running agents continuously.

A long-running system that repeatedly reconstructs the same context from scratch is wasteful even if the model is excellent.

The model stack I use now is less elegant than the model stack I wanted. It is more sustainable. That matters more.

3. The cheapest model is not cheaper if it makes the work worse

I also spent several days trying Kimi K3 and MiniMax M3 while running a pentesting task.

The experience made me skeptical of the current enthusiasm around Chinese models. In this particular workflow, moving down from Sol felt like a major drop in reasoning quality. It was easy to push the models across safety boundaries, but that was not a sign of useful intelligence. It was a sign that safety behavior was not doing much work as a differentiator.

I am not claiming that every Chinese model is bad, or that a model's nationality tells me anything useful about its quality. I am saying that headline benchmarks and low prices did not transfer automatically to the work I needed to do.

The real test is not whether a model can produce an impressive answer in a benchmark prompt. The real test is whether it can stay useful through a messy repository, ambiguous evidence, a failed test, a stale environment, a long task handoff, and a requirement not to invent success.

I still want cheaper models in the stack. But I want them selected through task-specific evaluations and cost per accepted result, not hype.

4. Without verification, you are just autocomplete with a longer timeout

The most important lesson is the simplest one.

If you run agents without a verification loop, you are not operating autonomous systems. You are autocompleting with a longer timeout.

I built BetaBots because implementation agents are bad at grading their own work. BetaBots use products through visible interfaces, follow different user goals, preserve evidence, and fail when a real journey does not produce what the user needed. They do not replace deterministic tests, security review, accessibility checks, or human research. They add an independent user-like layer that the builder does not control.

The contrast that convinced me was uncomfortable. An earlier run launched fifty broad synthetic users and reported 50 out of 50 happy. A later, stricter cohort launched only three users. All three failed. The median score was 49 out of 100.

The second result was much better.

The failed cohort gave the coding agent something concrete to investigate. The agent could preserve the journey, classify the failure, decide whether the product or evaluator was wrong, repair the relevant layer, deploy the exact revision, and run the journey again.

I wrote about that operating model in How to Use BetaBots Without Building an Infinite Repair Loop.

The point is not that BetaBots magically created autonomy. I later audited the skills behind the long-running mission and found that BetaBots was only one judge inside a larger protocol stack. Planning, debugging, testing, code review, deployment, and revision verification all had to know what to do with its evidence.

A failed verification should usually hold a release, not stop the mission. The supervisor decides whether the next task is a product repair, an evaluator repair, an infrastructure investigation, a new test, or an escalation to me.

The reusable loop is:

Supervisor delegates -> workers build -> independent verifier judges -> supervisor interprets the failure -> workers repair -> verifier judges again.

That is the difference between an agent that reports a problem and an agentic system that can own the next step.

5. Long-running agents need a durable home

My first attempt at reliable agents was too focused on secure, reproducible containers. I tried to build agents that lived inside Docker containers on a server. It backfired. I spent more time solving environment problems than getting useful work from the agents.

Then I moved the operation to my Mac.

The headache dropped by roughly 99 percent. Everything just worked. But the machine became the next bottleneck. My M1 Pro could not handle the load comfortably. I had to keep restarting it, keep it charged, and avoid taking it with me because agents needed to keep running.

That is not freedom. It is a desktop computer becoming an employee who cannot leave the office.

I moved back to a server, this time installing the agents directly into a stable server environment instead of trying to make Docker the center of the workflow. Some environment problems returned, but they were manageable compared with the benefit of not loading my laptop and being able to travel with it.

The exact networking label is not the point. I mean a private, durable server environment. The underlying lesson does not:

If the machine running the agent has to remain physically next to you, the agent is not really running independently.

The server does not make the agent intelligent. It makes the agent durable. It gives the work somewhere to continue when I close the laptop, leave the apartment, restart the local interface, or go to sleep.

A private server is not glamorous infrastructure. It is the part that turns a demo into an operating environment.

6. Choosing the coding tool is as important as choosing the model

I used to think the model was the main choice and the coding tool was just a shell around it. I no longer believe that.

I started with Cursor, and Composer worked well for me. Then I moved to Claude Code in the terminal. When Codex CLI appeared, I realized that the workflow was becoming difficult to sustain through a single terminal-oriented interface.

The same day I had that realization, Zed 1.0 was released. I was excited. I used Zed for roughly three months, and the experience was good in almost every way except remote development. Zed did not fit the way I wanted to run long-lived agents on remote machines. I had to keep my Mac active to let the remote work continue.

That was a deal breaker.

I recently moved to T3 Code. So far, I am impressed. It lets me work with multiple remote development environments and keep agents running there even when my Mac is closed. The mobile app matters as much as the desktop interface. I no longer need to ask Hermes in Discord to perform computer use just to supervise a Zed window.

My current setup is two Hetzner servers, both with Codex and Hermes configured. Both Hermes agents are connected to my private Discord server. I use them for operational and creative work that requires heavy context, research, and computer use. T3 Code is configured on my laptop and phone with access to both servers for development work.

I also experimented with Buzz because the idea is attractive: a native environment for humans and agents to work together. But it is still raw and buggy. Existing tools like Discord currently do a better job for the communication layer.

The tool determines whether the model can reach the work, whether the work survives your absence, and whether you can supervise it from wherever you are. That makes tool choice part of the architecture.

7. Skills matter, but only when they are the right skills

I recently ran an experiment without the roughly sixty skills I had accumulated over the years.

The difference was immediate.

The model still knew how to write code. It still knew how to debug. It still had taste, could ask questions, and could produce plausible plans. But the quality of the operating behavior dropped. It became easier for the agent to skip a step, choose a weak verification path, or treat a completion report as proof.

This matches what I saw in the longer telemetry audit. The week-long Self-Degree mission loaded 33 distinct skills 1,664 times. The most frequently loaded skills were test-driven development, systematic debugging, Using Superpowers, code review, verification before completion, and BetaBots. BetaBots mattered, but it was not the most-loaded skill. Test-driven development and systematic debugging were loaded far more often.

The conclusion is not that everyone should install 60 or 100 skills.

The conclusion is that skills are operating guidance. Tools are the interfaces through which an agent inspects, changes, coordinates, and verifies a system. A skill only helps when it is relevant, selected at the right time, and connected to a handoff another role understands.

A large skill folder can become another context problem. My current inventory analysis found that more than half of the installed skills had no detected load in the measured period. Some were simply irrelevant to the work. Others overlapped. Several deployment skills were doing similar jobs under different names.

The next problem is not collecting more procedures. It is routing the right procedure to the right failure.

I wrote the deeper analysis in A Long-Running AI Agent Is a Protocol Stack, Not a Superprompt.

8. Parallel work is no longer a bad practice

For most of my life, splitting attention across too many tasks ended badly. Context switching destroyed momentum. I learned to focus on one thing and finish it.

Long-running agents changed the economics.

If an agent is going to work for six hours, waiting beside it is wasteful. Starting a second independent mission is rational. Starting a third may also be rational. Eventually, however, I create more output than I can review.

My current capacity is roughly five parallel full-time agents. They often end up waiting for me. The bottleneck is not always verification. It is specifications.

One agent needs a product decision. Another needs a missing edge case. A third has reached an ambiguity in the PRD. A fourth has produced a candidate that needs a real-world judgment rather than another coding prompt. The machines are ready to continue, but the human has not defined the next boundary clearly enough.

This is becoming one of my personal benchmarks for being AI-native: not how many agents I can launch, but how many missions I can keep moving to successful completion without creating an expensive queue of work I do not understand.

That number will differ for every operator. Earlier I estimated that three or four concurrent missions were a reasonable working range. After more practice, I am around five. These are snapshots of my own capacity, not a universal recommendation.

I wrote more about the human side of this in What Do You Do While Your AI Agents Are Working?.

9. Specifications and verification are what humans need to do

There are two jobs I do not want to delegate completely: writing the specification and verifying the result.

You cannot tell an agent "make more money" and expect it to run productively for days. That is not a mission. It is a desire.

A useful PRD gives the agent an outcome, scope, constraints, edge cases, authority boundaries, acceptance criteria, verification instructions, and stop conditions. It explains what the agent may change, what it must preserve, what evidence counts, and what it must bring back when the work is blocked.

The better the specification, the longer the agent can work without returning to ask what the product is supposed to mean. The better the verification loop, the longer it can work without quietly converting plausible implementation into fake completion.

The human role is moving upward. I do not need to write every line. I do need to decide what good means, what risk is acceptable, what is irreversible, and what evidence can change my mind.

That is why long-running agents are not primarily a coding breakthrough. They are a specification and quality-control breakthrough.

If your agents cannot run for days yet, do not begin by buying a stronger model. First improve the PRD. Then improve the verification loop. Then make sure failures have owners and can create the next task without immediately becoming your task.

10. The mental gap is real

There is one cost that does not appear in the dashboard.

When agents work faster and longer, I start to lose my understanding of what they are doing. Reading every log is not sustainable. Watching every tool call creates the illusion of control while consuming the time I was trying to recover.

But ignoring the logs is not a solution either. If I only accept finished features, more decisions harden into the system while I understand less about the architecture. Eventually I am approving a product I can no longer explain.

I am starting to think that long-running agents need to teach their operator while they work.

Not by dumping a giant summary on me. By explaining decisions in a way that tracks what I already understand, what I have forgotten, and what I need to learn next. The agent should be able to say why it chose a library, why it moved responsibility into the backend, which alternatives it rejected, and what trade-off it accepted.

This is still an experiment. I do not have a finished teaching system. But the problem is becoming obvious enough that I no longer think ordinary logs are sufficient.

The human needs to remain a participant, not a spectator and not a rubber stamp.

What the $40k actually bought me

The raw token number is attention-grabbing, but it is not the deepest lesson.

I learned that autonomous agents can now run for days, but only when the surrounding system can survive days of work. That means model routing, cache-aware context, durable servers, remote-first tools, relevant skills, bounded parallelism, detailed specifications, independent verification, and a human who still understands what is being built.

The expensive model helped me cross the one-hour ceiling. It did not solve the operating problem.

The server kept the work alive. It did not decide what good meant.

The skills improved behavior. They did not create handoffs by themselves.

BetaBots found failures. It did not repair them.

The supervisor connected the pieces. I still owned the goal, the permissions, the risk, the irreversible decisions, and the final judgment.

That is the real transition. I am not trying to build agents that run forever. I am trying to build systems that keep ownership until the evidence says the mission is complete, unsafe, blocked, too expensive, or no longer worth continuing.

If you cannot run agents for days yet, work on your specifications and verification loops.

If you can run them for days but cannot keep up, work on model routing, durable infrastructure, and your capacity to operate several missions without losing understanding.

The future of autonomous agents is here.

The invoice is only the first lesson.

Public references

• I Used to Run AI Agents for an Hour. Now They Run for Days
• A Long-Running AI Agent Is a Protocol Stack, Not a Superprompt
• How to Use BetaBots Without Building an Infinite Repair Loop
• BetaBots on GitHub
• What Do You Do While Your AI Agents Are Working?