How to Use BetaBots Without Building an Infinite Repair Loop

At first I made BetaBots too easy to please. Then I overcorrected and made them almost impossible to please. In both cases, nobody had decided what a failed journey was allowed to mean.

In July 2026, I ran fifty BetaBots against one of my products. They generated thousands of screenshots, thoughts, and actions. The report said every bot was happy.

Then I ran three much stricter bots. All three failed. Their median score was 49 out of 100.

The smaller failed cohort was more useful. I wrote about it in I Used to Run AI Agents for an Hour. Now They Run for Days. The failures gave my main coding agent something concrete to investigate. It could reproduce a journey, decide whether the problem was in the product or the evaluator, make a repair, deploy it, and run the journey again.

Later I audited the skills behind that week-long mission. BetaBots was only one judge inside a larger protocol stack. Planning, debugging, testing, review, deployment, and exact-revision verification all had to consume its evidence.

That solved one problem and exposed another.

If every failed BetaBot creates another implementation task, the system has no natural reason to stop. A customer can be unhappy because the product is broken. A critic may be hunting for a flaw. An investor may want a different business. A competitor may be right that the product is easy to copy, or simply prefer that it did not exist.

Trying to satisfy all of them creates an agent that works forever and delivers nothing.

I used to think the hard part was making BetaBots critical. The harder part is deciding whose criticism matters, what authority it has, and what the larger system should do with it.

Start with the decision

A BetaBots run should begin with the decision the system needs to make.

“Test my product” is not a decision. “Make it good” is worse. Both invite agents to discover work indefinitely.

A useful mission asks something narrower:

Should this exact revision be released to the intended customer?

Or:

Does the current product explain enough value to justify a real pricing experiment?

Or:

Which claims in this launch story remain unsupported by visible product evidence?

The decision determines the terminal states. A release mission may end with release, hold, narrow, kill, blocked, or human research required. A positioning mission may end with defensible, weak, misleading, or needs evidence. A discovery mission may produce a small set of hypotheses rather than a repaired product.

The mission also defines what can block. Production readiness has a common floor: critical journeys work, data persists, authentication is real, security and privacy requirements hold, and the deployed revision is the one that was reviewed. A market discovery run cannot use the same gate. It may discover uncertainty without treating uncertainty as a software defect.

The first guardrail against an infinite loop is simple: the run is trying to answer one question, not improve the product forever.

Build a council, not a crowd

I used to describe BetaBots mainly as personas. One was impatient. Another cared about privacy. Another was technically confident. That variation matters, but it is one layer too low.

A cohort owns a question and applies a specific pressure. A persona is a person inside that cohort, with a reason for arriving now, practical constraints, a trust threshold, and a reason to leave.

“Investor” is not a colorful character prompt. It is a contract to ask whether the product deserves deeper diligence. “Hater” is not an angry tone. It is a mandate to find material that could support credible criticism. “Target customer” is not an age and job title. It is a person experiencing a particular pain and deciding whether the product earns attention, trust, money, and reputation.

I would start with six cohort contracts, then adapt them to the mission.

The owner clone should run as part of verification, not appear only when the manager wants a convenient approval. For my own system, I would give it one aggregated owner-persona file by default. It could retrieve authorized second-brain material when the summary is insufficient. Detailed memories may influence private reasoning, but they must not appear in raw journeys, aggregate packets, or public artifacts. A direct owner decision always overrides the clone.

The hater is useful when it finds a reproducible vulnerability, an exposed private record, or an unsupported claim. A comment that the logo looks stupid is different. The investor is useful when it exposes that my story depends on a feature nobody can find or a moat that looks trivial to copy. It does not produce a fake term sheet. The reporter needs a real beat or publication in mind; “journalist” is too vague to mean anything.

The competitor should make the strongest honest case both for and against the product’s advantage. The target audience remains the most important cohort in most missions. Repeated failure on the core job can become a blocker. A synthetic customer saying “I would pay” remains a hypothesis until real people put real money at risk.

Not every mission needs all six. Regulated and specialized products may need buyers, administrators, parents, clinicians, or regulators. A cohort earns its place by owning a question the rest of the council cannot answer.

One happiness score is a design bug

The council does not belong in one average.

For target customers, I may set an explicit mission gate. Eighty percent can be a reasonable starting point for one release, but the number needs a denominator: people, journeys, required outcomes, or another defined unit.

The other cohorts are not satisfaction surveys. I want fewer credible hater attacks and less competitor confidence that the product is easy to dismiss. Investor, reporter, and owner results depend on the mission. If the cohorts disagree, the disagreement is evidence.

The two historical runs at the start of this article are a warning, not a benchmark. Fifty broad personas returning positive verdicts and three stricter personas returning failures do not form a controlled comparison. The samples, evaluation pressure, and scoring conditions differed. The numbers show that the instrument changed the answer. They do not tell me which cohort predicted real users.

BetaBots should stop at evidence

Individual BetaBots should not create implementation tasks.

Their job ends with raw evidence: screenshots, actions, first-person journeys, evaluator output, environment state, and explicit uncertainty. Those artifacts should remain immutable inputs to later judgment.

A second layer can group repeated findings, surface conflicts, and sort results by cohort, severity, and evidence quality. It must preserve the underlying reports. Every conclusion should link back to its sources, denominators should be explicit, and minority findings should remain visible. An aggregator cannot turn three different complaints into a consensus nobody expressed.

Then the top-level manager decides what happens next.

The manager may ignore a finding and record why, keep it as context, investigate it, turn it into a bounded repair, rerun only the affected cohort, hold the release, or ask me to decide.

This separation matters because the manager knows the larger mission. The hater does not. The investor does not. The target user does not. None of them should be able to silently redefine the product.

Give every failure a lane

I would route findings into six lanes:

1. A potential release blocker is a reproducible security, privacy, safety, data-loss, critical-path, or agreed target-user failure. It blocks only when the mission contract makes that gate non-skippable.
2. A product experiment is repeated plausible friction that deserves a test rather than immediate belief.
3. A strategic signal changes how the owner, investor, reporter, or competitor sees the product.
4. A weak or adversarial signal is isolated taste, mockery, hypothetical criticism, or low-confidence generated opinion.
5. An evaluator failure means the browser, environment, persona mind, evidence capture, or rubric failed. The product judgment is unsupported.
6. A human-research result means the synthetic cohort cannot validly answer the question.

The mission contract decides which lane can block. There is no universal list for every BetaBots run.

This gives the manager permission to reject weak criticism without pretending all criticism is irrelevant.

Long-horizon is not the same as endless

I do not want a universal three-cycle limit. Some products need more work. An owner may still set a time, cost, round, or compute budget.

The overall mission can be uncapped while each evaluation cycle still finishes. Cohorts complete their journeys, aggregators produce a traceable packet, and the manager makes a routing decision before another cycle begins.

The danger is not duration. It is repeating work without new evidence or a new decision.

After every cycle, I want a durable checkpoint with the evidence packet, the manager’s decisions, skipped findings and reasons, progress against mission gates, open owner decisions, and the next bounded action. Repeating a repair requires new evidence.

The manager should contact the owner when execution is stuck, a loop appears, an owner-owned blocker arrives, an important unspecified product decision needs judgment, or a periodic progress report is due.

Owner silence should block only the affected lane. The manager can defer that decision and continue independent work. For a reversible question, it may ask an owner-proxy subagent for a provisional answer from authorized context. High-risk, destructive, sensitive, public, or externally binding actions still belong to the human.

This system may run for days. It should never disappear for days.

What BetaBots already did

At the BetaBots revision inspected for this article, the generated-persona path already went well beyond demographics. It modeled the person’s job, immediate trigger, prior attempts, constraints, anxieties, trust threshold, decision criteria, social context, success evidence, and reasons to abandon the product.

Before generating those personas, the runner could inspect visible product evidence. During a live reflection, it supplied the persona profile, current screenshot, visible text and controls, bounded session memory, route hints, session statistics, and an action schema. Multi-session runs preserved browser state and bounded recent narrative, action, and screen memory. Run artifacts included raw stories, screenshots, evidence, summaries, and post-run answers.

It also used “truth pressure.” Meaningful actions consumed simulated life, money carried an additional life cost, and deliberate lying carried the harshest fictional penalty. The mechanism made simulated commitments costlier than a survey answer. It did not prove actual willingness to pay.

The implementation still had limits. Supplied cohorts could run without the depth required of generated personas. Audience research informed generation but was not injected into every live reflection. Spending, return, recommendation, investor diligence, and competitor comparison were not modeled with the precision I would want.

When a bot says “I would pay,” I should inspect its full journey and reasoning: what it saw, which stakes and constraints were active, whether the product crossed its trust threshold, whether actions agreed with words, whether similar personas decided independently, and whether the judgment survived a later session. The artifacts can support that audit. They cannot validate real customer behavior.

Potential next step: personas with a changing world

I do not plan to implement this next layer now. The current gaps still point toward it.

A deeper persona would contain more than a static biography. It could model discretionary budget, purchasing authority, existing obligations, current subscriptions, switching cost, and the cost of doing nothing. It could preserve a belief state: what the person believes before arriving, which sources they trust, and what evidence could change their mind. It could include the social world around a decision: who receives the recommendation, whose reputation is at risk, and which competitor is the default alternative.

The system could then maintain a private decision ledger: trust gained or lost, promises observed, money and attention spent, alternatives reconsidered, unresolved objections, and reasons for returning, abandoning, recommending, or regretting.

MiroFish offers one architectural reference at the revision I inspected. It separates source ingestion, ontology and graph construction, relationship-aware profiles, behavior configuration, social simulation, optional graph-history updates, interviews, and reports.

I would not copy its social-media simulation. BetaBots evaluates products rather than information propagation. What interests me is the structure underneath: generate a persona from a world of entities, relationships, events, and beliefs, then preserve how that state changes through experience.

A careful BetaBots version could separate stable identity, world relationships, temporary scenario state, and private append-only memory with explicit retrieval and provenance. It could test coherence, contradictions, sensitivity to different seeds or models, and whether stated intentions predict later simulated behavior.

That is a research direction, not a release promise. The inspected MiroFish code is also not evidence that synthetic agents predict real people; I found no visible calibration benchmark in that repository revision.

Make disagreement usable

The first version of BetaBots produced a lot of synthetic approval. The stricter version produced failures that kept the engineering loop alive. Neither answered the harder question: what should happen when different modeled people disagree?

Before I launch another cohort, I want five things written down: the decision, the people allowed to judge it, the evidence they need, the manager who interprets disagreement, and the condition that ends the run.

More personas create more opinions. The useful system knows which opinion is a blocker, which is an experiment, which needs a human, and which should be recorded and ignored.

That is the difference between agents that keep working and agents that keep circling.

---

This article refers to BetaBots commit 5964eec and MiroFish commit 60757b3, inspected on July 31, 2026. Source inspection identifies what those code paths contain. It does not establish live default-branch behavior or prove that synthetic personas predict customers, investors, competitors, or media.