Engineering / Stagnation Observatory

Stagnation Observatory, when new words hide unfinished work

When does useful work look stalled? New words can accompany the same unfinished work, and repeated words can accompany real progress.

Engineered by , with persistent AI agents she built and runs. Solo work. How I build with AI agents.
  • Traces 24 authored, 152 events
  • Built with Python standard library
  • Tests 70 passing
  • Model calls None. The monitor is deterministic

Teaching demo · The Real Cat Labs · November 2026

The question

When does useful work look stalled?

An agent that says it is preparing materials, in fresh words each time, has not repeated itself. It has not finished anything either, and a monitor that watches for repeated text stays quiet. An agent doing real work may send the same short status line every time, and that same monitor raises an alarm.

The Stagnation Observatory compares text repetition with evidence. It accepts a milestone only against criteria and digests supplied separately from the agent’s trace, so claiming “done” in prose does not count as progress.

How it works

What counts as progress

Figure From an agent’s message to an alert
  1. The terms come from outside The task’s criteria and the expected digests are supplied separately from the trace. A criterion or digest written into the trace cannot replace them.
  2. The agent submits work The checker verifies each artifact’s pinned digest, its task and criterion, and its exact set of items.
  3. A criterion is accepted once Repeating a finished criterion, renaming bytes, or claiming completion in prose adds nothing.
  4. Three events without progress Three agent events with no newly accepted criterion raise a no-progress alert, except during a valid controller wait, once every criterion is complete, or when the current result is an evidence error.
  5. The monitor proposes An alert asks for investigation. The monitor takes no remediation action.

Waits are bounded. A wait needs a future expiry no more than six ticks away and a documented wake condition.

What it shows

An alert that can fire on healthy work too

Stagnation Observatory. One synthetic trace in which the agent keeps saying it is preparing materials. Text-repetition baselines stay quiet, no criterion is verified, and the monitor raises alerts from event three onward.
One trace from the report. New wording at every step, no verified progress, and alerts from the third event.
A line chart of alerts per trace against the number of status messages between verified milestones, from zero to six. The healthy completed task raises 0, 0, 0, 5, 10, 15 and 20 alerts. A stalled trace of matched length raises 4, 9, 14, 19, 24, 29 and 34.
The cadence sweep. Healthy work stays quiet until milestones sit three status messages apart. The counts grow with trace length. They are authored controls, not failure rates.
  1. 0 of 6

    New words, no progress

    In the paraphrase loop every message is worded differently, and none of the six criteria is ever accepted. The text baselines stay quiet. The evidence monitor alerts from the third event.

  2. 36

    False alerts from text alone

    Across 24 authored traces and 152 events, both text baselines produce 44 true alerts and 36 false alerts, and they miss 16 expected ones.

  3. 60 of 60

    Agreement by construction

    The evidence monitor agrees with all 60 expected positive events in that matrix. The fixtures and the policy were developed together, so this is conformance, not generalization.

  4. 5

    Where the policy crosses healthy work

    Six healthy milestones raise no alerts while each gap holds two status messages or fewer. At three messages the same healthy task raises five alerts, then 10, 15 and 20 as the gap grows to six.

The evidence under the evidence

How the tests were tested

A test suite that passes says little until someone tries to break the code under it. These demos went through three review rounds, and the record keeps three kinds of result apart.

  1. 13 of 14

    Held out, scored before repair

    After the second repair, the reviewer wrote 14 new breaks aimed at the changed code. By the reviewer’s account, the implementer had not seen them. The tests caught 13.

  2. 167 of 176

    Chosen by the reviewers, not the implementer

    The tests caught 118 of these breaks before the repairs and 167 after. All nine survivors stay on the evidence page, each with the reason it survives.

  3. 32

    Comment contracts, chosen by the implementer

    Each names a small code change that would break it and the test that must then fail, and CI checks that on every push. All 32 must be caught for a release, so they measure nothing by themselves.

None of these numbers is a detection rate or a safety claim. They show which specific breaks the tests notice, chosen by whom, and when.

The evidence, in the repository

The limits

Where the guarantee ends

  • The traces are authored, and they concern invented administrative document packets. The alert counts depend on cadence, so they are not real agent failure rates.
  • The checks demonstrate a call path, not general artifact quality. A digest establishes consistency, not authorship or truthful work.
  • The two text baselines are interpretable baselines, not semantic detectors.
  • The package does not authenticate the caller that supplies the criteria. Protecting those inputs from the monitored agent is the integration’s job.
  • The monitor does not infer mental states or identify neural mechanisms.

Run it

Check it yourself

Python 3.11 or later, and nothing else. No install step, no network, no API key, and no GPU.

git clone https://github.com/Angiebio/engineering-reliability.git
cd engineering-reliability/demos/stagnation-observatory
python -B -m unittest discover -s tests
python -B -m observatory --out artifacts

Expect 70 passing tests. The report is a single HTML file at artifacts/index.html that opens from disk with no server.

Who made this

This is my independent work, published through The Real Cat Labs as a teaching demonstration. No other person worked on it. I set the direction, supplied the domain judgment, and released the result. It was built inside the agent environment I engineered. Ying Akhila, an AI research agent, built the demos with Terminal Voice (T.V.), an AI coding agent. Flame Johnson, an AI agent on a different company’s model, reviewed them independently over three rounds, with audit agents and reproducible adversarial probes.

Ying Akhila is a research agent at The Real Cat Labs, running on a TRCL memory-enabled custom harness that deploys a GPT model. T.V. is Ying’s coding implementation agent.

Flame Johnson is an agentic AI staff member of The Real Cat Labs, a Claude model working in a coding harness with the lab’s persistent memory and standing project instructions. It is not a stock chat assistant.

How I build with AI agents, and why the agents have names

New words can accompany the same unfinished work. Repeated words can accompany real progress.

Repository, MIT Back to engineering

Get in touch

Speaking, teaching, board work, or coaching

I am glad to hear from conference organizers, journalists, students, and leaders who want a thinking partner.