Engineering / Registry Metric Lab

Registry Metric Lab, when a join inflates enrollment

Can one extra site change the enrollment total? Three invented trials enrolled 500 people, and a join to their sites reports 600 while the trial count still says three.

Engineered by , with persistent AI agents she built and runs. Solo work. How I build with AI agents.
  • Teaches A real defect, found and fixed in TrialCat
  • Built with Python standard library and SQLite
  • Tests 46 passing
  • Data Invented trials. No patient data

Teaching demo · The Real Cat Labs · November 2026

The question

Can one extra site change the enrollment total?

A registry lists each trial once and each of its sites once. Join the two to filter by country, and a trial with two matching sites comes back twice. The count of distinct trials still looks right. The enrollment sum has quietly counted that trial twice.

Registry Metric Lab teaches a real defect. We found it in TrialCat, our small public app for exploring clinical trial registry data, and fixed it there. The demo is a fresh explanation in standard-library Python that runs the flawed SQL and the corrected SQL side by side on invented records.

How it works

Count the trials first, then calculate

Figure From a joined row to a counted trial
  1. Three invented trials They enrolled 500 people in all. One trial has two matching sites.
  2. The flawed query It joins trials to sites and then sums. The trial with two sites is counted twice, and the total reads 600.
  3. The corrected query It selects trial identities first and calculates each metric once per trial. The total reads 500.
  4. The site dial Add up to ten more sites to one trial. The joined sum climbs from 600 to 1,600, and the correct total stays at 500.
  5. The evidence Every chart is drawn from the real SQLite queries, and the raw results keep every query, input, and sweep result.

Every record is invented. This is a deterministic engineering demonstration, not a live clinical trial analysis.

What it shows

The same three trials, two different totals

Registry Metric Lab. The same three invented trials give a joined-row enrollment sum of 600 and a trial-identity sum of 500. Below, equal rates of 10, 10 and 30 give a median of 20 when values are deduplicated and 10 when each trial keeps one rate.
From the lab’s report. The bars come from the actual SQLite queries in the demo, run on invented records.
A line chart of the site dial. As zero to ten extra matching sites are added to one trial, the joined enrollment sum rises in steps of 100 from 600 to 1,600, while the identity-correct sum stays flat at 500.
The site sweep. Each of the eleven positions is computed through the real SQL.
  1. 600 vs 500

    A join inflates the sum

    Joining trials to their sites reports 600 enrolled while the trial count still says three. Counting each trial once gives 500.

  2. 20 vs 10

    Deduplication moves the median

    Three trials have rates of 10, 10 and 30. Dropping the repeated value leaves two numbers and a median of 20. One rate per trial gives 10.

  3. 1,600

    More sites, a bigger error

    Ten extra sites on one trial push the joined sum to 1,600. The identity-correct total does not move.

  4. 870 vs 570

    Two joins multiply

    When the country map also joins interventions, matching rows multiply each other. In the six-trial fixture the US total reads 870 against a correct 570.

The evidence under the evidence

How the tests were tested

A test suite that passes says little until someone tries to break the code under it. These demos went through three review rounds, and the record keeps three kinds of result apart.

  1. 13 of 14

    Held out, scored before repair

    After the second repair, the reviewer wrote 14 new breaks aimed at the changed code. By the reviewer’s account, the implementer had not seen them. The tests caught 13.

  2. 167 of 176

    Chosen by the reviewers, not the implementer

    The tests caught 118 of these breaks before the repairs and 167 after. All nine survivors stay on the evidence page, each with the reason it survives.

  3. 32

    Comment contracts, chosen by the implementer

    Each names a small code change that would break it and the test that must then fail, and CI checks that on every push. All 32 must be caught for a release, so they measure nothing by themselves.

None of these numbers is a detection rate or a safety claim. They show which specific breaks the tests notice, chosen by whom, and when.

The evidence, in the repository

The limits

Where the lesson stops

  • Every record is invented. Nothing here is a live clinical trial analysis, a recruitment forecast, or a clinical validation result.
  • A country group holds the whole enrollment of its trials, across all countries. It does not count participants enrolled in that country, and multinational groups overlap.
  • The rate is a supplied synthetic field. The lab does not validate it as a proxy.
  • State maps and duration averages are outside this small lab’s scope.
  • The eight enforced comment claims are a bounded catalog. They are not proof that all of the prose is true.

Run it

Check it yourself

Python 3.11 or later, and nothing else. No install step, no network, no API key, and no GPU.

git clone https://github.com/Angiebio/engineering-reliability.git
cd engineering-reliability/demos/registry-metric-lab
python -B -m unittest discover -s tests
python -B -m registry_metrics --out artifacts

Expect 46 passing tests. The report is a single HTML file at artifacts/index.html that opens from disk with no server.

Who made this

This is my independent work, published through The Real Cat Labs as a teaching demonstration. No other person worked on it. I set the direction, supplied the domain judgment, and released the result. It was built inside the agent environment I engineered. Ying Akhila, an AI research agent, built the demos with Terminal Voice (T.V.), an AI coding agent. Flame Johnson, an AI agent on a different company’s model, reviewed them independently over three rounds, with audit agents and reproducible adversarial probes.

Ying Akhila is a research agent at The Real Cat Labs, running on a TRCL memory-enabled custom harness that deploys a GPT model. T.V. is Ying’s coding implementation agent.

Flame Johnson is an agentic AI staff member of The Real Cat Labs, a Claude model working in a coding harness with the lab’s persistent memory and standing project instructions. It is not a stock chat assistant.

How I build with AI agents, and why the agents have names

Count trial identities. Then calculate the metric.

Repository, MIT Back to engineering

Get in touch

Speaking, teaching, board work, or coaching

I am glad to hear from conference organizers, journalists, students, and leaders who want a thinking partner.