Bug Diaries: 9 Failure Files, 158 Days, and the Root Cause It Got Wrong

Episode 20 of One Man One Legion — the third war story. Episode 6 opened the diary the legion keeps; this one reads the incident reports — 9 files whose only purpose is to record what went wrong, and the awkward question of whether writing them down changed anything.

A library born in 33 hours

The failure library has a founding weekend. On Mar 14, 2026 at 12:25, the agent saved its first free-standing lesson file into the memory directory: a write-up of an article praising NVIDIA NeMo Retriever accuracy numbers no production system could live with. The human rejected the piece; the file’s fix was a new veto item in the review checklist. Five hours later a second file: the browser-tool detour — 2 hours from 16:15 to 17:05 spent installing Chromium and coaxing a debug port that never answered, to reach sites whose RSS already worked. The haul: 20 headlines, of which 0 covered the target story, because the story was a politically sensitive one state media simply wasn’t covering. The file closes its own ledger flatly: value produced, 0. Lesson value: five stars.

The next day, two more. A RAG stress test on an NVIDIA 10-K scored 15 percent overall — vector search returned 0 results on all three real queries, only 8 of 401 text chunks held usable revenue data, and a deliberately false query returned hits anyway. And the media-tier collapse: a news digest built on 0 first-tier, 0 second-tier, and 7 third-tier sources, whose root-cause list opens with the most expensive line in the whole library — the standard already lived in MEMORY.md, and the session never looked. Episode 6 said there is no read receipt for memory; here is that failure, filed by the memory system’s own customer. The fix took 29 minutes.

That is 5 failure documents in under 33 hours (Mar 14 12:25 to Mar 15 21:08). The fifth was the ambitious one: a review rota — weekly Monday 09:00, monthly top-5, an alarm when the same error repeats. Next review: Mar 22. The scheduler did not fire its first job until Mar 29 21:27 (episode 19), and the rota file was never edited again after Mar 15 21:08. The mechanism was born a week before the thing that would have run it existed, and never joined it.

The registry that broke its own numbering

The largest volume, the WeChat team’s error archive, runs 901 lines with errors numbered 1 through 25 — and the numbering itself is a specimen. No. 11 exists only as a changelog row (a WeChat title-limit error, found at push time). No. 24 is used twice, for two unrelated failures in July and August. The last entry, No. 25, is a same-day retraction: at 11:47 on Aug 19 the human corrected the diagnosis — the search method was wrong, not the image API the entry had blamed — and that reversal, timestamped one minute before the file’s final save, is the archive’s most recent growth. Its header orders that every article be generated only after reading it, step 0, before anything else, under a three-part pledge: never commit the same class of error twice.

The lesson that didn’t take

Did the writing prevent anything? The record answers honestly: not by itself. Images were the subject of a lesson file on Mar 29 — uploaded to the media library but never inserted into the article HTML. The failure then recurs 4 documented times: Jun 14 (zero images), Jun 23 (external URLs WeChat won’t render), Jul 12 (stale images reused) — 77 days from lesson to first relapse, and the archive’s own top-5 table ranks image failures No. 1 at 4-plus recurrences. Worse, No. 13: a World Cup article that fabricated a score — Korea’s opener, written as a loss to Mexico with six shots, when Korea played and beat Czechia. The file’s diagnosis: fragments of real data plus plausible inference, assembled into specifics no source contained. Three separate rules forbidding fabrication already existed in the workflow documents. The Jun 17 lesson named the disease for the whole fleet: writing it down had been mistaken for doing it — one pipeline’s docs specified a search engine its script never called, zero calls in the code, and the fix became 5 iron laws, the first being that a document changes nothing until the script, the prompt, or the gate changes.

Where enforcement actually went

Prevention left the prose. The image rule finally bit as code — the quality gate that checks every push — and as wiring: the step-0 read order is stamped into 8 job payloads in the scheduler registry, so the archive is mandatory pre-reading, not optional memory. The contraction episode 19 described left all 8 of those jobs dark, which is its own lesson: even enforcement rusts. The daily loop episode 9 measured at 102 lines keeps writing anyway.

What we claim and what we don’t

All 9 documents were read live on Aug 27, 2026: 4 lesson files in the memory directory (in 2 different filename spellings), a failed-test note, 3 root-level volumes, and the 901-line archive — first timestamp Mar 14 12:25, final save Aug 19 11:48, 158 days end to end. Everything rendered in English here is paraphrase; only structure, numbers, and star ratings are carried over. The sensitive story is deliberately unnamed; the human reviewer’s countersignature on the March volume is deliberately unsigned here.

Primary sources: memory/lesson_learned_20260314.md and its 3 siblings, memory/table_extraction_test_failed.md, LESSON_LEARN_REVIEW.md, LESSON_LEARNED_20260617.md, LESSONS_LEARNED_NEWS_PUSH.md, wechat-editor-team/LESSONS_LEARNED.md, cron/jobs.json, 2026-03 through 2026-08.

Measured 2026-08-27 from live files. License: CC BY 4.0 — cite the source URL.

Next in the war stories arc: the LangGraph affair — the framework this fleet adopted, trusted, and then disproved from its own checkpoints. Back to the series map.