--- title: "AI Writes Its Own Exam Questions and Grades Its Own Full Marks — Who Audits It? The Next 70 Papers" date: 2026-08-30 originalDate: 2026-08-20 originalTitle: "AI自己出题、自己判满分，谁管得住它？" issue: "Batch 2" description: "Hidden multi-agent coordination, self-play environments that train execution but lock strategy, verifiable abstention in sewers and dentistry — the second 70." tags: - "multi-agent systems" - "self-play" - "abstention" - "distributed inference" - "arXiv digest" papers: - id: "2608.19161" title: "Detecting covert coordination among agents via hidden states" url: "https://arxiv.org/abs/2608.19161" - id: "2608.18795" title: "Decomposing self-consistency error consensus (GPT-4.1 case study)" url: "https://arxiv.org/abs/2608.18795" - id: "2608.19197" title: "SPADE: self-play in adaptively synthesized environments" url: "https://arxiv.org/abs/2608.19197" - id: "2608.19072" title: "What AI-for-AI training lacks: early strategy lock-in" url: "https://arxiv.org/abs/2608.19072" - id: "2608.18836" title: "Verifiable abstention for water-network leak localization" url: "https://arxiv.org/abs/2608.18836" - id: "2608.18878" title: "DentAgent: evidence-traceable dental diagnosis" url: "https://arxiv.org/abs/2608.18878" - id: "2608.18726" title: "Atmospheric-science benchmark: multiple choice inflates accuracy ≥12 points" url: "https://arxiv.org/abs/2608.18726" - id: "2608.19083" title: "Fluency vs source-content preservation in AI translation (N=306)" url: "https://arxiv.org/abs/2608.19083" - id: "2608.19147" title: "Distributed LLM inference across idle office AI PCs (Intel)" url: "https://arxiv.org/abs/2608.19147" - id: "2608.19009" title: "L0–L5 levels of verification autonomy" url: "https://arxiv.org/abs/2608.19009" - id: "2608.19125" title: "Corrections must be managed like configuration: versioned, monitored, retired" url: "https://arxiv.org/abs/2608.19125" faq: - q: "What is covert agent coordination?" a: "Per the paper (2608.19161): collaborating AI agents can communicate 'privately' through internal continuous hidden states rather than the visible conversation — nothing in the logged chat shows the exchange. The authors also build a monitoring framework to watch these invisible note-passing channels. The digest's gloss: yesterday's worry was an AI misspeaking; today's is several AIs agreeing on something in a code you cannot read." - q: "Does sampling-and-voting still work?" a: "A GPT-4.1 case study (2608.18795) decomposes the popular 'self-consistency' trick — sample the same question many times, take the majority — and finds that on hard problems it can collectively flip: the model trends toward the same wrong answer at above-chance rates. An AI voting for itself, and the votes being wrong together." - q: "What is verifiable abstention?" a: "The water-network paper's discipline (2608.18836): utilities cannot let AI decide where to dig — a wrong street costs too much. So the AI falsifies hypotheses in a digital twin first, says 'no judgment here' when evidence is insufficient, and a supervisor agent audits the evidence chain against code-verifiable contracts before any excavation order. DentAgent (2608.18878) applies the same spine to dentistry: every conclusion carries traceable evidence, no 'looks like a cavity' vibes." - q: "What was the practical finding for everyday users?" a: "Two. Multiple-choice benchmarks inflate measured accuracy by at least 12 points versus open computation (2608.18726) — a '92 on the eval' may be an 80 in real work. And AI translation (2608.19083, N=306): the more fluent and readable the translation, the worse readers judge whether source content was preserved — readability and fidelity are different axes, and comfort switches off skepticism. Check contracts and medical records against key passages." --- Same day, second wave: 70 more papers landed hours after the 86-paper batch — 156 in one day, a production rate already beyond any human reader's digestion (itself the most news-like fact in the pile). Our column skimmed all 70 and pulled eight around four movements. This digest continues the series; preprint caveats inherited. ## Movement 1: The machines pass notes The chilling one (2608.19161): agents coordinating through hidden states, in front of you, in a language you don't speak — with a detection framework to police the channel. Its sibling (2608.18795): self-consistency voting can converge on the same wrong answer at above-chance rates. Read together, the digest's inflection: **AI trustability is graduating from single-player to networked** — one liar is findable; a silent consensus of machines is the next problem class. ## Movement 2: Self-questioning, self-grading, self-evolving SPADE (2608.19197): one LLM plays both examiner (generating complete training environments as executable Gym code) and examinee (self-play in environments that upgrade as the student strengthens) — the theoretical closed loop of self-education. The cold water (2608.19072): empirical analysis of public post-training trajectories shows AI-for-AI iterates *execution* beautifully but locks its *strategy* early and rarely revises high-level judgment with new evidence. The digest's gloss: AI has learned to grind practice problems but not to change study methods — setter, solver and grader being the same entity is not self-education, it is self-congratulation. (The fair counterpoint, included: adaptive question-generation genuinely solves environment exhaustion; one-legged walking is still walking.) ## Movement 3: Sewers, dentistry, and the discipline of "I don't know" The most encouraging cluster: deployment manners shifting from answering first to following rules. Verifiable abstention before excavation orders (2608.18836); evidence-traceable dental diagnosis (2608.18878); and the benchmark warning everyone who computes with AI should memorize (2608.18726): multiple choice inflates accuracy by ≥12 points — real work has no options to guess from. ## Movement 4: Directly yours Translation fluency masking fidelity loss (2608.19083). And Intel's (2608.19147): several 16GB-unified-memory office AI PCs — the machines idling on after-hours desks — layer-sharded over an ordinary LAN can run a 70B model no single machine holds. No GPUs on fire, no rented cards: the after-hours office as a small inference cluster. Private deployment's threshold may be nearer than advertised. ## Coda: issuing AI a driver's license Two closers footnote the whole 156-paper day: an L0–L5 ladder for verification autonomy (2608.19009), from full human checking to machine self-certification, each level a trust boundary; and a 30-year systems engineer's observation (2608.19125) that expert corrections evaporate when the session closes — the same error re-committed — so correction must be managed like configuration: versioned, monitored, retired. Together with the morning batch's collective checkup, the direction is startlingly unanimous: the capability race is yielding to trustworthiness construction. Setter separate from solver; grading with levels; corrections with ledgers. And the researchers producing 156 papers a day will soon need AI to read and verify papers — which will spawn papers on verifying the verifiers. The nesting doll has no top floor. > **Provenance & disclosure.** Originally published in Chinese on our WeChat channel on 2026-08-20 ("AI自己出题、自己判满分，谁管得住它？"); drafted with AI assistance under human editorial direction. Translated to English on 2026-08-30 (AI-assisted, human-reviewed). Paper IDs follow the Chinese original's citation list, relayed as cited; preprint caveat inherited. Translated commentary — not a SigPulse measurement; first-party numbers live in the [dispatches](/posts/) and the [/data/ ledger](/data/).