Skip to content
SIGPULSE
Papers Batch 1 3 min read raw .md ↗

Do New arXiv Papers Really Show That Less Training Data Beats More? Four August Papers, Read Inside China

Chinese original 2026-08-29 · 「烧钱堆算力过时了？AI圈流行起"少即是多"」 · translated to English 2026-08-29

The dominant folk-model of AI progress — more data, more GPUs, bigger models — took four body shots from late-August arXiv submissions, at least as our Chinese-language column reads them. Its roundup ran under “Burning money on compute is out? The AI world’s new ‘less is more.’” This digest translates the reading and pins each paper claim to its abstract (SWE-Prime and the retrieval-heads paper were verified against their arXiv pages at translation time; figures below follow the abstracts).

The claims

  • SWE-Prime (2608.27449): successful trajectories are noisy supervision; two-stage, two-granularity selection of SFT data — and a 10% subset outperforms the full resolved dataset, relative gains up to 12.2% (SWE-Bench Pro) and 24.2% (SWE-Bench Verified).
  • Weak-model guidance in RLVR (2608.27420): strong models over-trained on verifiable rewards collapse into narrow strategies (entropy collapse). Forcing the target model to continue from a weaker model’s partial reasoning reintroduces diversity — the study-grade student teaching the exam-grade one where the side paths are.
  • TTPO (2608.27448): at test time, rollouts inconsistent with pseudo-labels are almost certainly wrong; that asymmetry licenses policy optimization during inference, with no ground-truth labels — training and deployment stop being separate rooms.
  • Retrieval Heads Meet Vision (2608.27417): in vision-language models, roughly 1.7–2.6% of attention heads are causally responsible for grounding text to image regions; masking the top 20 such heads cuts grounding accuracy by up to 80 points, while random masking barely moves it.

How the Chinese column reads it

Four lessons, in the column’s own voice. Do one hundred problems well instead of ten thousand badly — SWE-Prime validated the homeroom teacher’s edict with a few million dollars of compute. The weak student’s wild paths can save the strong student’s life — its one-line footnote: diversity is AI’s oxygen; a model that is too strong suffocates itself. Supplementary tutoring inside the exam hall — TTPO erases the line between training and testing. And the org-chart joke that carried the piece on Chinese social media: while bosses keep hiring, the model already cut its own headcount to 2% — two percent of attention heads doing all the visual grounding.

The column’s closing move is personal, not technical: the same logic at individual scale — cut ten 60-point tasks to three done thoroughly, and swallow the short-term cost of looking less busy, because scorecards see quantity, not depth.

Cold water

The column supplies its own: Deloitte’s 2026 forecast that compute demand only rises; Nature-energy reminders that every token of inference cleverness was bought with training-phase electricity. Its synthesis — foundation by scale, craft by precision — is the correct reading, and our FAQ’s weaker restatement (“curation beats accumulation”) is the version safe to cite.

What to watch

Whether selection methods like SWE-Prime’s survive contact with non-SWE domains (the paper’s domain is narrow); whether test-time optimization (TTPO) changes serving economics — optimization during inference means inference gets more expensive per call, not less; and whether the 2% finding graduates from interpretability curiosity to pruning practice (a 50x head-count reduction in the grounding pathway is a compression target someone will chase).

Sources

Provenance & disclosure. Originally published in Chinese on our WeChat channel on 2026-08-29 (“烧钱堆算力过时了？AI圈流行起’少即是多’”); drafted with AI assistance under human editorial direction. Translated to English on 2026-08-29 (AI-assisted, human-reviewed). Paper claims were checked against arXiv abstract pages (2608.27449 and 2608.27417 verified line-by-line at translation; 2608.27420 and 2608.27448 follow the abstracts as listed). The FAQ’s “weaker restatement” is our cross-check, not the column’s phrasing. This is translated commentary — not a SigPulse measurement. Our first-party measurements live in the dispatches and the /data/ ledger.

Papers covered in this digest (machine-readable in /papers.json)

  • 2608.27449 — SWE-Prime: Fewer Trajectories, Better Performance
  • 2608.27420 — Boosting LLM Exploration via Weak-Model Guidance in RLVR
  • 2608.27448 — TTPO: Test-Time Policy Optimization
  • 2608.27417 — Retrieval Heads Meet Vision: Uncovering How VLMs Locate and Extract Visual Information

Cross-checked sources (machine-readable in the raw markdown)

FAQ — Direct Answers

What did SWE-Prime actually show?
Per the abstract (checked against the arXiv page): successful agent trajectories still contain ineffective, redundant, or risky steps, so SFT on all of them injects noisy supervision. SWE-Prime screens at two granularities — trajectory-level, then segment-level — keeps every segment in sequence for context but counts only selected ones in the loss. Training on a 10% subset beat training on the full resolved dataset, with relative gains up to 12.2% on SWE-Bench Pro and 24.2% on SWE-Bench Verified.
Do these papers refute scaling laws?
No, and the Chinese take says so explicitly: its own cold-water section cites Deloitte's 2026 predictions (next-phase AI needs more compute, not less) and Nature-published energy analyses. The synthesis offered: scale settles 'does it exist at all'; precision settles 'how well' — the papers price the second scoreboard, they do not cancel the first.
What is the weakest link in the 'less is more' reading?
These papers show that careful selection beats unfiltered bulk on the same budget — selection is itself an added process with its own costs. None of the four claims data stopped mattering; the honest summary is 'curation beats accumulation,' which is a weaker (and more useful) claim than 'less is more.'