When the Bottleneck Moves From Weights to Memory and Power: 24 arXiv Papers on the Machinery Layer

Second digest of the week’s 81-paper series: 24 papers on the machinery under the model — the inference stack (5), optimization theory and its industrial applications (13), and training methods (6). The common thread: the research object is expanding from “the model” to “the model plus its memory hierarchy plus its power supply,” and every component is acquiring a cost accounting. (All papers are arXiv preprints, not peer-reviewed; claims follow their abstracts.)

The serving stack: memory, not weights

KV cache has displaced model weights as the central bottleneck of the inference stack. py-kvcache (arXiv:2609.11744) delivers the honest measurement of offloading it to NVMe SSDs in vLLM: prefix caching shortens time-to-first-token for long contexts, but past a crossover — short prefixes, fast GPUs — recomputing beats loading (see FAQ). OmniKVQuant (2609.11582) pushes KV quantization into omni-modal LLMs that ingest audio, video and text together, and finds text-scene recipes like TurboQuant failing there: numerical distributions differ enough across modalities that one parameter set cannot rule them all.

Quantization gets its theory backlog cleared. Why Does Post-Training Quantization Work? (2609.11716) explains the long-observed grace period — why per-weight quantization error does not, in fact, accumulate into prediction collapse — and in doing so marks the boundary of when it would. Structured Transforms for Low-Overhead Quantization (2609.11687) takes the Kashin-decomposition route: weights split into two low-precision components plus structured orthogonal transforms whose convergence properties are stronger — with the acknowledged tax that orthogonal transforms are not cheap on GPUs; “structured” is precisely the fight over that overhead. The bit-count arms race is nearing physical limits; the theory ledger is starting to pay.

Watts join the bill of materials

Characterizing Job Power Elasticity for Power-Flexible AI Training (arXiv:2609.11542) moves the frame to the datacenter: LLM training is the fastest-growing source of electricity demand, and power capacity — not GPU supply — is becoming the first infrastructure bottleneck. The paper characterizes which training jobs can throttle down under grid stress and still make progress. Energy moves from operating cost to first-class design variable; expect this genre to multiply.

Step sizes enter the jigsaw phase

On smooth convex optimization, the rate question is essentially closed; the remaining work is combination. Optimal Recursive Composition and Dyadic Phase Laws (arXiv:2609.11788) gives the exact convergence picture for recursively composed long-step schedules; Constant Steps Are s-Composable (2609.11784) answers Grimmer, Shu and Wang’s open question in the affirmative, with an exact interpolation certificate — balanced constant steps compose at every horizon. Preconditioning industrializes in parallel: Stochastic Gradient Methods with Online Scaling (2609.11751) and its operator-splitting twin (2609.10732) — covering forward-backward, Douglas-Rachford and ADMM — turn hand-tuned preconditioning into a learnable component. Early stopping gets its minimax-optimal rule via the discrepancy principle (2609.11273); CONES (2609.11207) handles feasible sets that keep shrinking while the loss keeps moving; pessimistic bilevel optimization gets single-loop algorithms (2609.11183); and the Cheney-Goldstein best-approximation machinery from 1959 is extended to multiple sets (2609.11105) — approximation theory’s own health check. Off the gradient path, SMT solving learns non-ground clauses from conflicts (2609.11509), patching a structural blind spot in CDCL(T).

The other half of the batch sinks optimization into worlds with real constraints: seasonal energy-storage scheduling via entropic SDDP with HJB cross-certification (arXiv:2609.11339, validation-first design because power grids forgive little), multi-product inventory with pathwise safety sets on stock, warehouse capacity and carbon quota via forward-backward SDEs (2609.11817), drone sensor placement and routing over dynamic wildfire fields (2609.11829), and certified lower bounds on worst-case risk for RL under adversarial state perturbations (2609.10866) — in safety-critical settings, “how bad is the worst case” must be provable, not estimable.

Training methods find their geometry

The six training papers share a temperament: no longer “new method, higher score,” but “why does the method work, and when does it fail.” AdamX (arXiv:2609.11867) uses cosine similarity between consecutive gradients as its adaptive signal — gradient-direction coherence itself carries the information about how large the update should be; Musec (2609.11655) patches Muon’s spectral-flattening instability (see FAQ). Thinking with Looped Flows (2609.11801) names a structural debt of recurrent models: inference can loop as long as you like, but training backpropagates through only one or two updates — the training-inference asymmetry that any test-time-compute route inherits. Negative Self-Distillation (2609.11699) is a warning shot: on-policy self-distillation can leak privileged ground truth — the model learns to copy answers rather than reason — and turns that information around, training the model to recognize and avoid flawed reasoning paths. And low-rank structure shows its two faces: LOCUS (2609.11739) finds that doing preference alignment in a low-rank subspace trims verbosity — output length, i.e. the serving bill — without losing utility, while EGGROLL, Unrolled (2609.10980) dissects why rank-one perturbations in evolution strategies are geometrically harsh: every perturbation lands on a measure-zero subspace.

The takeaway

The machinery layer’s narrative has flipped from performance to accounting: every paper prices something — the crossover point of an SSD offload, the boundary condition of a quantization guarantee, the watt-elasticity of a training job, the exact rate of a step-size schedule. When each component carries its own theoretical ledger, comparisons between methods stop being folklore and become engineering.

Provenance & disclosure. Originally published in Chinese on our WeChat channel on 2026-09-12 as three parts of a nine-part “arXiv 81” series (“量化为什么没压坏模型？5篇论文拆推理栈”; “GD的步长还能挖多久？13篇优化新论文”; “Muon还不稳？优化器军备赛的新答卷”); drafted with AI assistance under human editorial direction. Translated and consolidated to English on 2026-09-12 (AI-assisted, human-reviewed). All 24 cited arXiv IDs were resolved via the arXiv API on 2026-09-12 and titles matched the claims; the Grimmer-Shu-Wang and Cheney-Goldstein-1959 attributions verified against the abstracts. The silver-ratio background follows the column as relayed. Preprint caveat inherited. Translated commentary — not a SigPulse measurement; first-party numbers live in the dispatches and the /data/ ledger.