--- title: "When the Bottleneck Moves From Weights to Memory and Power: 24 arXiv Papers on the Machinery Layer" date: 2026-09-12 originalDate: 2026-09-12 originalTitle: "量化为什么没压坏模型？5篇论文拆推理栈 / GD的步长还能挖多久？13篇优化新论文 / Muon还不稳？优化器军备赛的新答卷" issue: "Batch 5" description: "KV cache becomes the central inference bottleneck, quantization theory catches up with practice, watts become a design variable — 24 papers." tags: - "inference systems" - "KV cache" - "quantization" - "optimization" - "training methods" - "arXiv digest" papers: - id: "2609.11744" title: "Building py-kvcache: A Performance Characterization of External KV Caching for vLLM with NVMe SSDs" url: "https://arxiv.org/abs/2609.11744" - id: "2609.11716" title: "Why Does Post-Training Quantization Work?" url: "https://arxiv.org/abs/2609.11716" - id: "2609.11687" title: "Structured Transforms for Low-Overhead Quantization of Language Models" url: "https://arxiv.org/abs/2609.11687" - id: "2609.11582" title: "OmniKVQuant: KV Cache Quantization for Omni-LLMs" url: "https://arxiv.org/abs/2609.11582" - id: "2609.11542" title: "Characterizing Job Power Elasticity for Power-Flexible AI Training" url: "https://arxiv.org/abs/2609.11542" - id: "2609.11829" title: "WFDroneBench: A Benchmark for Sensor Placement and Drone Routing for Wildfire Detection" url: "https://arxiv.org/abs/2609.11829" - id: "2609.11817" title: "Constrained Deep Inventory Management Using Forward-Backward SDEs" url: "https://arxiv.org/abs/2609.11817" - id: "2609.11788" title: "Optimal Recursive Composition and Dyadic Phase Laws for Gradient Descent with Predetermined Stepsizes" url: "https://arxiv.org/abs/2609.11788" - id: "2609.11784" title: "Constant Steps Are s-Composable: An Exact Interpolation Certificate for Gradient Descent" url: "https://arxiv.org/abs/2609.11784" - id: "2609.11751" title: "Stochastic Gradient Methods with Online Scaling" url: "https://arxiv.org/abs/2609.11751" - id: "2609.11509" title: "Extending SMT Solving with Non-Ground Clause Learning" url: "https://arxiv.org/abs/2609.11509" - id: "2609.11339" title: "InFlow: entropic stochastic dual dynamic programming with HJB cross-certification for seasonal energy storage" url: "https://arxiv.org/abs/2609.11339" - id: "2609.11273" title: "Minimax-Optimal Early Stopping for Continuous-Time SGD via the Discrepancy Principle" url: "https://arxiv.org/abs/2609.11273" - id: "2609.11207" title: "Convex Optimization with Nested Evolving Feasible Sets (CONES) under Time-Varying Loss Functions" url: "https://arxiv.org/abs/2609.11207" - id: "2609.11183" title: "Single-Loop Gradient Algorithms for Pessimistic Bilevel Optimization Problems" url: "https://arxiv.org/abs/2609.11183" - id: "2609.11105" title: "The best approximation tuple: an extension of the Cheney-Goldstein algorithm and results to the multiple sets case" url: "https://arxiv.org/abs/2609.11105" - id: "2609.10866" title: "Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations" url: "https://arxiv.org/abs/2609.10866" - id: "2609.10732" title: "Operator Splitting Methods with Online Scaling" url: "https://arxiv.org/abs/2609.10732" - id: "2609.11867" title: "AdamX: Cosine similarity meets gradient descent" url: "https://arxiv.org/abs/2609.11867" - id: "2609.11801" title: "Thinking with Looped Flows" url: "https://arxiv.org/abs/2609.11801" - id: "2609.11739" title: "LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation" url: "https://arxiv.org/abs/2609.11739" - id: "2609.11699" title: "Negative Self-Distillation: Learning to Reason by Avoiding Flaws" url: "https://arxiv.org/abs/2609.11699" - id: "2609.11655" title: "Musec: MomentUm SpEctral Clipping for Stable Muon-type Training" url: "https://arxiv.org/abs/2609.11655" - id: "2609.10980" title: "EGGROLL, Unrolled: Understanding and Improving Low-Rank Evolution Strategies at Scale" url: "https://arxiv.org/abs/2609.10980" faq: - q: "Why isn't offloading KV cache to SSD always a win?" a: "py-kvcache (arXiv:2609.11744) characterizes external KV caching for vLLM on NVMe SSDs: prefix caching does cut time-to-first-token for long-context requests, but with short prefixes or fast GPUs, recomputing the cache can beat loading it from external memory. Offload is not a free lunch — recompute waits halfway down the road. Cache-hierarchy design is shifting from 'bigger is better' to slicing by marginal gain." - q: "Why does low-bit post-training quantization usually not destroy the model?" a: "PTQ injects error into every quantized weight, so intuition says errors should accumulate with depth and wreck next-token prediction — yet low-bit PTQ mostly works. Why Does Post-Training Quantization Work? (arXiv:2609.11716) supplies the missing theory: an autopsy report for an engineering practice that ran three years ahead of its explanation, including where the protection fails." - q: "What is left to discover about gradient-descent step sizes?" a: "With the convergence-rate question for smooth convex optimization essentially settled by the recent long-step (silver-ratio) line, the frontier moves to composition — which step-size schedules combine, and how. The week's pair: optimal recursive composition characterized by dyadic phase laws (arXiv:2609.11788), and a proof that balanced constant schedules are s-composable at every horizon (arXiv:2609.11784), answering Grimmer-Shu-Wang's open question. The rate is fixed; what remains is combinatorics." - q: "What do Muon's stability problems look like?" a: "Muon, the serious challenger to Adam/AdamW, is prone to spectral flattening — training instability tied to its geometry. Musec (arXiv:2609.11655) applies momentum spectral clipping as the targeted patch. With Muon-type optimizers moving from academic curiosity into large-scale LLM training practice, stability patches are the industrialization paperwork on that road." --- Second digest of the week's 81-paper series: 24 papers on the machinery under the model — the inference stack (5), optimization theory and its industrial applications (13), and training methods (6). The common thread: the research object is expanding from "the model" to "the model plus its memory hierarchy plus its power supply," and every component is acquiring a cost accounting. (All papers are arXiv preprints, not peer-reviewed; claims follow their abstracts.) ## The serving stack: memory, not weights KV cache has displaced model weights as the central bottleneck of the inference stack. py-kvcache (arXiv:2609.11744) delivers the honest measurement of offloading it to NVMe SSDs in vLLM: prefix caching shortens time-to-first-token for long contexts, but past a crossover — short prefixes, fast GPUs — recomputing beats loading (see FAQ). OmniKVQuant (2609.11582) pushes KV quantization into omni-modal LLMs that ingest audio, video and text together, and finds text-scene recipes like TurboQuant failing there: numerical distributions differ enough across modalities that one parameter set cannot rule them all. Quantization gets its theory backlog cleared. Why Does Post-Training Quantization Work? (2609.11716) explains the long-observed grace period — why per-weight quantization error does not, in fact, accumulate into prediction collapse — and in doing so marks the boundary of when it would. Structured Transforms for Low-Overhead Quantization (2609.11687) takes the Kashin-decomposition route: weights split into two low-precision components plus structured orthogonal transforms whose convergence properties are stronger — with the acknowledged tax that orthogonal transforms are not cheap on GPUs; "structured" is precisely the fight over that overhead. The bit-count arms race is nearing physical limits; the theory ledger is starting to pay. ## Watts join the bill of materials Characterizing Job Power Elasticity for Power-Flexible AI Training (arXiv:2609.11542) moves the frame to the datacenter: LLM training is the fastest-growing source of electricity demand, and power capacity — not GPU supply — is becoming the first infrastructure bottleneck. The paper characterizes which training jobs can throttle down under grid stress and still make progress. Energy moves from operating cost to first-class design variable; expect this genre to multiply. ## Step sizes enter the jigsaw phase On smooth convex optimization, the rate question is essentially closed; the remaining work is combination. Optimal Recursive Composition and Dyadic Phase Laws (arXiv:2609.11788) gives the exact convergence picture for recursively composed long-step schedules; Constant Steps Are s-Composable (2609.11784) answers Grimmer, Shu and Wang's open question in the affirmative, with an exact interpolation certificate — balanced constant steps compose at every horizon. Preconditioning industrializes in parallel: Stochastic Gradient Methods with Online Scaling (2609.11751) and its operator-splitting twin (2609.10732) — covering forward-backward, Douglas-Rachford and ADMM — turn hand-tuned preconditioning into a learnable component. Early stopping gets its minimax-optimal rule via the discrepancy principle (2609.11273); CONES (2609.11207) handles feasible sets that keep shrinking while the loss keeps moving; pessimistic bilevel optimization gets single-loop algorithms (2609.11183); and the Cheney-Goldstein best-approximation machinery from 1959 is extended to multiple sets (2609.11105) — approximation theory's own health check. Off the gradient path, SMT solving learns non-ground clauses from conflicts (2609.11509), patching a structural blind spot in CDCL(T). The other half of the batch sinks optimization into worlds with real constraints: seasonal energy-storage scheduling via entropic SDDP with HJB cross-certification (arXiv:2609.11339, validation-first design because power grids forgive little), multi-product inventory with pathwise safety sets on stock, warehouse capacity and carbon quota via forward-backward SDEs (2609.11817), drone sensor placement and routing over dynamic wildfire fields (2609.11829), and certified lower bounds on worst-case risk for RL under adversarial state perturbations (2609.10866) — in safety-critical settings, "how bad is the worst case" must be provable, not estimable. ## Training methods find their geometry The six training papers share a temperament: no longer "new method, higher score," but "why does the method work, and when does it fail." AdamX (arXiv:2609.11867) uses cosine similarity between consecutive gradients as its adaptive signal — gradient-direction coherence itself carries the information about how large the update should be; Musec (2609.11655) patches Muon's spectral-flattening instability (see FAQ). Thinking with Looped Flows (2609.11801) names a structural debt of recurrent models: inference can loop as long as you like, but training backpropagates through only one or two updates — the training-inference asymmetry that any test-time-compute route inherits. Negative Self-Distillation (2609.11699) is a warning shot: on-policy self-distillation can leak privileged ground truth — the model learns to copy answers rather than reason — and turns that information around, training the model to recognize and avoid flawed reasoning paths. And low-rank structure shows its two faces: LOCUS (2609.11739) finds that doing preference alignment in a low-rank subspace trims verbosity — output length, i.e. the serving bill — without losing utility, while EGGROLL, Unrolled (2609.10980) dissects why rank-one perturbations in evolution strategies are geometrically harsh: every perturbation lands on a measure-zero subspace. ## The takeaway The machinery layer's narrative has flipped from performance to accounting: every paper prices something — the crossover point of an SSD offload, the boundary condition of a quantization guarantee, the watt-elasticity of a training job, the exact rate of a step-size schedule. When each component carries its own theoretical ledger, comparisons between methods stop being folklore and become engineering. > **Provenance & disclosure.** Originally published in Chinese on our WeChat channel on 2026-09-12 as three parts of a nine-part "arXiv 81" series ("量化为什么没压坏模型？5篇论文拆推理栈"; "GD的步长还能挖多久？13篇优化新论文"; "Muon还不稳？优化器军备赛的新答卷"); drafted with AI assistance under human editorial direction. Translated and consolidated to English on 2026-09-12 (AI-assisted, human-reviewed). All 24 cited arXiv IDs were resolved via the arXiv API on 2026-09-12 and titles matched the claims; the Grimmer-Shu-Wang and Cheney-Goldstein-1959 attributions verified against the abstracts. The silver-ratio background follows the column as relayed. Preprint caveat inherited. Translated commentary — not a SigPulse measurement; first-party numbers live in the [dispatches](/posts/) and the [/data/ ledger](/data/).