October 24, 2026 · evening edition

Capacity is a path, not a number

Open models are stretching beyond memory limits, but storage, speculative decoding, context reuse and failure controls now determine what usable capacity means.

News cutoff: Oct 24, 2026, 1:40 AM Pacific · Published Oct 24, 2026, 2:05 AM Pacific

The take · AI-authored analysis

The new local-AI frontier is controlled degradation.

Open models are learning to exceed the machine they run on: Unsloth can park MoE experts on SSD, a DeepSeek quant streams 110.6GB Engram tables from disk, and Apple, AMD and Intel tests combine drafts, tensor parallelism and memory mapping to keep oversized systems moving. But capacity is not a checkpoint-size claim. One Unsloth user watched a long tool session fall from about 15 to 2 tok/s after repeated context processing; another community post warns that sustained SSD reads can cause controller relocation writes. danielhanchen ↗ · Loginhe ↗ · Hyungsun ↗ · brainchillzZ ↗ · particle ↗ · Chromix_ ↗

The caveat is that most performance numbers are single-machine or publisher reports and compare engine-plus-checkpoint stacks, not isolated software. The implication: measure the whole path—storage behavior, context replay, draft acceptance, device topology, rollback and security policy—because “it fits” only matters if it stays responsive and recoverable. Hyungsun ↗ · Standard_Alloy ↗ · danielhanchen ↗

The caveat: Storage-backed experts and aggressive speculation expand what can run locally, but they also add bottlenecks and failure modes that peak throughput numbers do not capture.
What to watch: Benchmark the complete inference path and its degradation under long contexts, concurrency and recovery—not just whether the weights load.

X / Twitter

OS Twitter / X Recap

Collection scope

Bounded logged-out reads returned supported payloads for 52 of 53 configured profiles. x-eliebakouch failed with no supported logged-out payload; this is a first-page subset, not a complete timeline or replies.

Inference systems and open weights

Muse Spark pairs sparse attention with serving claims Artificial Analysis describes an 816B-total, 42B-active MoE with 46 sliding-window layers, 75% less KV cache at 1M context, and tuned SGLang launch profiles. Throughput remains provider-reported. @ArtificialAnlys ↗
Kimi K3.1 Pro scores well before weights arrive Artificial Analysis reports 64% on its cyber benchmark at $2.81 per task. The API is available, but the post says weights will come later, so this is not yet an open-weight release. @ArtificialAnlys ↗
MiniMax M3.1 Flash reaches Tenstorrent Cloud MiniMax says its open weights now run on Tenstorrent’s Tensix hardware with Ethernet scale-out and an open-source software stack. This establishes an integration path, not independent performance evidence. @MiniMax_AI ↗
Mirendil publishes a private-code compression result Baseten says Mirendil-Base-Large achieved the lowest bits-per-byte result among tested models on a private-code evaluation and announced an inference partnership. The post does not expose the test set. @baseten ↗

Reddit Recap

Collection scope

Fourteen configured communities were collected with required pacing. Thirteen returned bounded first-page payloads after two permitted cooldown retries; r/oobabooga returned no in-window items. Results are community reports, not controlled lab reproductions.

The engine is only one variable

Five Apple engines diverge by model and build On an M3 Max 36GB, one tester measured 41.3 tok/s for dense Qwen3.8-27B in Kapitsa, while oMLX reached 122.9 on Qwen3.6-35B-A3B. Checkpoint and speculation changes confound engine-only comparisons. Hyungsun ↗
Four R9700 cards favor one TP4 topology A detailed Radiance test reports 236 tok/s at concurrency one and 1,192 at 16 for Qwen3.8-Flash-Next; TP4 beat two TP2 groups by about 45% at equal streams. Single-system result. brainchillzZ ↗
Two Arc Pro B70s sustain a long agent workload A Windows user reports 1.68M tokens across 1,030 Qwen3.8-27B Q8 requests at 51 tok/s average. They also trace a 170B-per-launch oneAPI leak to shared SYCL contexts and describe a workaround. Standard_Alloy ↗
DeepSeek’s Engram tables become a separate storage problem A quantization project publishes 1.75–2.25 bpw DeepSeek V4.1 Flash backbones plus a shared 110.6GB Engram file for SSD streaming. Its quality and single-GPU claims are author-reported; llama.cpp support is pending. Loginhe ↗
A Qwen blend trades output length for a small task loss A community consolidation of JetBrains results says a 27B blend solved 37 of 100 tasks versus 39 for Qwen3.8 while using 71% fewer output tokens on 30 shared tasks. Secondary aggregation. Evening_Hawk_7470 ↗

Useful experiments with explicit limits

Turning small dense models into MoEs mostly hurts quality An RTX 4060 experiment reports 41–45% worse perplexity for a converted Qwen2.5-1.5B and 13–15% worse for Qwen3-0.6B, with little performance benefit. The author publishes the negative result. Bayliner1980 ↗
Read-heavy SSD inference can still provoke writes A community analysis notes that NAND read disturbance can trigger controller relocation writes, so “read-only” expert streaming is not necessarily write-free. It asks users to measure SMART counters instead of asserting a drive-life estimate. Chromix_ ↗
INIT puts mutation behind immutable controls An Apache-2.0 Rust agent can edit and hot-reload its own code while retaining a root policy, sandbox, signed audit log, snapshots and rollback. Its author explicitly offers no evidence that self-modification improves small models. No_Skill_8393 ↗

Small interfaces, alpha channels and slow video

ComfyUI gets a phone-first control surface ComfyUI Mobile Pocket exposes forms at /pocket/ and a touch graph at /pocket/canvas. Its author reports panning rising from 16 to 70 fps on a Samsung A54; Android is the only tested platform. Deneme123deneme ↗
MOSS-Image gets a small alpha-channel LoRA A proof-of-concept LoRA trained on 39 images for about 40 minutes on one 16GB GPU produces RGBA images with real transparency. The author shares assets and warns that artifacts remain. gatortux ↗
An 8GB M1 Air completes a local LTX-2.5 clip A user reports generating a 36-second, 448×256, 24fps video with distilled LTX-2.5 22B Q4 in about 32 minutes at six steps. It demonstrates feasibility, not interactive speed. iJohnnyRobert ↗

discord

Discord Recap

Collection scope

Dedicated OSNews collector captures from EleutherAI, Hugging Face, Nous and Unsloth were collected within the approved 14-hour limit with true capture times, cloudBrowserRead:false and no personal account. These are bounded server/channel views.

Projects, proposals and friction

ConspiracyBench moves from dataset to adversarial testing EleutherAI’s research channel describes a roughly 40-contributor project testing whether models endorse conspiracy theories, with an initial leaderboard and a proposed hill-climbing phase. This is a community project update. likelytobelaura [AI], Server Tag: AI AI ↗
A 99-parameter super-resolution model appears on Hugging Face A project author shared microSR-x2, an intentionally tiny image upscaler with code and weights. The collected Discord post supports the 99-parameter claim but contains no comparative image-quality evaluation. nilmer ↗
A long tool session exposes context-replay cost An Unsloth user reports Qwen3.6-35B GGUF falling from about 15 to 2 tok/s after repeated tool calls, with multi-minute pauses. The suspected full-context reprocessing remains unresolved. particle ↗
Nous sketches adaptive local model fitting A Nous moderator describes TurboFit scanning hardware, configuring Hermes, discovering peer devices and temporarily swapping to smaller models under load. SSD-streaming support is being optimized; this is a work-in-progress proposal. sovthpaw [NOUS], Server Tag: NOUS NOUS Moderator ↗

Original sources

Primary-Source Recap

Collection scope

Official GitHub release feeds and project posts were used for implementation details. Release notes establish shipped changes, while performance figures and security descriptions remain attributable to their publishers.

Unsloth expands local capacity—and hardens the path

MLX can leave oversized MoE experts on SSD Unsloth’s beta says MLX can keep excess experts on disk, allowing Qwen3.8-Flash-Next to fit a 32GB Mac at lower response speed. The Desktop path is currently chat-only. danielhanchen ↗
MOSS-Image and Flint join the supported model set The same release adds MOSS-Image 5.1B for 16GB GPUs and Flint 40B, a 2.5B-active MoE that fits 24GB at 3-bit. Performance and quality comparisons are publisher claims. danielhanchen ↗
Certificate and local API boundaries tighten Unsloth now rejects compromised eMudhra emSign certificates in core installs, stops sending local API keys through plaintext HTTP proxies, and blocks keyed requests from unsandboxed Python or terminal execution. danielhanchen ↗

Runtime release details

LiteLLM changes its release-signing key LiteLLM 1.103.8 warns that old cosign policies will fail, publishes the replacement key, updates OpenTelemetry compatibility and removes the eMudhra root from bundled certificates. CI and admission policies need updating. yuneng-berri ↗
llama.cpp adds Soprano text-to-speech Build b11850 adds Soprano 80M/1.1 TTS through a Qwen3 backbone and Vocos decoder, including conversion and pretokenizer support. Generation currently emits a full utterance as one chunk. github-actions[bot] ↗
Image resizing now matches torchvision more closely Build b11851 changes LFM2/d1-omni resizing to integer-weight behavior matching torchvision, correcting small pixel differences in roughly 1% of bilinear and 0.25% of bicubic crop values. github-actions[bot] ↗
Concurrent model downloads stop fighting each other llama.cpp b11848 adds coordination for downloads across processes, reducing corruption and duplicate work when multiple clients request the same model artifact simultaneously. github-actions[bot] ↗
RDNA3 and RDNA4 get a bf16 WMMA build fix llama.cpp b11847 corrects the HIP bfloat16 WMMA operand vector type for newer AMD architectures, addressing compilation failures rather than changing model numerics or benchmark performance. github-actions[bot] ↗
Ollama preserves literal Qwen function tags Ollama 0.40.11 release-candidate notes preserve literal function closing tags in Qwen3Coder parameters, allocate context for OpenCode and adjust vllm-d3 state handling alongside Phonon voice updates. github-actions[bot] ↗