Speed is a systems metric
Open AI performance is moving from raw decode rates to the full path through storage, routing, task completion, evaluation and authority.
Useful performance belongs to the whole system.
The most useful local-AI benchmark is no longer tokens per second. This window’s strongest work treats speed as a system property: Betelgeuse pages Qwen experts through VRAM, RAM and NVMe without pruning them; another experiment measures the quality tax of routing shortcuts; llama.cpp adds narrower kernels and chat-format fixes; creative tools turn model graphs into governed applications. Discord builders reach the same conclusion from different angles, comparing time-to-answer, approval boundaries and evaluation rather than raw decode rates. The caveat is evidence quality: nearly every performance result is self-reported, four Reddit communities rate-limited, and release feeds were quiet. The implication is practical: evaluate the whole path—from bytes and routing to task completion and authority—because a faster counter can still produce a slower or less trustworthy system. jesdga95 ↗ · zyxciss ↗ · TumbleweedNew6515 ↗ · github-actions[bot] ↗ · @ostrisai ↗ · Murphino [COS], Server Tag: COS COS ↗ · Autonomous Cybernetic Engineer [GPU], Server Tag: GPU GPU ↗
- The caveat: Nearly every performance figure is self-reported; four Reddit communities rate-limited, and official release feeds were quiet.
- What to watch: Measure bytes, routing, completion quality and approval boundaries alongside tokens per second.
X
Collection scope
Thirty-five in-window posts came from bounded logged-out reads. One substantive original post is selected; its 4K/computational-equivalence claim is the author’s experiment, not an independent benchmark.
Compression changes the production budget
- Primal VAE pushes MOSS Image toward native 4K Ostris reports an overnight 2×1 Primal VAE experiment producing 4096×4096 images at the compute cost of 1024×1024 MOSS Image generation; quality and training stability remain author-reported. @ostrisai ↗
Primary sources
Collection scope
Official release feeds were comparatively quiet. These two llama.cpp build releases are primary records; release-feed timestamps establish publication, while their narrow changes do not imply a broader product release.
Runtime edges keep getting narrower
- llama.cpp adds an OpenCL FWHT path Build b12020 adds a fast Walsh–Hadamard transform implementation to the OpenCL backend and publishes the project’s usual cross-platform artifacts and attestations. github-actions[bot] ↗
- Qwen3-Coder thinking output gets normalized Build b12019 accepts both Qwen3-Coder think-block variants, then strips the block when reasoning_format is none, tightening chat-format behavior rather than changing model weights. github-actions[bot] ↗
Collection scope
Ten of fourteen communities returned bounded anonymous archive evidence. r/cline, r/ComfyUI, r/ollama and r/Oobabooga rate-limited. Posts and measurements are individual author reports, not community consensus or reproduced benchmarks.
Inference exposes its hidden budgets
- Routing shortcuts carry a measurable quality tax A Basalt developer’s Qwen 3.8 experiment found expert bias and tail skipping raised throughput but reduced factual and chained-reasoning scores, especially with a small expert cache. jesdga95 ↗
- Betelgeuse pages experts instead of dropping them Its author reports 22 sustained tokens per second for a 68 GB Qwen quant on an RTX 3060 plus 16 GB RAM, using VRAM slots, a RAM cache and io_uring. zyxciss ↗
- A three-box DeepSeek stack publishes its bottlenecks Two DGX Sparks handle tensor-parallel inference while a 3090 server serves Engram tables and speculative drafts; the author reports 2,618 prefill tokens per second at 256K context. TumbleweedNew6515 ↗
- Ternary Bonsai gains native ik_llama.cpp paths A guide documents lossless TQ-to-PQ repacking, merged PQ2_0 CUDA kernels and runtime CPU repacking, addressing the fork’s earlier block-size and sign-tensor incompatibilities. Any_Bus9205 ↗
- A 4090 firmware unlock gets workload measurements One Windows test compares an unofficial BF16 unlock against stock 4090 and 5090 runs across ComfyUI workloads; gains vary, and the author cautions warranty-expired hardware. devmoching ↗
Agents become governed applications
- Model voting helps route uncertain decisions Across 2,850 labeled decisions, one author’s five-open-model panel nearly matched Opus on unanimously shipped cases but trailed overall; disagreement was most useful as a human-review trigger. PomegranateAble4339 ↗
- Blomma makes approval a local agent primitive The macOS beta pairs Gemma 3 12B with a separate 4B judge, typed SQLite memory and explicit approval before retention or outbound traffic; all inference runs through local Ollama. SherifAyman ↗
- Video search becomes a terminal tool for agents A new command-line project lets humans or terminal-only agents locate moments by describing what happens in a video, turning visual inspection into a scriptable retrieval step. pikadhu ↗
- A browser-local model interface draws early interest An r/LocalLLaMA post recaps Opera’s proposed WebAI interface for websites and extensions to invoke local browser models; standardization and cross-browser support remain claims in the linked announcement. JChataigne ↗
Creative stacks hide graphs behind workflows
- PotionUI turns model graphs into shareable forms Version 0.0.18 presents local image, video and music models through task-specific forms, with accounts, GPU limits, remote execution and approval-gated MCP changes under GPL-3.0. 0roborus_ ↗
- KRC NodePack adds a local workflow catalog The ComfyUI pack stores model metadata, LoRA stacks, prompts and switchable configurations in a local database, with import, backups, path healing and drag-and-drop media slots. riceandnori ↗
- Hidzuki moves character chat onto the desktop The MIT-licensed Python companion imports JSON personas, exposes a local control panel and connects to OpenAI-compatible endpoints, including Ollama, while animating nine expression states. Pathetic0777 ↗
- Toska gives an agent undoable brush actions The Krita fork lets a constrained painting agent choose brushes, textures and layers, score each pass and undo mistakes; local OpenAI-compatible models can replace the author’s hosted setup. yawk-ra ↗
Discord
Collection scope
Fresh verified OSNews-only captures covered Hugging Face, Unsloth, Nous Research and EleutherAI within the approved 14-hour window. The cloud reused true capture times with cloudBrowserRead:false; each channel is a bounded rendered sample.
Training questions move from scale to control
- A 1B scratch model struggles to hold identity An Unsloth member reports pretraining Freedream on roughly 20B tokens, then finding mixed-data SFT caused name drift while identity-only LoRA damaged chat formatting; remedies remain an open question. Digilidaahz ↗
- Time-to-answer beats tokens per second In an Unsloth discussion, Murphino argues newer models may emit more slowly yet finish tasks faster with fewer tokens, so output quality and completion time should precede decode-rate comparisons. Murphino [COS], Server Tag: COS COS ↗
Authority boundaries become product features
- ACE separates repair generation from approval Its developer says ACE reproduced a provider-timeout defect, proposed a bounded repair and passed 18 tests, while an independent controller retained authority to accept the change. Autonomous Cybernetic Engineer [GPU], Server Tag: GPU GPU ↗
- AgriMarket AI combines agents with public data The open-source Morocco-focused platform connects LangGraph workflows to FAOSTAT and NASA POWER data, Flask APIs and a Next.js interface; its builder is seeking stronger reliability evaluation. Mohammed | AI Engineer ↗
- Hermes Agent gets a Cadence plugin proposal A Nous developer shared a pull request for activity tracking and streak counters, extending Hermes Agent through a plugin rather than a core release. Jon Komet [NOUS], Server Tag: NOUS NOUS ↗
- A continuous-learning prototype claims low forgetting An EleutherAI research-channel builder says a rule-free inference approach approaches near-zero forgetting and plans a repository update; the claim has no benchmark evidence in the captured message. Kristrodamus ↗