The deployment loop starts learning
Kernel agents, search-time retrievers, serverless training loops and local benchmarks move adaptation into the deployed stack.
The deployment loop is starting to learn.
The deployment loop is starting to learn. Together says agents now retune kernels for a model’s hardware and traffic; ARI turns reasoning preferences into search-time retriever updates; Baseten packages serverless SFT and RL beside production serving. The runtime layer is moving too: llama.cpp adds Kimi K3.1 Pro support and more Hexagon fusion, while LiteLLM’s LTS line tightens cost tracking and credential handling. Community evidence supplies the friction that launch posts omit. One five-model Qwen test finds equal tool-calling accuracy across different sizes, another user reports newer quants failing calls, and mixed-GPU builders show why whole-system measurements matter. The caveat is broad: speedups are vendor or single-author claims, five Reddit communities rate-limited, X reads were bounded, and no eligible Discord capture was available. Adaptive infrastructure only matters if its feedback loop is inspectable and reproducible. @togethercompute ↗ · @dianetc_ ↗ · @baseten ↗ · github-actions[bot] ↗ · FaithlessnessFar9647 ↗ · maddinek ↗
- The caveat: Speedups remain vendor or single-author claims; five Reddit communities rate-limited, X reads were bounded and no eligible Discord capture was available.
- What to watch: Treat adaptive inference as a feedback system: inspect the optimizer, workload, hardware, failure modes and rollback path before accepting headline gains.
X
Collection scope
Twenty-seven in-window posts came from bounded logged-out profile samples. Replies and full timelines were unavailable; performance and benchmark claims remain attributed.
Infrastructure begins adapting in production
- Together assigns agents to kernel optimization Together says its compiler tunes kernels for a model, hardware and traffic shape, reporting 12–20% median serverless TPS gains and lower production latency. @togethercompute ↗
- ARI updates a retriever during search Its authors derive gradients from a reasoning model’s ranking preferences, claiming a weak 400M cross-encoder improved held-out Recall@100 by as much as sevenfold. @dianetc_ ↗ · @lateinteraction ↗
- Baseten moves serverless training loops into GA Loops combines SFT and RL, pay-per-token training, always-warm sampling and one-click checkpoint deployment while promising numerical checks against training-serving mismatch. @baseten ↗
Optimization claims acquire workload context
- DeepSeek serving gains come from a stack of changes Artificial Analysis attributes GB300 improvements to vLLM kernels, FlashInfer autotuning, CUDA graphs and tuned speculation, reporting 848 tokens per second for one agent. @ArtificialAnlys ↗
- Accent changes the TTS leaderboard Artificial Analysis places Qwen-Audio-3.1-TTS-Plus first for US English but HeyGen Voice ahead for UK English and overall, underscoring evaluator dependence. @ArtificialAnlys ↗
Primary and business sources
Collection scope
Ten GitHub releases, nine RSS items and one Hugging Face registry record fell inside the window. Registry metadata alone was not treated as a launch.
Runtimes expose more of the machine
- llama.cpp adds Kimi K3.1 Pro’s sparse attention path Build b12180 reuses the K3 graph while adding 24 lightning-indexer layers and an additive cache mask; non-flash paths remain correct but expensive. github-actions[bot] ↗
- LiteLLM’s LTS line tightens credentials and accounting Version 1.108.2 fixes empty OpenAI keys falling back to an admin key and corrects Bedrock service-tier handling; signed images remain verifiable with cosign. litellm-release[bot] ↗
- Hexagon keeps a residual path inside HVX registers Build b12181 fuses post-norm residual addition into RMS_NORM_MUL and adds a dedicated fused opcode, reducing intermediate movement while simplifying fusion logic. ebateni ↗
- Hexagon generalizes its tiled convolution path to 3D Build b12179 folds depth and kernel height into a virtual row, reuses the HMX implicit-GEMM path and prevents a 256-entry DMA queue overflow. github-actions[bot] ↗
- Invalid UTF-8 no longer crashes the llama.cpp CLI Build b12177 hardens prompts and file inputs against malformed byte sequences, a small correctness fix with direct relevance to automated tool pipelines. github-actions[bot] ↗
Adaptive systems widen the operational perimeter
- Hundreds of Artex servers were reportedly exposed TechCrunch says researchers found publicly reachable installations of an AI penetration-testing tool, making deployment hygiene part of the security model around offensive agents. Zack Whittaker ↗
- Crusoe sells a troubled Wyoming data-center project TechCrunch reports Google as the buyer, using the transaction to illustrate permitting and execution risk behind the physical infrastructure promised to AI workloads. Marina Temkin ↗
Collection scope
Nine communities returned bounded anonymous evidence: r/DeepSeek, r/LocalLLaMA, r/LocalLLM, r/MachineLearning, r/OpenWebUI, r/Qwen_AI, r/SillyTavernAI, r/StableDiffusion and r/unsloth. Five rate-limited; community measurements remain unreproduced.
Local measurements reveal tradeoffs hidden by launch numbers
- Five Qwen configurations tie on one tool-calling test A 50-question BFCL V4 run on an RTX 4090 found three models at 70% accuracy despite size and speed differences; methodology is author-controlled. FaithlessnessFar9647 ↗
- A newer quant reportedly fails tool calls more often One user says Qwen Flash Next v3.1 IQ4_XS sometimes stops after malformed calls under Strata and two harnesses; the author has not isolated the cause. citrusalex ↗
- A mismatched GPU box parallelizes routed experts A Strata fork splits DeepSeek V4.1 Flash across Radeon, Arc and CPU, reporting 25.57 tokens per second versus 9.89 for a different stock configuration. maddinek ↗
- A Rust engine pages MoE experts into 16GB Its author runs Qwen3.6-35B-A3B on an RTX 5080 by keeping 64% of experts resident, reporting 38–53 tokens per second and KL 0.0079. External_Bat2889 ↗
Builders turn local models into whole systems
- zml/llmd targets one server across many accelerators The project advertises continuous batching, prefix caching, speculative decoding and automatic tensor or expert sharding across NVIDIA, AMD, Intel, TPU and Metal. siwu ↗
- Lamplight documents a 95% local workflow Its builder describes splitting inference, storage and interactive work across Apple and small-form-factor machines, emphasizing routing and operational lessons over a packaged release. CodeSamurai ↗
- A 2-bit DeepSeek build exposes long-thinking failure modes The author reports roughly 50 tokens per second with 120GB RAM, while unbounded 64K tests consumed 60,000 thinking tokens without answering. Ryyn_- ↗
Discord
Collection scope
No eligible recent capture was available from the approved OSNews-only account for Hugging Face, Nous Research, Unsloth or EleutherAI. No Discord coverage is claimed.