NVIDIA's 1.9x Local AI Gain Is a Reason to Delay Your Next GPU Purchase

NVIDIA used IFA 2026 to announce up to 1.9 times faster local inference from new llama.cpp and vLLM optimizations. The improvements are available directly and through LM Studio and Ollama. The obvious reaction is that local models got faster. The more useful conclusion is that the economic life

ChatGPT Desktop Can Now Run Ollama Models. The Boundary Matters

Ollama published version 0.34.0 release candidate one on September 5 with a feature that would have sounded contradictory a year ago: ChatGPT Desktop can now use Ollama models directly. The setup is available through the Ollama app on macOS. A user can keep the ChatGPT interface while sending

SGLang 0.5.19 Makes Hardware Specific Recipes Part of the Product

SGLang released version 0.5.19 on September 4 with 786 pull requests from 214 contributors. The release adds models, kernels, cache changes, beam search, diffusion work, and optimizations across NVIDIA, AMD, Intel, Apple, and Ascend hardware. That volume is impressive, but it is not the part I find most

NVIDIA PAIR Is a Router, Not One Giant GPU

NVIDIA released Personal AI Router, or PAIR, on September 3. It connects compatible Windows, Linux, and macOS systems on the same network, discovers participating inference nodes, and gives applications familiar Ollama compatible and OpenAI compatible endpoints. Independent requests can then move to an eligible machine according to engine availability, model

A Three Line llama.cpp Fix Explains Why Two GPUs Are Not One Bigger GPU

A llama.cpp change merged this morning removed one guard, added one explicit device selection, and enabled an existing CUDA graph optimization on systems using more than one GPU. The code change was three lines added and one removed. The result was not a universal speedup. It improved token generation

The Browser Is Becoming a Local AI Runtime, Not Just a User Interface

Hugging Face released a library on September 1 that loads and runs optimized WebGPU kernels from the Hub. The first collection includes 207 kernels, each packaged with an interface, shader templates, correctness cases, benchmark cases, and usage instructions. It also introduced Fleet, a browser tool that tests those kernels on

The Best Part of the New DeepSeek Model Is That It Ships Two Serving Paths

DeepSeek published the weights for DeepSeek V4 Flash Vision Exp on August 31. The headline is a 305 billion parameter experimental vision model under the MIT license. DeepSeek reports stronger multimodal agent results than its text model on its own evaluation setup, while keeping comparable text agent performance. The more

The Most Important Part of an Inference Release Is the Known Issues Section

NVIDIA published TensorRT LLM release candidate 25 on August 31. The headline feature is a new KV cache manager enabled by default across a long list of current models, including DeepSeek V4, GPT OSS, Kimi K3, Qwen 3.8, Gemma 4, and GLM 5. NVIDIA calls the new manager the

llama.cpp Just Tuned Two M3 Chips Separately. Local AI Buyers Should Notice

Two small llama.cpp releases landed today with a useful lesson for anyone buying a Mac for local AI. Build b10696 added Metal fa vec tuning for the M3 Pro. Build b10697 added separate tuning for the M3 Ultra. The underlying pull requests, 27963 and 27999, each supplied a tuning

Six Billion Active Parameters Can Still Demand a Huge Machine

Qwen opened the weights for Qwen3.8 Flash Next this week, and the headline number looks made for local AI: only 6 billion parameters activate for each token. That sounds like a small model. It is not. Qwen says the main mixture of experts model contains 125 billion parameters, plus

Your Local AI Benchmark Is Missing the First Minute

SGLang released version 0.5.18 this week with a startup optimization that deserves more attention than another tokens per second chart. The server can now stage checkpoint pages from storage while it captures CUDA graphs. In the project's Qwen3 32B test on an H100, the overlapped path

A Local Model Is Not a Local Agent: Perplexity Just Moved the Whole Runtime Onto DGX Spark

Perplexity launched Portable Computer on NVIDIA DGX Spark this week. The obvious headline is that its AI agent now runs locally. The important detail is what Perplexity means by locally: the orchestrator model, subagent model, agent harness, and execution environment all run on the machine. This is not a cloud