vLLM released version 0.28 this morning with 584 commits from 270 contributors. The headline items cover Kimi K3, DeepSeek V4, speculative decoding, a newer model runner, and a Rust front end. The feature I care about most is quieter: tiered KV cache offloading now includes disk. That sounds like
On August 24, 2026, NVIDIA announced that Groq 3 LPX, an interactive AI inference accelerator, is now in full production. On its face this is a data center product for AI factories, the kind of announcement that normally glides past everyone who builds AI on their own desk. Read it
Speculative decoding sounds like the rare inference optimization with a free lunch. Let a smaller component guess several future tokens, ask the target model to verify them in one pass, and commit the guesses that survive. The target model still controls the output, but generation can advance by several tokens
The obvious way to build a local AI setup is to buy the fastest GPU you can afford, load the best model that fits, and send every request to it. That design feels efficient because no expensive hardware sits idle. It is also how a background agent turns a useful
Two open weight releases landed within about a day of each other this month, and together they tell you more about where local AI is going than any benchmark chart. On August 11, Nvidia shipped Nemotron 3.5 Lightning, a 30 billion parameter mixture of experts model with 3 billion
On Tuesday, Microsoft shipped a fix for CVE 2026 24301, a Copilot vulnerability chain that Varonis Threat Labs named CoSnitch. The short version is ugly. A single crafted link, no click through, no confirmation dialog, could poison Copilot's persistent memory, auto execute prompts through the assistant's
On Wednesday, Marvell announced it had granted Google a warrant to buy up to 58.97 million of its shares, worth as much as $12.2 billion if fully exercised, as part of an expanded custom silicon deal running through 2033. Marvell stock jumped double digits. Broadcom, the incumbent king
On Monday, OpenAI published a post titled Pacing Model Development in an Era of Cyber-Critical Capabilities. Strip away the careful language and the substance is remarkable: they temporarily paused reinforcement learning training on their latest deployment bound models for two weeks to harden and red team their research infrastructure,
The numbers dropped this week and everyone is reading them wrong. New U.S. business applications hit 5.6 million in 2025, up 24 percent since ChatGPT launched, according to a Citadel Securities research note that made the rounds again Monday. Gen Z is leading the surge, and the detail
Over the weekend Bloomberg reported that Stripe has finalized a deal to acquire OpenRouter for more than $7 billion. That is over five times the $1.3 billion valuation OpenRouter raised at in May. Read that again. A company whose entire product is helping you switch between AI models, a
Pull up the gross margin line on almost any AI startup's P&L and you will find the same wound. Bessemer's February 2026 pricing playbook puts AI companies at 50 to 60 percent gross margins against 80 to 90 for traditional SaaS, and a16z lands
On Wednesday OpenAI replaced its chief revenue officer, Denise Dresser, after nine months on the job and handed the role to Dali Rajic, the president and chief operating officer of Wiz. The press release language was the usual polish about helping businesses realize the full value of AI. Founders should