The small-model shift is here: cheap, fast, and deployable

For the past two years, the AI narrative has been dominated by frontier models—400-billion-parameter behemoths that cost dollars per task and require datacenter-class hardware. But in the last month, two developments have made it clear that the small-model segment has crossed a threshold that matters for builders.

First, users experimenting with OpenAI's GPT‑5.6 Luna and GLM 5.3 report that these models run at roughly 100 tokens per second and cost tens of cents for complex, multi-step tasks—searching thousands of emails, generating personalized code reviews, assembling daily news digests [Source: Calvin French-Owen]. That cost structure changes the economic equation for any product that wants to embed AI. As one investor quoted in the post put it: "It's weird we're not seeing more consumer AI companies." The answer, until now, was token costs. A daily personalized news site might cost $1 per user per day with a large model. At $30/month, you'd need to deliver value equivalent to The Economist. With Luna, that same task costs cents, putting consumer AI within reach of subscription models that actually work.

Second, a NATO-backed startup called Scaleout Systems demonstrated this week that small models can run on autonomous drones for battlefield target identification and engagement [Source: Ars Technica]. Scaleout uses federated learning to train models on edge hardware—forward bases and drone computers—without relying on cloud APIs. The models are lean enough to run on embedded devices, yet capable enough for computer vision tasks like recognizing military vehicles and selecting targets. This is a concrete example of small models enabling low-latency, offline AI in hardware-constrained environments.

These two stories are connected by the same underlying trend: the small-model Pareto frontier has shifted. While large models like Multiverse Computing's Quasar 438B (400B+ parameters) still lead on composite benchmarks like the AI Intelligence Index (scoring 43 vs. Luna's likely lower score), they are overkill for many real-world tasks [Source: Multiverse Computing]. Quasar is fast for its size—500 tokens in 15.3 seconds—but it still requires significant infrastructure. Small models like Luna and GLM 5.3 offer 80–90% of the capability for 1–10% of the cost, and they run on a single GPU or even CPU.

For engineers and founders, the implications are concrete:

• Inference costs are no longer a blocker for consumer apps. You can now build a personalized, AI-powered product with a per-user cost of pennies per day.
• Edge deployment is viable for real-time, high-stakes tasks. Drones, cars, medical devices—anywhere latency or connectivity is a concern.
• The large-model gap is narrowing on practical tasks. Many coding and research workflows that previously required a frontier model can now be handled by small models.

The small-model shift isn't a revolution. It's an economic and practical evolution. But for anyone building with AI, it's the most important change this year.