The Small Model Revolution Is Happening Where You’re Not Looking
Adebayo Alonge was in a Cape Town hotel room in 2019, about to demo his startup’s pill-scanner. The RxScanner uses infrared spectroscopy to identify counterfeit medication — a technology that could save thousands of lives across Africa. The demo should have taken seconds.
It took five minutes per scan.
The data center was 14,000 kilometers away in the US. The bandwidth couldn’t handle it. So Alonge’s engineers spent two hours shrinking the AI model down to something that could run entirely on an Android phone. It worked. The demo was saved. And Alonge became an evangelist for “small AI.”
This story — reported by IEEE Spectrum’s David Berreby — is the best parable I’ve read this year about where AI actually needs to go.
The Mega-Trend That Misses Most of the World
While Silicon Valley is building billion-parameter models, trillion-dollar data centers, and debating whether GPT-7 is sentient, the World Bank quietly dropped a number worth sitting up for: only 0.7% of internet users in the world’s poorest countries have ever used ChatGPT. In the most developed nations, it’s 25%.
World Bank president Ajay Banga said it straight at Davos: “Most people are discussing AI from the LLM/generative side. But that needs a lot of computing power, electricity, massive data, and skilled people to manage it. Outside the developed world, other than maybe India and China, very few countries have that combination.”
He’s right. And the implication is uncomfortable for anyone who thinks the AI story is just about scaling up.
Small Isn’t a Compromise
The counterargument is obvious: bigger models are more capable. Benchmark scores go up with scale. Reasoning improves. Emergent abilities appear. Why settle for less?
Here’s why: the marginal capability gain from a 500-billion-parameter model over a 3-billion-parameter model is irrelevant if the big model can’t run on the device in your pocket. A model that doesn’t work is infinitely worse than a model that does 80% of what GPT-7 does but runs on a $100 phone with no internet.
Farmers in India are already getting soil analysis from small models running on drones. Pharmacies across Ghana and Kenya are authenticating medication with on-device AI. These aren’t pilot projects — they’re production systems.
The Corporate Side of the Same Coin
There’s another article making the rounds today on Hacker News that hits the same note from the enterprise angle. Unmeshed’s piece argues that “LLMs Are Not a Default Execution Engine” — that the most expensive prompt is the one that never needed to exist.
It’s a corporate blog from a company selling AI governance tools, so take the framing with salt. But the core observation is solid: too many teams start with “let’s put an LLM on it” before asking whether the problem needs intelligence at all. Sometimes a lookup table wins. Sometimes a rules engine wins. Sometimes a tiny model wins.
The counter-counterargument: sure, but enterprises will figure this out organically as the hype cycle cools. That’s probably true for well-resourced teams in San Francisco. It’s less true for the rest of the world, where every wasted API call is a real cost.
The Bet That’s Actually Interesting
The industry narrative has been about scaling — more parameters, more data, more compute. That narrative serves Nvidia’s stock price and the hyperscaler data center buildout. But it doesn’t serve the 5 billion people whose internet experience is “sometimes” and “if the power stays on.”
The bet that actually matters isn’t about whether we can build AGI in a warehouse full of H100s. It’s about whether we can make AI useful enough to run on a phone, on a farm, in a clinic without a server rack.
Two stories, same conclusion from different directions: the next phase of AI isn’t about getting bigger. It’s about getting smart enough to know when small is the answer.
Sources: IEEE Spectrum — Small AI Models Gain Traction Around the World, Unmeshed — Using AI Wisely Starts Before The First Prompt, World Bank Report on AI Adoption