Why The Need For AI R&D Is Greater Than Ever
Charles Yeomans is the chairman and founder of Atombeam.
gettyBack in January, an eternity in AI time, the Wall Street Journal published a piece by reporter Kate Clark titled “These Billion-Dollar AI Startups Have No Products, No Revenue and Eager Investors (paywall).” It described a new class of startups, dubbed “neolabs,” that “give priority to long-term research and developing AI models over immediate profits.” Billions of dollars are flowing into pure research, years ahead of any product, at valuations that would have seemed unserious a decade ago.
Reasonable people diverge on what this means. To some it is exactly how transformative technology gets built, and the upside justifies the price. To others, it looks like the late 1990s, when unbounded optimism about research paid for a great deal of pain. Both readings miss what I think is the useful question, which is not whether AI research deserves this much capital but which research programs do.
Ashu Garg, a general partner at Foundation Capital, put the standard sharply in Clark’s article: “The technical chasm to cross for each of these neolabs is very substantial and I think that risk is very real. The vast majority of them will not cross that at all. They will end up with something that is just incrementally better. And if you’re incrementally better than alternatives, you don’t matter.”
Garg’s point is the key. Incremental improvement is valuable, and most engineering progress is made of it. But incremental improvement has never aligned with stratospheric funding rounds, and it never will. The rounds only make sense if the research is aimed at something structural.
Something structural is available, and it is hiding in plain sight.
Large language models are powerful, and we are still early in discovering what they can do. But nearly all of them rest on a transformer architecture whose limits are not bugs awaiting patches; they are properties of the design. A transformer is frozen the day its training ends, and it learns nothing from the year it spends deployed at your company, unlike a human employee. It computes how plausible a sequence of words is, and plausibility is the only quantity it has; it contains no second quantity meaning true, which is why a fluent wrong answer is not a malfunction. And its cost per answer is constant by construction: the trillionth query costs what the first one did, and the model is no wiser for having answered it.
Problems that are properties of a design do not yield to tuning. They yield to different designs, and different designs are precisely what serious R&D exists to produce.
The clearest signpost for what a different design should do is the one every reader carries around. A human brain runs on roughly 20 watts, about the draw of a dim light bulb, and the reason it can afford to is that it learns. A person doing a job does not solve every problem from scratch; experience accumulates, the common cases become nearly free and attention is spent only on what is genuinely new. Learning is not just how we get better at our jobs. It is how we get cheaper at them, too. An AI that actually learned, permanently, from its own operation, would follow the same curve: more capable with every day in service and less expensive with every question answered because most of each answer would already exist. That is what makes today’s arrangement so strange. We have built systems that must be told everything, remember nothing and pay full price for every answer forever, and then we express surprise at the power bill.
Consider how strange the current situation is by the standard of computing history. For more than five decades, innovation in computing followed one rule so reliably that we stopped noticing it: Each generation did more, faster, for less. Moore’s Law was the famous case, transistor counts doubling every two years at little added cost, but the same curve ran through storage, networking and everything built on them. Every major computing technology we have ever adopted got cheaper per unit of work as we used it more.
Transformer AI inverts that. It is the first mainstream computing technology whose economics worsen with adoption: The more we use it, the more compute, power and capital it demands, with data centers whose power draw is now routinely compared to that of small cities. We have never had a technology that grows more expensive per unit of value the more we rely on it, and we have never asked existing infrastructure to absorb anything like it. Whatever else this is, it is not sustainable, and no amount of tuning makes it so.
Which brings us back to the neolabs and the capital pouring into them. The R&D that deserves funding at this scale is the kind aimed at the structural limits, and the good news is that we will not have to take anyone’s word for whether it has succeeded because the tests are concrete. Does the cost of an answer fall with use instead of holding constant? Does the system keep learning after it ships? Can it tell a true answer from a fluent one at the moment it matters rather than leaving the customer to discover the difference? In short: Does it improve the way a person does, getting better and cheaper at once?
Research that moves those needles is not incremental, and by Garg’s standard it is the only kind that matters. The potential of AI defies expectation. The status quo does not, and it will not suffice. The next chapter of AI belongs to the R&D aimed at foundations, and the need for it has never been greater.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
