Raja Koduri on Why AI Infrastructure Has a Cost Problem and How OXMIQ Plans to Fix It

A single hyperscale AI data center now costs approximately $50 billion to build. The silicon budget alone, just the GPUs, runs $30 to $35 billion per gigawatt. Add in energy, power delivery, networking, and cooling, and you’re looking at another $10 to $15 billion on top of that. As AI demand climbs, those figures climb with it. The industry, quite simply, cannot build fast enough.

Raja Koduri, who now leads architecture work at OXMIQ, framed this challenge in a recent conversation that covered everything from chiplet integration to the long-term future of how agents and silicon might communicate. His perspective carries weight. He has spent decades at the intersection of chip design, systems software, and product development at Apple, AMD, and Intel, and the ideas he is now advancing at OXMIQ are a direct product of what he has seen work, and what hasn’t.

The core argument is straightforward: faster chips alone will not solve this. What determines who wins over the next five years is utilization, getting more useful work out of every transistor already in the system. And for AI to reach a global scale, silicon costs need to drop by a third or more. That’s the problem OXMIQ was built to address.

Raja Koduri and the Case for a Unified Core

At the center of OXMIQ’s platform is OxCore, which Raja Koduri describes as “the new computing core.” The idea is to unify what most systems currently keep separate: scalar processing (CPU-style), vector processing (GPU-style, as in CUDA), and matrix math (tensor/TPU-style). Most chips today treat these as distinct abstractions that require different tools, different workflows, and different mental models.

OxCore collapses all three into a single architecture.

“You can think of it as a single core that encapsulates CPU, GPU, and TPU into one unified architecture,” he said. That unification has a practical payoff: it improves utilization and simplifies how developers think about execution. Instead of building for three separate paradigms, developers work within one.

The analogy that shapes OXMIQ’s broader mission is instructive. Mobile internet became affordable for a billion people in India not because one company solved everything, but because open, licensable architecture allowed a wide range of builders to compete. OxCore is designed with the same logic: enable more players to build competitive chips and chiplet systems, and costs come down through competition.

Chiplets Are Harder Than They Look

Chiplet-based design has become a popular topic in semiconductor circles, but Raja Koduri is less impressed by the enthusiasm than most. He has done the hard work, from AMD’s HBM1 integration through Intel’s 47-chiplet Data Center GPU, Ponte Vecchio, and the experience has made him clear-eyed about the gap between theory and execution.

“Everyone who is super excited about chiplets are the ones who haven’t done them yet,” he said.

The problem isn’t the concept. It’s that standards alone don’t produce working systems:

• High-bandwidth connections between chiplets generate significant heat, which makes standardization progressively harder as bandwidth increases.
• Even within a single team at Intel, aligning on architecture, power, thermals, and validation across chiplet interfaces proved difficult.
• Integration challenges don’t disappear with a published standard. They require disciplined execution at every layer of the stack.

OXMIQ’s chiplet quilting approach is built around this reality. The goal is not plug-and-play in the marketing sense. It’s a validated, partner-aligned system where architecture, power, thermals, and testing are coordinated from the start, not assembled and hoped for afterward.

The Vision: Agents Talking Directly to Silicon

The most forward-looking part of the conversation centered on a concept Raja Koduri called “agents to atoms,” and he didn’t hesitate to describe it as “probably the most visionary or most profound thing” discussed.

The current software stack between an AI agent and the hardware executing its work is thick. Programming languages, frameworks, drivers, runtimes, all of it was built for human programmers. Agents aren’t human programmers. So why should they be constrained by the same abstractions?

“They don’t need to talk through Python, C, all these intermediate languages,” he said. “We created them for humans to program. But when it’s an agent generating work, there will be new, more efficient forms of communication, where the agent can talk to what I call nano-agents in silicon directly.”

The implication is significant. An entire inference model can be expressed in a single page of math equations. Translating that into tens of thousands of lines of code, passed through layer after layer of the stack before the hardware finally executes, costs latency, energy, and efficiency at every step. OXMIQ’s long-term direction is to thin those layers, so that systems can express intent more directly to hardware. Fewer translations. Less energy lost. Higher efficiency.

When asked what one thing people should remember about OXMIQ, the answer was immediate: “Think agents to atoms.”

Licensable GPU IP and the OxCapsule Beta

OXMIQ’s business model differs from traditional fabless chip companies. Rather than designing and selling its own chips, the company licenses IP so that partners can build silicon tuned to their specific requirements. OxCore scales from edge devices to data centers, and from robotics to automotive, all within a shared architecture and software ecosystem.

“There is ARM for CPUs. But there is not ARM for GPUs,” he noted. “Anyone can license our IP and build a chip. That’s the problem we’re trying to solve.” The mobile analogy applies again: ARM’s availability is what allowed Apple to design its own chip for the iPhone. A world where Intel was the only CPU option, he argues, would have made that impossible.

On the software side, OXMIQ launched OxCapsule, its first software product, in public beta in November 2025 with V1.0 for Windows and Mac. V2.2 followed in December 2025 with Linux client support, and monthly updates are ongoing. The product gives developers access to continuous compute across GPUs, accelerators, and CPUs through a single interface, without requiring them to rebuild workflows when switching hardware platforms.

Participation in the beta already includes companies such as ARM, AMD, Intel, Infineon, Global Foundries, Tenstorrent, and Radisys, alongside universities including Boston University, NYU, Texas A&M, IIT Hyderabad, and the University of Utah.

About Raja Koduri

Raja Koduri is a semiconductor and computing industry veteran whose career spans leadership roles at Apple, AMD, and Intel. At Apple, he developed an approach to problem solving grounded in user perspective, an experience he has described as “almost like going to a university.” His time at Intel, which he has called “the PhD phase” of his career, gave him exposure to the full stack: transistor manufacturing, packaging, systems, and software. He now applies that accumulated experience at OXMIQ, where he leads work on compute architecture, chiplet integration, and the long-term vision of direct agent-to-silicon communication.