For years, the AI industry has largely accepted one basic reality: if you wanted to run frontier models at scale, you built around NVIDIA. OpenAI’s new inference chip, Jalapeño, is a serious attempt to change that equation – and its first published results suggest it may have landed a meaningful hit against the biggest names in AI hardware.

OpenAI says Jalapeño can deliver more AI work per watt while responding faster than the NVIDIA systems it compared against. That matters because AI is no longer just about training giant models in massive data centers. The business is increasingly about serving billions of everyday requests: ChatGPT conversations, coding suggestions, image generation, search answers, and AI agents that need to take several steps quickly without feeling sluggish.

The headline claim is striking. Across tests using GPT-OSS 120B, DeepSeek R1, and Kimi K2.5, OpenAI says Jalapeño produced 1.5 to 1.9 times more AI work per watt at peak throughput, while cutting end-to-end latency by 1.7 to 3.6 times versus the comparison systems. At highly interactive settings – the sort of setup that makes a chatbot or agent feel responsive rather than delayed – the claimed performance edge grew to 2.1 to 4.1 times.

That is a big deal, with one important caveat: these are OpenAI’s own results, not a broad independent test suite. The company used SemiAnalysis’ public InferenceX benchmark and compared the chip with NVIDIA’s GB200 and GB300 systems at different operating points, but hardware claims always deserve outside validation once more organizations can get hands-on with the product.

Still, the significance is not hard to see. OpenAI is not simply trying to make a faster chip. It is trying to reshape the economics of running AI.

Inference is the less glamorous half of the AI business, but it may become the more important one. Training a major model is expensive and headline-worthy, yet it happens periodically. Inference happens every time someone asks a question, generates an image, runs an agent, or sends another prompt into a model. At ChatGPT scale, even small improvements in speed or power use can turn into substantial savings.

Jalapeño was designed specifically for that job. OpenAI says conventional systems tend to make a tradeoff between high throughput and low latency. In plain English, an operator can serve lots of requests efficiently, or make individual responses very quick, but doing both at once is difficult. OpenAI’s pitch is that Jalapeño handles both sides of the problem with one architecture.

Its results on GPT-OSS 120B help explain why the company is making such a fuss. At peak efficiency, OpenAI reported 85,448 mixed tokens per second per kilowatt for Jalapeño, compared with 44,960 for the NVIDIA GB200 setup – roughly a 1.9-times advantage. More notably, Jalapeño reached 1,459 tokens per second per user at its fastest setting, versus 535 tokens per second on the comparison system.

Those are technical metrics, but the user-facing version is simpler: a capable model could feel quicker, and OpenAI could serve more people with the same amount of electricity and hardware. Faster outputs are nice. Lower operating costs are even more important, because they give OpenAI more room to offer cheaper products, absorb heavier usage, or spend its savings on the next generation of models.

The strongest numbers appeared when OpenAI matched Jalapeño to the existing systems’ top decoding speeds. On the GPT-OSS test, the company said Jalapeño delivered 53.7 times more mixed-token throughput per kilowatt at the GB200 system’s fastest decoding speed. On DeepSeek R1, it claimed a 104.3-times lead at the comparable point; on Kimi K2.5, it claimed 56.1 times more throughput per kilowatt.

Those enormous multipliers do not mean Jalapeño is simply 50 or 100 times better in every real-world scenario. They reflect a particular comparison: how much work each system could do while delivering the same fast per-user response rate. The bigger point is that NVIDIA’s platforms appear to lose efficiency quickly when pushed toward highly interactive workloads, while OpenAI says Jalapeño remains relatively balanced.

That balance is at the heart of the chip’s design. Modern language-model serving has two major phases. First comes prefill, where the system reads and processes the user’s prompt. Then comes decode, where it generates an answer token by token. Prefill is more compute-intensive; decoding is much more constrained by memory bandwidth. Moving data among chips and servers adds another source of delay.

This is why a generic accelerator can look extraordinary on a spreadsheet and still struggle with the feel of a real AI product. An AI agent does not perform one giant calculation and stop. It may search, reason, write code, check its work, call a tool, read the result, and repeat the process. If every step waits on memory or network traffic, the lag compounds. OpenAI’s argument is that Jalapeño was built around this increasingly common style of workload from the start.

The company says its system keeps model state, including the so-called KV cache used during generation, close to the compute resources that need it. Rather than treating networking as an add-on between accelerators, Jalapeño incorporates it directly into the architecture, allowing a workload to stay within one connected system and reducing expensive data movement.

That is also why the announcement is more than a standard custom-chip story. Google has long used its Tensor Processing Units, or TPUs, to power its own AI services. Amazon has developed Trainium and Inferentia chips for cloud customers. Microsoft has been investing in its Maia AI accelerator. Meta, meanwhile, has developed its own inference silicon as it looks for ways to lower the cost of serving AI at enormous scale.

OpenAI is now firmly joining that club. The company is signaling that it does not want its future to be dictated entirely by the release schedule, pricing, and supply constraints of outside chip suppliers – even one as central to the AI boom as NVIDIA.

That does not mean NVIDIA is suddenly out of the picture. OpenAI explicitly says it will continue deploying NVIDIA accelerators and hardware from other partners across both training and inference. Jalapeño is not a declaration of independence so much as an effort to gain leverage, specialization, and more control over the most expensive part of operating an AI platform.

There is another telling detail in the announcement: OpenAI says it used AI to help make the chip. The company says its models shortened design, measurement, and verification loops, helping the team go from initial design to tapeout in nine months. It also used Codex with GPT-Astra to optimize model kernels, reporting that selected AI-generated implementations ran 1.5 to 1.8 times faster than existing human-written versions for certain GPT-OSS attention and mixture-of-experts blocks.

Again, that does not mean AI independently designed a finished chip in nine months. Silicon development remains an extraordinarily complex engineering effort involving architecture, verification, physical design, manufacturing, packaging, cooling, systems software, and deployment. But it is a glimpse of the feedback loop OpenAI is chasing: use AI to design better AI infrastructure, then use that infrastructure to run more capable AI.

OpenAI has rated Jalapeño at 700 watts, though it says sustained measured power stayed at or below 550 watts in the workloads tested. The company plans to begin deploying the chip in its own compute infrastructure by the end of the year, with a second-generation version already deep in development and a third generation in early formation.

That roadmap is perhaps the clearest signal in the whole announcement. Jalapeño is not being positioned as a one-off internal project or a symbolic response to rivals. OpenAI is trying to establish a long-term silicon program, one designed around its own models, software, products, and real-world usage data.

For users, the short-term impact may be subtle: snappier answers, more capable agents, and fewer moments where demand makes an AI service feel slow or constrained. For OpenAI, the stakes are much larger. If Jalapeño’s results hold up at production scale, the company could reduce the cost of every answer it serves while making its products faster at the same time.

That is how an infrastructure story becomes a product story. And it is why OpenAI’s new chip deserves attention: the most consequential AI battle may no longer be only about who builds the smartest model. It is increasingly about who can afford to run it for everyone.