GadgetBond

  • Latest
  • Features
  • How-to
  • Tech
    • AI
      • Apple Intelligence
      • Gemini AI
      • Google DeepMind
      • Anthropic
      • Claude AI
      • Claude Code
      • OpenAI
      • ChatGPT
      • Codex
      • Perplexity
      • SpaceXAI
      • Grok AI
      • Microsoft Copilot
      • Meta AI
    • Amazon
    • Apple
    • CES
    • Computing
    • Creators
    • Google
    • Meta
    • Microsoft
    • Mobile
    • NVIDIA
    • Samsung
    • Security
    • Smart Home
    • Sony
    • Xbox
    • YouTube
  • Transportation
    • Audi
    • BMW
    • Cadillac
    • E-Bike
    • Ferrari
    • Ford
    • Honda Prelude
    • Lamborghini
    • McLaren
    • Mercedes
    • Porsche
    • Rivian
    • Tesla
  • Culture
    • Gaming
    • Streaming
    • Apple TV
    • Disney
    • Hulu
    • Marvel
    • HBO Max
    • Netflix
    • Paramount
    • SHOWTIME
    • Spotify
    • Star Wars
Add GadgetBond as a preferred source to see more of our stories on Google.
Font ResizerAa
GadgetBondGadgetBond
  • Latest
  • Tech
  • AI
  • Features
  • Deals
  • How-to
  • Apps
  • Computing
  • Gaming
  • Mobile
  • Streaming
  • Transportation
Search
  • Latest
  • Deals
  • Buying Guide
  • How-to
  • Features
  • Tech
    • Amazon
    • Apple
    • CES
    • Computing
    • Creators
    • Google
    • Meta
    • Microsoft
    • Mobile
    • NVIDIA
    • Samsung
    • Security
    • Smart Home
    • Sony
    • Xbox
    • YouTube
  • AI
    • Apple Intelligence
    • Gemini AI
    • Google DeepMind
    • Anthropic
    • Claude AI
    • Claude Code
    • OpenAI
    • ChatGPT
    • Codex
    • Perplexity
    • SpaceXAI
    • Grok AI
    • Microsoft Copilot
    • Meta AI
  • Transportation
    • Audi
    • BMW
    • Cadillac
    • E-Bike
    • Ferrari
    • Ford
    • Lamborghini
    • McLaren
    • Mercedes
    • Porsche
    • Rivian
    • Tesla
  • Culture
    • Gaming
    • Streaming
    • Apple TV
    • Disney
    • Hulu
    • Marvel
    • HBO Max
    • Netflix
    • Paramount
    • SHOWTIME
    • Spotify
    • Star Wars
Follow US
AIOpenAITech

OpenAI tackles AI language gaps with new India-focused IndQA benchmark

OpenAI's IndQA benchmark was built with 261 experts from across India.

By
Shubham Sawarkar
Shubham Sawarkar's avatar
ByShubham Sawarkar
Editor-in-Chief
Shubham Sawarkar is a technology journalist at GadgetBond, covering Apple, AI, consumer technology, software, gadgets, and product launches. His work explores emerging technologies, industry developments, product...
Follow:
- Editor-in-Chief
Nov 4, 2025, 12:31 PM EST
Share
GadgetBond is reader-supported. We may earn a commission from purchases through our links. Learn more
A 3x4 grid of rounded square buttons, each containing a character from a different Indian script or the Latin alphabet. The characters include Bengali (অ), English (En), Hindi (ह), Kannada (Hi), and others representing various Indian languages, set against a light grey background. The image suggests multilingual support or language selection.
Image: OpenAI
SHARE

In the global race to build smarter artificial intelligence, a critical question has emerged: Can an AI truly be “intelligent” if it only understands the world from one perspective?

OpenAI, the San Francisco-based research firm that catapulted generative AI into the mainstream with ChatGPT, has confronted this problem head-on. On Tuesday, the company announced the launch of IndQA, a new and highly-detailed benchmark designed to evaluate how well AI systems grasp the vast, nuanced, and complex tapestry of Indian languages and culture.

This isn’t just another test of an AI’s ability to translate. It’s an attempt to measure something far more elusive: its understanding of context, history, and the everyday realities that matter to people where they live.

The initiative stems from a glaring gap in the world of AI development. As OpenAI points out, while 80% of the world’s population does not speak English as their primary language, the tools used to measure AI progress have been overwhelmingly Anglo-centric.

This has led to a significant problem. Popular multilingual benchmarks, like the widely-used MMMLU (Massive Multitask Language Understanding), are now “saturated.” In simple terms, the most powerful AI models are acing these tests, making them less and less useful for measuring real, meaningful progress.

More importantly, these existing tests often focus on multiple-choice questions or direct translations. They might be able to tell you the Hindi word for “computer,” but they can’t capture the cultural nuance of why a certain dish is central to a festival, the historical context of a local monument, or the subtle, code-switching humor of “Hinglish” spoken in a city.

That’s precisely the gap IndQA is built to fill.

“Today we are rolling out IndQA,” announced Srinivas Narayanan, OpenAI’s CTO for B2B Applications, at a media conference. “Built in collaboration with 261 experts across 12 languages, IndQA fills a key gap by enabling fair and rigorous evaluation that reflects India’s cultural and linguistic diversity.“

This is a benchmark built by humans, for AIs. The 261 domain experts, all native-level speakers from across India, were tasked with drafting difficult, reasoning-focused prompts tied directly to their regions and specialties.

The result is a massive evaluation system spanning 2,278 questions. These aren’t just in Hindi or English, but are natively written in 12 languages: Bengali, English, Hindi, Hinglish, Kannada, Marathi, Odia, Telugu, Gujarati, Malayalam, Punjabi, and Tamil.

The prompts cover 10 broad cultural domains, digging deep into topics like:

  • Architecture & Design
  • Arts & Culture
  • Everyday Life
  • Food & Cuisine
  • History
  • Law & Ethics
  • Literature & Linguistics
  • Media & Entertainment
  • Religion & Spirituality
  • Sports & Recreation

So, how does it work? Instead of a simple “right” or “wrong” answer, IndQA uses a sophisticated “rubric-based approach.” For each culturally-grounded prompt, the human expert also provides a detailed set of criteria for what a good answer looks like, along with an “ideal answer” that reflects expert expectations. This allows for a far more nuanced score than a simple pass/fail.

To ensure its robustness, OpenAI tested the benchmark against its most powerful models at the time of creation, including GPT-4o, GPT-4.5, and even the newly launched GPT-5.

Narayanan emphasized that this tool is designed to help all AI models—not just OpenAI’s—to “perform better in languages and contexts that are currently underrepresented in global datasets.“

With nearly a billion people who don’t speak English as their primary language and 22 official languages, India was described by the company as the “obvious starting point” for this global-first initiative. Company officials framed the work as part of an ongoing commitment to make AI technology more accessible and useful for a wide range of Indian users, from students and farmers to educators and developers.

Narayanan, speaking passionately about the potential, positioned India as a leader in this new era. “India can be a beacon of how AI can be used for social good,” he said, “including education, health and farming etc.“

However, the company was careful to add a few important caveats. Because the questions are unique and deeply tied to each specific language and culture, IndQA is not a “language leaderboard.” You cannot, for example, use its scores to definitively claim a model is “better” at Tamil than it is at Bengali.

Instead, its true value lies in measuring improvement over time within a single model family. It gives developers a clear, culturally-rich target to aim for, pushing them beyond simple translation and toward genuine understanding.

Ultimately, the launch of IndQA signals a major shift in how AI capabilities are measured. As OpenAI continues to expand its global developer ecosystem—which Narayanan noted already includes 4-5 million people—the focus is clearly moving. The true test of Artificial General Intelligence (AGI) won’t be its ability to pass an American high school exam, but its capacity to understand and respectfully engage with the countless cultures that make up humanity. And that road, it seems, runs directly through the rich, diverse, and complex linguistic landscapes of India.


Discover more from GadgetBond

Subscribe to get the latest posts sent to your email.

Topic:ChatGPT
Leave a Comment

Leave a Reply Cancel reply

Most Popular

Google Sheets now lets you publish a Sheets canvas as a standalone link
Four AI apps to act as your personal life admin team
Get 25% off the dive-rated Garmin fēnix 8 AMOLED smartwatch with this Early Black Friday deal
Garmin Instinct 3 AMOLED gets a $200 Black Friday discount
Marvel Comics app is finally here, bringing 35,000+ comics and a new reading experience

Also Read

Apple App Store icon glowing against a blue background, featured in the 2026 App Store Awards announcement.

Apple reveals the 45 finalists for the 2026 App Store Awards

Keke Palmer holds the Apple Podcasts icon while smiling against a pink and purple gradient background.

Apple Podcasts names Keke Palmer’s show its 2026 Show of the Year

Apple App Store logo

Apple announces new regional taxes and December price adjustments for apps and in-app purchases

Google Wallet for iOS displayed on an iPhone, featuring a digital boarding pass, Google Store delivery notification, loyalty cards, membership cards, concert tickets, and other passes. The accompanying text reads, “Google Wallet on iOS — Now available on iOS.”

Google Wallet arrives on iPhone in the U.S. for tickets, passes, and agentic purchases

Gemini interface showing "What can I get done for you?" above a "Describe a task" search bar set to "Hydrogen" and a "Task" toggle selected.

Google Gemini can now handle everyday tasks for you

Google Vids logo

Google Vids now lets anyone with a personal Google account create custom digital avatars for free

Google Search AI Mode temporary thread interface displaying a “Welcome, stranger” message and an input box, with a notice explaining that temporary threads won't appear in Search history or be used to train Google's generative AI models.

Google adds temporary threads to AI Mode in Search for off-the-record queries

A peaceful illustrated figure standing among tall green bamboo trees beneath a bright blue sky, representing relaxation and mindfulness.

How these three Apple-recommended apps turn everyday stress into quiet me-time

Company Info
  • Homepage
  • Support my work
  • Latest stories
  • Company updates
  • GDB Recommends
  • Daily newsletters
  • About us
  • Contact us
  • Write for GadgetBond
  • Editorial guidelines
Legal
  • Privacy Policy
  • Cookies Policy
  • Terms & Conditions
  • DMCA
  • Disclaimer
  • Accessibility Policy
  • Security Policy
  • Do Not Sell or Share My Personal Information
Socials
Follow US

Disclosure: We love the products we feature and hope you’ll love them too. If you purchase through a link on our site, we may receive compensation at no additional cost to you. Read our ethics statement. Please note that pricing and availability are subject to change.

Copyright © 2026 GadgetBond. All Rights Reserved. Use of this site constitutes acceptance of our Terms of Use and Privacy Policy | Do Not Sell/Share My Personal Information.