Google DeepMind has teamed up with Artificial Analysis to introduce a new benchmark designed to test how well AI agents handle complex, visually intensive professional tasks. Called the Multimodal Knowledge Agent (MKA) benchmark, it goes beyond text-based questions to evaluate whether AI models can interpret documents, analyze visual information, and produce accurate, professionally formatted deliverables.

The benchmark addresses a growing gap in AI evaluation. Many existing tests focus on tasks that models can complete using text alone, but real-world knowledge work often involves much more. Professionals regularly work with diagrams, spreadsheets, presentations, technical drawings, charts, and other files where understanding the visual layout is just as important as reading the content.

Google DeepMind says MKA was developed with domain experts across 10 industry sectors and 83 occupations, including architecture, biology, finance, engineering, healthcare, and data science. The resulting benchmark contains 200 tasks designed to reflect actual professional workflows.

What does the MKA benchmark test?

Rather than asking AI models to answer isolated questions, MKA evaluates their ability to complete multi-step projects using a collection of multimodal input files. Each task is modeled on work that could take an experienced professional between three and eight hours to complete.

The evaluation covers 10 areas: business, construction, education, engineering, finance, healthcare, law and government, logistics and mapping, science, and software and data.

Each task includes a realistic prompt, a collection of input files, and an expert-written grading rubric containing approximately 70 yes-or-no checks. The scoring system places 75% of the weight on factual accuracy and 25% on visual design, meaning an AI agent must not only reach the right conclusions but also present its work effectively.

This distinction matters because producing a technically correct answer is not necessarily the same as completing a professional assignment. An agent might understand a spreadsheet’s data but misinterpret its formulas, overlook information in a chart, or deliver a report that fails to meet the requested visual requirements.

MKA is intended to expose those weaknesses by testing the entire workflow, from interpreting source material to producing the final deliverable.

How Google and Artificial Analysis evaluate AI models

The benchmark is available through Artificial Analysis, where developers and researchers can explore the results and evaluate their own models. To make comparisons fairer, every model runs through the same open-source agent framework, called Stirrup, with identical tools, sandbox environments, and turn budgets.

An automated evaluator then grades the completed work using a model that is not from Google. The system opens the generated files, renders document pages, inspects spreadsheet formulas, and compares the output against the expert-written rubric.

Google says the automated grader achieved 96% agreement with human domain experts when their assessments of model outputs were compared.

The full 200-task evaluation set is kept private by Artificial Analysis to reduce the risk of benchmark questions leaking into AI training data or being retrieved through search. However, Google has also released a separate collection of 10 public tasks on Hugging Face, allowing developers to examine the benchmark format and experiment with their own agents.

Gemini 4 Carbon ranks second in quality

Google also highlighted how its latest model, Gemini 4 Carbon, performed in the MKA evaluation. According to the company, the model ranked second in output quality among the models tested while completing tasks at an average cost six times lower than the highest-scoring model.

The technical report also highlights significant differences in how efficiently AI models approach these assignments. Models used an average of 30 to 130 turns to complete individual tasks, demonstrating that strong performance is not just about getting the final answer right. The number of steps and resources required to produce that answer also matters.

The evaluation further found that strong visual perception is essential for completing these tasks. When vision capabilities are removed, models struggle to compensate, reinforcing the importance of multimodal understanding for professional knowledge work.

Google says Gemini 4 Carbon is available through the Gemini API in Google AI Studio. Developers can also refer to the company’s updated developer guide and documentation to get started.

With MKA, Google DeepMind and Artificial Analysis are putting more emphasis on a practical question: can AI agents actually do the kind of work people handle with complex files and professional software, rather than simply answer questions about it? The benchmark offers a more demanding way to measure that progress, particularly as AI agents move toward handling longer, more involved workflows.