Q191. What is MMLU? Massive multitask language understanding primarily measuring A. The agents ability to use a web browser. B the foundational world knowledge and problem solving ability across 57 subjects like math, law, and history. C. The speed of the agent's GPU. D. How many languages the agent can speak? Correct answer B. Detailed explanation. MMLU is the SAT score for LLMs. While it doesn't measure agentic loops, it measures the raw intelligence of the brain powering your agent Q192. Which benchmark specifically tests an agent's ability to fix real world GitHub issues? A. Human level, B. S.WE bench, software engineering benchmark, C. GSM8K,

D. ImageNet. Correct answer B. Detailed explanation. SWEBench provides an agent with a codebase and a GitHub issued description. The agent is successful only if it writes a patch that passes the repo's existing unit tests. Q193. What is the GIA, General AI assistance benchmark known for? A. Testing agents on video games. B. Proposing tasks that are conceptually simple for humans. Example, find the price of this specific toy in 2022, but very hard for AI to automate v tools. C. Measuring the carbon footprint of AI. D. Testing agents in a vacuum. Correct answer B. Q1 194. What does the pass at K metric measure in coding agents?

A. How many times the agent passed a human interview? B. The probability that at least one of the K code samples generated by the agent will pass the unit tests. C. The speed of code execution. D. The number of lines of code written per second. Correct answer B. Q1 195. What is agent bench? A. A comprehensive framework that evaluates agents across diverse environments like OS, database, knowledge graph, and web. B. A physical bench where AI researchers sit. C. A tool for measuring the weight of a server. D. A list of all agents currently running in the cloud. Correct answer A. Q96. What is the Berkeley function calling leaderboard BFCL? A. A list of the fastest functions in

Python. B. A benchmark that specifically ranks how accurately different models can call external APIs tools without errors. C. A competition for human programmers. D. A way to measure the temperature of a data center in Berkeley. Correct answer B. Q1 197. What is human level? A. A data set of 164 handwritten Python programming problems used to test code generation logic. B. A psychological test for AI developers. C. A way for agents to evaluate human work. D. A metric for how humanike an agent's voice sounds. Correct answer A. Q 198. What is inference throughput? A. The total number of users on the platform. B. The number of tokens or requests a

system can process per second. Critical for scaling multi-agent swarms. C. The depth of the agent's reasoning. D. The price of a single API call. Correct answer. B. Q1 199. What is groundedness in rag based agents? A. A metric that checks if the agents answer is strictly based on the provided search results rather than its own internal hallucinations. B. Ensuring the agents server is properly electrically grounded. C. The agents ability to understand gravity. D. How realistic the agents personality is? Correct answer. A. Q200. What is a letterboard contamination risk? A. Spilling coffee on a computer. B. When benchmark questions are accidentally included in the model's

training data, leading to fake high scores. C. Too many people using the same model. D. An agent winning too many competitions. Correct answer