“Better than a person” sounds like a simple comparison until someone asks which person, doing what, with how much help. A student taking an exam without notes and a team of specialists with access to a library do not represent the same threshold. A claim about artificial intelligence needs that kind of detail before its result supports a conclusion.
AGI stands for artificial general intelligence: the expression points towards broad capabilities, although each proposal must specify its threshold. Superintelligence, in Bostrom’s sense as used here, requires greatly surpassing the most capable people across practically every intellectual field. A definition establishes what someone is trying to demonstrate. It is not evidence that anybody has achieved it. Source: Nick Bostrom
1. What is the competing unit?
A model can receive a question and produce an answer. A wider system can consult documents, run programs, ask other models and select among several attempts. Treating these as the same object hides resources that contribute to the outcome. The distinction does not invalidate the second system: it explains what would have to be acquired, operated and assessed to obtain a similar service.
Imagine a hypothetical demonstration in which an AI receives a mathematical problem and a correct solution is displayed. If a hundred attempts were permitted and someone selected the successful one, the demonstration tells us something different from a correct answer on the first attempt. Exploring many alternatives can be entirely useful for some tasks. Readers need to know the procedure to assess cost and reliability, particularly if no expert will be available afterwards to choose.
2. Which tasks were left out?
A test set selects problems that can be presented and scored. That selection makes results comparable, but also limits what is known. If every exercise has a verifiable answer, performance alone reveals little about how the system will handle an ambiguous request or disagreement over the objective. Generality requires examining breadth and transfer, rather than accumulating victories on closely related variations of one exercise.
Ask how cases were chosen and which failures accompany the successes. A commercial demonstration can show a real result without revealing how frequently it occurs. Adding difficult problems is insufficient too: if the intended application involves incomplete instructions, incorrect data or changing contexts, those conditions belong to the task. The useful difficulty is the one that reveals something about the service somebody intends to use.
3. Whose performance provides the comparison?
Beating an average person differs from beating a professional, which in turn differs from beating the best specialists. The cognitive evaluation framework proposed by Google DeepMind in March 2026 invites the assessment of separate capabilities against distributions of human performance. Breaking the comparison down helps prevent a single adjective from concealing different strengths and limitations. Source: Google DeepMind
The comparison also changes depending on whether accuracy, speed or cost is measured. A system might make a few additional errors yet prove useful for an initial screening stage that another procedure will review. Elsewhere, an infrequent error could have consequences that make that advantage insufficient. Deciding whether a tool is worthwhile requires connecting the measurement to its use; declaring a universal winner removes precisely the information needed to choose.
4. What is it allowed to do?
Capability and permission describe different relationships. An AI may draft a purchase order without being authorised to send it; a less capable system may hold credentials that allow it to spend money. Assessing intelligence alone overlooks the scope of action. Before delegating, identify accessible resources, permitted operations, consultation requirements and the means of stopping or correcting what has been done.
Human intervention has no fixed value either. Reviewing every operation may provide a safeguard or become a routine of approval without examination. Its usefulness depends on what reviewers can know and change within the time available. An excellent test result does not establish that this arrangement of permissions and checks works. It must be examined as part of the system that will operate in the world.
5. Does the result demonstrate a capability or announce a future?
An observed result supports a conclusion about the conditions examined. Claiming that the same progress will continue requires assumptions about resources, methods and obstacles. A date for the arrival of AGI is a forecast, not an already measured feature of a product. Changing an assumption can change that forecast without making the original result false.
When reading an announcement, follow the whole chain: what was observed, which interpretation is offered and which decision it is meant to justify. You may find an important advance accompanied by a weak prediction, or a limited tool that solves a valuable problem extremely well. That distinction allows progress to be recognised without handing it a blank cheque. The final question is which concrete decision improves with the available evidence and what still needs examining.