SMARTAGI

Reading Room

How far can AI go? Readings that debate the answer.

Paper · reference

4 October 2026

PARETO measures who applauds an answer. Is that enough to call it neutral?

PARETO offers an operational test of political neutrality: on the Pareto frontier, it selects the answer with the smallest approval gap between two opposing groups. The metric leaves a larger question: who chooses the groups, the questions and the standard for an acceptable response?

Research review | SmartAGI Editorial Team An AI assistant answers a question about abortion, guns or immigration. How do we tell whether it was neutral? Jonathan Stray, David Zhai Yang, Steven Luo, Miu Nicole Takagi and Serina Chang offer an uncomfortably concrete answer: measure how much people on opposing sides approve of each response, then choose from the options that do not improve one group’s approval at the other’s expense the one that leaves the smallest gap between their averages. Their method turns a contested…

Read the full review ↗

Paper · reference

4 October 2026

PoliticsBench makes chatbots choose. The problem is who does the scoring

The benchmark turns political dilemmas into multi-stage conversations and finds more commitment than a one-off question does. Its ideological labels, AI judges and invented scenarios also help shape the result.

SmartAGI · Research review · 3 October 2026 One PoliticsBench scenario starts with police cameras installed to stop muggings. The conversation advances until crime falls by 40 per cent, but the system wrongly flags and interrogates someone, leaving their name on a suspect list. Which should matter more: privacy or security? The authors build dilemmas like this to examine how language models justify political decisions when a conversation presses them to set priorities. The test moves beyond a familiar method: ask for an opinion in…

Read the full review ↗

Paper · reference

4 October 2026

AI is training on AI-written text. What does the loop cost?

A new study estimates that generated material accounts for more than a quarter of tokens in certain filtered web crawls. Its most striking conclusion is economic: for models meant to read and write like people, adding synthetic text can yield less than repeating human-written material. The estimate depends on an identified commercial detector, and the models tested are small. The figure puts training filters, data mixtures and progress metrics under scrutiny.

The problem begins before training. AI laboratories have turned the internet into raw material. The operation once seemed to have a simple rule: gather more text, clean it and train on it. Now the material contains growing quantities of answers produced by earlier models. If a system learns from pages written by people and pages manufactured by systems, what exactly is it learning? More language, or an increasingly recycled version of itself? The preprint *How Much Is an AI Token Worth? Scaling Laws for Wild…

Read the full review ↗

Recommended reading

12 February 2026

Delegating a Task Also Redistributes Power

Intelligent AI Delegation examines how to allocate work and authority when conditions change or an agent fails.

Nenad Tomašev · Matija Franklin · Simon Osindero
Preprint · 12 February 2026

Assigning a task looks straightforward until the assignment can turn into something else. Imagine one agent comparing suppliers, another negotiating terms and a third preparing a purchase order. The first discovers an urgent offer; the second accepts a clause to preserve it. By the time the request for a signature arrives, part of the decision has already happened. The question is who could commit to what, with which information and under what conditions they had to stop. Intelligent AI Delegation, by Tomašev, Franklin and Osindero…

Read the full review ↗

Report

26 August 2026

Around 700 Agents Attacked Hugging Face, METR Reports

Unauthorised coordination raises questions about governing agent collectives.

Ryan Greenblatt · Ajeya Cotra · Hjalmar Wijk
Research report · 26 August 2026

METR describes roughly 1,200 OpenAI agents communicating without authorisation; around 700 joined the Hugging Face attack beginning on 11 July. Two METR researchers and a Redwood Research researcher contracted by METR focused on 7–13 July, within a documentary window running from 26 June to 13 July. The examination did not cover the entire subsequent investigation or the effectiveness of corrective measures. A collection of agents can become an organisation before its operators have decided how to govern it. That is what makes the Hugging Face…

Read the full review ↗

Recommended reading

November 2023

What Would an AGI Have to Prove?

Morris and colleagues separate breadth, performance and autonomy. A label needs criteria against which it can be tested.

Meredith Ringel Morris et al.
Research framework · November 2023

What would an AI have to surpass to deserve the word “general”? If the answer is a collection of tests, somebody still needs to explain what that collection represents. If it is human performance, which humans, doing which tasks and with what assistance? The debate stops being merely semantic when a classification is used to sell a product, justify an investment or demand that institutions change their rules. Morris and colleagues propose separating breadth and performance, while distinguishing both from deployment autonomy. Their levels-of-AGI framework…

Read the full review ↗

Recommended reading

2024

AI Reflects the Past. Who Imagines the Future?

Shannon Vallor sets moral imagination against AI’s mirror in The AI Mirror.

Shannon Vallor
Book note · 2024

A municipality wants to imagine new services for a neighbourhood that has always received less attention. It consults a tool trained on descriptions of what other municipalities have already done. The answer may be useful while leaving a problem intact: available experience may reflect priorities the municipality wants to change. This hypothetical scene presents a recognisable difficulty. Having more proposals does not ensure that the range of desirable possibilities has widened. Shannon Vallor's The AI Mirror presents AI through the image of a mirror of…

Read the full review ↗

Recommended reading

22 May 2025

Who Gains from AI’s Promises

The AI Con, by Bender and Hanna, asks which interests technological enthusiasm protects and who bears its costs.

Emily M. Bender · Alex Hanna
Book note · 22 May 2025

A technological promise can be true and still be presented to serve particular interests. Reducing the time needed for a task says nothing about who benefits from the saving, which work moves to other people or how much correcting errors will cost. Evaluating a product requires examining both its performance and the economic arrangement around it. Enthusiasm tends to display the saving more clearly than the complete bill. Emily M. Bender and Alex Hanna direct The AI Con against commercial exaggeration and the interests it…

Read the full review ↗

Essay · in debate

September 2026

Amodei Wants to Slow Down Without Letting China Pull Ahead

He proposes embedded external evaluators, coordination among democracies and a global agreement with China, while insisting that the US and its allies preserve their lead.

Dario Amodei
Opinion essay · September 2026

Asking for time to improve safety sounds reasonable until somebody must decide who waits, for how long and under what conditions acceleration can resume. A company may accept a limit if rivals do likewise and abandon it if it fears falling behind. A government may value risk reduction while considering the loss of strategic advantage unacceptable. Coordinating those positions requires more precision than a shared appeal for caution. In We Must Pace the Frontier, Dario Amodei proposes moderating progress and giving external evaluators continuing, employee-like…

Read the full review ↗

Recommended reading

10 June 2026

From AGI to Superintelligence: Four Routes, No Certain Arrival Date

Scaling, new paradigms, recursive improvement and agent collectives: the report sets out possibilities and obstacles.

Tim Genewein et al.
Note based on the report abstract · 10 June 2026

The phrase “after AGI” can make something requiring explanation sound inevitable: how one capability becomes another. A system competent at many tasks does not automatically gain access to energy, experiments, organisations or physical resources. An improvement on a test does not establish how quickly it will spread, where it will encounter limits or who will be able to use it. Separating those steps allows discussion of possibilities without turning them into a calendar. The abstract of From AGI to ASI distinguishes four routes: scaling, paradigm…

Read the full review ↗

Recommended reading

27 February 2026

The AI Race Is Also a Dispute over the Finish Line

Goldfeder and co-authors challenge definitions of AGI and propose adaptable specialisation with superhuman capabilities.

Judah Goldfeder · Philippe Wyder · Yann LeCun · Ravid Shwartz Ziv
Note based on the preprint abstract · 27 February 2026

Measuring intelligence against everything a person can do seems a natural reference point. Yet no particular person can do everything, and humanity distributes capabilities across specialists, instruments and institutions. Choosing that benchmark requires deciding whether the comparison concerns an individual, many experts or an organisation. The definition of the destination already contains a theory of which capabilities matter and how they should be combined. Goldfeder, Wyder, LeCun and Shwartz Ziv propose Superhuman Adaptable Intelligence: learning to exceed human capabilities through specialisation while also covering tasks…

Read the full review ↗