OpenAI proposes that external assessors examine safety arguments, safeguards, capabilities and incidents. Its document, published on 22 September, sets out principles for that relationship, including independence and the access needed to investigate. Its usefulness will depend on what examiners can check and communicate, as well as what the developer promises to allow. Source: OpenAI
The problem is easy to state: reviewing a complex technology requires proximity to those who understand it, but a review wholly dependent on the supplier will struggle to contradict it. Institutional distance alone is no solution either. A highly independent assessor allowed to observe only a demonstration could deliver conclusions that are honest and narrow.
Independence to do what?
The starting point should be the question being asked. Testing whether a safeguard withstands a particular examination differs from certifying that a system is suitable for every use. The broader the conclusion, the harder it becomes to assemble sufficient evidence. A report must preserve the relationship between its question, the access granted and what was actually examined.
Suppose an external team receives a system prepared for testing while customers use another version with additional connections. The investigation could be impeccable within its supplied environment and leave the effects of those connections unexamined. The problem would not necessarily be the assessor’s competence: it might lie in presenting a conclusion as covering a product the team never encountered.
The scope of access should therefore appear alongside the result. Versions, available resources and restrictions on experiments all matter. So does whether the supplier selected every case or the team could devise its own attempts at refutation. A test that permits surprises provides different information from a guided tour.
A useful report may be uncomfortable
Funding deserves attention without becoming an automatic accusation. When a client pays, a contract can protect or restrict the assessor’s ability to publish disagreement. The concrete questions are who can change a conclusion, delay it or prevent its release, and which procedure can resolve disputes without hollowing out the promised independence.
Confidentiality creates a legitimate tension. An examination may require access to information that should not become public. It does not follow that every adverse conclusion must remain secret. A reasonable arrangement can distinguish confidential technical material, findings suitable for publication and warnings that should reach people deciding how to use a system.
NIST’s AI Risk Management Framework is a voluntary framework for managing AI risks. It provides a reference outside the developer, rather than a particular certification of OpenAI. Source: NIST
A conditional conclusion should retain its conditions when it reaches the public. If a developer presents the report as general approval, the assessor needs to be able to correct that presentation. Independence also matters after the document is delivered.
A signature does not end responsibility
A buyer might interpret an external report as permission to stop asking questions. That would go too far. An assessment has a scope and a date; the organisation using the system still decides what data to introduce, which permissions to grant and which tasks to assign. A new use may pose a question the previous assessor never considered.
Disagreement between assessors also deserves thought. When two teams reach different conclusions, the first task is to compare their questions and conditions, rather than average their opinions. They may have examined different versions, assigned different significance to a failure or received unequal access. Showing those differences can provide more information than a single seal of approval.
OpenAI’s proposal opens a useful discussion about organising external scrutiny. Turning it into evidence will require commitments applied to concrete assessments and results capable of contradicting the supplier’s expectations. Independence will become visible when it permits an uncomfortable conclusion to be sustained, explained and delivered to those who need to know it.