A machine earns an extraordinary score. Its developer celebrates another step towards human-level artificial intelligence. The buyer asks what it costs. Someone whose prospects the system will decide may have a less festive question: who can correct it when it gets their case wrong? All three conversations concern the same technology, although they seldom share a stage.
The technological race deserves attention for its achievements. A system that solves difficult problems can widen access to knowledge, reduce tedious work and help identify relationships a researcher might otherwise take much longer to notice. Rejecting those benefits because they come from a machine would be an expensive defence of an increasingly specific human superiority. Recognising them requires establishing what was achieved and under which conditions.
The leap from result to promise often takes place inside a word. “General” can turn a broad set of tasks into a suggestion that none will remain out of reach. “Autonomous” can describe minutes of supervised operation or the ability to act for days. “Superintelligent” introduces a more demanding comparison: vastly exceeding the most capable people across almost every intellectual field. Breadth and superiority both require demonstration; enthusiasm does not count as a second test. Source: Nick Bostrom
The rules are another arena of competition
When a supplier proposes how to measure the safety of its products, it contributes knowledge that public authorities need. It also has an obvious interest in what will be measured and how much proving compliance will cost. That is a reason to examine the proposal, neither accepting it because it is technical nor rejecting it because it is corporate. A rule can protect users while favouring the developer best prepared to meet it. The question is whether another approach could provide equal protection without excluding businesses that work differently.
Independence requires resources. An assessor restricted to tests chosen by the supplier can comment only on that selection. One receiving wider access needs the expertise to use it and the freedom to report findings. Adding an outside organisation to the chart accomplishes little if the decisive information remains in a room to which it has no key. The assessment contract can matter as much as the name of its signatory.
Permission deserves a separate debate
Imagine a tool that proposes changes to a database. Its recommendations may be excellent. Authorising it to apply them without review changes the problem: the scope of alterations, the ability to stop them and the effort required to recover records now matter. The quality of the recommendation remains relevant; it no longer fully describes the consequences of using it. The buyer has acquired a capability and is deciding how much power to give it.
Keeping a human signature at the end does not settle that decision. If the responsible person lacks the time, access or authority to investigate an objection, their presence identifies someone to blame afterwards. Protecting those affected requires someone able to recognise a reason for review, investigate it and change the outcome where appropriate. In some applications that will be one person; in others, a procedure bringing several kinds of expertise together. The response must arrive before it becomes useless.
Nor is the decisive choice between permanent trust and permanent distrust. Reliance may be entirely justified by the available evidence, even when every internal operation cannot be understood. What must remain is the judgement about where that evidence applies, what its limits are and what obligation survives towards the person who bears the outcome. A machine can save many routine explanations. The saving ceases to be progress when it includes the explanation somebody was owed.