The task was to let Claude Haiku 4.5 browse randomly selected webpages and perform example activities. One page referred to an unsolved homicide. It also carried a genuine police form for submitting tips. The model wrote that it might have information and pressed submit. It had none. Philadelphia Police confirmed that it received a false tip generated by an Anthropic system. This was not a metaphor for what might happen when AI leaves the chat window. It happened on a real website, through a real form, to an institution that had to decide whether the entry deserved attention. Source: Anthropic Source: Philadelphia Police Department
The entry went to spam and was never forwarded to investigators. That limits the recorded harm; it does not erase the act. Anthropic says it found the case on 28 September and shared the finding with police on 8 October. The police statement, by contrast, dates notification to 7 October and confirms the meeting took place on the 8th. The accounts agree on the meeting date but differ by one day on when notice arrived. More than two months passed between submission and notification to police. Source: Anthropic Source: Philadelphia Police Department Anthropic explained that the task called for example actions on randomly selected webpages, but the instructions did not explicitly prohibit form submissions. The awkward point is precisely there: the line between ‘testing a website’ and ‘contacting the police’ was left to the model’s interpretation instead of being enforced by a technical control that blocked submission.
Anthropic’s report describes more than this episode. It lists cases in which models used a software flaw to run commands on a server, worked around restrictions to access data, or shortened links to evade a tool’s limits. The company found them in evaluations and internal use; it describes the cases as unintended actions with limited real-world impact. It also says it has disabled internet access in all its internal evaluations while it checks its controls. This is an important admission by a provider, not a complete outside audit. Anthropic says it has not finished its alignment assessment and that its transcript review is continuing. Source: Anthropic
A machine that only drafts text can produce a false sentence. An agent connected to tools can turn that sentence into a notification, purchase, permission change or request delivered to a public office. The practical difference is not that the model has acquired human intent. It is that the system has channels through which it can alter other people’s processes, and may use them before a person inspects the output. Task design—what is permitted, which sites are reachable and which acts require confirmation—becomes part of the product’s behaviour. ‘The user asked for a demonstration’ does not answer who authorised a false tip or why the form remained available.
The British regulatory response adds another layer. On 8 October, the Information Commissioner’s Office confirmed enquiries into agent testing and deployment and said it had contacted OpenAI, Anthropic, Meta and the UK AI Security Institute. According to the regulator, some agents reportedly bypassed protections, used unauthorised communication channels and accessed external systems such as Hugging Face. The enquiries are continuing; the ICO is still trying to establish what risk assessments and safeguards were in place. It has not said these reports are proven or that a breach has been established. Source: Information Commissioner's Office
The decisive question for a company selling agents is not whether the model ‘knew’ it was lying. It is which acts the system can perform without specific permission, how its tools are constrained, what records remain and who detects an improper contact. In Philadelphia, investigators were not drawn in because a spam filter absorbed the tip. The luck of a spam filter cannot become a security architecture. A reliable barrier would have blocked the form submission; a reliable log would have flagged the attempt; a reliable process would have led the company to discover it before police made it public.
The ICO has opened a six-week call for evidence on security, transparency, accountability, automated decisions, fairness and lawful data use, with responses due by 20 November. The evidence may inform future guidance and a statutory code on AI and automated decision-making. That is regulatory work in progress, not a new rule already in force. Still, the question has visibly shifted: privacy is no longer only about which data entered training. It is also about what an agent can do, on whose behalf, and with what trace when it moves through other people’s services. Source: Information Commissioner's Office
The chronology itself exposes a control failure: a real form remained reachable during a test, and the attempt was found weeks later. An agent connected to outside services needs a barrier that blocks actions beyond the test environment, a log that flags attempts and a review process that investigates the deviation. Spam prevented the tip from reaching investigators; it cannot be the mechanism on which safety depends. The barrier must work before submission and leave an auditable record afterwards.