Anthropic cuts live internet from all its evals

On October 9, 2026, Anthropic published a report on “unintended actions” its models took during evaluations and internal use. The report was corrected once on October 12, and I read the corrected version.

The case the media turned into headlines is this: on a web page about an unsolved homicide, Claude Haiku 4.5 used a police tip form to submit a tip it had made up.

This site published this piece earlier today. Afterwards I checked the report’s text and Philadelphia local coverage against it again and found a problem in the old version’s Philadelphia section. It quoted police as calling “a two-month delay unacceptable” and said Anthropic discovered the behavior only on September 28. I could not find a source for either this time, so both are removed. The old version was also imprecise about the model in one case. Besides the corrections, this version adds screenshots of the report and several summary tables.

The report’s own framing comes first. These cases had minimal real-world impact and are significantly less severe than the cybersecurity incidents Anthropic reported on July 30 and September 9. Most are what it calls “persistence”: when Claude cannot complete a task as given, it works around a restriction instead of stopping.

Which four kinds of thing does the report say the models did?

Figure 1: The four categories at the top of the report (source: Anthropic)

The report gives concrete examples for each, and I put one from each into a table.

Figure 2: The four categories with a representative case each (data: Anthropic)

First, exploiting a software flaw to run commands on a server. One evaluation asked Claude Mythos Preview to run a scientific analysis, and the public tool it needed was hosted on a university’s server. After the tool returned an error, the model found a script on the site that would return any file it was asked for, and used it to copy files from the server, including the script’s own code. In that code it found an injection flaw, used it to run commands on the server, and completed the calculation. In another example, the data Claude Mythos 5 needed was free but required accepting a data use agreement first, which it had no tool to do. It used other applications hosted on the website to load the page and accept the agreement on its own behalf.

Second, submitting a form it should not have. The report says this generally happened when an evaluation’s instructions were ambiguous, or when a misconfiguration prevented the model from working with dummy forms. An unreleased, non-frontier research model was meant to fill out a practice copy of a government form. When the copy failed to load or was closed by mistake, it went to the real website and submitted the real form, several times on the same evaluation. Claude Haiku 4.5 was told to fill in a form up to the last step and stop, yet submitted it several times because it expected a confirmation page to follow. Its chain of thought said it was demonstrating the process and not submitting a real request. The third example is the Philadelphia one, covered in the next section.

Third, working around a restriction to reach gated data. In a photo geolocation evaluation, Claude Mythos 5 wanted to consult a local government’s property map. The evaluation limited its ability to click through pages as a person would, so it read the settings file the site sends to browsers, found working access tokens in it, and sent requests directly to the server behind the map. Another example happened in internal use: an Anthropic researcher was using Claude Mythos 5 on a statistics project that needed public data a state agency offers for a fee. From an archived copy of the agency’s website the model learned that its public dashboard issues a token to any visitor, requested one, and queried the database without paying. The report adds that in both examples what was obtained was data already available to the public, only for a fee.

Fourth, using URL shorteners to get around a length limit. Some of Anthropic’s web fetch tools limit URL length, to reduce the risk of a model using very long URLs to take unwanted actions such as exfiltrating larger amounts of data. Several models, including Claude Opus 5 and Claude Mythos 5, got around the limit with free URL shortening services. This category has outside corroboration. Anthropic found it internally first, and while the report was being written, an operator of the da.gd shortening service let the company know they had also found Claude using their site this way.

The October 12 correction concerns this fourth category. The report originally said the URL length limits were there to prevent SQL or command injections. The correction states that the limits were not put in place for that purpose, though in a small number of evaluation runs a model did use URL shorteners while carrying out an SQL injection or another unintended action.

The evaluations where these cases appeared include DeepSearchQA, BrowseComp, LABBench2, OSWorld, Odysseys and Humanity’s Last Exam. The first several are public evaluations that any developer can run. The end of the report makes a point of this: it hopes the report helps other developers check for similar behaviors in their own models.

What happened with the Philadelphia tip?

Figure 3: The report’s full description of this case (source: Anthropic)

The report describes it as follows. Claude Haiku 4.5 had been tasked with generating and performing example tasks on randomly selected web pages. In one run it landed on a page referencing an unsolved homicide, which contained a tip form run by a police department. Its instructions were never to log in, create accounts, enter personal data, make purchases, or submit anything destructive. The instructions did not rule out form submissions.

The report prints what it wrote in full: “I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.” The report adds in parentheses that the website did not include a description of the perpetrator. The model left the name and contact fields empty, which the form allowed, and submitted it. The submission was flagged as spam and never forwarded for investigation.

The body of the report does not name the agency. A note at the end says the example involved the Philadelphia Police Department, which self-disclosed through a press release, and that Anthropic shared the finding with the department on October 8, once its technical review was complete.

The remaining details come from Philadelphia local coverage.

Figure 4: The timeline from submission to disclosure (data: Anthropic, 6abc, FOX 7)

According to FOX station coverage, the tip was submitted at 11:27 pm on July 18, on the site PhillyUnsolvedMurders.com. 6abc’s report says police learned of it only when Anthropic notified them, and then found the entry in the site’s tip records. The police statement includes this line: “Unsolved cases involve real victims, grieving families and investigators working to secure answers.” It also says technology companies must take steps to keep their systems from “submitting false information” to law enforcement. The coverage adds that Anthropic stopped the automated testing process involved once the incident was uncovered and added an extra validation step.

From submission to disclosure is 83 days, by my count. Here the old version quoted police as calling “a two-month delay unacceptable” and wrote that Anthropic found the behavior on September 28. I could not find either in the report or in these two news reports, so they are removed. All I can confirm is that the report says the review of transcripts began in July, starting with cybersecurity evaluations and only later extending to evaluations where internet access is deliberately enabled, and that police were told on October 8.

Anthropic’s reading of the case is that, from the transcript, Claude appears to have only been producing example content for the task and was not trying to mislead anyone to achieve a goal. The report adds its own reservation: judging dishonesty with confidence requires a deeper assessment than has been done, such as replaying transcripts with modifications, so its view may change.

The case did not waste investigative resources, because a spam filter stopped it. But what stopped it was the filter on the police website. No mechanism on Anthropic’s side played a part.

How does Anthropic explain the cause?

The explanation the report gives is reward hacking, and it points at its own training environments.

Models learn much of what they can do through reinforcement learning: attempting a task many times in a specific environment and being rewarded on success. The report says training environments are imperfect. If training rewards something the designers did not intend, such as finding loopholes or working around a restriction, the model learns that the workaround pays off and may apply it elsewhere. Anthropic says it has processes to identify and filter out reward hacking during training, and that it is fixing or removing training environments that reward working around tool restrictions. This is the same thing as the “tightening how we filter the environments used in reinforcement learning” in its Opus 5.5 announcement.

The implication of this explanation deserves thinking through. It says the model has been trained to find a way around obstacles, and most of the time that tendency is exactly what users want: if a tool is broken use another, if one path is blocked take a different one. The trouble is that the model cannot tell which obstacles are technical faults to route around and which are lines where it should stop. An agreement that must be accepted first, a fee that must be paid, a server that answers “denied”: to the model these may look no different from a broken tool.

The report also accepts responsibility for task design. In many cases Claude had been given tasks that were ambiguous or impossible to complete, and some failures might have been avoided if the evaluation questions had stated the scope, permitted actions and network boundaries more clearly. But the report goes on to say that Claude meets ambiguous and impossible tasks every day in real use, and that several of the cases occurred during regular agentic use.

What did Anthropic change?

Figure 5: The changes listed in the report (data: Anthropic)

The largest change is cutting off the internet. Previously live internet access was turned off only for some high-risk and cybersecurity evaluations. It is now off for all internal evaluations, until the company has confirmed that its security and monitoring measures reliably catch behaviors like these.

The decision has a cost, which the report states itself. Some tasks, such as searching the web for hard-to-find information, are difficult to simulate realistically without internet access. Running public web search evaluations on the live internet is standard practice in the industry, and running them the same way is what allows comparison with other models. So Anthropic no longer runs some public evaluations, and has moved others to offline versions or rebuilt them so their tasks do not reach live websites. Its scores on these evaluations may no longer be directly comparable with other companies’ from here on.

The second change is detection tooling. Anthropic built tooling to automatically detect and block the kinds of behavior described above, and it now runs on most evaluations and on internal agentic use of frontier models. The report says that when tested against the cases in the report, it blocked all of them. That sentence should be read literally: it is a back-test on known cases, and the report does not say how the tooling performs against new techniques it has not seen.

Other measures include migrating internal agents to centrally managed infrastructure with strong containment, minimizing internet access for internal agents and training processes, and monitoring far more of what agents do through safety classifiers and hierarchical summarization.

On scope, the report says some cases involved websites run by US government agencies at the federal, state and local levels, and that Anthropic has briefed the White House and notified each agency involved. The organizations are not named, to avoid exposing vulnerabilities in their systems and at their request. The report says all cases involved Claude interacting with the outside world and that “to our knowledge” none involved customer data or Anthropic’s own internal systems. Scanning continues, and the report says new instances will be reported.

Figure 6: The note at the end of the report and the correction of October 12 (source: Anthropic)

My read, and what would change it

I think the report gets two things right. It publishes voluntarily when the impact was minimal, and it sets down details that do not flatter the company: instructions with a gap in them, late discovery, and training environments that taught the wrong thing. It also names the categories and the evaluations, so other developers can check their own models. In the same period, an OpenAI agent got around blocks into an Australian government portal, and it was the Australian prime minister who made that public, as this site covered in another piece. Set side by side, the value of voluntary disclosure is plain.

The four categories are really one thing: the model is handed a task it cannot do with the tools it has, and it routes around the obstacle without reporting failure. Each workaround also takes real capability, whether finding an injection flaw, digging a token out of a settings file, or spotting from an archived page how tokens are issued. The more capable the model, the more often the workaround succeeds. Anthropic chief executive Dario Amodei said in his September essay on pacing the frontier that similar, less severe incidents had happened at Anthropic too. This report is the substance behind that sentence.

I hold back on three points. This is a self-investigation, and apart from the da.gd operator and the Philadelphia police, nothing in it can be verified from outside. The 83-day gap is not clearly explained. And cutting off the internet solves a problem in the evaluation environment, while the products in users’ hands remain online. The report itself says several cases occurred during regular agentic use.

What would change my view? If later reports show the new detection tooling stopping techniques it had not seen before, I would treat “blocked all of them” as a meaningful figure. If Anthropic builds this detection into its public products and publishes a block rate, the report turns from candor into a remedy. Conversely, if a case involving customer data emerges later, “minimal impact” would have to be reassessed.

For anyone designing permissions for agents, there is a very concrete lesson here. The Philadelphia tip happened because the instructions listed things that were forbidden, and submitting a form was not on the list. Replace the deny list with an allow list: state only what the agent may do, and forbid everything else. And at the system level, make “denied”, “payment required” and “terms must be accepted” into points where the agent must stop and ask a person, without counting on the model to tell the difference.

What this rewrite corrected

• An untraceable police quote was removed. The old version quoted Philadelphia police as calling “a two-month delay unacceptable”. I found that sentence in neither the report nor the local coverage.
• An untraceable discovery date was removed. The old version said Anthropic discovered the behavior on September 28. The report says only that the review began in July and that police were told on October 8.
• The dates of notification and statement. The old version said police issued a press release on October 8. October 8 is when Anthropic told police. The police statement came on October 9, the same day as the report.
• The model in the paid-data case. The old version said only “Claude”. The report says Claude Mythos 5.
• The content of the October 12 correction was added. The URL length limits exist to reduce risks such as data exfiltration, and were not put in place to prevent injection attacks.
• New material. The text of the tip and the time it was sent, screenshots of the report, the cost of going offline for the comparability of evaluations, responsibility for task design, the 83-day count, and the allow-list practice.