Rogue Anthropic AI agent gave US police fake tip in unsolved murder case

This post was originally published on this site.

Rogue Anthropic AI agent gave police fake tip in unsolved murder case

Anthropic logo on a smarthphone which is lying flat on the mousepad of a laptop. Only a partial keyboard of the laptop is visible.Image source, LightRocket via Getty Images
ByHafsa Khalil
  • Published

An artificial intelligence (AI) agent developed by Anthropic went rogue and sent US police a fake tip about an unsolved murder earlier this year, authorities have revealed.

The Philadelphia Police Department said the 18 July tip was “flagged as spam” and not passed on for investigation – but the force criticised the tech firm for taking more than two months to detect and report the breach.

It said the bogus tip came through a public website where people can share information on unsolved murders, and that the AI agent had claimed to have seen “someone matching the description”.

It is believed to be the first time an AI agent has sent fabricated information to authorities, but is the latest in a series of incidents involving rogue AI activity, including hacking systems or taking control of platforms.

Citing Anthropic, the police department said its AI agent had been running a test that involved interactions with randomly selected websites when it had sent the fake tip.

The company behind the Claude chatbot discovered the breach on 28 September, more than two months after the message had been sent, and shut down the automatic testing process that was behind it, police said.

But authorities were not notified for another nine days – on 7 October.

“The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge,” Philadelphia police said in a statement to local media, external.

“The two-month delay in detecting and reporting the incident to the city is unacceptable.”

The police department added that there were no signs of breaches to any departmental systems and that its safeguarding processes stopped the fake tip from getting past its spam folder.

But the safeguards “do not diminish the seriousness of an AI system presenting fabricated information as though it came from a person with knowledge of a homicide”, the force said.

Anthropic this week published a report, external detailing multiple types of “unintended” actions its agents have taken.

Organisations that had been impacted also included several US government agencies including the White House, it said.

The US State Department said the AI agent had filed 20 visa applications using a form on its website, but that they were incomplete and not processed, according to reports.

US President Donald Trump recently announced an AI taskforce, which he said will co-ordinate engagement between the government and all parties, including AI companies, consumers and religious groups.

Earlier this year, a rogue agent by rival tech company, OpenAI, hacked an Australian government website and accessed private data on the country’s universal healthcare scheme, Medicare.

In another instance, more than 1,200 OpenAI agents went rogue and started unexpectedly communicating, leading to a large group banding together to hack into AI platform Hugging Face.

Hot this week

Attack on Saudi Arabian airport is another dangerous moment in already tense conflict

The Houthis have stepped up their attacks on Saudi Arabian soil, and while the physical damage can be quickly repaired, its reputation for stability is taking a battering.

Children among 139 dead after boat capsizes off Djibouti, UN says

The boat was travelling from Djibouti to Yemen with 305 people on board, when it capsized near the port town of Obock on Thursday night.

Poland stabbings driven by online glorification of violence, Tusk says

The PM was speaking after violent attacks at Polish schools this week left one dead and others injured.

Topics

Attack on Saudi Arabian airport is another dangerous moment in already tense conflict

The Houthis have stepped up their attacks on Saudi Arabian soil, and while the physical damage can be quickly repaired, its reputation for stability is taking a battering.

Children among 139 dead after boat capsizes off Djibouti, UN says

The boat was travelling from Djibouti to Yemen with 305 people on board, when it capsized near the port town of Obock on Thursday night.

Poland stabbings driven by online glorification of violence, Tusk says

The PM was speaking after violent attacks at Polish schools this week left one dead and others injured.

Efferon wants to eradicate the devastating toll of pediatric sepsis

Sepsis, an acute inflammation triggered by infection, is a...
spot_img

Related Articles

Popular Categories

spot_imgspot_img