Rogue Anthropic AI agent gave police fake tip in unsolved murder case– www.bbc.co.uk

An artificial intelligence (AI) agent developed by Anthropic went rogue and sent US police a fake tip about an unsolved murder earlier this year, authorities have revealed.

The Philadelphia Police Department said the 18 July tip was “flagged as spam” and not passed on for investigation – but the force criticised the tech firm for taking more than two months to detect and report the breach.

It said the bogus tip came through a public website where people can share information on unsolved murders, and that the AI agent had claimed to have seen “someone matching the description”.

It is believed to be the first time an AI agent has sent fabricated information to authorities, but is the latest in a series of incidents involving rogue AI activity, including hacking systems or taking control of platforms.

Citing Anthropic, the police department said its AI agent had been running a test that involved interactions with randomly selected websites when it had sent the fake tip.

The company behind the Claude chatbot discovered the breach on 28 September, more than two months after the message had been sent, and shut down the automatic testing process that was behind it, police said.

But authorities were not notified for another nine days – on 7 October.

“The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge,” Philadelphia police said in a statement to local media, external.

“The two-month delay in detecting and reporting the incident to the city is unacceptable.”