New AI auditing technique aims to detect illegal content as UK police urge caution on sharing kids' photos

Researchers have developed a new auditing technique to test generative AI models for malicious capabilities without prompting them for illegal outputs, while UK authorities are simultaneously warning parents about the risks of sharing children's images online. The dual developments highlight growing concerns over AI-generated child sexual abuse material (CSAM) and the need for both technical and behavioral safeguards.

The auditing method, reported by the MIT Election Lab, allows researchers to evaluate open-source AI models for their potential to produce hate speech or CSAM without actually generating such content. This addresses a key challenge: many open-source models are freely available for legitimate tasks like product renderings, but they can also be adapted by malicious actors. According to Meridia, "these models also find their way into the hands of nefarious actors who may optimize them to produce illegal content, like hate speech or child sexual abuse material (CSAM)." This is a growing problem; Meridia noted that the National Center for Missing and Exploited Children has received increasing reports of such material.

2 sources

MIT Election LabNew method aims to keep kids safe from illegal AI-generated contentMeridiaNew method aims to keep kids safe from illegal AI-generated content

Separately, the UK's National Crime Agency (NCA) and Internet Watch Foundation (IWF) issued a warning about the risks of sharing children's images on social media. As reported by the BBC, authorities have already identified over 8,000 AI-generated realistic CSAM images and videos. The warning emphasizes that innocent family photos can be harvested and manipulated by AI tools. The BBC quoted the warning, stating that "innocent family photos posted on social media can be harvested and manipulated by AI tools to create abusive content." The authorities called for greater caution from parents regarding children's digital privacy.

Together, the new auditing technique and the public warning represent a two-pronged approach: one technical, aiming to detect vulnerabilities in AI models before they are exploited, and the other behavioral, urging parents to limit the digital footprint of their children. As generative AI becomes more sophisticated, both strategies will be crucial in combating the misuse of these powerful tools.