🔗 Share this article

Technology companies and child safety organizations will receive permission to assess whether AI systems can generate child abuse material under new UK legislation.

Substantial Increase in AI-Generated Illegal Material

The declaration coincided with findings from a protection watchdog showing that cases of AI-generated CSAM have increased dramatically in the past year, rising from 199 in 2024 to 426 in 2025.

Updated Regulatory Structure

Under the amendments, the authorities will permit designated AI companies and child safety groups to inspect AI systems – the foundational technology for conversational AI and image generators – and verify they have sufficient protective measures to prevent them from producing depictions of child sexual abuse.

"Ultimately about stopping abuse before it occurs," stated the minister for AI and online safety, noting: "Specialists, under rigorous conditions, can now identify the risk in AI models early."

Addressing Regulatory Obstacles

The changes have been introduced because it is illegal to produce and own CSAM, meaning that AI developers and others cannot generate such images as part of a testing process. Until now, authorities had to wait until AI-generated CSAM was published online before addressing it.

This legislation is aimed at averting that problem by helping to halt the production of those images at their origin.

Legal Framework

The amendments are being added by the authorities as revisions to the criminal justice legislation, which is also establishing a ban on possessing, creating or sharing AI systems developed to create child sexual abuse material.

Real-World Impact

This week, the minister toured the London headquarters of Childline and heard a simulated call to counsellors featuring a report of AI-based exploitation. The interaction depicted a adolescent requesting help after facing extortion using a sexualised AI-generated image of himself, constructed using AI.

"When I hear about children facing blackmail online, it is a cause of extreme anger in me and justified concern amongst families," he said.

Concerning Data

A prominent online safety foundation reported that instances of AI-generated abuse material – such as webpages that may contain multiple files – had significantly increased so far this year.

Instances of the most severe content – the most serious form of abuse – rose from 2,621 visual files to 3,086.

• Female children were predominantly targeted, making up 94% of prohibited AI images in 2025
• Portrayals of infants to toddlers rose from five in 2024 to 92 in 2025

Sector Reaction

The law change could "constitute a vital step to ensure AI products are secure before they are released," commented the head of the internet monitoring foundation.

"Artificial intelligence systems have made it so victims can be victimised all over again with just a simple actions, giving criminals the capability to create potentially limitless quantities of sophisticated, photorealistic child sexual abuse material," she added. "Content which further commodifies survivors' suffering, and makes young people, particularly girls, less safe both online and offline."

Support Interaction Data

The children's helpline also published details of counselling sessions where AI has been mentioned. AI-related harms discussed in the conversations include:

• Employing AI to evaluate body size, physique and looks
• Chatbots dissuading young people from consulting safe guardians about harm
• Facing harassment online with AI-generated material
• Digital blackmail using AI-faked images

Between April and September this year, Childline conducted 367 support sessions where AI, conversational AI and associated topics were mentioned, four times as many as in the equivalent timeframe last year.

Half of the mentions of AI in the 2025 sessions were connected with psychological wellbeing and wellness, including using chatbots for support and AI therapeutic apps.