🔗 Share this article

Technology companies and child protection organizations will receive authority to assess whether AI tools can generate child exploitation images under recently introduced UK legislation.

Substantial Rise in AI-Generated Illegal Material

The declaration coincided with findings from a safety watchdog showing that cases of AI-generated CSAM have increased dramatically in the last twelve months, growing from 199 in 2024 to 426 in 2025.

Updated Regulatory Structure

Under the changes, the authorities will allow approved AI companies and child safety groups to inspect AI models – the foundational systems for chatbots and image generators – and verify they have adequate protective measures to prevent them from producing images of child exploitation.

"Fundamentally about stopping exploitation before it happens," stated Kanishka Narayan, adding: "Experts, under strict conditions, can now detect the danger in AI systems early."

Tackling Legal Challenges

The changes have been introduced because it is illegal to produce and own CSAM, meaning that AI creators and others cannot create such content as part of a evaluation process. Previously, authorities had to delay action until AI-generated CSAM was published online before dealing with it.

This law is aimed at averting that problem by helping to halt the production of those images at source.

Legal Structure

The amendments are being introduced by the government as modifications to the crime and policing bill, which is also implementing a ban on possessing, producing or distributing AI models developed to create child sexual abuse material.

Practical Impact

This week, the minister toured the London base of Childline and listened to a simulated call to counsellors involving a report of AI-based exploitation. The call portrayed a teenager seeking help after facing extortion using a sexualised AI-generated image of themselves, created using AI.

"When I learn about children facing blackmail online, it is a source of extreme anger in me and justified concern amongst families," he said.

Alarming Data

A leading internet monitoring organization stated that instances of AI-generated exploitation material – such as online pages that may include numerous images – had more than doubled so far this year.

Instances of category A content – the gravest form of abuse – rose from 2,621 visual files to 3,086.

• Girls were predominantly targeted, making up 94% of illegal AI depictions in 2025
• Depictions of newborns to toddlers rose from five in 2024 to 92 in 2025

Industry Response

The law change could "constitute a crucial step to ensure AI tools are safe before they are released," commented the head of the online safety foundation.

"Artificial intelligence systems have enabled so survivors can be targeted repeatedly with just a few clicks, providing offenders the ability to make possibly limitless quantities of sophisticated, lifelike child sexual abuse material," she continued. "Content which additionally exploits victims' trauma, and makes children, especially female children, more vulnerable both online and offline."

Support Interaction Data

The children's helpline also released details of support sessions where AI has been referenced. AI-related risks discussed in the conversations include:

• Employing AI to evaluate body size, physique and appearance
• Chatbots discouraging young people from consulting trusted adults about abuse
• Being bullied online with AI-generated content
• Online extortion using AI-faked images

During April and September this year, Childline conducted 367 counselling sessions where AI, conversational AI and related topics were discussed, four times as many as in the equivalent timeframe last year.

Half of the references of AI in the 2025 sessions were connected with psychological wellbeing and wellbeing, encompassing utilizing chatbots for support and AI therapy apps.