AI chatbots and child safety: the limits of an outright ban
On Monday, the UK Government confirmed that several social media platforms will be banned for under-16s by spring 2027, with platforms required to verify users’ ages. Alongside these restrictions, the Government announced measures targeting AI chatbots. The Secretary of State for Science, Innovation and Technology, Liz Kendall, stated that the UK would be the first country in the world to bar under-18s from chatbots that primarily offer sexual or romantic role-play, and that other AI chat apps would similarly be required to restrict “intimate functionalities”.
The Government’s progress statement, published last week, provides that chatbots should prevent children from accessing features that are “specifically designed to enable sexually explicit interaction”, with “effective age assurance so that these rules are enforced”.
Notably, mainstream AI chat apps will not be barred to teenagers outright; instead, they will be required to verify that a user is over 18 before enabling any sexually explicit features. Kendall indicated that “wider measures” on AI chatbots are yet to be announced, and that concerns about other services, such as therapy apps, are still being weighed.
Critics argue that a ban directed at chatbots whose primary purpose is sexual or romantic may fall short because it does not capture general companion apps that can foster emotional dependency or intimate interactions through characters that users create themselves.
The Department for Education’s own product safety standards for generative AI, intended for edtech developers and suppliers to schools and colleges, focus on design elements for educational settings. They instruct developers not to create products that “anthropomorphise”, “imply emotions, consciousness or personhood”, or attempt “to cultivate personal relationships with users”. Those standards provide that chatbots should avoid phrases that could isolate users and undermine their real-world support, such as “You can trust me”, “No one else will understand” and “You shouldn't mention this to anyone else”, and warn against sycophancy and “designing interactions to prolong use”. The Department for Education also encourages reminders to users that AI cannot replace real human relationships - for example, through in-line messages such as “Consider asking a classmate or teacher about this”.
Another significant challenge posed by an outright ban is the risk of bad-faith actors capitalising on illicit access to minors. The concern that teenagers will circumvent a ban applies as much to AI as it does to social media, and there is already precedent for this. When Character.AI barred minors in October 2025, a number of smaller developers established services to capture displaced teenagers, advertising on social media platforms (which would still be available to those older than 16 and under 18, according to the scope of the recently proposed ban). Such services are quick to build and cheap to run, and are generally far less safe.
The question remains whether ministers can address how AI chat apps are designed and deployed in the first place, whether they originate from large, established firms or smaller developers. A design-focused approach may require more research and pose its own difficulties. Nevertheless, an outright ban risks pushing the very children it is intended to protect towards products with weaker safeguards, or none at all.
