
Towards AI Supported Moderation to Promote Child Safety Online
Project team
Dr Lauren Scott
Principal Investigator
Lecturer in Computer Science, Northumbria University
Dr William Seymour
Co-Investigator
Lecturer in Cybersecurity, King’s College London
Bo Wei
Co-Investigator
Senior Lecturer in Computer Science, Newcastle University
Prof. Julie Gore
Co-Investigator
Professor of Organizational Psychology, Birkbeck University of London
Prof. Mohamed Khamis
Co-Investigator
Professor of Human-Computer-Interaction and Cybersecurity, University of Glasgow,
Summary
Children spend much of their time in online games and chat spaces like Minecraft, Fortnite, Roblox and Discord. These platforms offer real benefits: connection, learning and play. However, they carry serious risks, including sexual abuse, exploitation and harassment from adults and other children.
Most safety tools today only react after harm has already happened, through blocking, muting or reporting. They also assume children can easily spot danger as it unfolds, which isn't always true. This project takes a different approach: understanding how children themselves notice something is wrong and decide whether to stay in an interaction or leave it, before harm escalates.
We will work directly with children to explore what signs make an interaction feel unsafe, and when they would welcome support (such as a prompt to pause and think) rather than feeling controlled or judged.
This will produce a framework, grounded in children's own experiences, that can guide the future design and training of AI tools built to encourage critical thinking in risky moments, without taking away children's independence.
The project will also build lasting connections between researchers, policymakers, charities, and children and parents, laying the groundwork for further research and, ultimately, safer online spaces for young people.