Project overview

Content moderation is a contentious topic, with public concerns about mass surveillance potentially undermining privacy and free speech. However, the proliferation of illegal content, particularly child sexual abuse materials, means that some form of moderation is needed for safety online. This challenge becomes even more complex in end-to-end encrypted (E2EE) environments, where users expect higher levels of privacy from service providers, with encryption often seen as a technical safeguard. The Online Safety Act 2023 makes digital service providers legally responsible for user safety on their platforms and appoints Ofcom as the regulator for online safety in the UK. A key challenge for Ofcom in implementing the Act is evaluating potential interventions for content moderation on E2EE platforms, not only for their technical robustness but also for trustworthiness from the user perspective.

This project addresses this challenge by exploring the trust relationship implications of content moderation interventions in E2EE environments. While applied cryptographers and computer scientists have developed and evaluated approaches to content moderation in E2EE from privacy and security perspectives, the socio-technical complexities of trust remain underexplored (Scheffler and Mayer 2023; Stockwell et al. 2025). Trust is a relational concept about ‘who trusts whom and for what’ (Balsa, Nissenbaum, and Park 2022, 171, original emphasis). Even when a privacy-preserving solution is cryptographically proven, trust is not automatically conferred, nor is it rendered obsolete by cryptographic guarantees; rather, trust (or mistrust) is distributed across the broader socio-technical arrangement in which the intervention is embedded (Balsa, Nissenbaum, and Park 2022; Pizio and Spencer 2025).

This research project draws on digital trust scholarship to examine proposed content moderation interventions specific to E2EE. It applies a trust mapping methodology (Spencer, 2023) to identify and map the socio-technical elements of trust and relationships entailed in these proposed interventions, and critically asks what implications they may hold for trust relationships. The analysis is based solely on publicly available materials.

Overall, this project extends discussions of trust in cryptography through a socio-technical lens, offering an understanding of E2EE, privacy and trust that supports policymakers in anticipating challenges in the social acceptance of content moderation interventions if introduced into E2EE services.

Activities

The project entailed a three-month research placement at Ofcom, aimed at developing the secondee’s engagement skills and providing experience in evidence-based policy work. Key activities included delivering research presentations, carrying out the trust mapping research project, and working in an embedded role within Ofcom’s Online Safety Group to achieve impact through engagement with colleagues across functional teams.

Using the trust mapping methodology, the research examined two interventions in detail: client-side hash-matching and on-device machine learning, both illustrated through Apple’s proposed implementations. The analysis and findings were presented to Ofcom colleagues in a research presentation and documented in a research report submitted to Ofcom at the end of the placement.

Impact

Through an in-depth examination of two content moderation interventions that are arguably operable in E2EE environments, and by comparing the networks of trust relationships each intervention entails, the project has elucidated key factors shaping trust across areas of technical design and proof, information-flow arrangements, and broader societal and relational contexts. It demonstrates that trustworthiness of an intervention is not merely a technical property, but must be evaluated relationally, within both the socio-technical arrangement it constructs and the socio-technical network in which it is situated.

Key findings are summarised as follows:

1. E2EE is often relationally defined and open to interpretation across different implementations

   Implications: It is important to attend to the interpretative nuances of E2EE definitions, as these underpin arguments about what preserves or undermines E2EE

2. Privacy and trust depend on information-flow design, not solely on cryptography

   Implications: Assessment of privacy-preserving technologies must go beyond cryptographic proofs to consider user agency, whether information flows are appropriate for the context, and how these flows operate in practice

3. Interventions can be implemented differently within different socio-technical arrangements, each structuring different trust relationships

   Implications: Trust must be evaluated relationally—both within the socio-technical arrangement an intervention constructs and the socio-technical network it is situated in—rather than as a technical attribute of the intervention itself

4. Trust in sophisticated technologies often rely on intermediaries (e.g. privacy advocates or other technical experts) to validate claims and proofs for users

   Implications: Trust mapping can be used to identify which intermediaries are trusted, contested or missing, and engage with these actors in the development of regulation

5. Trust stability is contingent and dynamic; instabilities indicate sites of mistrust

   Implications: Trust should be treated as something to be monitored in terms of relational stability. Methods such as trust mapping can help identify where trust may become unstable and anticipate challenges to the social acceptance of proposed interventions

Future work

Planned follow-on activities include publishing the research findings in a peer-reviewed academic journal as a contribution to the scholarly discussion on cryptography and trust, and exploring opportunities to present them to academic researchers and non-academic stakeholders.

Outcomes/outputs

Realised outputs include a research presentation delivered to Ofcom colleagues and a research report submitted to Ofcom at the end of the research placement. Future updates will be provided as additional outputs are realised.