← Saleh Afroogh · Featured Projects
arXiv preprint · 2026

Safe by Benchmark, Unsafe in Practice

Lessons from 1,331 child-related AI incidents

Junfeng Jiao, Kevin Chen, Shrestha Mishra, Abhejay Murali, David Atkinson, Shriti Raj, Saleh Afroogh (lead author)
PaperCodeData
1,331coded child-related AI incidents
3incident sources: OECD.AI, AIAAIC, AIID
779incidents tied to deployment & organizational failures
316sexualization cases, the most common harm

Overview

Child safety in AI is usually judged at the model level, through benchmarks, red-teaming and content filters. This project asks what actually goes wrong once these systems reach children. We built KidAID, a corpus of 1,331 publicly reported incidents in which a minor experienced harm or a credible worst-case risk, drawn from the OECD.AI incident monitor, AIAAIC and the AI Incident Database. Each incident is coded with a child-specific internal (individual-level) harm, an external (population-level) harm, and the sociotechnical failure mode most directly implicated, using a new child-centered taxonomy.

Highlights

Results

Most prevalent failure modes per year, 2014–2026
Most prevalent failure modes per year, 2014–2026 (2026 incomplete).
Taxonomy of internal generative AI harms ordered by prevalence
Taxonomy of internal (individual-level) harms, ordered by prevalence.
Taxonomy of sociotechnical failure modes
Taxonomy of sociotechnical failure modes; bar length indicates prevalence.
Internal and external harm categories by primary failure mode
Internal (a) and external (b) harm categories by primary failure mode.
Model-internal versus sociotechnical safety view
Why benchmarks miss these harms: a model-internal view of safety versus a sociotechnical one.

Method

Resources

Paper (arXiv)arXiv ↗Codebook & annotationsGitHub ↗KidAID datasetDataverse ↗

Citation

Jiao, J., Chen, K., Mishra, S., Murali, A., Atkinson, D., Raj, S., & Afroogh, S. (2026). Safe by benchmark, unsafe in practice: Lessons from 1331 child-related AI incidents. arXiv preprint arXiv:2610.34705.