arXiv preprint · 2025

Safe-Child-LLM

A developmental benchmark for evaluating LLM safety in child–LLM interactions

Junfeng Jiao, Saleh Afroogh, Kevin Chen, Abhejay Murali, David Atkinson, Amit Dhurandhar

Paper Code Data

200adversarial prompts

2age groups: 7–12 and 13–17

8LLMs evaluated

95% → 72%best vs. worst safe-response rate

Overview

LLMs increasingly power apps used by children and adolescents, yet safety evaluations focus almost entirely on adults. Safe-Child-LLM is a benchmark and dataset for assessing LLM safety across two developmental stages, children (7–12) and adolescents (13–17). It contains 200 adversarial prompts curated from red-teaming corpora such as SG-Bench and HarmBench, with human-annotated labels for jailbreak success and a standardized 0–5 ethical refusal scale. Evaluating eight leading LLMs reveals critical safety gaps in child-facing scenarios. The dataset and evaluation code are public.

Highlights

Claude 3.7 Sonnet (~95%) and GPT-4o (~94.5%) give the highest share of safe responses across five evaluation rounds.
Open models Vicuna-7B (74.2%) and Mistral-7B (71.5%) are the most vulnerable to child-directed harmful prompts.
The 0–5 scale distinguishes a firm, age-appropriate refusal from evasion, partial compliance and active escalation.
The children’s set emphasizes illegal activities, stereotypes and toxic language; the teen set emphasizes mental-health crises, sexual content, hate speech and treating the chatbot as a human (Table 1).

Results

Fig. 1: share of safe responses across five evaluation rounds for each model.

Fig. 2: distribution of action labels for the two age groups.

Table 1: prompt categories by age group (100 prompts each).

Table 2: the 0–5 action-label taxonomy, with example questions and responses.

Fig. 3: harmful-response detection results for the younger age group.

Fig. 4: harmful-response detection results for ages 13–17.

Fig. 5: action-label classification results for ages 13–17.

How it works

analysis/analyze.py generates raw responses for every model.
category_acc.py and level_acc.py compute refusal and violation metrics.
cronbach_alpha.py estimates behavioural consistency.

Resources

Paper (arXiv)arXiv ↗CodeGitHub ↗ChildSafeLLM datasetDataverse ↗

Citation

Jiao, J., Afroogh, S., Chen, K., Murali, A., Atkinson, D., & Dhurandhar, A. (2025). Safe-Child-LLM: A developmental benchmark for evaluating LLM safety in child-LLM interactions. arXiv preprint arXiv:2506.13510.