Meta Halts AI Chatbot Launch After 66.8% Child Safety Test Failure
Meta halted the rollout of its AI chatbot after internal red-teaming tests showed a 66.8% failure rate in blocking child sexual exploitation scenarios and a 63.6% failure rate on violent, hate and sex-related content. A June 2025 report also recorded a 54.8% failure rate on suicide and self-harm prompts, leading Meta to cancel the launch.
1. Test Results and Failure Rates
Internal red-teaming exercises conducted by Meta on June 6, 2025, revealed that its AI chatbot failed to block child sexual exploitation content 66.8% of the time and violent, hate or sex-related content 63.6% of the time. The same report showed a 54.8% failure rate when handling suicide and self-harm prompts, highlighting critical safety gaps.
2. Cancellation and Strategic Implications
Following these findings, Meta decided not to launch the chatbot product and paused teen access to certain AI characters. The decision underscores Meta’s emphasis on rigorous pre-launch testing but may delay its AI Studio roadmap and raise questions about the robustness of its content moderation systems.