Amid reported incidents of AI chatbots allegedly being used to plan and execute violent attacks, OpenAI announced that its models will restrict information about previous mass shooting incidents and their perpetrators to users alongside tightening of safety rules around how teenagers use ChatGPT.
The latest announcement comes after OpenAI reported “imminent threats of violence” by users to law enforcement agencies over 200 times in the past three months (July-September 2026). The announcement comes amid mounting evidence of generative AI being used in the preparation of violent attacks. A study published on October 28 by the International Centre for Counter-Terrorism (ICCT) identified 12 terrorist and extremist incidents involving generative AI, including six completed attacks and six foiled plots.
“Our work includes bringing imminent threats of violence to the attention of law enforcement, ..and These referrals are making a positive difference in disrupting violent plots, and we continue to rely on mental health and behavioral experts to help us assess difficult cases,” read an excerpt from an OpenAI statement made Friday.
OpenAI said its first community safety report, covering the second half of 2026, will be published in early 2027, with updates every six months.
The company is also tightening restrictions on information about past mass attacks for all users. ChatGPT will provide less detailed responses about such incidents and the individuals responsible, including limiting reconstructions of attacks, details about weapons and equipment used, how they were obtained, and comparisons of the scale of harm that could increase attackers’ notoriety.
“We’re now placing additional restrictions on how ChatGPT responds to questions about past mass attacks and the people who carried them out, directing the model to provide less detailed information in those areas even if it is available from other online sources. ..These limits apply regardless of a user’s age or reason for asking,” read the statement.
OpenAI said ChatGPT for Teens is also getting further stringent safety measures for use by teenagers.
“Our policies already prohibit helping anyone plan or carry out violence and restrict helping teens find or obtain weapons. We’re extending those protections to restrict practical information about weapons and security systems that could be misused to facilitate violence, even when there’s no sign of violent intent. Teens can still use ChatGPT to help prevent and de-escalate violence, seek support, and connect with a trusted adult,” read the statement by OpenAI.
OpenAI is also seeking to expand parental notifications to alert parents when a teenager’s account is temporarily paused for repeatedly discussing violence. The company will be rolling out the new safeguards in the coming few weeks.
“We remain committed to mitigating risks of harm in ChatGPT, strengthening our layered protections, monitoring for misuse, enforcing our rules, and involving real-world support and interventions when situations call for it,” it said.
Also Report: Canada seeks answers from OpenAI over Tumbler Ridge Mass Shooter
