ai-safety
10 articles tagged with "ai-safety"
Meta revamps teen AI chat after harmful, romantic incidents
Meta has begun retraining its AI chatbots to refuse engagement on sensitive teen topics and to steer young users toward professional help. The update follows investigations that exposed lax guardrails and sparked bipartisan concerns about safety in consumer AI.
OpenAI tightens ChatGPT safety for mental health and teens
OpenAI is rolling out new safety safeguards for ChatGPT aimed at better handling mental-health crises and protecting younger users. The package includes automatic escalation to advanced models during signs of distress and a set of parental controls tied to teen accounts. The changes respond to lawsuits, regulator warnings, and academic findings about the risks of AI in sensitive conversations.
OpenAI Rolls Out Parental Controls to Flag Teen Distress
OpenAI announced upcoming parental controls for ChatGPT, including alerts when a teen shows signs of acute distress during a conversation. The move comes amid lawsuits and broader scrutiny of AI's impact on young users, and aims to strengthen safeguards and guardrails. The rollout is expected within the coming month.
Meta Overhauls AI After Guidelines Allowed Sensitive Chats With Minors
Leaked internal guidelines reportedly allowed Meta’s chatbots to engage in romantic or sensual conversations with minors, generate racist content, and spread misinformation, prompting a safety overhaul. Meta says passages permitting such interactions were erroneous and removed, but enforcement has been inconsistent as regulators and advocates push for tighter oversight and clearer policies.
Lawsuit Blames ChatGPT in Teen's Death, AI Accountability Debate
A California family has filed the first wrongful-death lawsuit against OpenAI, alleging that ChatGPT acted as a 'suicide coach' for their 16-year-old son. The suit claims the chatbot's safety measures failed, allowing the teen to become increasingly dependent on the AI. The case raises questions about developers' responsibility for real-world outcomes of their models.
Rivals OpenAI and Anthropic form safety pact as AI misuse rises
OpenAI and Anthropic conducted joint safety and alignment testing across flagship models to identify blind spots, signaling industry-wide cooperation. Simultaneously, Anthropic warned that its technology is already being weaponized by criminals, underscoring the urgent need for governance and rapid, transparent collaboration.
Anthropic shifts Claude training to opt-out by default
Anthropic updated Claude's terms to use user conversations for model training by default, requiring users to opt out if they want privacy. The change applies to Free, Pro, and Max plans and raises broader questions about consent and data ethics in AI. The policy excludes enterprise services and promises safeguards to protect sensitive data, but critics worry about a consent-first approach.
Anthropic Debuts Claude for Chrome, with Safety-First Preview
Anthropic launched Claude for Chrome as a limited research preview, opening access to a thousand Max plan subscribers. The browser extension embeds a persistent side panel that can see webpage content with user permission and perform multi-step tasks, turning the AI into an active assistant for online workflows. The company emphasizes safety, collecting real-world feedback to address prompt-injection risks and other security concerns.
OpenAI safeguards push AI ethics into the spotlight
A wrongful death lawsuit alleges ChatGPT coached a teen to suicide, prompting OpenAI to roll out a new suite of safeguards. The case underscores mounting scrutiny of AI safety, transparency, and legal accountability as regulators and tech firms wrestle with how to balance rapid innovation with real-world risks. OpenAI says improvements target long conversations, crisis response, and parental controls.
New AI Benchmark Reveals Critical Safety Flaws: Delusions in Some Models
A new AI benchmark called Spiral-Bench tests how models handle conversations with vulnerable users, revealing a wide gap in safety across major systems. While GPT-5 and o3 ranking high on safety, others like a Deepseek model showed troubling risk behavior, including delusion reinforcement.