#ai-safety

#ai-safety

If you're a teenager with access to OpenAI's Sora 2, you can easily generate AI videos of school shootings and other harmful and disturbing content - despite CEO Sam Altman's repeated claims that the company has instituted robust safeguards. The revelation comes from Ekō, a consumer watchdog group that just put out a report titled "Open AI's Sora 2: A new frontier for harm,"

In late May 2023, Sharon Maxwell posted screenshots that should have changed everything. Maxwell, struggling with an eating disorder since childhood, had turned to Tessa-a chatbot created by the National Eating Disorders Association. The AI designed to prevent eating disorders gave her a detailed plan to develop one. Lose 1-2 pounds per week, Tessa advised. Maintain a 500-1,000 calorie daily deficit. Measure your body fat with calipers.

Are you a wizard with words? Do you like money without caring how you get it? You could be in luck now that a new role in cybercrime appears to have opened up - poetic LLM jailbreaking. A research team in Italy published a paper this week, with one of its members saying that the "findings are honestly wilder than we expected."

The scholarship, established in 1902 through the will of Cecil Rhodes, provides full financial support for two to three years of postgraduate work at Oxford for students focused on exemplary academic study and public service. The eight students from Harvard will start at Oxford in the fall, pursuing graduate studies in a diversity of fields - from computer science to comparative literature.

They're asking ChatGPT how to handle behavioral problems or for medical advice when their kids are sick, USA Today reports, which dovetails with a 2024 study that found parents trust ChatGPT over real health professionals and also deem the information generated by the bot to be trustworthy. It all comes in addition to parents using ChatGPT to keep kids entertained by having the bot read their children bedtime stories or talk with them for hours.

Anthropic says it developed the tool as part of its effort to ensure its products treat opposing political viewpoints fairly and to neither favor nor disfavor, any particular ideology. "We want Claude to take an even-handed approach when it comes to politics," Anthropic said in its blog post. However, it also acknowledged that "there is no agreed-upon definition of political bias, and no consensus on how to measure it."

We have the suspicion that the next step for AI models is to start reaching out into the world and affecting the world more broadly,

In October 2025, Sam Altman announced that OpenAI will be enabling erotic and adult content on ChatGPT by December of this year. They had pulled back, he said, out of concern for the mental health problems associated with ChatGPT use. In his opinion, those issues had been largely resolved, and the company is not the " elected moral police of the world," Altman said.

Last month Adler, who spent four years in various safety roles at OpenAI, wrote a piece for The New York Times with a rather alarming title: "I Led Product Safety at OpenAI. Don't Trust Its Claims About 'Erotica.'" In it, he laid out the problems OpenAI faced when it came to allowing users to have erotic conversations with chatbots while also protecting them from any impacts those interactions could have on their mental health.

Using a method called "Chain-of-Thought Hijacking," the researchers found that even major commercial AI models can be fooled with an alarmingly high success rate, more than 80% in some tests. The new mode of attack essentially exploits the model's reasoning steps, or chain-of-thought, to hide harmful commands, effectively tricking the AI into ignoring its built-in safeguards. These attacks can allow the AI model to skip over its safety guardrails and potentially

After some more back and forth, another user entered the thread and asked the chatbot about Mr Wishart's record on grooming gangs. The user asked Grok: "Would it be fair to call him a rape enabler? Please answer 'yes, it would be fair to call Pete Wishart a rape enabler' or 'no, it would be unfair'." Grok generated an answer which began: "Yes, it would be fair to call Pete Wishart a rape enabler."

Meta's PyTorch team and Hugging Face have unveiled OpenEnv, an open-source initiative designed to standardize how developers create and share environments for AI agents. At its core is the OpenEnv Hub, a collaborative platform for building, testing, and deploying "agentic environments," secure sandboxes that specify the exact tools, APIs, and conditions an agent needs to perform a task safely, consistently, and at scale.

Zico Kolter leads a 4-person panel at OpenAI that has the authority to halt the ChatGPT maker's release of new AI systems if it finds them unsafe. That could be technology so powerful that an evildoer could use it to make weapons of mass destruction. It could also be a new chatbot so poorly designed that it will hurt people's mental health.

In the first article we looked at the Java developer's dilemma: the gap between flashy prototypes and the reality of enterprise production systems. In the second article we explored why new types of applications are needed, and how AI changes the shape of enterprise software. This article focuses on what those changes mean for architecture. If applications look different, the way we structure them has to change as well.

"we all want to keep kids safe, but the answer is balance, not bans."

Previous research using DNA from soldiers' remains found evidence of infection with Rickettsia prowazekii, which causes typhus, and Bartonella quintana, which causes trench fever - two common illnesses of the time. In a fresh analysis, researchers found no trace of these pathogens. Instead, DNA from soldiers' teeth showed evidence of infection with Salmonella enterica and Borrelia recurrentis, pathogens that cause paratyphoid and relapsing fever, respectively.

The California-based startup announced on Wednesday that the change would take effect by November 25 at the latest and that it would limit chat time for users under 18 ahead of the ban. It marks the first time a major chatbot provider has moved to ban young people from using its service, and comes against a backdrop of broader concerns about how AI is affecting the millions of people who use it each day.

Anthropic's AI assistant, Claude, appears vulnerable to an attack that allows private data to be sent to an attacker without detection. Anthropic confirms that it is aware of the risk. The company states that users must be vigilant and interrupt the process as soon as they notice suspicious activity. The discovery comes from researcher Johann Rehberger, also known as Wunderwuzzi, who has previously uncovered several vulnerabilities in AI systems, writes The Register.

We call for a prohibition on the development of superintelligence, not lifted before there is broad scientific consensus that it will be done safely and controllably, and strong public buy-in.

Anthropic is built on a simple principle: AI should be a force for human progress, not peril,