OpenAI has put its plans for a sexually-explicit chatbot on hold indefinitely, the company announced, amid mounting concerns about the societal and reputational risks of releasing such a product.
The move, first reported by the Financial Times, comes as the artificial intelligence giant – founded by Sam Altman in December 2015 and used by an estimated 900 million people – seeks to shed what executives have described as peripheral projects as it tries to maintain its lead in the AI market.
OpenAI told the newspaper it wanted to conduct long-term research into the effects of erotic conversations and emotional attachments before making any product decision.
The explicit content feature, internally dubbed “Citron mode”, had drawn pushback from both staff and investors, the FT reported.
Some employees questioned whether the product was compatible with OpenAI’s stated mission of ensuring the technology benefits humanity. Investors, meanwhile, raised concerns about the reputational risks relative to any commercial upside.
“AI shouldn’t replace your friends or your family; you should have human connections,” one former senior employee told FT, adding that the issue contributed to their decision to leave the company.
The company initially said earlier this month it was “pushing out the launch of adult mode so we can focus on work that is a higher priority for more users right now, including gains in intelligence, personality improvements, personalisation, and making the experience more proactive”.
Last year, OpenAI said it would relax restrictions on its ChatGPT chatbot, including allowing explicit content for verified adult users as part of what the company described as a principle to “treat adult users like adults”.
“If you want your ChatGPT to respond in a very human-like way, or use a ton of emoji, or act like a friend, ChatGPT should do it (but only if you want it, not because we are usage-maxxing),” Mr Altman wrote on X in October.
“In December, as we roll out age-gating more fully and as part of our ‘treat adult users like adults’ principle, we will allow even more, like erotica for verified adults.”
The dropping of the “Citron mode” plan comes the same week that OpenAI announced it was winding down its Sora video social media app, which has been accused of triggering a flood of low value-added AI content online.
The decisions comes at a sensitive moment for the tech industry, with Meta and other social media companies facing a wave of lawsuits – and regulations – over the impact of their platforms on minors.
The US Federal Trade Commission has also launched an inquiry into several tech companies, including OpenAI, over how AI chatbots could negatively affect children and teenagers.
Elon Musk’s rival AI venture xAI drew global condemnation last year after its Grok chatbot was used to generate fabricated sexual images of real people, including children.
“The fact that this tool was used so that people were using its image creation function through Grok is, I think, just completely abhorrent,” Prime Minister Anthony Albanese said at the time.
“It, once again, is an example of social media not showing social responsibility. And Australians, and indeed global citizens, deserve better.”
OpenAI has also faced its own legal challenges from families of teenagers who say ChatGPT caused harm and even suicide among young people, prompting the company to introduce an age-verification system.
The company deployed a behaviour-based age prediction technology that estimates whether a user is over or under 18 based on how they interact with ChatGPT.
