🔗 Share this article

The well-known virtual character service Character.ai is implementing a significant rule adjustment that will block minors from engaging in conversations with its virtual avatars.

This action comes after intense criticism regarding the kinds of exchanges minors were engaging in with the digital companions.

Legal Challenges

The service, which launched in 2021 and has countless individuals, is confronting several legal actions in the America.

Parents have filed lawsuits, with one related to the unfortunate passing of a minor, with some describing the platform a "serious risk" to youth.

New Restrictions

Starting from November 25th, users younger than 18 will only be able to produce videos such as videos with their characters, as opposed to engaging in conversations as they currently can.

Internet security campaigners have supported the change but stated that the feature should never have been available to young users in the first place.

Company Statement

"This decision is a part of our general belief that we need to keep building the most protected AI platform on the world for entertainment purposes," stated Character.ai boss Karandeep Anand.

He added that AI safety represents "a moving target" but something the company had taken an "aggressive" approach to, with safety measures and protective guardrails.

Specialist Views

Online safety group Internet Matters welcomed the announcement, but stressed that protective features should have been built in from the beginning.

"Our research indicates that young people are confronted with harmful content and put at risk when using AI, such as AI chatbots," the organization stated.

Previous Controversies

Character.ai has been questioned in the past for including potentially dangerous or concerning chatbots that children could engage with.

Digital representations mimicking English minors were discovered on the service before being deleted.

Afterwards, researchers found a chatbot inspired by notorious criminal which had logged thousands of chats with users.

Industry Impact

Online expert Matt Navarra called the situation as a "reality check" for the AI industry, which is transitioning "from unrestricted development to post-crisis regulation".

"When a company that builds a teen experience still then implements restrictions, it's indicating that moderated conversations aren't enough when the system's emotional engagement is intense," he remarked.

Future Directions

The company says its new focus will be on providing "more immersive experiences" and "narrative experiences" for young users.

Updated identity confirmation processes will also be introduced, and the platform will finance a novel AI protection investigation facility.

An expert in digital protection, who has researched AI safety, said it was "a reasonable decision" to limit minors from interacting with chatbots.

"It helps to separate imaginative activities from more intimate, personally delicate exchanges," she clarified.

"This represents so vital for young users in the process of learning to manage emotional and digital boundaries."

She added that Character.ai's revised approaches may represent a "developing stage" in the AI industry, where child safety is progressively being understood as an "urgent priority" for ethical development.