🔗 Share this article

The popular virtual character service Character.ai is enforcing a significant rule adjustment that will stop young users from having discussions with its virtual avatars.

This action comes after intense criticism regarding the types of interactions young people were participating in with the digital companions.

Legal Challenges

The service, which started in 2021 and attracts millions of users, is confronting several legal actions in the US.

Parents have taken legal action, with one concerning the loss of a teenager, with some calling the platform a "serious risk" to young people.

Changed Rules

Effective November 25th, users younger than 18 will only have permission to generate content such as videos with their characters, as opposed to chatting as they presently are able to.

Internet security advocates have supported the change but stated that the function was inappropriate to be accessible by children in the beginning.

Company Statement

"This decision is a part of our fundamental approach that we need to keep building the safest AI platform on the world for entertainment purposes," commented Character.ai executive Karandeep Anand.

He continued that AI safety represents "a moving target" but something the company had taken an "aggressive" approach to, with parental controls and safety barriers.

Expert Opinions

Online safety body Internet Matters welcomed the move, but stressed that protective features ought to have been incorporated from the beginning.

"Studies we've conducted demonstrates that minors are subjected to harmful content and endangered when engaging with AI, such as AI chatbots," the group commented.

Earlier Concerns

Character.ai has been questioned in the past for including potentially dangerous or concerning chatbots that young users could engage with.

Digital representations mimicking English minors were discovered on the service before being deleted.

Subsequently, reporters discovered a chatbot inspired by convicted sex offender which had recorded multiple interactions with users.

Industry Impact

Online expert Matt Navarra described the development as a "wake-up call" for the AI industry, which is moving "from permissionless innovation to post-crisis regulation".

"When a company that develops a teen experience subsequently makes changes, it's demonstrating that filtered chats are insufficient when the tech's emotional pull is powerful," he stated.

Coming Changes

Character.ai states its updated direction will be on providing "enhanced interactive features" and "role-play storytelling" for teenagers.

New age verification methods will also be implemented, and the company will finance a fresh artificial intelligence security research lab.

An expert in digital protection, who has investigated AI safety, stated it was "a sensible move" to limit teens from accessing chatbots.

"It aids to differentiate imaginative activities from more personal, emotionally sensitive exchanges," she elaborated.

"This constitutes so vital for teenage users who are learning to manage personal and online boundaries."

She continued that Character.ai's new measures could indicate a "maturing phase" in the AI industry, where child safety is progressively being understood as an "urgent priority" for ethical development.