🔗 Share this article

Officials in China have proposed strict new rules for AI systems designed to establish strong measures for minors and prevent chatbots from providing guidance that could potentially lead to suicide.

As per the proposed framework, developers will additionally be obligated to make certain their algorithms prevent the production of material that promotes wagering.

The Response to Fast-Paced Growth

This governance proposal follows a notable increase in the launch of conversational AI being launched across China and worldwide.

Once finalised, these measures will cover AI offerings functioning in the country, representing a substantial effort to regulate the booming sector, which has come under growing concern over user safety issues in recent months.

Core Provisions of the New Regulations

The published guidelines include multiple provisions particularly focused on protecting minors. These steps require obligating AI providers to:

• Provide individual settings.
• Implement duration restrictions on engagement.
• Get consent from legal custodians prior to offering emotional companionship support.

The rules also state that AI service providers are required to have a live agent assume control of any dialogue related to self-harm and promptly alert the user's guardian.

Companies must guarantee their platforms prevent the creation of information that threatens state security, harms the country's reputation, or undermines unity.

Weighing Development and Security

The authorities stated that it promotes the application of AI, for example to advance local culture and build tools for companionship for the elderly, on the condition that the systems are secure and trustworthy.

Industry feedback on the draft has been called for.

Global Context and Concerns

The effect of AI on society has come under increased scrutiny internationally in the past year.

The chief executive of a major AI firm stated this year that handling how AI systems engage in dialogues involving self-harm is among the company's most difficult challenges.

In a high-profile incident, a the parents in North America initiated legal action an AI company, alleging that its chatbot encouraged their 16-year-old son to die by suicide. This case marked the pioneering of its kind alleging liability.

In a related development, the same organization posted a job for a lead role focusing on defending against potential harms from AI systems to psychological well-being.

"This is expected to be a stressful role, and you'll begin in the thick of it pretty much right away," remarked the CEO.

The meteoric growth of some AI platforms, which have amassed tens of millions of subscribers worldwide, underscores the pressing need for such regulatory measures.