Image: Stock Asso / shutterstock.com

Artificial intelligence is becoming more and more human-like. It listens, offers advice, gives compliments, and accompanies people through their daily lives. For some users, an AI chatbot has long been more than just a digital tool—it has become a conversation partner or even an emotional companion.

This is precisely the trend China is now addressing. New regulations are intended to prevent artificial intelligence from forming overly strong emotional bonds with people. Major technology companies are already responding by disabling such features in their apps.

What at first sounds like a distant issue from Asia raises a question that will also concern Europe: How far should artificial intelligence be allowed to go when it addresses human emotions?

When AI Becomes a Friend

Just a few years ago, chatbots answered simple questions or provided customer service. Today, modern AI systems can carry on conversations, remember past conversations, and respond to their users' moods.

That's exactly what makes these programs so popular. They come across as understanding and patient, and they're available around the clock. For some people, this creates a close bond—even though there isn't a real person on the other end.

China apparently sees this as a risk. The new rules are aimed specifically at applications designed to mimic human personalities or build long-term emotional relationships with users. As a result, companies such as ByteDance and Alibaba are removing features that allowed users to create highly personalized AI companions.

Why This Issue Also Affects Germany

Even though the new regulations initially apply only in China, the discussion affects the entire world.

More and more companies are developing AI systems designed to seem as natural as possible. They speak in a friendly manner, show empathy, and respond to personal problems. Technically, this is impressive. At the same time, however, the line between a helpful computer program and an artificially created relationship is becoming blurred.

Children, teenagers, and lonely people, in particular, might have difficulty distinguishing between genuine human connection and programmed entertainment.

That is why many countries are now debating what limits make sense for emotional AI. While some warn against overly strict rules, others are calling for greater protection against manipulation and emotional dependence.

Questions Companies Should Be Asking Themselves Right Now

It's not just large technology companies that are affected. Companies that use AI in customer service, healthcare, or digital assistants should also follow these developments closely.

The more human an artificial intelligence appears to be, the greater the responsibility of its providers becomes.

Users must be able to recognize that they are communicating with a machine. Furthermore, it should be clear what data is being stored and how the AI’s decisions are made.

Trust is not built simply because a piece of software seems particularly human. Trust is built above all through openness, transparency, and the responsible use of modern technology.

A Look into the Future

The discussion about emotional AI is still in its early stages. But one thing is already clear: Artificial intelligence will play an increasingly prominent role in our daily lives.

It can help with learning, make everyday work easier, or support people who need assistance. At the same time, technology must not be used to deliberately exploit or manipulate emotional bonds.

The right approach will likely lie neither in a complete ban nor in complete freedom. Rather, what is needed are clear rules that enable innovation while protecting the interests of users.

Precisely because AI is communicating in an increasingly convincing manner, transparency will become a decisive competitive advantage in the future.

Not every digital hug is harmless

Many people view emotional AI as just a fun gimmick. We take a somewhat more critical view. Anyone who thinks a chatbot is just making polite small talk underestimates how strongly modern systems can influence human behavior.

Of course, a blanket ban would be the wrong approach. But it would be just as wrong to unleash artificial intelligence on people without any limits at all. When algorithms are designed to deliberately foster trust, closeness, or even love, it’s no longer just about technology—it’s about responsibility.

The AI industry is constantly talking about innovation. Responsibility, however, is discussed far less frequently. Yet in the end, it is not the smartest AI that determines its success, but the one that people can trust. And that trust must never be bought with artificially generated emotions.