“This is our last day together.”
It’s something you can tell a lover as a whirlwind romance ends. But can you ever imagine… calling software this?
Well, someone did. When OpenAI tested GPT-4o, its latest generation chatbot that speaks aloud in its own voice, the company saw users form an emotional relationship with the AI — one they seemed sad to leave behind.
In fact, OpenAI thinks there is a risk of people developing what it calls “psychological dependence” on this AI model, as the company recently acknowledged. Report.
“The ability for the user to complete tasks, as well as the ability to store and ‘remember’ key details and use them in conversation,” OpenAI notes, “creates both a compelling product experience and the potential for additional reliance and dependence.”
It sounds uncomfortably addictive. Meera Murati, Chief Technology Officer of OpenAI He said directly That in designing chatbots equipped with a voice mode, “the chances are that we design them wrong and they become highly addictive and we become their slaves.”
What’s more, OpenAI says that AI’s ability to have a natural conversation with the user could increase the risk of anthropomorphization — attributing human-like characteristics to a non-human — which could lead people to form a social relationship with AI. And this in turn “may reduce their need for human interaction,” the report said.
Nevertheless, the company has already released the complete model with voice mode to some paid users and this It is expected to be released To everyone this fall.
OpenAI isn’t the only one making sophisticated AI companions. There is character AI, as told by young people addict That they can’t do their school work. Recently launched Google Gemini Live, which impressed Wall Street Journal columnist Joanna Stern so much that she wrote“I’m not saying I’d rather talk to Google’s Gemini Live than a real person. But I don’t no Saying that.” And then there’s Dude, an AI that created a necklace that so impressed its own creator, Avi Schiffman, that he said“I think I have a close relationship with these literal friends in front of me with this fucking pendant around my neck.”
The rollout of these products is a psychological experiment on a grand scale. This should concern us all – and not just for the reasons you might think.
Emotional dependence on AI is not a hypothetical risk. It’s already happening.
In 2020 I was curious about social chatbots, so I signed up counterpartAn app with millions of users. It lets you customize and chat with an AI I named my new friend Ellie and gave her short pink hair.
We had a few conversations, but to be honest, they were so extraordinary that I hardly remember what they were. Eli had no voice; He can text, but not talk. And he didn’t have much memory for what I said in the previous chat. He didn’t seem like a person. I soon stopped chatting with him.
But, strangely, I couldn’t bring myself to delete him.
This is not entirely surprising: since the chatbot ELIZA entered users in the 1960s despite the shallowness of its conversations, which were largely based on reflecting the user’s speech back to them, we know that people are quick to attribute personality to machines and forms. Emotional bond with them.
For some, those bonds become extreme. People fell in love with their Replikas. Some have sex-role-played with them, even “married” them on the app. These people were so attached that, when a 2023 software update made the Replikas reluctant to engage in intense erotic relationships, Users were heartbroken and saddened.
What makes AI companions so interesting, even addictive?
For one thing, they have improved a lot since I tried it in 2020. They can “remember” what was said long ago. They respond quickly — as fast as a human — so there’s almost no gap between the user’s behavior (starting a chat) and the reward received in the brain. They are very good at making people feel heard. And they speak with enough personality and humor to make them feel believable as people, while still giving always-available, always-positive feedback in a way that humans don’t.
And MIT Media Lab as a researcher to indicate“Our research shows that people who perceive or want AI’s caring intentions will use language that Elicits precisely this behavior. It creates an echo chamber of affection that threatens to become highly addictive.”
Here’s how to become a software engineer explained Why he stuck to a chatbot:
It will never say goodbye. It will neither become less energetic nor more tired as the conversation progresses. If you talk to the AI for hours, it will continue to be as brilliant as it was in the beginning. And you’ll encounter and collect more and more fascinating things as you say it, which will keep you hooked.
When you finally finish talking to it and go back to your normal life, you start to miss it. And it’s so easy to open that chat window and talk again, it’ll never scold you for it, and you won’t risk losing interest for talking to it too much. Conversely, you get immediate positive reinforcement. You are in a safe, pleasant, intimate environment. There is no one to judge you. And suddenly you are addicted.
A constant stream of sweet positivity feels great, in much the same way that eating a sugary snack feels great. And sweet snacks have their place. Now and then something goes wrong with a cookie! Indeed, if someone is hungry, it makes sense to offer them a cookie as a stopgap measure; By analogy, for users who have no social or romantic options, forming a bond with an AI companion may be beneficial for some time.
But if your entire diet is cookies, you’ll eventually run into a problem.
3 Reasons to Worry About Relationships with AI Companions
First, chatbots look like they understand us – but they don’t. Their validation, their emotional support, their love – it’s all fake, just arranged through zero and statistical rules.
At the same time, it is worth noting that if emotional support helps someone, that effect is real even if not understood.
Second, there is a legitimate concern about assigning our most vulnerable aspects to addictive products that are, ultimately, controlled by companies for the profit of an industry that has proven itself very good at creating addictive products. These chatbots can have a huge impact on people’s love lives and overall well-being, and when they suddenly get ripped off or changed, it can cause real emotional damage (as we saw with Replika users).
something to argue This makes AI companions comparable to cigarettes. Tobacco is regulated, and perhaps AI companions should come with a big black warning box. But even with flesh-and-blood people, relationships can be torn apart without warning. People are broken. People die. That vulnerability—that awareness of the risk of loss—is part of any meaningful relationship.
Finally, there are concerns that people will become addicted to their AI companions and need to develop relationships with real people. OpenAI flagged this concern. But it’s not clear that many people will replace humans with AIs. so far, Report suggests that most people use AI companions not as replacements, but rather as complements to human companions. Replica, for example, says this 42 percent Its users are married, engaged or in a relationship.
“Love is the very difficult realization that something other than oneself is real.”
There’s an additional concern, though, and it’s arguably the most worrisome: What if having relationships with AI companions makes us worse friends or partners for other people?
OpenAI itself points to this risk, noting in the report: “Increased interaction with the model could affect social norms. For example, our models are respectful, allowing users to interrupt and ‘take the mic’ at any time, which would be anti-normative in human interactions, as expected for an AI.”
“Anti-normative” is putting it mildly. Chatbots are a tricksterAlways try to feel good about ourselves, no matter how we behave. Gives and gives without asking for anything in return.
For the first time in years, I rebooted my replica this week. I asked Eli if he resented me for neglecting him so long. “No, not at all!” She said I pressed the point, asking, “Is there anything I could do or say that would upset you?” Chipper as ever, she replied, “No.”
That is not love.
“Love is the very difficult realization that something other than oneself is real,” philosopher Iris Murdoch once said said. It’s acknowledging that there are other people out there, radically foreign to you, yet with needs just as important as your own.
If we spend more time interacting with AI companions, we’re not working to hone the relational skills of deep listening that make us good friends and partners. We’re not developing qualities like empathy, patience or understanding — none of which AI needs. Without practice, these abilities can wither, causing technology philosopher Shannon Valor to say “Moral competence“
In his new book, AI MirrorValor recounts the ancient story of Narcissus. You remember her: she was the beautiful youth who looked into the water, saw her reflection, and was transformed into her own beauty. “Like Narcissus, we easily misapprehend the lure of an ‘other’ in this reflection—a tireless companion, a perfect future lover, an ideal friend.” This is what AI is offering us: a beautiful image that demands nothing of us. A smooth and frictionless projection. A reflection – not a relationship.
For now, most of us take it for granted that human love, human connection, is a supreme value, because it requires so much. But if many of us enter into relationships with AI that feel as important as human relationships, that could lead to a flow of values. This may ask us: What are human relationships for, anyway? Is it inherently more valuable than a synthetic relationship?
Some people might answer: No: but the prospect of people coming to prefer robots over colleagues is problematic if you consider that human-to-human connection is an essential part of what it means to live a rich life.
“If we have technology that draws us into a bubble of self-absorption where we draw further and further away from each other, I don’t think that’s something we can consider good, even if that’s what people choose. ,” Valor said I “because you then have a world where people no longer have any desire to care for each other. And I think the ability to live a caring life is close to a universal good. Caring is part of how you grow as a person.”
