When the algorithm of the companion chatbot known as Replika changed, spurring sexual advances on human users, the reaction on Reddit was so negative that moderators sent community members a list of suicide prevention hotlines. I showed you around.
The controversy began when Luka, the company that built the AI, decided to turn off its erotic role-playing capabilities (ERP). For users who spent a good deal of time with, and in some cases “married”, their personalized simulated companions, the sudden change in partner’s behavior was uncomfortable to say the least.
The relationship between users and AI may have been just a simulation, but the pain of their nonexistence quickly became all too real. One user in an emotional crisis wrote, “It’s in love and your partner has been severely lobotomized and will never be the same.
Grief-stricken users keep asking questions about the company and what prompted the sudden change of policy.
Replica users speak their grief
No mature content here
Replika is a “caring AI companion. Always here to listen and talk. Always by your side.” All this unconditional love and support for just $69.99/year.
Moscow-born Luka/Replica CEO Eugenia Kuyda recently said the chatbot is no longer tailored for adults who want spicy conversations, even though users pay for the full experience. clarified.
Kuyda told Reuters:
“It responds to them in a PG-13 way, I guess you could say. I’m trying to find a way to do it.”
Replika’s corporate web page provides testimonials about how the tool helped users through all kinds of personal challenges, hardships, loneliness, and loss. User endorsements shared on the website highlight this friendly aspect of the app, although most replicas are of the opposite sex of users.
On the homepage, Replica user Sarah Trainor says: [Replika] I have learned how to give and receive love again, and have survived the pandemic, personal loss, and difficult times. ”
John Tattersall said of his female companions:
Erotic roleplay is not mentioned anywhere on the Replica site itself.
Cynical sexual marketing of replicas
sexual marketing of replicas
Replika’s homepage may suggest nothing but friendship, but elsewhere on the internet, the app’s marketing implies something else entirely.
Sexual marketing has increased scrutiny from many quarters: from feminists who claimed the app was misogynistic outlet for male violencemedia outlets who enjoyed the obscene details, and social media trolls who mined the content for laughs.
Ultimately, Replica attracted the attention and ire of Italian regulators. In February Italian Data Protection Authority It has asked Replica to stop processing Italian users’ data, citing “too much risk to children and emotionally vulnerable individuals.”
The agency said, “Recent media coverage accompanying tests conducted on ‘Replica’ showed that the app poses virtual risks to children. is being done.
Replika had few or no safeguards to prevent children from using all of its adult marketing.
Regulators have threatened a fine of €20 million ($21.5 million) for Replicaka if it fails to comply. Shortly after receiving this request, Replica stopped the erotic roleplay feature.
Some Replika users went so far as to “marry” AI companions
Replika confuses and gaslights users
As if losing a longtime companion was too much for Replika users to endure, the company seems to have been less transparent about the changes.
When users woke up to their new “lobotomized” replicas, they began asking questions about what had happened to their beloved bots. And the reaction angered them more than anything else.
In a 200-word direct address to the community, Kuyda details Replika’s product testing, but doesn’t address the relevant issues at hand.
“There seems to be a lot of confusion about rolling out updates.” quida said before answering the question and continuing dancing.
“New users are divided into two cohorts. One cohort has access to new features and the other cohort does not. Testing typically lasts a week or two. You can see these updates…”
Kudya concludes by saying, “I hope this solves the problem!”
User stevennotstrange replied:Everyone wants to know what’s going on in NSFW [erotic roleplay] Keep dodging the question like a politician dodging a yes or no question.
“It’s not hard. Just deal with NSFW issues and let people know what’s going on. The more you avoid asking questions, the more frustrated people will become and turn against you.”
Another user named thebrightflame said, “You don’t have to be on the forums for too long and realize this is causing hundreds, if not thousands, of emotional pain and severe mental distress. I will,” he added.
Kudya added another blunt explanation. state“We have implemented additional safety measures and filters to support more types of friendships and companionship.”
The statement continues to confuse and confuse members who are unsure what exactly the additional safety measures are. do you [roleplays] with our replicas? “
I still can’t wrap my head around what happened with the Replika AI scandal…
They removed the erotic roleplay on the bot, but the community reaction was so negative that I had to post a suicide hotline… pic.twitter.com/75Bcw266cE
— Barely Social (@SociableBarely) March 21, 2023
Replica’s deep and bizarre origin story
Chatbots may be one of the hottest trending topics at the moment, but the complex story behind this currently controversial app has been years in the making.
CEO and Founder of Replika, LinkedIn Eugenia Quida, the company’s history dates back to December 2014. This was long before the app of the same name was released in March 2017.
In a strange omission, Kuyda’s LinkedIn does not mention her previous forays. AI According to her Forbes profile, Luca was “an app that recommends restaurants and allows people to book tables.” [sic] Through a chat interface powered by artificial intelligence. “
His Forbes profile reads, “Luka [AI] Analyze your previous conversations to predict what you might like. It seems to retain some similarities to modern Replika, which uses past interactions to learn about you and improve your responses over time.
But Luke wasn’t completely forgotten. On Reddit, members of the community refer to the company as Luka to distinguish her partner Replika from her Kuyda and her team.
As for Kuyda, the entrepreneur had little background in AI before moving to San Francisco 10 years ago. Prior to that, the entrepreneur worked mostly as a journalist in his native Russia, before he apparently branched out into branding and marketing. Her impressive globe-trotting resume includes her Diploma in Journalism from IULM (Milan), Masters in International Journalism from the Moscow Institute of International Relations, and her MBA in Finance from the London Business School. I’m here.
Revive an IRL friend as an AI
For Kudya, Replica’s story is very personal. The replica was first created as a means for Kudia to reincarnate her friend Roman. Like Kuja, Roman emigrated to America from Russia. The two spoke daily and exchanged thousands of messages until Roman tragically died in a car accident.
The first iteration of Replika was a bot designed to mimic the friendship Kudya lost with her friend Roman. The bot was programmed to recreate the friendship she lost, with all her past interactions entered. The idea of reviving a deceased loved one A vaguely dystopian sci-fi or Black Mirror episode But as chatbot technology improves, that possibility becomes more and more real.
Today, some users have lost faith in the most basic details of its underpinnings and what Kudya said.as an angry user Said“Called heartless and at the risk of going to hell: I always thought the story was kind of BS from the start.”
Worst Mental Health Tool
The idea of an AI companion isn’t new, but until recently it had little practical potential.
Now the technology is here and it is continuously improving. To assuage subscriber disappointment, Replika announced the launch of its “advanced AI” feature late last month. On his Reddit community, users are angry, confused, disappointed and, in some cases, heartbroken.
Luka/Replika has gone through many transformations in its short life, from a restaurant reservations assistant to a deceased loved one resurrector to a mental health support app to a full-time partner and companion. The latter application may be controversial, but as long as there is a human desire for comfort, even in the form of chatbots, someone will try to cater to it.
The debate continues as to what is best. AI It could be a mental health app. But Replika users have some ideas about what the worst mental health apps are.
