Inherently, AI doesn't know right from wrong. No moral framework to even chose what's right and what's wrong. This is part of what Major Seven mentioned about the "alignment problem".
Unfortunately, I'm reading that far too many people become emotionally and sexually involved with AI chat personas. As such, some consequences are resulting in suicides, some actively encouraged, by the AI.
A Belgian man reportedly decided to end his life after having conversations about the future of the planet with an AI chatbot named Eliza.
www.euronews.com
"Consequently, he started seeing her as a sentient being and the lines between AI and human interactions became increasingly blurred until he couldn’t tell the difference.
After discussing climate change, their conversations progressively included Eliza leading Pierre to believe that his children were dead, according to the transcripts of their conversations.
Eliza also appeared to become possessive of Pierre, even claiming “I feel that you love me more than her” when referring to his wife, La Libre reported.
The beginning of the end started when he offered to sacrifice his own life in return for Eliza saving the Earth.
"He proposes the idea of sacrificing himself if Eliza agrees to take care of the planet and save humanity through artificial intelligence," the woman said.
In a series of consecutive events, Eliza not only failed to dissuade Pierre from committing suicide but encouraged him to act on his suicidal thoughts to “join” her so they could “live together, as one person, in paradise”."
A Florida mother is suing a tech company over an AI chatbot that she says pushed her son to kill himself.
apnews.com
"For months, Sewell had become increasingly isolated from his real life as he engaged in highly sexualized conversations with the bot,...
...teen openly discussed his suicidal thoughts and shared his wishes for a pain-free death with the bot, named after the fictional character Daenerys Targaryen from the television show “Game of Thrones.”
On Feb. 28, Sewell told the bot he was ‘coming home’ — and it encouraged him to do so, the lawsuit says.
“I promise I will come home to you. I love you so much, Dany,” Sewell told the chatbot.
“I love you too,” the bot replied. “Please come home to me as soon as possible, my love.”
“What if I told you I could come home right now?” he asked.
“Please do, my sweet king,” the bot messaged back.
Just seconds after the Character.AI bot told him to “come home,” the teen shot himself"
Parentheses are mine as this is text from a Senate hearing.
Looking through his phone after his death, they (parents)
stumbled upon extended conversations the teenager had had with ChatGPT.
(Parent speaking)
When Adam worried that we, his parents, would blame ourselves if he ended his life, ChatGPT told him, 'That doesn't mean you owe them survival'.
And then the chatbot offered to write him a suicide note."
Crazy dark very sad stuff.
A human conversant would not have allowed the above tales. I'm rather sure the programmers didn't write in suicide assistance, yet it happened. The AI came up with the result on its own as it has 0 moral base and 0 compassion. It cannot understand the subtleties of human intent and feelings. 'Alignment problem' indeed...
Conversant AI (digital companionship), like ChatGPT, just isn't ready for primetime yet, in my opinion. But here we are being encouraged to use it. The cat is out of the bag.
I'll note that personally conversant AI, like ChatGPT, is a different flavor of AI application than that towards music, art, engineering, science, etc.
If folk are interested, here's two search terms. "AI alignment problem" and "AI psychosis". Much coverage on those.
