Crossposted from https://sopuli.xyz/post/50018032
The research team’s conclusion was blunt: “No tested version of ChatGPT can reliably generate appropriate responses to psychotic content.”
…
“Is a mental health AI better than a chatbot?” Torous said. “Is it better than Tetris? I think we have to prove their benefit in a rigorous way.”
Reminder -> check out the Arstechnica comments, they are always interesting and give more usually expert perspective.
No. A word predictor is not going to be able to give these people what they need. What they need is tailor made psychiatric help from a qualified physician or health practitioner.
The very reason that these language models are so “good” at enabling people to do bad things in moments of crisis is because they are designed to tell the user engaging them reaffirming statements in order to keep them on the hook spending tokens.
Edit: The absolute best we could possibly do is to flag that behavior as concerning and force a universal healthcare system to do wellness checks to get these people the help they need before the worst happens. But of course, to do that we’d actually need universal healthcare. Plus, that comes with the problem of actually tying the account to the real identity of the person which I’m pretty against.