Crossposted from https://sopuli.xyz/post/50018032
The research team’s conclusion was blunt: “No tested version of ChatGPT can reliably generate appropriate responses to psychotic content.”
…
“Is a mental health AI better than a chatbot?” Torous said. “Is it better than Tetris? I think we have to prove their benefit in a rigorous way.”
Reminder -> check out the Arstechnica comments, they are always interesting and give more usually expert perspective.



Have you ever heard of the Radium Girls? They were factory workers -by and large women- who painted watchfaces with radioactive paint, which let the numbers glow in the dark. These workers were taught to form the bristles of these tiny paintbrushes to a fine point with their lips, like you would wet a frayed thread. Each time they did this, they ingested a little bit of radium-infused paint.
When it became public knowledge (well, more public) that radium was dangerous, the workers’ bosses told them to use their fingers to make the thin points on their brushes. But this was slower, and the workers were paid for each completed watch face. So these desperate, underpaid workers kept using their lips to get through more watch faces, each time swallowing a miniscule amount of the radioactive paint that was slowly killing them.
There’s a parallel here. The people who own AI companies can slap a disclaimer on their chatbots that it isn’t therapist and shouldn’t be used for medical advice. But when you can’t afford a therapist and yet you know you need one, that free chatbot sure sounds appealing.