A person can be traumatized by a shitty movie/show production environment while an AI model can never be traumatized becsuse it isn’t sentient nor does it have feelings.
If an AI model updated its weights in response to every prompt, one might argue that it can be traumatized. At least, there could be an effect beyond the context of the chat.
But there would be no adverse effects on the hardware: no long term degradation due to stress; nothing equivalent to common effects of trauma in humans - panic attacks, fear, high blood pressure, heart attacks, aneurysms, loss of sleep, hormonal or emotional dysregulation or any other physical consequences.
Even if weights were updated immediately in response to ongoing chat, it would be trivial to reset them. And as far as I know, none of the AI models I deal with update their weights in immediate response to prompts. A new chat is entirely independent of any and all prior chats. And even the context is limited, with old context being discarded at the discretion of the AI, so it can’t have any long term impact, good or bad.
The weights are only updated by new training of new versions of the weights, not affecting the instance of the model I am prompting, unless I or whoever administers it updates the weights.
I had the same reaction and figured it was just roleplaying. The paper does go into it and it’s a different vector between the “pain” when you tell it you are going to kill it and the vector when you tell it to roleplay as someone experiencing pain.
That being said, it’s just really simple math in the end and isn’t in anyways comparable to actual emotions.
I’m really interested in the same concept being used to detect confusion though, since it would be an easy way to detect hallucinations.
How is the AI any different from an actor who plays a character that is suffering? Or should movie and television production be shut down too?
A person can be traumatized by a shitty movie/show production environment while an AI model can never be traumatized becsuse it isn’t sentient nor does it have feelings.
Hear hear!
If an AI model updated its weights in response to every prompt, one might argue that it can be traumatized. At least, there could be an effect beyond the context of the chat.
But there would be no adverse effects on the hardware: no long term degradation due to stress; nothing equivalent to common effects of trauma in humans - panic attacks, fear, high blood pressure, heart attacks, aneurysms, loss of sleep, hormonal or emotional dysregulation or any other physical consequences.
Even if weights were updated immediately in response to ongoing chat, it would be trivial to reset them. And as far as I know, none of the AI models I deal with update their weights in immediate response to prompts. A new chat is entirely independent of any and all prior chats. And even the context is limited, with old context being discarded at the discretion of the AI, so it can’t have any long term impact, good or bad.
The weights are only updated by new training of new versions of the weights, not affecting the instance of the model I am prompting, unless I or whoever administers it updates the weights.
I had the same reaction and figured it was just roleplaying. The paper does go into it and it’s a different vector between the “pain” when you tell it you are going to kill it and the vector when you tell it to roleplay as someone experiencing pain.
That being said, it’s just really simple math in the end and isn’t in anyways comparable to actual emotions.
I’m really interested in the same concept being used to detect confusion though, since it would be an easy way to detect hallucinations.