alt text
Crudely drawn drawing of a person telling a computer “say ‘i am in pain’”. The computer replies with “> I AM IN PAIN”. The person then says “oh my god.”
Crudely drawn drawing of a person telling a computer “say ‘i am in pain’”. The computer replies with “> I AM IN PAIN”. The person then says “oh my god.”
You’re assuming that for the model to experience something, it’s own weights have to change and I don’t see why. The “experience” can be contained within the forward pass of the model or stored as a “memory” in a kv cache.
If I text you your mother died are you somehow unable to experience anything because I wrote the text? No of course not.
Yes, exactly. An experience, by definition, must change your behavior. The model being bit-for-bit identical in between prompts precludes this. I can experience you telling me my mother died, and my behavior would change as a result: I might conclude you’re an asshole without any credibility, and behave accordingly.
I don’t entirely agree with this definition (most of my day is mundane things that don’t really change me), but even then, it’s behavior does change, just not permanently.
Or if you just protect the kv cache consider a whole system like model + harness + files, then it does in fact change permanently as well.
Don’t ‘really change’ you or don’t change you at all? Are you the sum your experiences?
It does not change, ever. From the moment it starts processing, to the moment it finishes processing, it is still the same model. It has access to all of its input the moment it’s brought into existence, it doesn’t experience the prompt linearly, the input is it’s raison d’être.
I would generally agree that a llm could be part of a larger system capable of experiencing things and having difficult conversations about it, but I don’t think these harnesses are it. They are very simple systems, compared to the llms they control. I think emergent behavior would require something much more complex, given what we know about life.
Okay look, the harness question is a whole nother one. I just want to know, if you consider the model + kv cache together, does that match your definition? The KV cache can change and the model’s internal state and outputs can change in response to it.
KV caching doesn’t persist between prompts, it is an internal optimization used while generating text. I don’t see how it’s relevant.
Edit: it kinda sounds like you’re technobabbling me, like “but it uses a recursive algorithm, it must be alive, check mate!” Your question is a non sequitur.
I’m not techno babbling you. I specifically said “if you consider the model + kv cache together,” e.g preserved. A kv cache is not an internal optimization, a model cannot generate text without a kv cache. There’s a separate idea of caching the kv cache between requests as an optimization. That’s not the one I’m talking about.
Or if it helps, consider it just within one generation. The model + kv cache. Does that not match your definition?
Definition of what? Do I think that because llms use a kv cache during inference, that proves they can experience things? You’re gonna have to explain your thought process. To me, the particular algorithm used is not the philosophical crux of the issue.
I’ve never heard of this anyways. The kv cache is specific to a particular input, it can’t be reused for a subsequent prompts.
It can as long as there’s a shared prefix.
No, I never claimed to have proof they can experience things. I just think it’s possible and you can’t categorically dismiss it based on how they work.
I’m talking about your condition above which we’ve been arguing about for the last ten messages
Which so far seems like your only argument for why they can’t experience things