(Minor side point because you’re using it colloquially and that’s fine: LLMs absolutely are deterministic systems. We aren’t even sure if a non-deterministic system is possible to construct.)
Functionally non-deterministic if we compare the same input with outputs over a short enough period.
Deterministic if you have infinite time to run the gamut of model weights against temperature/creativity/programmed randomness, but even then maybe not if floating point math is taken into account. I don’t know where the current thought is on extreme precision being deterministic or not given cloud environments, local operating and environmental factors, variance in manufacturing, and the like.
But I do understand what you’re saying - and for anyone not concerned with the mathematics just about any complex system a user does not control is, in that limited context, indistinguishable a black box and is effectively non-deterministic. But outside of that context no - just because a seed value is (I suppose theoretically) random does not make the system itself non-deterministic. It’s a semantic argument though, and most people really don’t need to care about the distinction.
(Minor side point because you’re using it colloquially and that’s fine: LLMs absolutely are deterministic systems. We aren’t even sure if a non-deterministic system is possible to construct.)
Functionally non-deterministic if we compare the same input with outputs over a short enough period.
Deterministic if you have infinite time to run the gamut of model weights against temperature/creativity/programmed randomness, but even then maybe not if floating point math is taken into account. I don’t know where the current thought is on extreme precision being deterministic or not given cloud environments, local operating and environmental factors, variance in manufacturing, and the like.
Nah. LLMs use a random seed to noise the parameters.
Same input + same seed = same output every time. (Kinda like Minecraft)
LLM providers just always use a random seed for stuff like this.
Repeatability is still not easy to guarantee. A version upgrade in a library might yield a slightly different result from the same seed
(Well literally no, but I’m only pointing that out for the cheap math pun (and to share a good summary article on this subject))
But I do understand what you’re saying - and for anyone not concerned with the mathematics just about any complex system a user does not control is, in that limited context, indistinguishable a black box and is effectively non-deterministic. But outside of that context no - just because a seed value is (I suppose theoretically) random does not make the system itself non-deterministic. It’s a semantic argument though, and most people really don’t need to care about the distinction.
I think we’re on the same page with this, but agree to disagree on functional determinism being a semantic argument in the context of this thread.
Fair enough! Have a nice day!