Functionally non-deterministic if we compare the same input with outputs over a short enough period.
Deterministic if you have infinite time to run the gamut of model weights against temperature/creativity/programmed randomness, but even then maybe not if floating point math is taken into account. I don’t know where the current thought is on extreme precision being deterministic or not given cloud environments, local operating and environmental factors, variance in manufacturing, and the like.
I’d argue repeatability is impossible to guarantee. Any cloud based computing with floating point math is subject to variance. Bare metal local LLM computing can be relatively stable, but still suffers from precision error from your GPU.
But I do understand what you’re saying - and for anyone not concerned with the mathematics just about any complex system a user does not control is, in that limited context, indistinguishable a black box and is effectively non-deterministic. But outside of that context no - just because a seed value is (I suppose theoretically) random does not make the system itself non-deterministic. It’s a semantic argument though, and most people really don’t need to care about the distinction.
Functionally non-deterministic if we compare the same input with outputs over a short enough period.
Deterministic if you have infinite time to run the gamut of model weights against temperature/creativity/programmed randomness, but even then maybe not if floating point math is taken into account. I don’t know where the current thought is on extreme precision being deterministic or not given cloud environments, local operating and environmental factors, variance in manufacturing, and the like.
Nah. LLMs use a random seed to noise the parameters.
Same input + same seed = same output every time. (Kinda like Minecraft)
LLM providers just always use a random seed for stuff like this.
Repeatability is still not easy to guarantee. A version upgrade in a library might yield a slightly different result from the same seed
I’d argue repeatability is impossible to guarantee. Any cloud based computing with floating point math is subject to variance. Bare metal local LLM computing can be relatively stable, but still suffers from precision error from your GPU.
(Well literally no, but I’m only pointing that out for the cheap math pun (and to share a good summary article on this subject))
But I do understand what you’re saying - and for anyone not concerned with the mathematics just about any complex system a user does not control is, in that limited context, indistinguishable a black box and is effectively non-deterministic. But outside of that context no - just because a seed value is (I suppose theoretically) random does not make the system itself non-deterministic. It’s a semantic argument though, and most people really don’t need to care about the distinction.
I think we’re on the same page with this, but agree to disagree on functional determinism being a semantic argument in the context of this thread.
Fair enough! Have a nice day!