I’m kind of gobsmacked by this. I’ve heard younger developers compare LLM with compiler and chalked it up to ignorance, but I don’t get what this is.
Can I write code in assembly? Yes. Of course. It’s true that I never do it, but if there was a rule that you needed to write some assembly now and then to certify as a kernel developer I don’t see how it would be any kind of impediment.
Comparing more directly: if you hand me a compiled binary and the source code and ask “why is the bit at address 0xXXXXXXXXX a 0?” I can, with 100% certainty, answer that. Probably in less than a day of effort. If I ask "why did the LLM output “for” as the XXXXth word of its utterance, can Linus say why?
And “can you write a good commit message without an editor?” Uhhh? At least when I was in school, this was something people did for fun/messing around. I still do cat > somefile when I need to put some quick text together. In fact, Claude will sometimes write a Git commit by cat-ing into a file–which it’s doing because there’s so many online examples of this. Fucking weird things for Linus to pick. Makes me think his brain’s rotted.
If I ask "why did the LLM output “for” as the XXXXth word of its utterance, can Linus say why?
If you have the model that was used, and the seed used for the generation, then yes you can trace the weights and figure out how everything was generated.
AI is deterministic. And compilers can be just as “nondeterministic” if you don’t know what compiler was used and what flags were enabled.
I had the misfortune of working a lot with claude skills for the past few months.
I run the exact same skill, on the exact same codebase and it doesn’t yield the same results. Sometimes it straight up ignores parts my prompt. Other times it ignores the skill file entirely. And then I ask Claude, why did it do X thing that it wasn’t supposed to do? The answer is “I apologize, I should have blabla”. It is extremely unreliable and unpredictable.
Now I’m stuck with a feature that I’m responsible for that occasionally goes rogue. Fun times.
Nearly all evaluation steps for an LLM I produce some level of entropy. Yes you can evaluate an LLM model deterministically, but we don’t. We even tune the levels to get good results. Model cards have sampling parameters supplied to give “the best” results. For example, Qwen 3.8 flash next gives
We recommend using the following sets of sampling parameters for generation:
Replaying the seed will only get you the same result using a particular engine and a particular random number generator. Run it on llama.cpp Vs Strata and you can get different results. Different libc, different results.
A compiler is systematically deterministic. No calls to rand()
Replaying the seed will only get you the same result using a particular engine and a particular random number generator
And the seed and rng are all part of the LLM. If you use the same inputs you get the same output. That’s the definition of determinism.
Determinism does not care whether or not the user/programmer understands the program or not. The word you’re probably looking for is “unpredictable” or “less predictable”.
Everybody understands what you’re saying. Yes, the source of randomness in a computer is deterministic (unless you have a physical source of entropy). Every computer program is deterministic.
Nothing about the operation of LLMs relys on that determinism or views it as useful. LLMs are probabilistic in nature and require randomness (emulated or real). When people are saying it’s not deterministic they are stating that there are multiple outputs possible for a single input (excluding the seed to the RNG because it’s not necessary to use a deterministic source of randomness).
That is fundamentally different to a compiler, which is the overall point being made.
All I’m saying is that determinism has a specific meaning in computer science. In an LLM, the rng value is determined by the seed. Given the same inputs, an LLM always produces the same output. It does not matter whether the user understands the code or operation at all. It is purely dependent on the program itself.
A non-deterministic program, is one where the output is determined by factors outside the code itself. This usually requires non-deterministic programming language constructs. For example, if you convert a set to a list in python, the order of the elements is non-deterministic. Out of the control of the program. Same inputs, different output.
Because weight x = y isn’t really a satisfactory answer to the “why” in question though. To really answer that you need to refer to the pretraining and context, which in a lot of cases today, and likely forever, since there’s no reason for anyone to cold start model training ever again, is necessarily opaque. A lot of the best why answers are going to be given by the model itself (which… obviously circular). Trust and attribution are just really hard in this paradigm. Not necessarily so different from the challenge of trusting/attributing human output, except that humans are more diverse and in general difficult to manipulate.
You make a good point about capability. However, I’m trying to say that in practical capability, I do know exactly what compiler flags were enabled (it’s whatever the Makefile passed, which is whatever configure determined, the whole thing is checked in as source code), but the exact model+seed isn’t being recorded for this Linux changes.
To make the compiler analogy, we should be saving model, seed, prompt, etc and then the LLM output becomes the equivalent of the compiled code. I think that’s an extremely defensible argument, while “bet you can’t write assembly!” is not.
For local AI image generation, often the input parameters are indeed recorded into the output image. You can then use the metadata of the image to get the parameters and re-generate the exact same image. This is the definition of determinism in computer science.
I’m kind of gobsmacked by this. I’ve heard younger developers compare LLM with compiler and chalked it up to ignorance, but I don’t get what this is.
Can I write code in assembly? Yes. Of course. It’s true that I never do it, but if there was a rule that you needed to write some assembly now and then to certify as a kernel developer I don’t see how it would be any kind of impediment.
Comparing more directly: if you hand me a compiled binary and the source code and ask “why is the bit at address 0xXXXXXXXXX a 0?” I can, with 100% certainty, answer that. Probably in less than a day of effort. If I ask "why did the LLM output “for” as the XXXXth word of its utterance, can Linus say why?
And “can you write a good commit message without an editor?” Uhhh? At least when I was in school, this was something people did for fun/messing around. I still do
cat > somefilewhen I need to put some quick text together. In fact, Claude will sometimes write a Git commit by cat-ing into a file–which it’s doing because there’s so many online examples of this. Fucking weird things for Linus to pick. Makes me think his brain’s rotted.If you have the model that was used, and the seed used for the generation, then yes you can trace the weights and figure out how everything was generated.
AI is deterministic. And compilers can be just as “nondeterministic” if you don’t know what compiler was used and what flags were enabled.
I had the misfortune of working a lot with claude skills for the past few months. I run the exact same skill, on the exact same codebase and it doesn’t yield the same results. Sometimes it straight up ignores parts my prompt. Other times it ignores the skill file entirely. And then I ask Claude, why did it do X thing that it wasn’t supposed to do? The answer is “I apologize, I should have blabla”. It is extremely unreliable and unpredictable. Now I’m stuck with a feature that I’m responsible for that occasionally goes rogue. Fun times.
Nearly all evaluation steps for an LLM I produce some level of entropy. Yes you can evaluate an LLM model deterministically, but we don’t. We even tune the levels to get good results. Model cards have sampling parameters supplied to give “the best” results. For example, Qwen 3.8 flash next gives
Replaying the seed will only get you the same result using a particular engine and a particular random number generator. Run it on llama.cpp Vs Strata and you can get different results. Different libc, different results.
A compiler is systematically deterministic. No calls to
rand()And the seed and rng are all part of the LLM. If you use the same inputs you get the same output. That’s the definition of determinism.
Determinism does not care whether or not the user/programmer understands the program or not. The word you’re probably looking for is “unpredictable” or “less predictable”.
Everybody understands what you’re saying. Yes, the source of randomness in a computer is deterministic (unless you have a physical source of entropy). Every computer program is deterministic.
Nothing about the operation of LLMs relys on that determinism or views it as useful. LLMs are probabilistic in nature and require randomness (emulated or real). When people are saying it’s not deterministic they are stating that there are multiple outputs possible for a single input (excluding the seed to the RNG because it’s not necessary to use a deterministic source of randomness).
That is fundamentally different to a compiler, which is the overall point being made.
AI is not deterministic at all, only if you keep the seed values for generating random numbers the same all over the pipeline.
It’s literally a huge stochastic model.
All I’m saying is that determinism has a specific meaning in computer science. In an LLM, the rng value is determined by the seed. Given the same inputs, an LLM always produces the same output. It does not matter whether the user understands the code or operation at all. It is purely dependent on the program itself.
A non-deterministic program, is one where the output is determined by factors outside the code itself. This usually requires non-deterministic programming language constructs. For example, if you convert a set to a list in python, the order of the elements is non-deterministic. Out of the control of the program. Same inputs, different output.
Because weight x = y isn’t really a satisfactory answer to the “why” in question though. To really answer that you need to refer to the pretraining and context, which in a lot of cases today, and likely forever, since there’s no reason for anyone to cold start model training ever again, is necessarily opaque. A lot of the best why answers are going to be given by the model itself (which… obviously circular). Trust and attribution are just really hard in this paradigm. Not necessarily so different from the challenge of trusting/attributing human output, except that humans are more diverse and in general difficult to manipulate.
You make a good point about capability. However, I’m trying to say that in practical capability, I do know exactly what compiler flags were enabled (it’s whatever the Makefile passed, which is whatever configure determined, the whole thing is checked in as source code), but the exact model+seed isn’t being recorded for this Linux changes.
To make the compiler analogy, we should be saving model, seed, prompt, etc and then the LLM output becomes the equivalent of the compiled code. I think that’s an extremely defensible argument, while “bet you can’t write assembly!” is not.
For local AI image generation, often the input parameters are indeed recorded into the output image. You can then use the metadata of the image to get the parameters and re-generate the exact same image. This is the definition of determinism in computer science.
There was another article posted on Lemmy somewhere where recently other kernel developers had to clean up after Torvalds as of recent.
So yeah. I’d say his brain has rotted.
Would be proper to attach some links for such a claim, I think. (I’m not saying you’re wrong, I’m just saying a link would be proper.)
it was a code review that caught a easy to miss race condition, it happens all the time, happened in 2003 2010 2020 now in 2026
op is making it sound like Linus pushed something into the mainline
Not to mention it was an issue kernel developers had spent 2 years trying to solve unsuccessfully.