The “problem” is that any standard compliant code uniquely defines the output for any given input. A bug-free compiler translates that execution path into the exact same behaviour, just in a different formulation.
An LLM translates a more or less vague natural language description of an execution path into a precise one formulated in code.
The difference is pretty fundamental: In one case you give an exact description of the inputs and outputs, and let a compiler translate that to a different but logically equivalent description. In the other you give a non-exact description and let a stochastic machine translate that into one of many possible exact descriptions.
That doesn’t mean it’s impossible to produce good code with an LLM. It means you need to inspect the output from the LLM with far more scrutiny than you would ever inspect the assembly output from a compiler.
A calculator that produces incorrect output from some input is broken. An LLM that produces broken code from a precise description is working as advertised.