Because weight x = y isn’t really a satisfactory answer to the “why” in question though. To really answer that you need to refer to the pretraining and context, which in a lot of cases today, and likely forever, since there’s no reason for anyone to cold start model training ever again, is necessarily opaque. A lot of the best why answers are going to be given by the model itself (which… obviously circular). Trust and attribution are just really hard in this paradigm. Not necessarily so different from the challenge of trusting/attributing human output, except that humans are more diverse and in general difficult to manipulate.
Because weight x = y isn’t really a satisfactory answer to the “why” in question though. To really answer that you need to refer to the pretraining and context, which in a lot of cases today, and likely forever, since there’s no reason for anyone to cold start model training ever again, is necessarily opaque. A lot of the best why answers are going to be given by the model itself (which… obviously circular). Trust and attribution are just really hard in this paradigm. Not necessarily so different from the challenge of trusting/attributing human output, except that humans are more diverse and in general difficult to manipulate.