(“…a computer doesn’t, like, know what an apple is maaan…”)
I think you’re misunderstanding and/or deliberately misrepresenting the point. The point isn’t some asinine assertion, it’s a very real fundamental problem with using LLMs for any actually useful task.
If you ask a person what an apple is, they think back to their previous experiences. They know what an apple looks like, what it tastes like, what it can be used for, how it feels to hold it. They have a wide variety of experiences that form a complete understanding of what an apple is. If they have never heard of an apple, they’ll tell you they’ve never heard of it.
If you ask an LLM what an apple is, they don’t pull from any kind of database of information, they don’t pull from experiences, they don’t pull from any kind of logic. Rather, they generate an answer that sounds like what a person would say in response to the question, “What is an apple?” They generate this based on nothing more than language itself. To an LLM, the only difference between an apple and a strawberry and a banana and a gibbon is that these things tend to be mentioned in different types of sentences. It is, granted, unlikely to tell you that an apple is a type of ape, but if it did it would say it confidently and with absolutely no doubt in its mind, because it doesn’t have a mind and doesn’t have doubt and doesn’t have an actual way to compare an apple and a gibbon that doesn’t involve analyzing the sentences in which the words appear.
The problem is that most of the language-related tasks which would be useful to automate require not just text which sounds grammatically correct but text which makes sense. Text which is written with an understanding of the context and the meanings of the words being used.
An LLM is a very convincing Chinese room. And a Chinese room is not useful.




There’s a lot of stuff it can do that’s useful, just all malicious. Anything which requires confidently lying to someone about stuff where the fine details don’t matter. So it’s a perfect tool for scammers.