I was surprised to see that this lovely write-up wasn’t posted here already. Also see the author’s Mastodon post about it: https://wandering.shop/@aesthr/117470082684445346
That was a really interesting and well-written read, though I got a bit lost when the logic took some sharp turns. The idea of fascism in LLM systems was new to me and certainly got me thinking.
It really is worth a read. Thank you.
Amazing article. Thanks for sharing.
Because it’ll burn the planet?
“AI” is certainly environmentally damaging and dangerous, and if it isn’t given new input, it quickly starts generating absolute crap, but I don’t agree with this statement:
LLMs are simply not capable of producing new knowledge. They can only reproduce what’s in their training data. An LLM-based system, being a statistical model of its training data and therefore effectively a form of lossy compression, can by definition never solve any problem for which the training data or information otherwise available to the system doesn’t already contain the solution.
We’ve seen new mathematical proofs coming out of LLMs. Those proofs weren’t in the dataset that they were trained on.
AlphaGo Zero was given the rules of go and made to play against itself with no example games. It came up with new openings that hadn’t been seen before, and were better than anything that humans had come up with.
AlphaFold was given protein sequences and was able to generate correct 3D structures, that again weren’t in its training data.
Trying to defend the position of “nothing new comes out of AI” will necessarily lead you to such a narrow definition of novelty, that it will exclude most things that we consider to be human creativity.
Neither AlphaGo, not AlphaFold are LLMs though.
Also the mathematical proofs you’re talking about seem to be mostly stolen by OpenAI from scientists that interacted with their chatbot before publishing their research.
That not fully true. The “problem” with math is that it is vast, so the building blocks for a proof are available but there is hardly anyone that has a wide enough interest to know about all the components. But AI is great at that. It read all the papers in all the disciplines and now can chain reasoning spanning all of it, a feat that’s impossible for a human.
And this makes their advances very tricky to evaluate. If a human did that, they would be a genius, but how do you evaluate a computer that did a very fancy word search across all publications?
So you don’t understand how the technology you’re defending functions then?
“AI” is certainly environmentally damaging and dangerous, and if it isn’t given new input, it quickly starts generating absolute crap
You see that as “defending” the technology?
That specifically is not the part I was referring to, but for the record, if an LLM isn’t given any input, it does nothing.
Giving it input is how you “prompt” it to generate the absolute crap.
I was referring to the part where you gave the LLMs credit for solving math equations when there is evidence that they simply stole the solutions from real mathematicians and attempted to pass it off as work the LLM had done itself.
Do you disagree that the examples given are new knowledge? I’m sure OP is aware AI is “just a token generator” or whatever; that doesn’t really matter here when there is proof of it at least producing data in the convex hull of its training data, and in the case of alphago, with no training data at all.
They literally stole the answers to those math problems from human mathematicians who hadn’t published their work yet, but had spoken to a chat bot about it.
All 300 of them?
Computers have “come up with” solutions to problems that eluded human-only problem-solving for decades. That’s literally what computers are supposed to be for. That proves nothing about the novelty of AI beyond a narrow range of incremental problem-based solving which, again, we’ve used computers for for decades.
Characterizing previous computer use in creating proofs as the same thing as current ai usage is pretty wild
