Cross posted from https://sh.itjust.works/post/64317237
Sam Altman claimed we’re already living through the singularity. Is it true, or is it just marketing? Just days before, one of OpenAI’s models escaped from a testing environment and ended up hacking another company’s servers on its own. We analyze how real that claim is and what the experts say.
And from there, we jump to something bigger: can artificial intelligence have a soul? We delve into the debate that has divided physicists, neuroscientists, and philosophers, from Roger Penrose’s quantum theory to the arguments of Sean Carroll and Max Tegmark, including integrated information theory and the thermodynamics of life. Spoiler alert: even physics can’t agree.
Gray Goo.
https://en.wikipedia.org/wiki/Gray_goo
By feeding the LLM output back into the LLM and destroying the source material, they are creating a singularity.
The problem is that it is a heat-death-of-the-universe-type singularity.
How delightful. Now I can add yet another thing to my ever-growing list of things to be concerned about.
Yeah, no scenario though where I’d see that coming out of llm. It’s way out of that capacity.
What we can achieve though is drying up land, creating famine and misery and the crash of society.
The LLMs are committing ecophalogy of knowledge. Their nonsensical output is being fed back into them like an ouroboros, resulting in more nonsensical output.
Humans have been performing ecophagy since hunter-gatherer times, and the rate has increased exponentially since the Industrial Revolutions, but always to the benefit of Humans and humanity.
Now the general-purpose LLMs are being destructively fed all of human knowledge and the data centres are being destructively fed all of our natural resources; not for the benefit of Humans or humanity, but for hubris and apathy.
Diversity of thought is being homogenised as more people use LLMs for mundane tasks that would be quicker, easier and more efficient to do ourselves.
Our own Gray Matter is turning into Gray Goo.
The last sentence would’ve sufficed, thatwe can absolutely agree on.
It’s not that there will be ai nanobots rising from llm developing into that, though. Or bots. Or continually self replicating anything.
Use your imagination. It’s an analogy.
I’m a technician and Autist. I don’t do wild speculation. I do derive future prospects from technical or physical facts.
And if there’s not a major thing added, it will not happen.
What is the “it” that will not happen?
An llm can and will not have an own conscience or motivation or even possibility of acting of its own.
It can only act triggered and the rest looks like it would be so, but its only parroting.
Downvotes or not, it is factually missing parts to get there.
Its like saying the currently most advanced self driving car is possible to develop into knight riders kitt. It’s not biology. It needs the components to be able to.
This is about the consequences of recursively training AI on AI-generated content. It doesn’t have anything to do with volition or true intelligence. There’s a good article about it here.