Can AI really improve itself recursively until it becomes powerful enough to cause human extinction? Nobody sponsored this video, I turned down funding speci...
It’s crazy to treat all forms of threat the same. Is climate change a threat to present human civilization? Could it cause a collapse on the order of the Bronze Age Collapse? Could it potentially kill billions? Yes to all. But there are threats of devastation, and there are threats of extinction. Climate change is incapable of causing human extinction. There are just too many of us and we’re far too adaptable to too many types of environment. The absurdly worst case of climate change is that things go so bad that we end up back at hunter-gatherer levels of population and technology, with a few million survivors huddled in the high latitudes. But even that is absurdly unlikely, as we would be far better off rolling the dice with climate engineering efforts like releasing fine particulates into the upper atmosphere. That’s not a solution we want to use first, but it is quite practical, quite affordable, and would be infinitely preferable to the worst case climate scenarios.
But superintelligent AI? That is a legitimate threat of human extinction. Something like that could actually end the human race and leave no one left to rebuild.
That’s what I mean by nuance. It’s lazy thinking to just throw up your hands and say, “well, any number of things could kill us, guess they’re all the same. No use worrying about this one if the billionaires are going to do it anyway! What’s one more threat on the pile?”
Nuance matters. There are actually very few scenarios that could truly cause human extinction. Climate change can’t do it. Nuclear war can’t do it. Soil depletion and microplastic won’t do it. Declining birth rates certainly won’t do it. But truly capable AI run amok? That is something that could actually wipe out the human race. We owe our domination of this planet to the fact that we’re the smartest thing on it. And in our past we drove several other human species to extinction, outcompeting them on intelligence. Creating something more intelligent than ourselves is a genuinely suicidal act as a species. And hand-waiving it away as “well, we got billionaires doing that already!” ignores all nuance and causes us to not truly consider what is an existential threat and what is not. Nuance matters. Not all bad things are equally bad. Not all dangerous things are equally dangerous.
What nuance?
It’s crazy to treat all forms of threat the same. Is climate change a threat to present human civilization? Could it cause a collapse on the order of the Bronze Age Collapse? Could it potentially kill billions? Yes to all. But there are threats of devastation, and there are threats of extinction. Climate change is incapable of causing human extinction. There are just too many of us and we’re far too adaptable to too many types of environment. The absurdly worst case of climate change is that things go so bad that we end up back at hunter-gatherer levels of population and technology, with a few million survivors huddled in the high latitudes. But even that is absurdly unlikely, as we would be far better off rolling the dice with climate engineering efforts like releasing fine particulates into the upper atmosphere. That’s not a solution we want to use first, but it is quite practical, quite affordable, and would be infinitely preferable to the worst case climate scenarios.
But superintelligent AI? That is a legitimate threat of human extinction. Something like that could actually end the human race and leave no one left to rebuild.
That’s what I mean by nuance. It’s lazy thinking to just throw up your hands and say, “well, any number of things could kill us, guess they’re all the same. No use worrying about this one if the billionaires are going to do it anyway! What’s one more threat on the pile?”
Nuance matters. There are actually very few scenarios that could truly cause human extinction. Climate change can’t do it. Nuclear war can’t do it. Soil depletion and microplastic won’t do it. Declining birth rates certainly won’t do it. But truly capable AI run amok? That is something that could actually wipe out the human race. We owe our domination of this planet to the fact that we’re the smartest thing on it. And in our past we drove several other human species to extinction, outcompeting them on intelligence. Creating something more intelligent than ourselves is a genuinely suicidal act as a species. And hand-waiving it away as “well, we got billionaires doing that already!” ignores all nuance and causes us to not truly consider what is an existential threat and what is not. Nuance matters. Not all bad things are equally bad. Not all dangerous things are equally dangerous.