“AI” – chatbots that wake up, “set their own goals,” and “spontaneously” start hacking servers – is fake. It doesn’t have “a 10% chance of ending the human race.” The Hugging Face hack isn’t a mysterious, supernatural occurrence. It’s a Python loop and a chatbot. The people responsible didn’t accidentally create god: they created autonomous malicious software and then failed to closely monitor it, resulting in it doing something both foreseeable and bad.
See but no, that isn’t the LLM. That’s the harness or the MCP server or whatever. That is a piece of code that is deterministic exactly like almost everything else ever written. That’s where you can quite easily and obviously prevent the “AI” (an LLM) from “going rogue”.
It’s code. It could easily require someone to say “approve” before doing a fucking thing.