“AI” – chatbots that wake up, “set their own goals,” and “spontaneously” start hacking servers – is fake. It doesn’t have “a 10% chance of ending the human race.” The Hugging Face hack isn’t a mysterious, supernatural occurrence. It’s a Python loop and a chatbot. The people responsible didn’t accidentally create god: they created autonomous malicious software and then failed to closely monitor it, resulting in it doing something both foreseeable and bad.
But LLMs can’t update databases?
Sure they can, look up harnesses and tool APIs. They gave LLMs the ability to take actions if they deem that action as an answer.
They don’t act on their own though, they’re a chain reaction that a human must start.
It’s an autocomplete the size of a football field.
See but no, that isn’t the LLM. That’s the harness or the MCP server or whatever. That is a piece of code that is deterministic exactly like almost everything else ever written. That’s where you can quite easily and obviously prevent the “AI” (an LLM) from “going rogue”.
It’s code. It could easily require someone to say “approve” before doing a fucking thing.
At this point I just let codex run as root and control entire server clusters. It does a better job than I used to, and we finally get to live in the sci fi universe where I can talk to my computer and it does stuff