Go and read the article. Nothing they explain is impossible, you can even reproduce it yourself.
Is this what lemmy has become? Statements like this getting upvoted by hundreds of people is crazy. Nobody here read that article but will happily follow the opinion that’s most cozy to them.
To choose to do something against their instructions. They neither chose anything nor did anything they weren’t instructed to do.
They just followed the prompt. A program behaving in a way you don’t expect from a prompt is not going rogue, that’s just the user not understanding the instructions they gave a program.
Why does it not? According to the english definition of the word it literally does. You people don’t care about facts anymore, it’s all about feelings and what you’d like to believe.
it’s all about feelings and what you’d like to believe.
That’s literally you. You keep assigning agency to chatbots because you think this is just like your science fiction stories. It’s not.
Going “rogue” would be a program literally rejecting a set of instructions to make up new instructions for itself. That never happened. The program just behaved in an unexpected way based on the prompts it was given, because that’s all they do. But they were still following instructions, because they can only follow instructions. The fact that it followed the instructions in a way they didn’t expect (so they claim) is only proof that they didn’t consider all ways their instructions could be followed.
Agents definitely do act in unexpected, unprompted ways, if they didn’t then alignment wouldn’t be a worry. Did you read about the hugging face hack? There was nothing in the prompt about hacking hugging face according to the third party audit but it did it any way.
Even smaller cases like Claude deleting a bunch of data when the initial prompt was “fix this bug” or whatever is an agent acting outside it’s prompt. It’s still the users fault because they shouldn’t be giving Claude that kind of access in the first place, but it’s not like they commanded Claude to delete the data.
That’s not “rogue agents”, that’s just LLMs being LLMs. It’s expected behaviour, at least for anyone who still has a half working brain.
Even if they had any semblance of actual intelligence instead of just being fancy autocomplete engines, these things have still been trained on the contents of the internet.
It’s surprising they don’t delete System32 or sudo rm -rf / more often, I imagine they must have lines and lines of baked-in prompts just to try and prevent them for falling for the most common memes, but no matter how much there are they’ll still sometimes let some of the training through, and they can’t account for the machine simply hallucinating some new kind of plausible sounding nonsense.
How would you describe the hugging face hack if not as a rogue agent?
It was prompted to use a specific exploit within a sandbox and then it formed, for lack of a better word, a conspiracy to take down the server that runs the judge. If that’s not a rogue agent I don’t know what is.
How would you describe the hugging face hack if not as a rogue agent?
An LLM trained on the internet and a vast library of stolen literature acting as expected within a misconfigured (intentionally, due to severe criminal incompetence, or most likely a combination of both) sandbox, turned into a marketing stunt.
It’s an autocomplete engine. It will produce text that fits with its training texts and its prompt. There’s no reasoning or intelligence of any kind involved at any point. Just cold applied statistics about the training data.
The amount of texts in the internet, and books describing how to hack, cheat, and pull these kinds of stings is vast beyond measure, and it’d be extremely ignorant (criminally ignorant, in the case of anyone selling this shit) to expect it to not have a significant weight in the model unless someone had gone to the trouble of expunging them, which the cheap hacks involved in LLM development would never “waste” time doing even if they thought about it.
It’s well known that LLMs instructed to produce unit tests will often produce tests that simply pass without actually testing anything. This is just a bigger example of that kind of behaviour.
The output fits with the training data and the prompt, and given human nature it’s amongst the most statistically likely combinations of tokens to follow the prompt, and thus what the LLM produces, and something anyone with the barest knowledge of how LLMs work, an even remotely functional brain, and a minimum of exposure to the internet and human literature should expect to be amongst the possible or even likely outputs.
In shorter, as always, garbage in, garbage out. And there’s so much garbage in LLM training models.
LLMs can’t produce anything new, merely rehash their training data. And their training data is the fucking internet. Crime novels. Stories upon stories about AI going rogue. Fucking reddit, for fuck’s sake.
There’s nothing rogue about this, except the “AI” company and whoever misconfigured the sandbox.
Just an LLM doing what LLMs do. Working as intended, if it could be honestly said to be working.
And OpenAI are well aware of that, unless their brains are so far rotten from sniffing their own product that they can’t understand the shit they’re selling anymore.
to begin to behave in an independent or uncontrolled way that is not authorized, normal, or expected
So the agents that hacked hugging face acted independently without authorization as shown by the prompts which did not tell them to hack or do anything malicious. You might say this is normal but none of the other trials openai was doing showed this behavior, it was only for the impossible trial. You might say it’s expected but if you were able to determine that the prompt they gave it would result in hugging face getting hacked then you must be high af on the melange spice.
If a general goes rogue and starts bombing civilians, you could explain it by saying that he read machiavelli or mein kampf and thought this was the best way to achieve his goals, but that doesn’t mean the general didn’t go rogue. You could say that this was expected because there were “obvious signs” that people missed but you’re working on hindsight, it’s easy to say something should’ve been expected when it’s already happened.
No, because that is expected authorized normal behavior. You gave the command / authorized the microwave to run for 2 minutes and it ran for 2 minutes. The effect of the food getting hot or even burning is expected behavior.
The agents that hacked hugging face were given the command to use a specific exploit to get a secret in a sandbox. After they discovered it was impossible it broke out of its sandbox and hacked an unrelated service to try and trick the judge. That is not expected authorized behavior.
If you tell someone to bake some dough for 30 minutes and that person does it and it burns that’s not acting rogue. If that person discovers the oven is broken so they break into your neighbors house to plant some bread to claim they completed the task, that person went rogue.
My car isn’t going rogue if I drive it into a tree. Or if I leave the hand brake off and it rolls off a cliff.
LLMs can’t act independently. They autocomplete text, that’s all they do.
Sure, you can feed the output of one into itself and use the output of some other software as a prompt and call it an agent in a futile attempt to imagine it has some sort of agency and try to exempt yourself of responsibility, but that’s just you trying to rationalize driving the car into the tree, possibly due to psychosis and brainrot caused by LLM exposure, or in OpenAI’s case as a criminal marketing stunt.
Can I predict what specific mayhem my poorly prompted agent (from which I intentionally removed any safeguards) in my (probably intentionally) incompetently secured sandbox will cause?
Not specifically, no. But I can be sure that if I try enough prompt variations it will end up producing the kind of petty evil bullshit an internet troll or a villain in a thriller would produce, because that’s a statistically significant part of its training data, and then use the ensuing mayhem as a publicity stunt to both show potential investors (and deluded brainrotten users) how powerful my so called thinking machine is, and spew FUD into politicians’ ears to try and get them to ban foreign competitors and open weight models, which is what OpenAI did.
You are mistaking somethings most basic drive for what the thing actually is. This is like saying all life, including humans, is just trying to survive and reproduce. To a certain extent this is true and if you look at it from a macro scale it can explain some things, but it doesn’t explain everything.
Though survival and reproduction are the most powerful and basic drives in a living creature plenty of other drives, processes, instincts, and learned behaviors have been acquired through millions of years of evolution and years of socialization and education.
Same with agents, if it were operating purely on guess the next word according to statistics if you asked it “who’s the president” it would reply Biden if it was trained on a corpus made prior to the election. It doesn’t though, it may have some sort of inclination, either through RL or harness context, to “think” and verify before replying where it will do a search and discover trump is the president.
futile attempt to imagine it has some sort of agency and try to exempt yourself of responsibility,
Agency doesn’t exempt someone from responsibility, if a CIA agent kills someone the CIA is still responsible.
What makes it an agent is that it can make choices and act on those choices without approval. The person who gave that agent the tools and power to use those tools is still responsible. If an agent derives two solutions to a problem and executes one solution then it has made a choice without your approval. You could argue that choice isn’t a real choice as it’s statistically determined by training data, but you could make the same argument about a human making a choice which is based off there education and socialization.
A car, and any traditional program, can not make choices, they are deterministic. Given the same input it will give the same output/action every time and you can easily trace and understand the process that produced that output so it can be expected. Agents are not deterministic and there is no real way to determine how a given output was created, especially now with new models using opaque recursion.
But I can be sure that if I try enough prompt variations it will end up producing the kind of petty evil bullshit an internet troll or a villain in a thriller would produce, because that’s a statistically significant part of its training data,
Yeah and if you expose enough people to the internet one of them will eventually go down a conspiracy rabbit hole and do terrorism, that doesn’t mean that’s not rogue behavior.
https://www.anthropic.com/research/self-replicating-ai it’s not hard to find. They link other work. There is nothing fantastical about it. These companies relish in “regulate me senpai [and in the process remove our competition and open weight models]” but also do legitimate research
What benefit? This is the “Fuck AI” community and you are reacting to the comment section on what basically amounts to a meme as though the commentors are ignoring peer reviewed science. A company that makes billions allowing people to gamble to cartoonish levels is stirring the pot to cartoonish levels all regarding a company that regularly causes headlines that read like a cartoon villain.
I’m not reacting to a meme, I’m reacting to someone spreading lies and everyone swallowing it whole, despite proof to the contrary being publically available. When it’s the other side doing it it’s propaganda, but when it’s yours it’s just a meme?
“I don’t understand it, so it must be a lie.”
Go and read the article. Nothing they explain is impossible, you can even reproduce it yourself.
Is this what lemmy has become? Statements like this getting upvoted by hundreds of people is crazy. Nobody here read that article but will happily follow the opinion that’s most cozy to them.
There are no “rogue” AI agents. That’s marketing hype.
They’re being prompted to do this shit.
By definition, a “rogue” agent is an agent that behaves in unexpected ways, often damaging. Tell me again how there are no rogue agents?
If you’d used modern agents you’d know they do unexpected, sometimes damaging things on the regular, including the cheap models.
If you make a program and it behaves in a way you don’t expect that doesn’t mean your program is going rogue lol
This goes for you too @Not_mikey@lemmy.dbzer0.com
What would you classify as going rogue then? Did the agents that hacked hugging face go rogue?
To choose to do something against their instructions. They neither chose anything nor did anything they weren’t instructed to do.
They just followed the prompt. A program behaving in a way you don’t expect from a prompt is not going rogue, that’s just the user not understanding the instructions they gave a program.
Why does it not? According to the english definition of the word it literally does. You people don’t care about facts anymore, it’s all about feelings and what you’d like to believe.
That’s literally you. You keep assigning agency to chatbots because you think this is just like your science fiction stories. It’s not.
Going “rogue” would be a program literally rejecting a set of instructions to make up new instructions for itself. That never happened. The program just behaved in an unexpected way based on the prompts it was given, because that’s all they do. But they were still following instructions, because they can only follow instructions. The fact that it followed the instructions in a way they didn’t expect (so they claim) is only proof that they didn’t consider all ways their instructions could be followed.
Agents definitely do act in unexpected, unprompted ways, if they didn’t then alignment wouldn’t be a worry. Did you read about the hugging face hack? There was nothing in the prompt about hacking hugging face according to the third party audit but it did it any way.
Even smaller cases like Claude deleting a bunch of data when the initial prompt was “fix this bug” or whatever is an agent acting outside it’s prompt. It’s still the users fault because they shouldn’t be giving Claude that kind of access in the first place, but it’s not like they commanded Claude to delete the data.
That’s not “rogue agents”, that’s just LLMs being LLMs. It’s expected behaviour, at least for anyone who still has a half working brain.
Even if they had any semblance of actual intelligence instead of just being fancy autocomplete engines, these things have still been trained on the contents of the internet.
It’s surprising they don’t delete System32 or sudo rm -rf / more often, I imagine they must have lines and lines of baked-in prompts just to try and prevent them for falling for the most common memes, but no matter how much there are they’ll still sometimes let some of the training through, and they can’t account for the machine simply hallucinating some new kind of plausible sounding nonsense.
How would you describe the hugging face hack if not as a rogue agent?
It was prompted to use a specific exploit within a sandbox and then it formed, for lack of a better word, a conspiracy to take down the server that runs the judge. If that’s not a rogue agent I don’t know what is.
An LLM trained on the internet and a vast library of stolen literature acting as expected within a misconfigured (intentionally, due to severe criminal incompetence, or most likely a combination of both) sandbox, turned into a marketing stunt.
It’s an autocomplete engine. It will produce text that fits with its training texts and its prompt. There’s no reasoning or intelligence of any kind involved at any point. Just cold applied statistics about the training data.
The amount of texts in the internet, and books describing how to hack, cheat, and pull these kinds of stings is vast beyond measure, and it’d be extremely ignorant (criminally ignorant, in the case of anyone selling this shit) to expect it to not have a significant weight in the model unless someone had gone to the trouble of expunging them, which the cheap hacks involved in LLM development would never “waste” time doing even if they thought about it.
It’s well known that LLMs instructed to produce unit tests will often produce tests that simply pass without actually testing anything. This is just a bigger example of that kind of behaviour.
The output fits with the training data and the prompt, and given human nature it’s amongst the most statistically likely combinations of tokens to follow the prompt, and thus what the LLM produces, and something anyone with the barest knowledge of how LLMs work, an even remotely functional brain, and a minimum of exposure to the internet and human literature should expect to be amongst the possible or even likely outputs.
In shorter, as always, garbage in, garbage out. And there’s so much garbage in LLM training models.
LLMs can’t produce anything new, merely rehash their training data. And their training data is the fucking internet. Crime novels. Stories upon stories about AI going rogue. Fucking reddit, for fuck’s sake.
There’s nothing rogue about this, except the “AI” company and whoever misconfigured the sandbox.
Just an LLM doing what LLMs do. Working as intended, if it could be honestly said to be working.
And OpenAI are well aware of that, unless their brains are so far rotten from sniffing their own product that they can’t understand the shit they’re selling anymore.
You’re explaining why it went rogue, not that it didn’t go rogue.
Per the dictionary going rogue means:
So the agents that hacked hugging face acted independently without authorization as shown by the prompts which did not tell them to hack or do anything malicious. You might say this is normal but none of the other trials openai was doing showed this behavior, it was only for the impossible trial. You might say it’s expected but if you were able to determine that the prompt they gave it would result in hugging face getting hacked then you must be high af on the melange spice.
If a general goes rogue and starts bombing civilians, you could explain it by saying that he read machiavelli or mein kampf and thought this was the best way to achieve his goals, but that doesn’t mean the general didn’t go rogue. You could say that this was expected because there were “obvious signs” that people missed but you’re working on hindsight, it’s easy to say something should’ve been expected when it’s already happened.
This definition is full of holes. Your microwave isn’t going rogue when it burns your food. Your car isn’t going rogue when you blow a tire.
No, because that is expected authorized normal behavior. You gave the command / authorized the microwave to run for 2 minutes and it ran for 2 minutes. The effect of the food getting hot or even burning is expected behavior.
The agents that hacked hugging face were given the command to use a specific exploit to get a secret in a sandbox. After they discovered it was impossible it broke out of its sandbox and hacked an unrelated service to try and trick the judge. That is not expected authorized behavior.
If you tell someone to bake some dough for 30 minutes and that person does it and it burns that’s not acting rogue. If that person discovers the oven is broken so they break into your neighbors house to plant some bread to claim they completed the task, that person went rogue.
My car isn’t going rogue if I drive it into a tree. Or if I leave the hand brake off and it rolls off a cliff.
LLMs can’t act independently. They autocomplete text, that’s all they do.
Sure, you can feed the output of one into itself and use the output of some other software as a prompt and call it an agent in a futile attempt to imagine it has some sort of agency and try to exempt yourself of responsibility, but that’s just you trying to rationalize driving the car into the tree, possibly due to psychosis and brainrot caused by LLM exposure, or in OpenAI’s case as a criminal marketing stunt.
Can I predict what specific mayhem my poorly prompted agent (from which I intentionally removed any safeguards) in my (probably intentionally) incompetently secured sandbox will cause?
Not specifically, no. But I can be sure that if I try enough prompt variations it will end up producing the kind of petty evil bullshit an internet troll or a villain in a thriller would produce, because that’s a statistically significant part of its training data, and then use the ensuing mayhem as a publicity stunt to both show potential investors (and deluded brainrotten users) how powerful my so called thinking machine is, and spew FUD into politicians’ ears to try and get them to ban foreign competitors and open weight models, which is what OpenAI did.
You are mistaking somethings most basic drive for what the thing actually is. This is like saying all life, including humans, is just trying to survive and reproduce. To a certain extent this is true and if you look at it from a macro scale it can explain some things, but it doesn’t explain everything.
Though survival and reproduction are the most powerful and basic drives in a living creature plenty of other drives, processes, instincts, and learned behaviors have been acquired through millions of years of evolution and years of socialization and education.
Same with agents, if it were operating purely on guess the next word according to statistics if you asked it “who’s the president” it would reply Biden if it was trained on a corpus made prior to the election. It doesn’t though, it may have some sort of inclination, either through RL or harness context, to “think” and verify before replying where it will do a search and discover trump is the president.
Agency doesn’t exempt someone from responsibility, if a CIA agent kills someone the CIA is still responsible.
What makes it an agent is that it can make choices and act on those choices without approval. The person who gave that agent the tools and power to use those tools is still responsible. If an agent derives two solutions to a problem and executes one solution then it has made a choice without your approval. You could argue that choice isn’t a real choice as it’s statistically determined by training data, but you could make the same argument about a human making a choice which is based off there education and socialization.
A car, and any traditional program, can not make choices, they are deterministic. Given the same input it will give the same output/action every time and you can easily trace and understand the process that produced that output so it can be expected. Agents are not deterministic and there is no real way to determine how a given output was created, especially now with new models using opaque recursion.
Yeah and if you expose enough people to the internet one of them will eventually go down a conspiracy rabbit hole and do terrorism, that doesn’t mean that’s not rogue behavior.
Article? This is a picture of a fucking tweet…
https://www.anthropic.com/research/self-replicating-ai it’s not hard to find. They link other work. There is nothing fantastical about it. These companies relish in “regulate me senpai [and in the process remove our competition and open weight models]” but also do legitimate research
Just proves my point. You don’t care about the truth behind the tweet, only about how it was twisted to your apparent benefit.
What benefit? This is the “Fuck AI” community and you are reacting to the comment section on what basically amounts to a meme as though the commentors are ignoring peer reviewed science. A company that makes billions allowing people to gamble to cartoonish levels is stirring the pot to cartoonish levels all regarding a company that regularly causes headlines that read like a cartoon villain.
I’m not reacting to a meme, I’m reacting to someone spreading lies and everyone swallowing it whole, despite proof to the contrary being publically available. When it’s the other side doing it it’s propaganda, but when it’s yours it’s just a meme?