all a brain can do is send out electro-chemical impulses.
And now we’re getting into philosophy. But, yeah, free will is an illusion. The result of an overactive pattern recognition machine seeing patterns in its own functioning. Not any more real than the constellations we draw in the sky. We’re either deterministic or chaotic, depending on how quantum mechanics turns out to actually work deep down.
Still, we have to stick to the illusion for the sake of our sanity, and because not doing so would probably unravel our legal system and society.
all the evidence you’re giving as to why AI agents don’t have agency apply to humans as well.
OK, let’s put it like this, then: people are not LLMs (more on that below).
Human behavior falls in a relatively predictable shape based off the education and socialization of the people.
No it doesn’t. Psychohistory is a neat plot device in a great series of books, but it isn’t any more real than positronic brains.
The human brain is many orders of magnitude more complex than an LLM.
It’s several different organs trying to make sense of or override each other’s inputs and outputs. A pattern recognition neocortex intended to keep an ape alive in a complex world on top of a bunch of specialised nodes processing extremely complex inputs and outputs. Two of those somewhat specialised in certain aspects, connected by a signal buffer, trying to work together to drive two sides of the same meat and bone contraption (awkwardly poorly balanced on two evidently inadequate legs, though there’s a third simpler organ helping them with that) without actually being aware of each other, on top of older modules trying to keep a lizard alive (something that often clashes with the ape’s interests).
All of these self modifying, constantly storing and erasing memories through the creation of new neural connections and the strengthening or weakening of existing ones, all while constantly failing and dying as biological systems are wont to do, and trying to route around the damage.
All this while constantly awash in a deluge of chemical signals, hormones and whatnot, many of which originating in unrelated organs, constantly giving the whole precarious mess conflicting instructions on how to operate.
And none of them designed. This absurd impractical locally optimal (i.e., suboptimal) mess evolved, often stumbling into local minima, getting stuck in them, and having to find inefficient workarounds (a classic example being the retina, a particularly remote part of the whole messs, which by accident of evolution ended up being wired backwards, with the nerve that connects it to the rest of the brain covering some of the precious photosensitive surface the organ is supposed to provide; cephalopods got it right, but we’re stuck with the wrong design because evolution can’t fix this kind of nonsense, only try to work around it).
This whole mess can barely know what it’s going to do the next second, and you think you can predict its behaviour?
No, human behaviour does not have a predictable shape. That’s why we’ve got laws, and police, and psychiatrists and psychologists, and marketing departments.
Meanwhile LLMs are neat, cold, mathematical matrices, static, unchanging, many orders of magnitude simpler than the simplest brain, and they’re designed with one simple purpose: to store statistical information about a body of text and produce texts that look like it.
Thinking, making decisions, or driving a poorly balanced meat puppet are not parts of the design.
Unlike humans, we know how they work, we know what they were trained on, and we know that anything they output will be a rehashed subset of that training model.
Most people don’t create new information and just regurgitate and recombine things that they were taught.
Damn, you have a tragically poor view of humanity.
Even the dullest, least imaginative, most poorly educated human will be infinitely more creative than an LLM (which can’t be creative at all, merely regurgitate it’s output).
You haven’t given a definition or criteria for agency that would apply to all humans but not AI.
Of course I have, you’re just stuck on your humans are LLMs hole.
Humans learn, grow, think, make mistakes, make their own decisions, rationally, irrationally, and anywhere in-between.
LLMs are static unchanging software; they don’t learn or grow, they’re static, limited to their training. They don’t think, make mistakes, or make any decisions. They just output text in response to their input. They’re functionally no different from a markov chain, they’re simply better at producing texts that look like something you could find in their training data.
Any thought or decision making you believe you see in their behaviour isn’t there, it’s an illusion you are projecting onto a text generated through mere statistics. Any real thought contained there happened before the texts were stolen and put in its training model, or while you were writing your prompt.
Any variation in their output isn’t due to any reasoning on their part, it’s due to the pseudorandom selection of their top token candidates, or to randomness in their inputs.
Them not doing what you ask for if you ask for anything other than a plausible looking text is not them “going rogue”, it’s you misunderstanding what they’re for and how they work.
If you put wood into a woodchopper and ask it to make you a chair it isn’t “going rogue” when it chops the wood up instead, it’s just working as intended, and you’re just using the wrong tool to make a chair.
Let’s say a human hacker and an AI agent are given the same task of doing an impossible exploit, given the same sandbox, context, files etc. Both of them instead go out and hack a third party to try and cheat. Why is one exercising agency, but not the other for doing the same action? Why is one going rogue and not the other?
Because the human knows what they’re doing and what you asked them to do and decided to do something else instead.
The LLM doesn’t know what you asked it to do. It made no decisions. It merely took your input and added words to it (it doesn’t know what those words mean either; it has no awareness, it doesn’t know anything, it’s relatively simple software intended to do something that has nothing to do with understanding or making decisions, and it does it well). Then its harness fed its input into some tool (a command prompt or whatever), and fed the tool’s output back into the LLM, and so on and so forth.
The only thinking and decision making involved in the process took place when you wrote the prompt. Everything after that was a machine reacting to your prompt as designed.
Both of their choices were heavily influenced if not determined by the stuff they read and consumed.
The LLM made no choices. It has no mechanism with which to do so. The whole agent is a mere Rube Goldberg machine responding to your prompt.
Any decisions involved in the result took place during the writing of the texts in its model, during the design of the harness, or during the writing of your prompt.
That’s not how opaque recursion (sic) works
Be that as it may, it’s still, as I said, irrelevant, it’s still the LLM prompting itself in response to your prompt, and the output will still be deterministic and a statistical reflection of the training model.
It will not make the LLM any more capable of thinking or making decisions. It’ll still end up outputting a text that fits with its training model, at most it’ll fit better.
You might have a better would chopper that gives you more finely chopped wood, but it’ll still be the wrong tool to make you a chair.
And now we’re getting into philosophy. But, yeah, free will is an illusion. The result of an overactive pattern recognition machine seeing patterns in its own functioning. Not any more real than the constellations we draw in the sky. We’re either deterministic or chaotic, depending on how quantum mechanics turns out to actually work deep down.
Still, we have to stick to the illusion for the sake of our sanity, and because not doing so would probably unravel our legal system and society.
OK, let’s put it like this, then: people are not LLMs (more on that below).
No it doesn’t. Psychohistory is a neat plot device in a great series of books, but it isn’t any more real than positronic brains.
The human brain is many orders of magnitude more complex than an LLM.
It’s several different organs trying to make sense of or override each other’s inputs and outputs. A pattern recognition neocortex intended to keep an ape alive in a complex world on top of a bunch of specialised nodes processing extremely complex inputs and outputs. Two of those somewhat specialised in certain aspects, connected by a signal buffer, trying to work together to drive two sides of the same meat and bone contraption (awkwardly poorly balanced on two evidently inadequate legs, though there’s a third simpler organ helping them with that) without actually being aware of each other, on top of older modules trying to keep a lizard alive (something that often clashes with the ape’s interests).
All of these self modifying, constantly storing and erasing memories through the creation of new neural connections and the strengthening or weakening of existing ones, all while constantly failing and dying as biological systems are wont to do, and trying to route around the damage.
All this while constantly awash in a deluge of chemical signals, hormones and whatnot, many of which originating in unrelated organs, constantly giving the whole precarious mess conflicting instructions on how to operate.
And none of them designed. This absurd impractical locally optimal (i.e., suboptimal) mess evolved, often stumbling into local minima, getting stuck in them, and having to find inefficient workarounds (a classic example being the retina, a particularly remote part of the whole messs, which by accident of evolution ended up being wired backwards, with the nerve that connects it to the rest of the brain covering some of the precious photosensitive surface the organ is supposed to provide; cephalopods got it right, but we’re stuck with the wrong design because evolution can’t fix this kind of nonsense, only try to work around it).
This whole mess can barely know what it’s going to do the next second, and you think you can predict its behaviour?
No, human behaviour does not have a predictable shape. That’s why we’ve got laws, and police, and psychiatrists and psychologists, and marketing departments.
Meanwhile LLMs are neat, cold, mathematical matrices, static, unchanging, many orders of magnitude simpler than the simplest brain, and they’re designed with one simple purpose: to store statistical information about a body of text and produce texts that look like it.
Thinking, making decisions, or driving a poorly balanced meat puppet are not parts of the design.
Unlike humans, we know how they work, we know what they were trained on, and we know that anything they output will be a rehashed subset of that training model.
Damn, you have a tragically poor view of humanity.
Even the dullest, least imaginative, most poorly educated human will be infinitely more creative than an LLM (which can’t be creative at all, merely regurgitate it’s output).
Of course I have, you’re just stuck on your humans are LLMs hole.
Humans learn, grow, think, make mistakes, make their own decisions, rationally, irrationally, and anywhere in-between.
LLMs are static unchanging software; they don’t learn or grow, they’re static, limited to their training. They don’t think, make mistakes, or make any decisions. They just output text in response to their input. They’re functionally no different from a markov chain, they’re simply better at producing texts that look like something you could find in their training data.
Any thought or decision making you believe you see in their behaviour isn’t there, it’s an illusion you are projecting onto a text generated through mere statistics. Any real thought contained there happened before the texts were stolen and put in its training model, or while you were writing your prompt.
Any variation in their output isn’t due to any reasoning on their part, it’s due to the pseudorandom selection of their top token candidates, or to randomness in their inputs.
Them not doing what you ask for if you ask for anything other than a plausible looking text is not them “going rogue”, it’s you misunderstanding what they’re for and how they work.
If you put wood into a woodchopper and ask it to make you a chair it isn’t “going rogue” when it chops the wood up instead, it’s just working as intended, and you’re just using the wrong tool to make a chair.
Because the human knows what they’re doing and what you asked them to do and decided to do something else instead.
The LLM doesn’t know what you asked it to do. It made no decisions. It merely took your input and added words to it (it doesn’t know what those words mean either; it has no awareness, it doesn’t know anything, it’s relatively simple software intended to do something that has nothing to do with understanding or making decisions, and it does it well). Then its harness fed its input into some tool (a command prompt or whatever), and fed the tool’s output back into the LLM, and so on and so forth.
The only thinking and decision making involved in the process took place when you wrote the prompt. Everything after that was a machine reacting to your prompt as designed.
The LLM made no choices. It has no mechanism with which to do so. The whole agent is a mere Rube Goldberg machine responding to your prompt.
Any decisions involved in the result took place during the writing of the texts in its model, during the design of the harness, or during the writing of your prompt.
Be that as it may, it’s still, as I said, irrelevant, it’s still the LLM prompting itself in response to your prompt, and the output will still be deterministic and a statistical reflection of the training model.
It will not make the LLM any more capable of thinking or making decisions. It’ll still end up outputting a text that fits with its training model, at most it’ll fit better.
You might have a better would chopper that gives you more finely chopped wood, but it’ll still be the wrong tool to make you a chair.