When I’ve seen my parents use it, it brings up things it could not possibly know from the current conversation, so its definitely inserting previous context at the very least.
I have literally no idea what you’re so confidently talking about by “pure LLM”. It’s hooked into other stuff too that lets it e.g. properly do math (I’d assume it can by this point?), generate/analyze images, etc. But that has fuck-all to do with whether it can bring previous context into a conversation. Go read the 2017 paper “Attention is All You Need”; literally the whole point of a transformer model is that it can assess the surrounding context of input tokens out to a basically arbitrary depth thanks to its heavy parallelization. It’s obviously imperfect; go watch the most recent Kitboga video (clickbait, sorry) where he creates Gertrude the Pig by leading a scambot away from its training data.
In fact, older LLMs (similarly “pure”, again whatever that means) had far worse problems with being cursed by memory. That’s why the LSTM was invented from the RNN: because the model needed to have a way to forget accrued past context over time, or the results would quickly corrupt on longer inputs. (Keep in mind that with LSTMs, I’m mainly talking about performing more basic things like sentiment analysis.) That didn’t make it “impure”; it just made it not as good at its job as a GPT.
Anyway, unless previously prompted to change to this tone: no, in 2026, it’s not just going to start saying “I’d call the police 😂” in response to high component prices.
The ChatGPT app injects history and all sorts of stuff into context automatically, I’ve seen it do it, and that might include something (like the user’s previous chats) that unintentionally give a style for the model to adopt.
To be specific, I saw ChatGPT reference a specific appointment of a family member in a context that was absent of that information, on a topic that had nothing to do with it. The only way it could have possibly known that was injecting the previous chat into context, silently.
That’s all I’m saying.
LLM architecture has nothing to do with that; its being done outside the LLM. By “pure”, I meant a chat interface where one can see the full prompt as input.
What I’m saying is, we don’t know if that’s the cause or not until we see the context. But it could plausibly be unintentional on the user’s part.
I promise I was not being “hostile”. I led with “I have literally no idea what you’re talking about”, and that’s because I frankly did not. I appreciate you clarifying, although I think dividing it into “pure” and (consequently) “impure” isn’t sensible terminology. An LLM is just that: a model trained on a huge text corpus.
I still get what you’re going for now, namely that output can be “tainted” by previous input, hence “purity”. I think something like “isolated input” would be more apt, although that’s also ad hoc, and there’s probably real, established terminology out there somewhere (that maybe I’m even forgetting).
there’s probably real, established terminology out there somewhere (that maybe I’m even forgetting).
Maybe… there may not be one, though. The ChatGPT app quite purposely obscures the mechanics of the LLM underneath, and they aren’t keen on advertising that to users.
“Isolated input” makes sense to me.
In a technical sense, I suppose an adjacent term would be a raw completion API? Which is rarely offered anymore, sadly.
I have literally no idea what you’re so confidently talking about by “pure LLM”. It’s hooked into other stuff too that lets it e.g. properly do math (I’d assume it can by this point?), generate/analyze images, etc. But that has fuck-all to do with whether it can bring previous context into a conversation. Go read the 2017 paper “Attention is All You Need”; literally the whole point of a transformer model is that it can assess the surrounding context of input tokens out to a basically arbitrary depth thanks to its heavy parallelization. It’s obviously imperfect; go watch the most recent Kitboga video (clickbait, sorry) where he creates Gertrude the Pig by leading a scambot away from its training data.
In fact, older LLMs (similarly “pure”, again whatever that means) had far worse problems with being cursed by memory. That’s why the LSTM was invented from the RNN: because the model needed to have a way to forget accrued past context over time, or the results would quickly corrupt on longer inputs. (Keep in mind that with LSTMs, I’m mainly talking about performing more basic things like sentiment analysis.) That didn’t make it “impure”; it just made it not as good at its job as a GPT.
Anyway, unless previously prompted to change to this tone: no, in 2026, it’s not just going to start saying “I’d call the police 😂” in response to high component prices.
Whoa. What’s with the jump to being so hostile?
The ChatGPT app injects history and all sorts of stuff into context automatically, I’ve seen it do it, and that might include something (like the user’s previous chats) that unintentionally give a style for the model to adopt.
To be specific, I saw ChatGPT reference a specific appointment of a family member in a context that was absent of that information, on a topic that had nothing to do with it. The only way it could have possibly known that was injecting the previous chat into context, silently.
That’s all I’m saying.
LLM architecture has nothing to do with that; its being done outside the LLM. By “pure”, I meant a chat interface where one can see the full prompt as input.
What I’m saying is, we don’t know if that’s the cause or not until we see the context. But it could plausibly be unintentional on the user’s part.
I promise I was not being “hostile”. I led with “I have literally no idea what you’re talking about”, and that’s because I frankly did not. I appreciate you clarifying, although I think dividing it into “pure” and (consequently) “impure” isn’t sensible terminology. An LLM is just that: a model trained on a huge text corpus.
I still get what you’re going for now, namely that output can be “tainted” by previous input, hence “purity”. I think something like “isolated input” would be more apt, although that’s also ad hoc, and there’s probably real, established terminology out there somewhere (that maybe I’m even forgetting).
Maybe… there may not be one, though. The ChatGPT app quite purposely obscures the mechanics of the LLM underneath, and they aren’t keen on advertising that to users.
“Isolated input” makes sense to me.
In a technical sense, I suppose an adjacent term would be a raw completion API? Which is rarely offered anymore, sadly.