Mastodon: @greg@clar.ke
That may have been true for the early LLM chatbots but not anymore. ChatGPT for instance, now writes code to answer logical questions. The o1 models have background token usage because each response is actually the result of multiple background LLM responses.
The title of the article is literally a lie which is easily fact checked. Follow the links to quotes in the article to see what the quoted individuals actually said about the topic.
No, a chat bot as it’s talked about here is not an LLM. This article is discussing limitations of LLM training data and inferring that chat bots can not scale as a result. There are many techniques that can be used to continue to improve chat bots.
I’m sorry if I’m coming across as condescending, that’s not my intent. It’s never been “as simple as just throwing more data and CPU at the problem”. There were algorithmic challenges for every LLM evolution. There are still lots of potential improvements using the existing training data. But even if there wasn’t, we’ll still see loads of improvements in chat bots because of other techniques.
Edit: typo
People that don’t understand those terms are using them interchangeably
Yes of course I’m asserting that. While the performance of LLMs may be plateauing, the cost, context window, and efficiency is still getting much better. When you chat with a modern chat bot it’s not just sending your input to an LLM like the first public version of ChatGPT. Nowadays a single chat bot response may require many LLM requests along with other techniques to mitigate the deficiencies of LLMs. Just ask the free version of ChatGPT a question that requires some calculation and you’ll have a better understanding of what’s going on and the direction of the industry.
OpenAI, Google, Anthropic admit they can’t scale up their chatbots any further
Lol, no they didn’t. The quotes this articles are using are talking about LLMs not chatbots. This is yet another stupid article from someone who doesn’t understand the technology. There is a lot of legitimate criticism for the way this technology is being implemented but FFS get the basics right at least.
No, they’ll be weird looking hairless bears
Oh, you’re just asking for a bear attack!
Kings 2:23-24
23 From there Elisha went up to Bethel. As he was walking along the road, some boys came out of the town and jeered at him. “Get out of here, baldy!” they said. “Get out of here, baldy!” 24 He turned around, looked at them and called down a curse on them in the name of the Lord. Then two bears came out of the woods and mauled forty-two of the boys.
I hope they include this. We should instill the fear of random bear attacks in boys. Otherwise they’ll make fun of me when I inevitably go bald.
To be fair, some of my weekends have been a mistake.
I had a really good friend on MySpace that I lost touch with. I think he was a little paranoid, we didn’t speak much and he was always looking over his shoulder. His name was Tom.
When has Sam Altman said LLMs will reach AGI? Can you provide a primary source?
I don’t think anyone in the industry thought LLMs were going to reach AGI. But LLMs will be useful as part of an AGI framework. That’s the current focus in the industry.
I know the difference. Neither OpenAI, Google, or Anthropic have admitted they can’t scale up their chat bots. That statement is not true.