To be fair, yes, I’m just another person on the fediverse for all of you, but for me, this is the reality I live in.
I’ve always wondered about all the AI hate around here, and if this is just a fediverse thing, but realized recently that it’s not. In fact, people are generally getting tired of constant LLM and genAI use. I’ve already noticed people getting turned off by AI flyers more and more, sentences like “oh, they switched to an AI flyer”, in a voice of disappointment. And recently, I was at a trade show, talking to one of the reps at a booth, and it was really interesting to hear what he had to say on the topic. Apparently, two years ago, he constantly got questions about AI, wanting to hear that the products contain AI. This year, people also ask him fairly often if his products have AI, hoping to hear that they don’t have AI. And he was able to clearly say, there is no AI in his products (at least not the LLM kind, only the classic, non-neural network kind).
Those numbers would plummet if the users were actually paying the true cost to run these models. These companies can kick that can down the road, but not indefinitely.
I think that’s a false impression brought about by the endless articles using bad API math. They aren’t running the APi at cost, they are also rerouting easy task to smaller models and build fine tuned quants in the first week or two that they replace their models with. It’s very noticeable, the models are only at their best in the first few weeks than there is noticable degradation.
I don’t get how people can readily distrust OpenAI but drink the “there’s no profit in it” koolaid. Granted, they aren’t in the best of spots if businesses start running their own models but I think it’s overblown a bit personally.
I can run a small model that’s about as good as SOTA a year ago on my 8 GB card and run it for 15 people at a comfy rate, those giant GPUs they have in datacenters are each serving 100+ users, but their APIs are priced as if they needed one GPU per person.
I don’t have time to get into the nitty-gritty with this, but there’s a lot more going on behind-the-scenes than just running simple queries. I could see a future in which individuals and businesses are running open LLMs on their own hardware at cost for very specific purposes, with the understanding that relying on LLMs for everything will knee-cap them in the long-term. I’m not entirely against the technology itself, as long as people use it responsibly. However, that’s not what’s happening.
What is happening is a handful of AI companies, backed by tech giants and VC funding, are committing gross amounts of copyright infringement and intellectual property theft in an effort to monopolize all human knowledge and sell it back to us at a subscription. In order to do this, they are spending billions of dollars on influence and marketing to the public, government, and businesses. On the public side, they are trying to get everyone addicted to using their LLM chatbots and replace everything from search engines to genuine human interaction with LLMs. On the government side, they are trying to get the military to use LLMs in their war machine and domestic surveillance to identify and target anyone against the regime, while simultaneously cozying up to political leaders to convince them not to regulate them for fear of losing some kind of imaginary AI race with China. On the business side, they are trying to convince every executive and manager that AI is the future and can replace most if not all of their workers, even if it requires the current workers to train their model-powered replacements prior to being laid-off.
As a side note, the inability for regular users to purchase computing hardware at reasonable prices (RAM, GPUs, etc), thus being unable to run open models on their own hardware, is considered a benefit by these AI companies. Similarly, it benefits the AI companies for users to switch from simple queries to agents for basic tasks, because agents consume far more tokens.
The monopolization of this technology by a handful of tech companies, along with the build-out of data centers, hoarding compute hardware, and marketing of it to the public, government, and businesses, is unprofitable at the current prices. The market projections to achieve profitability are somewhere between laughable and insane. Almost all users of the technology, including individuals, government agencies, and businesses, are being heavily subsidized by a combination of dwindling VC funding and increasingly convoluted financing deals. Eventually, the free money is going to run out for the AI companies and their companions (data center builders, AI startups, etc), and they will be forced to raise prices, assuming they don’t immediately go under when the bubble bursts, which is looking increasingly likely. Businesses will be forced to have a serious discussion on ROI for LLM subscriptions, as well as the long-term effects of over-dependency on LLMs for core tasks, such as unmaintainable infrastructure, knowledge loss due to layoffs, and cult-like mindsets driving poor business decisions.
I really hope I’m not right about this, because when the bubble pops, it’s going to be bad, and the repercussions are going to be felt everywhere. I’m reminded of the scene from the Big Short:
You are. The upside is that SSDs and GPUs could be affordable again. The downside is I’ll probably be unemployed and trying to steal groceries from Walmart and it won’t matter.
From the information available, the only companies making a profit from AI are the ones selling the shovels (the hardware to run it).
https://isaiprofitable.com/
I dont know if you being right or wrong changes the predictable commercial future.
If you are wrong then the prevailing narrative holds, these companies are subsidising costa to an unprecedented level and a revenue correction is inevitable.
If you’re right, the actual (today) cost of tokens is not utterly horrendous compared to pricing… but that raises the question: what in the name of god do we need all the planned but not yet online datacentres for?
I can only think that it is to train and serve frontier models at an increased token cost that is not predicated on capacity based quantisation, and surely that will make the token costs horrendous because all that investment has to create a return somewhere, so a revenue correction will follow.
Best case scenario; the revenue correction arises after efficient open source models are capable of sustaining most commercial activity. Then the world does not need the datacentre buildout that’s propping up nvidia etc., (i kinda hope thats already the case). Frontier level models will stroll (have strolled?) on past the point of diminishing returns and will become commercial black holes…
All the datacenter build are more about squeezing supply imo. They build them so their competitors can’t, and in this case, that also includes retail.
At the rate that it’s being used, a business could recoup their cost in a year if we still had the old hardware prices. OpenAI literally bought wafers and are just sitting on them, I don’t get why there isn’t a lawsuit already.
Same for all the danger talk and copyright lawsuits. Sam Altman loves that he only has to pay a few hundred million for a monopoly, they have 1 billion a month in revenue [edited, I was counting all users and not just paying users], it’s a drop in the bucket.
And sure, open models won’t disappear from the internet, but their use in a business context is tied to these lawsuits. We are heading towards half of our economy being forced into subscription services. Most of the lawsuit money goes straight to shareholders instead of the actual content creators too (the book one being a notable exception tho). It drives me nuts.
Hoping for your best case too tho. Fingers crossed.
It’s seven trillion dollars. That is gonna need a lot of return on investment not to crack the economy like an egg.
yeah, i wish we were getting fast trains instead.
But… Their own numbers say their revenues aren’t close to enough. Why do you not believe their own stats?