valaramech

Absolutely. We’re still figuring all this out and it’s going to take time. I believe that LLMs have a place in the software development lifecycle but where exactly that is depends largely on the project.

For example, I think that open source projects should continue to ban LLM generated code. The barrier to entry and quality of submissions is simply too low for there to be any benefit. Code review, on the other hand, might actually supply some useful findings.

More or less, though I would argue that some of these aren’t actually a problem with the LLMs themselves. Specifically, the issues with management and “enslopification”. I would also push you to think more on which parts of this are purely your own dislikes - I, for example, actually enjoy reviewing other people’s code.

That said, I don’t disagree that LLMs are largely being used in ways that are not useful or helpful. However, I believe that time will show those uses as the harm they are. Businesses that went too hard on it will have to spend huge amounts of money to backtrack and some will not survive the endeavor. Aside from the scale, this isn’t too different from the crypto and blockchain nonsense that popped up everywhere when those were the hot new tech. A lot of companies will fail and a lot of time and money will be wasted in the process.

I’m trying to have a nuanced opinion about this. I do think that LLMs are pretty bad, but I don’t find myself landing on thinking they’re entirely bad. Time will tell and I’d actually be pretty happy to be proven wrong.

My opinion is that there isn’t a one-size-fits-all answer to this question. There exists a class of software for which the detriments of using LLMs in a non-disciplined way is kinda irrelevant. Generally, this is the kind of software that isn’t worth the money that would be spent on an actual developer. These would be, like, very small user-base internal applications.

I might also argue against the “meant to be used for all” part of that statement. That’s the way that AI companies - and many execs - are trying to position LLMs, but I don’t think that’s the reality of their usefulness. Kent Beck did a pretty good talk about this recently that I think summarizes the position well. LLMs let you make mistakes in days that used to take months. The corollary to that is that they also enable you to fix those mistakes in days instead of months, but that requires understanding, thought, and patience.

We’re in the same boat, friend. Believe me, I detest the slop as much as you do. However, it is possible to use the LLMs in a way that isn’t miserable for everyone - I’ve been slowly cajoling my coworkers into it. It takes a certain kind of discipline that I haven’t found particularly common from people that just want the robot to “do the thing”.

One of our product people happens to have that discipline. Their PRs are thoughtful and understandable and they take the time to actually understand their changes before slinging them. So, I know it is possible, if rare, currently.

The reality is that LLMs are massive force multipliers for a competent dev team. They’re largely garbage in other areas, but programming is one where they have clear, obvious benefits when used in a thoughtful way.

It is important to note that there are a lot of load bearing adjectives in those sentences and any usage of LLMs also needs to be weighed against it’s clear, obvious societal harms. It’s going to take a little bit of time for legislature and society to catch up to technology, but I have some hope still.