Are LLMs capable of writing good code?

understanding what the machine spits out

This is exactly why people will still need to learn to code. It might write good code, but until it can write perfect code every time, people should still know enough to check and correct the mistakes.

After a certain point, learning to code (in the context of application development) becomes less about the lines of code themselves and more about structure and design. In my experience, LLMs can spit out well formatted and reasonably functional short code snippets, with the caveate that it sometimes misunderstands you or if you're writing ui code, makes very strange decisions (since it has no special/visual reasoning).

Anyone a year or two of practice can write mostly clean code like an LLM. But most codebases are longer than 100 lines long, and your job is to structure that program and introduce patterns to make it maintainable. LLMs can't do that, and only you can (and you can't skip learning to code to just get on to architecture and patterns)

Great question.

is there any legit reason anyone should learn advanced coding techniques?

Don't buy the hype. LLMs can produce all kinds of useful things but they don't know anything at all.

No LLM has ever engineered anything. And there's no sparse (concession to a good point made in response) current evidence that any AI ever will.

Current learning models are like trained animals in a circus. They can learn to do any impressive thing you an imagine, by sheer rote repetition.

That means they can engineer a solution to any problem that has already been solved millions of times already. As long as the work has very little new/novel value and requires no innovation whatsoever, learning models do great work.

Horses and LLMs that solve advanced algebra don't understand algebra at all. It's a clever trick.

Understanding the problem and understanding how to politely ask the computer to do the right thing has always been the core job of a computer programmer.

The bit about "politely asking the computer to do the right thing" makes massive strides in convenience every decade or so. Learning models are another such massive stride. This is great. Hooray!

The bit about "understanding the problem" isn't within the capabilities of any current learning model or AI, and there's no current evidence that it ever will be.

Someday they will call the job "prompt engineering" and on that day it will still be the same exact job it is today, just with different bullshit to wade through to get it done.

I use it to write code, but I know how to write code and it probably turns a week of work for me into a day or two. It's cool, but not automagic.

LLMs are just computerized puppies that are really good at performing tricks for treats. They’ll still do incredibly stupid things pretty frequently.

I’m a software engineer, and I am not at all worried about my career in the long run.

In the short term… who fucking knows. The C-suite and MBA circlejerk seems to have decided they can fire all the engineers because wE CAn rEpLAcE tHeM WitH AI 🤡 and then the companies will have a couple absolutely catastrophic years because they got rid of all of their domain experts.

I'm my experience they do a decent job of whipping out mindless minutea and things that are well known patterns in very popular languages.

They do not solve problems.

I think for an "AI" product to be truly useful at writing code it would need to incorporate the LLM as a mere component, with something facilitating checks through static analysis and maybe some other technologies, maybe even mulling the result through a loop over the components until they're all satisfied before finally delivering it to the user as a proposal.

No, because that would require it being trained on good code. Which is rather rare.

They can write good short bits of code. But they also often produce bad and even incorrect code. I find it more effort to read and debug its code then just writing it myself to begin with the vast majority of the time and find overall it just wastes more of my time overall.

Maybe in a couple of years they might be good enough. But it looks like their growth is starting to flatten off so it is up for debate as to if they will get there in that time.

This question is basically the same as asking "Are 2d6 capable of rolling a 9?"

No, a large part of what "good code" means is correctness. LLMs cannot properly understand a problem so while they can produce grunt code they can't assemble a solution to a complex problem and, IMO, it is impossible for them to overtake humans unless we get really lazy about code expressiveness. And, on that point, I think most companies are underinvesting into code infrastructure right now and developers are wasting too much time on unexpressive code.

The majority of work that senior developers do is understanding a problem and crafting a solution appropriate to it - when I'm working my typing speed usually isn't particularly high and the main bottleneck is my brain. LLMs will always require more brain time while delivering a savings on typing.

At the moment I'd also emphasize that they're excellent at popping out algorithms I could write in my sleep but require me to spend enough time double checking their code that it's cheaper for me to just write it by hand to begin with.

A broken clock is right twice a day.

I think your wording is something to consider. If you want something that's written professionally, by definition it needs to be written by a professional. So that's clearly not what you're asking for, but that's what you wrote. And that kind of detail does matter, because LLMs are very good at getting part of the format correct and then messing up small details in random places, which makes them precisely useless on their own. But if you want to use them to produce templates that you're later going to modify, of course you can do that.

I'm not clear what you think an advanced coding technique would be. But if your system breaks and you don't understand it well enough to fix it, then I sure hope a competent programmer is on staff who can help you.

Finally, if you rely on automation to write your programs for you and somehow they magically seem to work most of the time, how do you know that they actually work all of the time? If they're giving you numbers, can you believe the numbers? When? Why? Who is guaranteeing you quality in product? Of course nobody is.

That all depends on where the data set comes from. The code you'll get out of an LLM is the average code of the data set. If it's scraped from the internet (which is very likely) the code you'll get will be an amalgam of concise examples from one website, incorrect examples from another, bits from blogs with all the typos and all the gunk and garbage that's out there.

Getting LLM code to work well takes an understanding of what the code it gives you actually does and why it's bad. It will always be bad because it cannot be better than the dataset and in order for a dataset to be big enough to train an LLM it'll have to have everything they can get including all the trash. But it can be good for providing you a framework to start with. It is however never going to replace actual programming and understanding of programming. The talk of LLMs completely replacing programers is mostly coming from people who do not understand coding or LLMs at all.

Of course it can. It can also spit out trash. AI, as it exists today, isn't meant to be autonomous, simply ask it for something and it spits it out. They're meant to work with a human on a task. Assuming you have an understanding of what you're trying to do, an AI can probably provide you with a pretty decent starting point. It tends to be good at analyzing existing code, as well, so pasting your code into gpt and asking it why it's doing a thing usually works pretty well.

AI is another tool. Professionals will get more use out of it than laymen. Professionals know enough to phrase requests that are within the scope of the AI. They tend to know how the language works, and thus can review what the AI outputs. A layman can use AI to great effect, but will run into problems as they start butting up against their own limited knowledge.

So yeah, I think AI can make some good code, supervised by a human who understands the code. As it exists now, AI requires human steering to be useful.

For small boilerplate or very common small pieces of code, for instance a famous algorithm implementation. Yes. As they are just probably giving you the top stack overflow answer for a classic question.

Anything that the LLM would need to mix or refactor would be terrible.

Yes, in small bits, after several tries, with human supervision. For now.

No in large amounts, too hard to human review, they're still doing it anyway.

No LLM is trust worthy.

Unless you understand the code and can double check what it’s doing I wouldn’t risk running it.

And if you do understand it any benefit of time saved is likely going to be offset by debugging and verifying what it actually does.

Technically it's possible, but it's neither probable nor likely, and it's especially not effective. From what I understand, a lot of devs who do try to use something like ChatGPT to write code end up spending as much or more time debugging it, and just generally trying to get it to work, than they would have if they'd just written it themselves. Additionally, you have to know how to code to be able to figure out why it's not working, and even when all of that is done, it's almost impossible to get it to integrate with a larger project without just rewriting the whole thing anyway.

So to answer the question you intend to ask, no, LLMs will not be replacing programmers any time soon. They may serve as a tool of dubious value, but the idea that programmers will be replaced is only taken seriously by by people who manage programmers, and not the programmers themselves.

I don't know how to program, but to a very limited extent can sorta kinda almost understand the logic of very short and simplistic code that's been written for me by someone who can actually code. I tried to get to get chat GPT to write a shell script for me to work as part of an Apple shortcut. It has no idea. It was useless and ridiculously inconsistent and forgetful. It was the first and only time I used chat GPT. Not very impressed.

Given how it is smart enough to produce output that's kind of in the area of correct, albeit still wrong and logically flawed, I would guess it could eventually be carefully prodded into making one small snippet of something someone might call "good" but at that point I feel like that's much more an accident in the same way that someone who has memorised a lot of French vocabulary but never actually learned French might accidentally produce a coherent sentence once in a while by trying and failing 50 times before succeeding and then failing again immediately after without ever having even known.

AI can only really complete tasks that are both simple and routine. I'd compare the output skill to that of a late-first-year University student, but with the added risk of halucination. Anything too unique or too compex tends to result in significant mistakes.

In terms of replacing programmers, I'd put it more in the ballpark of predictive text and/or autocorrect for a writer. It can help speed up the process a little bit, and point out simple mistakes but if you want to make a career out of it, you'll need to actually learn the skill.

Dunno. I'd expect to have to make several attempts to coax a working snippet from the ai, then spending the rest of the time trying to figure out what it's done and debugging the result. Faster to do it myself.

E.g. I once coded Tetris on a whim (45 min) and thought it'd be a good test for ui/ game developer, given the multi disciplinary nature of the game (user interaction, real time engine, data structures, etc) Asked copilot to give it a shot and while the basic framework was there, the code simply didn't work as intended. I figured if we went into each of the elements separately, it would have taken me longer than if i'd done it from scratch anyway.

Theoretically, I would say yes it’s possible, insofar as we could break down most subtasks of the development process into training parameters. But we are a long way from that currently.

ETA: I suspect LLM’s best use-case in this hypothetical would not be in architecting or implementation, but rather limited to tasks with human interfaces (requirements gathering, project planning and logistics, test scaffolding, feedback collection/distribution, etc).

If the unironic goal is to develop things without any engineering oversight (mistake) then there’s no point to using programming languages at all. The machine might as well just output assembly or bin code.

What’s more likely in the short term are software LLMs generating partial solutions that human engineers then are asked to “finish” (fix) and maintain. The effort and hours required to do so will, at a guess, balloon terribly and will often be at best proportional to the resources saved by the use of the automatic spaghetti generator.

I eagerly await these post mortems.

I've tried Copilot and to be honest, most of the time it's a coin toss, even for short snippets. In one scenario it might try to autocomplete a unit test I'm writing and get it pretty much spot on, but it's also equally likely to spit out complete garbage that won't even compile, never mind being semantically correct.

To have any chance of producing decent output, even for quite simple tasks, you will need to give an LLM an extremely specific prompt, detailing the precise behaviour you want and what the code should do in each scenario, including failure cases (hmm...there used to be a term for this...)

Even then, there are no guarantees it won't just spit out hallucinated nonsense. And for larger, enterprise scale applications? Forget it.

Its the most ok’est coder with the attention span of a 5 year old.

Yes and no. GPT usually gives me clever solutions I wouldn’t have thought of. Very often GPT also screws up, and I need to fine tune variable names, function parameters and such.

I think the best thing about GPTis that it knows the documentation of every function, so I can ask technical questions. For example, can this function really handle dataframes, or will it internally convert the variable into a matrix and then spit out a dataframe as if nothing happened? Such conversions tend to screw up the data, which explains some strange errors I bump into. You could read all of the documentation to find out, or you could just ask GPT about it. Alternatively, you could show how badly the data got screwed up after a particular function, and GPT would tell that it’s because this function uses matrices internally, even though it looks like it works with dataframes.

I think of GPT as an assistant painter some famous artists had. The artist tells the assistant to paint the boring trees in the background and the rough shape of the main subject. Once that’s done, the artist can work on the fine details, sign the painting, send it to the local king and charge a thousand gold coins.

Writing code is probably one of the few things LLMs actually excell at. Few people want to program something nobody has ever done before. Most people are just reimplimenting the same things over and over with small modifications for their use case. If imports of generic code someone else wrote make up 90% of your project, what's the difference in getting an LLM to write 90% of your code?

For repetitive tasks, it can almost automatically get a first template you write by hand, and extrapolate with multiple variations.

Beyond that… not really. Anything beyond single line completion quickly devolves into either something messy, non working, or worse, working but not as intended. For extremely common cases it will work fine; but extremely common cases are either moved out in shared code, or take less time to write than to "generate" and check.

I've been using code completion/suggestion on the regular, and it had times where I was pleasantly surprised by what it produced, but even for these I had to look after it and fix some things. And while I can't quantify how often it happened, there are a lot of times where it's convincing gibberish.

The LLM can type the Code, but you need to know what you want / how you want to solve it.

my dad uses this LLM python code generation quite routinely, he says the output's mostly fine.