Last night I was working on this site, and wanted to change the banner at the top of the page to be more inkeeping with the rest of the site, I wanted a pixellated sky towards the top (which I’m still not 100% happy with), so after creating a gradient in Aseprite1, I loaded up the CSS file of this site in Visual Studio Code, as part of the project.
I use VS Code to do all my coding. I love the tool, it just works, but lately I’ve noticed the AI Code completions picking up a lot more hallucinations. Here’s one that was suggested to me last night.
You can see the image within the “images” folder of the theme called top-bar-background.webp, but the AI Code completion suggested the file to be top-bar-bg.webp. It was late, I’m developing my cold, and it’s my site – so I accepted it, without proper checks and uploaded it.
It didn’t work.
Now there is the point where AI proponents should say “It’s AI! You should check this”, but here’s the thing – I really shouldn’t. Copilot already has access to my theme and file library. Surely a “check if that file exists” with the returned data should be done? You don’t need AI. There’s a function in most modern languages, here’s PHP’s for example. Surely AI can check if a bloody file exists?
Other AI proponents will say “Cursor (for example) won’t do this”, but that’s my point. Cursor is built on VS Code, but there’s probably some weird quirks that are different that I would have to learn.
That friction adds time, and slows me down.
And that’s me as an 20+ year developer who can spot things. I shudder to think what those who blindly vibe code miss.
- Aseprite is one of those programs that really should be three times the cost, it does one thing and one thing very well. I’ve never glided through a graphics program like it before. ↩︎
I make occasional use of AI in my programming. I’m no fanboy: I did two experiments in building applications entirely by “vibe coding” and while both worked… eventually… I wasn’t happy with the code quality of either.
Autocomplete’s nice, but yes, like any AI output it can catch you out.
The biggest challenge IMHO is learning that the kinds of mistakes an AI will make… are not the same as the kinds of mistakes a human will make. A human might get the wrong syntax for the background: directive, mess up the escaping of a URL, or accidentally try to put an image into a background-color:. An AI is unlikely to make those mistakes, but it might well make up something that superficially looks-convincing, but isn’t remotely-correct.
I think that’s why it catches people out. People like you, who were momentarily distracted by tiredness. People who think ChatGPT is their friend or that Siri is doing a good job of condensing the news for them. We’re all susceptible to getting caught out by AI… but the underlying reason, I believe, is because the ways in which it makes mistakes are so fundamentally alien to the ways in which humans make mistakes that it takes a special level of focus to detect them.
Anyway, that’s my thinking.
You raise a good point. I look for certain mistakes in code, because I know the mistakes I make in code. I’m not thinking of other mistakes that could occur :).