I love code-assist tools. I am able to navigate repos faster, implement simple solutions quicker, and be useful engineer even when I have limited framework or language knowledge. Yet, after a year of using Github Copilot and Cursor, I’ve noticed how these tools can also make inexperienced engineers worse.
There are times when offloading development to the LLM makes sense. Time constraints and throwaway solutions, make code-assist tools attractive short-cuts. But when you're writing enterprise code that needs to be maintainable and reliable, dependence on AI can lead to deterioration of the codebase and an engineer’s familiarity with it. With that in mind, an obvious sign that you are depending on AI too much is …
Last week I was scratching my head while looking at three methods that all seemed to do the same thing but differed by a small condition. Visually the code looked something like the following.
getItemsIfApple(type) {
if (type != Apple) return
getItem(type)
}
getItemsIfBanana(type) {
if (type != Bananas) return
getItem(type)
}
getItemsIfZuchinni(type) {
if (type != Zuchinni) return
getItem(type)
}
This is the hallmark of uncritical AI use. Large files that span in the thousands of lines of code, where large chunks of the code can be simplified or abstracted. Instead of catching instances where repeating code blocks can be replaced with helper methods, the code assist tool will continue to append new methods that reuse existing code. This results in thousand line files where half the code is near-identical copies.
This shouldn’t be surprising from a tool that is primarily designed to interpret and spoof language. Large language models are indeed fantastic translators. Asking the LLM to write a function or query that handles a simple task is a translation problem from english to the programming language. But foundational concepts of object oriented programming, are more mathematical than linguistic. They require designing code less like a linear series of steps and more like a graphical tree. Parent code blocks declare common patterns, while child code blocks handle branching cases and unique types. It should not be a surprise that code-assist tools struggle with this aspect of coding.
By handing over code-quality to AI, I see engineers submit code that is void of best practices. Instead of early returns, massive code blocks are inserted into single conditional statements. Advanced elements of a language that allow for inheritance and abstraction are largely unused. While we still maintain our code bases, it’s best to assist our code assistants with writing maintainable and clean code.
I routinely catch junior engineers in a query spiral. The engineers will keep asking the code-assist tool to “plz fix” or “summarize this section”, while making zero progress towards their goal. This persistence is all done to avoid the frightening activity of ... reading the code base.
When the code base itself follows consistent patterns, LLM’s are good at summarizing them. They even act as a decent replacements for key word or logic search in the repository. Where I find engineers most often entrap themselves into a query spiral is when the code base is simply poor or their question is out of the context of the repository.
Say you are asking the code-assist tool to set-up the configuration files for an environment deployment. The LLM doesn’t have access to your actual environment or cloud infrastructure. It will do its best from context-clues, such as Docker and Terraform files, but its response will likely be influenced by best-practices from developer guides used to train it. But I’ve worked on few teams that follow best-practices for anything, so the AI’s response may not suit your project. Not recognizing this will result in repeated attempts to massage config files into formats that don’t work for your project.
More importantly, not developing an intimate understanding of your code-base makes your future contribution to that code-base slow and potentially dangerous. If each time an engineer opens their repo, they describe their limited understanding of the code to the AI, ask it to devise a solution, then write a test, then push it to git, and have someone else review the code that they haven’t looked at, you are building a system that nobody is a subject matter expert on. Unfortunately, when that system fails and you’re asked to fix it, “The AI wrote it” won’t be a valid excuse.
I recently watched an engineer present the prompt files they wrote to improve their code-assist tools one-shot chances – when the LLM produces the correct result on its first try. They had written out all of these files to provide instructions on API standards, testing practices, project layouts, documentation styling guides, and much more. The code-assist tools use these prompt files with each question to inform their answer. My colleague leaned over to me and joked, “so when are we actually going to build something?”
Creating all these “meta” files that attempt to guide LLM’s will result in a slew of documentation bloat that will only grow stale and serve to annoy engineers in the future. For instance, my code-assistant recently insisted on writing a markdown document with every big request I asked. I would ask it to write a set of API controllers, and after completing the task it would proceed to generate an API Readme in the same folder. Turns out, someone had added a prompt file that instructed the code-assist tool to emphasize documentation for large asks, and it took that request quite seriously.
But more so, if we find ourselves spending so much time writing documentation to try to increase some untracked metric of the code-assistant’s performance, it might be a sign that the we’re forcing the tool to do something it’s not good at. Yes, AI tools translate language into code snippets very well, but perhaps single agents were never made to write whole applications in one go. Ultimately, as software engineers, our goal should be to build quality software products. Inundating our repo’s with style guides for bipolar LLM models is a distraction from that.
I don’t want to denounce the utility of code-assist tools, they are incredibly useful. But we’re beyond the peak of AI hype and we need to be honest about the short comings of these tools. Established companies are shipping buggier software. Startups are riddled with security holes. Vibe coding was fun while it lasted, but the industry needs engineers who actually understand what they're building. It's time to bring craftsmanship back to software engineering.
As always, I’d love to know what you think about code-assist tools and AI usage in the workplace. Have you encountered any of my anecdotes and how have you addressed them in your environment?