How efficiently you can get a base model to be aware of your codebases, of your business domain and of your intentions, to do what you want. The keyword here is TO DO. Doing is, still, all that matters. And doing means having agency, and that means agents.
Agents are the new “hot” thing in AI. A few years ago, using LLMs through an API was amazing, then, prompt engineering as a “semiformal” engineering discipline appeared, where the focus was on writing comprehensive (read, too over-engineered and sometimes brittle) prompt templates to guide models better.
Then, RAG appeared, and with it the resurgence of search and information retrieval as an important discipline. Here, it was all about how you embed your information and how to enrich your over-engineered prompts from a few months ago with yet more stuff.
What is the common denominator here? It’s all about reliability.
Agents are the current de facto standard for interacting with LLMs, roughly since the beginning of this year, I would say. It’s not good enough to call an LLM via an API or to use RAG if it means leaving your tasks unattended.
You want to send that email. You want to refactor that code, OCR that document, compile that project, etc., etc. So, without agents, you have very limited options.
First, before we go into what makes a good agent, we can ask first what actually is an agent? Well, there are many definitions around, but, the one I personally like to use, and it’s actually my own, is quite simple: an agent is a strong, base LLM that has access to tools and can do things on your behalf.
Of course, there are levels to this definition: Some base LLMs have a (very) strong base training in using tools, and can be extremely effective when embedded inside an environment where they can use some of those base tools. Think, models that can read and write files naturally become very capable when embedded inside an agentic IDE where they can use those tools liberally to plan their next moves. They also gain extra “dynamic context” that neither RAG nor prompt engineering could give them.
So, if you then take these base LLMs and then start giving them tailored tools to your own needs, then you unlock a whole new world of capabilities.
So, a good agent is a strong, base model that can use tailored tools that serve your needs. If you’re a pastry chef, you become much more effective when you have a good whisker, than when you need to fold your ingredients with a wooden spoon. The concept is similar here.
It’s as easy as it can be: a strong base model, tailored tools, and someone who loves writing documentation. Three key ingredients, of which one is given to you for free (the strong base model, and well, “free”) so you can actually just focus on what you can control: tailored tools and writing good documentation.
These are the aspects we’ll look into in depth for the rest of this post.
The first thing is ensuring the agent can use strong tools to just do things you care about. You know your business better than anyone, and you know how people can leverage it best and consume it, so, you are the only one who can actually craft the tools you want, because you are the only one who really knows what it takes to be effective with it. It’s your pet, after all!
While modern models become more and more capable with each iteration, it has now also become evident that garbage in, garbage out has never been more applicable. If your codebase, APIs, documents and unique value proposition are a tangled mess of legacy code, mixed with critical documentation, outdated APIs and key IP pieces of code, you will have a very rough time getting anything useful out of any agent.
Essentially, clean up your house before you invite people over.
Focus on designing code that’s SOLID, use meaningful names, keep your interfaces small and segregated and defining APIs that follow REST principles, but do not fret about irrelevant details. It does NOT matter in the slightest if you write a POST by the book, or if something must be a PUT or a PATCH or, you know… whatever. So, what does matter? Well, what has always mattered: don’t do stupid things: keep GET for retrieving data, do not DELETE any data outside a delete endpoint, keep request bodies meaningful and use descriptive names in the paths, bodies, responses. Do small, simple things with intention, because intention reveals taste, even if it’s not “technically correct”. And taste is more important here.
No, you definitely DON’T need an Open API spec either. Why? Well, if your endpoints are named clearly and if your data flows intentionally through your backend systems via a decently written API, that’s all you need. Again, keep it simple, use common sense, and NEVER OVERENGINEER stuff.
Now, for the last part, which is arguably related to the above, but is somehow harder: You need good people to write good docs.
Writing clearly is very hard, and it’s not the same about writing too much (I am guilty of that and try to improve every time I write) but it’s about writing what matters well.
Two good paragraphs that surface key issues are better than two pages of overly verbose slop that dilute the important pieces of information that you really need to convey to these models.
Focus on highlighting the details and internals of your “tailored tools” from a usability perspective and not a technical one. “Ok, I have this data in my hands, what will I use it for?” is a better question to answer than “Ok, with this data can I make that specific pain point go away?” because the first focuses on the actual downstream LLM “target” rather than on the usability of the API itself, which is more of a technical thing, that is less relevant from a user perspective.
It’s a very subtle distinction but, if you can really nail it down, you will be amazed at how good your agentic workflows can be.
It’s like the virtual and modern version of the old adage: “clear desk for a clear mind” that teachers and researchers liked to use to convey that something isn’t fully clear. You can’t hide your flaws in understanding things behind brilliant code if your documentation is bad. Think about it.
The final takeaway from this short post is: simple things done very well often beat complex workflows built on top of shaky foundations.
If you take time to build clean APIs and write great (not good, great) documentation, the dividends you’ll reap can be enormous, especially because these things actually pay themselves many times over by keeping your codebase easier to follow, predictable and give incentives to everyone to keep documentation top of mind, which in turn can make a flywheel of innovation kick off, where having that “clear desk” can provide the “clear mind” that’s often needed to innovate in ways that are truly rewarding or groundbreaking! The best part? If the foundations are there, agents can actually code your innovations for you too!