Architecting Agentic Systems – part II
I heard someone use this Analogy several months ago. I have no recollection who it was (if you are reading this and came up with this analogy, please let me know so I can give proper credit). At this point, I am working under the assumption that this is lore and everyone has heard this analogy, and I just missed it till recently in my life. Well, with that disclaimer in place, here is the analogy: Refrigerators were a revolutionary product when they were first invented. No more need to lug large blocks of ice miles away from where it could be made or harvested from frozen lakes. However, refrigerators were initially only used to prolong the shelf life of what people already used ice to preserve – vegetables, meat, kegs of beer, etc. They enhanced the businesses of the vendors of these products by allowing them to store them for longer without buying expensive ice blocks and waiting for them to come in on the next pony express. This was, however, not transformational. Just an enhancement. An improvement to existing business methods and processes. Refrigerators did not immediately result in the creation of new business models. That took time. New business models that came to be were brand new product offerings like beer in single-serve bottles and cans, Ice cream that could be stored at home, milk that did not need to be delivered fresh every day but could be sold in larger sized containers, and many more. These products or mechanisms of delivering products to customers could not have existed before Refrigerators. It was a transformational moment for new businesses.
Gen AI today, and especially AI Agents, are still in the early phase of ‘refrigerator’ adoption. We have the fridges but not yet the single-serve beer cans. Organizations are still just looking at AI Agents to enhance – automate and improve – existing processes. Very few have as yet developed entirely new processes or business models that could not have existed before AI Agents. Even my example in part I of this blog series that was defining the art of the possible with AI Agents being used by a smart refrigerator did not present any novel possibilities. Just an enhanced set of actions that are all possible today with my smart refrigerator by leveraging humans instead of AI agents. (I really have to have a talk with someone about this fixation on fridges…). All the example did was replace a human executing processes working with other humans with AI Agents executing pretty much the same processes collaborating with other Agents. Sure, by taking the tasks away from humans, we made life better. We freed the humans of one less thing to worry about – ordering groceries and worrying about how much to order and when to order before and after a vacation. AI Agents can do that error-free. They will not forget. They do not need a human to tell them when plans change. This is, however, not an example of using AI Agents to develop a new business model. We made our current life with current business models better. Not created something that was not even possible prior to agents.
An overwhelming percentage of examples of AI Agents being developed or talked about are really just taking current human processes and workflows and automating them with minimal enhancements leveraging AI and specifically LLMs. Where are the new innovations? Is that all there is with AI Agents? At a recent gathering of Technology executives where I had the opportunity to be on a panel, I posed a question to the room: Who is currently even prototyping an AI-based solution that delivered a business process or model that is truly novel? Something that was not even possible with just humans in play? Not a single hand went up. From Chatbots to marketing copy creation, to sales support, to preventive maintenance, to coding Agents, these are all processes and workflows that humans perform today. Yes, humans may be very inefficient and error-prone at many of these, and AI Agents will be better, but these are not transformative to the business.
There are infinite examples of where companies have had to put in what is often referred to as ‘people putty’ in place to execute complex processes and workflows that requires humans connecting the dots, so to speak, between several siloed systems to make something as simple as a sales order or a support ticket happen. AI Agents are a great way to eliminate the people putty and bring efficiencies where there was not. This is still not developing a new business model or process. This is just making your food last longer. This is not inventing beer in a single-serve can.
To truly leverage the power and potential of AI Agents, we will need to stop looking at existing processes and asking how we can make them better but explore new processes to solve existing or new problems.
To truly leverage the power and potential of AI Agents, we will need to stop looking at existing processes and asking how we can make them better but explore new processes to solve existing or new problems. One way to identify this lack of imagination when it comes to AI Agents is to look at the agents you are developing and ask the question – is each AI Agent here representing a human or an atomic human process that already exists in the current world? In the example from Part I of this blog series, here are the Agents I said we needed:
- Shopping Agent
- Calendar Agent
- Travel Agent
- Budget Agent
Each and of these exists today as a task I already perform myself or ask another human to do so. Not novel. We are able to do all of this without agents. We may suck at it, as attributed by all the milk we have had to throw away after returning from past vacations, or the need to rush to a grocery store within hours of arriving back from said vacation.
Now, what could be an example of a process or workflow that we could not do without AI Agents? I would like an AI Agent that tells me that I need to go on a vacation and sets it up by itself. It either monitors my health data with feeds coming from my wearable devices and tells me I am in dire need of a break before I burn out, or just looks at my company portal and finds I have unused vacation that will expire if I do not use it. It then looks at my family’s calendars and their wish lists of travel destinations and provides a list of options of vacations with locations, dates, including with or without kids, availability of hotels, and flights with cost. Can this be done today without Agents? Yes, with a very smart and dedicated and highly paid personal assistant like we see in shows. But impossible for us mere mortals not named ‘Harvey’, with no ‘Donna’ working for us.
True Business Transformation
To develop AI Agents that are novel in the business processes they execute, businesses need to think in a truly transformative manner. They need to not just look at existing processes and ask, ‘How can I save 30%?’, or ‘Can AI help me go 30% faster’? They need to ask what it is that they have wanted to deliver to their customers but could not. Either because the technology was non-viable or the cost with humans as the resources was too high. AI can help us find paths that humans today cannot deliver.
I want to take a pause here to look at the statements we see business leaders make today about the Return in Investment (ROI) they expect from AI Agents and ask if they are even realistic. ’30% productivity improvement of engineers developing software’ – unrealistic. All we have today is AI Agents that write, document, and test code. If Agents inside a developer’s IDE can reduce their time spent coding by 30%, we did not make software development 30% faster. There is much more a developer, especially a developer in a large enterprise, has to do every day before they even get to write a single line of code. If a developer is spending only 20% of their daily time actually in the IDE, then a 30% productivity improvement in code generation only made the developer 6% more productive (30% * 20%). Meh… Where are the real productivity sinks for developers? One huge area is knowledge management – what has been developed and why? What architectural decisions have been made and under what constraints? What are our coding standards for public APIs? What do we use for scanning code for security defects? Has anyone seen this bug before? All these questions are time sinks for developers. But as this knowledge is scattered across Confluence pages, wikis, Slack channels, file shares, meeting minute documents, and email threads between developers who no longer even work at the company, getting to it wastes more time than a developer will ever spend in her IDE. This is what AI Agents can truly help fix.
The same can be said for anything that humans do in a large enterprise today. Marketing copywriters do not write copy 8 hours a day. Sellers do not spend their entire day closing sales despite ‘Always being Closing’ being the ‘ABC’ of sales. They all have time sinks that span multiple (mostly not very useful) processes they have to engage with.
Back to the previously scheduled programming…
Self-driving cars are probably the best example of AI-driven business innovation that was impossible before the current state of AI. Self-driving cars have been in the human psyche for years, as evidenced by all the self-driving cars in every futuristic Sci-fi book or movie. We already have self-flying aircraft with auto-pilots navigating aircraft across the globe every day by the thousands. Cargo aircraft can even take off and land by auto-pilot (not yet permissible with human passenger aircraft). But the complexity of driving in dynamic road conditions was not possible without the ability to train AI models with millions of hours of video data. FSD is here – go ride a Waymo. Another area where AI has the potential of bringing truly new business processes is in the space of Digital Twins. Whether a Digital Twin of complex machinery like an aircraft or a factory or a digital twin of living organisms, trained AI models on how complex systems behave allow us to examine millions of paths the complex system can take under variable conditions, allowing us to observe the impact of the variables over time. This, in turn, would allow us to test mitigating controls that could be made to the physical systems or prophylactic steps that can be taken with biological systems to prevent negative scenarios from occurring. With AI, we can spawn off millions of Agents that introduce myriad preventive measures to the systems on each possible path, and also have observer Agents study their effectiveness before investing in actually inventing or implementing the preventive measure for real-world testing. The NVIDIA Cosmos Model is an example of a step towards a model of the physical world, and the new Protein Language Models (pLMs) and AI Cell Models being developed are exciting steps towards the development of AI Agents that can interact with digital twins of biological systems.
True Business Transformation requires thinking of the business as a whole and not just individual processes like we are working on improving today. Software development with AI Agents is just one business process in the Enterprise. Even if you are a single product, pre-revenue, pre-released product ‘day-0’ software company, you have business processes other than writing software. You pay your employees, you hire/fire people, you raise money, and you spend money on tools and other stuff you use. You have many business processes. You have to look at how your entire business can be transformed with AI Agents. Not just Agents that write software.
I remember reading that Nest, the smart thermostat company, had to have its interns research local codes for every locality in the US and comply to it county by county and eventually do that same for every country where they wanted to launch the product. With AI Agents, could they have ingested all the building, electrical, and heating codes for every locality in the planet and have other AI Agents design an interface for their product that would satisfy all codes at the same time? That would have been transformative and something that was impossible to do without AI. I am sure we can come up with several such scenarios. Do share any you think of in the comments below. Just don’t use travel booking as an example – it has become the Hello World of AI Agents. Even I used travel booking in my Smart Refrigerator example in my last post. Sorry.
In conclusion, AI Agents are extremely powerful. Anthropic has an awesome paper and videos on how to build effective Agents. But to truly leverage their power, we need to be looking to transform our business. Throw away the box; don’t just think outside it. The current harvest of Agents appears to be just making existing processes faster, cheaper, and less error-prone. While that adds value, it is not transformative. Build Agents that invent the next single-serve beer can. Don’t just chill my keg.