There’s a mistake that repeats itself every time a new tool makes work faster: people confuse the artifact with the value it represents. The output of a process is treated as the product itself.
This happens across industries. Lines of code get mistaken for software. Jira tickets get mistaken for project management. A 75-page financial diligence package gets mistaken for understanding. It’s easy to see why. The output is what you can count. Pages of diligence, lines of code, tickets closed. That’s what gets measured, and that’s what people optimize for.
One of the latest episodes in Silicon Valley drama was a compliance startup caught autogenerating 90% of all the evidence that went into a SOC2 report. The company, run by a pair of 21 year olds, followed a very clear line of thought:
Companies need SOC2 to close enterprise deals
SOC2 is a 50 page PDF
LLMs are great at making 50 page PDFs!
All true statements. One fatal flaw:
The product is not the 50 page SOC2 report. The product is the added security that comes from auditing your internal systems.
To be clear, this is not a problem with AI. Nor is it a problem with 21 year old Silicon Valley engineers (though they are frequent offenders). It’s a classic misalignment problem that happens anytime the original intent behind a process quietly gets replaced by the output it produces.
The SOC2 example feels obvious. Of course you shouldn’t offload that to an LLM. But in reality these processes are scattered across an organization, side by side with another 50 bad processes that absolutely should be offloaded to an LLM (or deleted entirely).
Even from the inside it’s hard to tell which stacks of paper are a waste of time and which stacks of paper secretly run the entire business.
Executives with decades of experience will forget how they gained that experience in the pursuit of cost savings. They see an analyst spending 80 hours on a diligence package and think: that’s 80 hours of waste we can eliminate. After all, LLMs are great at making 75 page PDFs!
I spent the early part of my career as a middle market M&A analyst. I was often frustrated to see the 75 page diligence package that cost me nights and weekends being scooted around like a paperweight. It’s easy to think the diligence was just a fluff task. And make no mistake, parts of it certainly were.
But the real product was understanding. The two weeks of digging through financials and industry research is what enables you to defend a thesis under pressure. Sitting across from the business owner, you’re speaking the same language. When every bank’s term sheet looks roughly the same, the deals go to whoever the borrower trusts.
Eight years later and I still have those EBITDA margins and entity diagrams burned into my memory. The 75 page report really was a paperweight at the end of the day. A forcing function to make you learn about the company. Had I remained in that career, I’d now have hundreds of those deals in the back of my mind. Informing my understanding of the industry. Helping me spot risks and opportunities that aren’t obvious at first glance. In short: expertise.
The work builds expertise. Expertise builds trust. Trust builds the business.
But if expertise is built through this type of work, what happens when that work is entirely handled by the AI?
The writing was never just the deliverable. It was evidence that the process happened. Before LLMs, a 75-page diligence package told you someone spent two weeks thinking about this business. Good or bad, the writing always told a story. Today it’s a coin toss.
And when the writing can no longer be trusted as proof of thought, the process doesn’t disappear. It just shifts. Automating the writing moves the burden to the reviewer, who now has to verify the output without the context that comes from having produced it. You haven’t saved 80 hours of work. You’ve transferred them to someone less equipped to spend them.
Why are these reports so bloated in the first place? And can you unwind it?
Every requirement in that 75-page template is scar tissue. Someone, somewhere, lost the bank money, and a new section got added to make sure it didn’t happen again.
A company got acquired and the buyer inherited a Superfund cleanup obligation that wiped out the equity. Now every deal has Section 44c: Environmental Liability Review. No one remembers why, but you have to fill it out before you can go home for the day.
Repeat this across a couple decades and you pretty quickly end up with these bloated bureaucratic reports and Bill Lumbergh leaning over your desk asking if you’ll have them done by Friday.
Until recently, if a student asked what they should study for the best career, I would answer computer science without hesitation. Today I’m less sure.
The real engineering expertise required had little to do with the minutiae of programming languages anyway. It’s the ability to take a problem, break it down into its atomic bits, see the pattern, and construct a solution.
There is a Matrix moment that occurs several years into any software engineering career where you break free of the syntax and start to “see the code”. Languages blur together. You realize it’s all the same 47 patterns just restyled and renamed.
But you get there through struggle. LLMs are an absolute force multiplier for engineers, and in theory they can work as excellent tutors. But they’re also a very tempting “do my work for me” button. Especially in a time crunch.
I realize I’ve spent this entire piece arguing against automating work. So let me be clear: I’m not saying AI shouldn’t touch these industries.
In fact, this isn’t even a choice you get to make. If your business runs on written analysis (memos, reports, diligence) it’s already AI-generated. Perhaps not with your blessing. But people default to doing their job the easiest way possible. Especially if they perceive that job as taking up their weekends to produce paperwork that no one reads.
The question is how you preserve the parts of the process that actually build expertise. And every industry has work that’s more friction than thought.
Finance
Identify the bad deals. For every good investment you review there are 10 that are non-starters. But it can still take a day or two of work to piece that together from the data dump of 50 random PDFs and Excel files.
Identifying the bad deals is part of expertise building, but the incremental value you get from reviewing the hundredth poorly formatted excel file for inconsistencies is quite low. And people are often worse at this task than they think. Computers excel at monotony, people do not.
Software
Never get stuck. I remember spending days blocked by some minor bug deep within a library, or trying to map out some legacy API. Some of this was good learning experience. Most was not.
Engineers tend to romanticize those days. Holed up in a room untangling some obscure problem; the feeling of victory when you discover the one line that fixes it. Some of those moments were truly great!
But be honest. Mostly it was terrible. And for every inspired solution, there were a dozen more hacky patches that just pushed the problem another year down the line. Remember, people were writing bad code long before AI started doing it.
Legal
I’ll admit my knowledge of legal is thinner here, but I had the opportunity to work with a law firm handling class action medical claims. The discovery phase meant going through tens of thousands of pages to identify common providers, procedures, recovery timelines.
This is a task where LLMs are better than humans by a wide margin. Do you really learn that much more from retyping the hospital address from the 50th medical record? The real knowledge work is looking at the aggregate data and determining whether there’s a case to be made.
I mostly started writing this to complain about AI generated diligence reports. But the real takeaway is that the artifacts that used to prove someone did the thinking no longer mean what they used to.
AI will produce most of the writing. That’s already settled. What isn’t settled is how organizations will verify the real work got done, and how individuals gain expertise when the “do it for me” button sits so enticingly on everyone’s desk.
Luckily all problems are solvable! The software world has been wrestling with this for a couple years now and real patterns are emerging. Mostly it looks like good old-fashioned best practices, just adopted with new urgency. And I think the industry is becoming better for it.
The point isn’t to protect busywork. It’s to know the difference. Challenge the process, understand it, then decide what stays. Don’t just copy the output.