The Compounding Isn't the AI

Three months ago I sat down to answer a straightforward question: why were we fielding the same support tickets, week after week?

The answer took about two hours to find. I pulled 240 tickets from a four-month window, read through them, and categorised what I saw. The patterns were obvious in hindsight. Access requests, report management, data provisioning — the kind of work that touches tools like Snowflake and Looker but requires no real judgement. Repetitive, manual, low-stakes for any individual ticket and quietly expensive in aggregate.

That analysis became a ranked backlog. The backlog became seven production bots. The bots, over Q2, handled roughly 180 individual requests across 64 batches, hands-off, while I was doing something else entirely.

None of that is the interesting part.

The move that actually matters

I used Claude Code for all of it: 78 sessions over roughly thirteen weeks. If you'd asked me at the start what I expected to get from that, I'd have said "faster code". That's not what I got.

What I got was a forcing function for structure.

Every time I started a new session, I had to explain the context. The rules. The edge cases. The things I'd learnt from the last time something went wrong. The first few times, I did that in the chat window and it was lost the moment the session ended. By session five or six, I'd started writing it down properly: a CLAUDE.md file, sitting at the root of the project, updated every time I learnt something new.

That file took about an hour to write the first time. After that, every session started warm. Thirteen weeks of work, and I never had to re-explain the same rule twice.

That's the thing I didn't expect: the compounding isn't the AI. It's the scaffolding you build around it.

What I actually built

The bots are fairly straightforward automations. Someone submits a form; a ticket gets created; the bot validates the request, queues it, opens a pull request against the relevant repository, and notifies the team. A human reviews and merges. Nothing reaches production without a sign-off.

What made the difference wasn't any individual bot. It was the shared infrastructure underneath them: a single Slack helper used by all seven bots, a shared config file, a consistent folder structure, a test scaffold that every new bot inherits. When I needed to change the way the bots post to Slack, I changed it once.

The one time I broke that rule, I copy-pasted a validation rule instead of centralising it. It silently diverged. A bot acted on stale logic for two weeks before I caught it. That was an expensive lesson in why DRY — Don't Repeat Yourself — isn't just an engineering principle.

By the end, there were roughly 770 automated tests across the project, all running in CI on GitHub Actions. I'm a PM. I didn't write most of that from scratch. But I understood every decision in it, because I'd had to explain each one clearly enough for Claude to implement it correctly.

The honest cost

I want to be careful not to oversell this.

The Q2 numbers are real: 64 batches, ~180 requests processed, mostly without me touching anything. But getting there took 16 sessions just on running, monitoring, and fixing the bots after they went live. That's not a small number. The automation saves time at scale; the maintenance cost is real and ongoing.

There's also a different kind of cost. Using AI to write and maintain production code means you carry the cognitive load of understanding what it's doing, even when you didn't write it line by line. The moment you stop doing that, things break quietly. You have to stay close.

And there are things I genuinely wouldn't use this for: one-off questions where thinking it through yourself is faster, anything with high stakes and no easy way to verify the output, actions that are hard to reverse. The bots work because every outward action, every PR, every Slack message, goes through a human review step. That's not an accident; it's a design decision I made early and held to throughout.

What I'd do differently

A few things. I'd let the bots report their own health from day one, rather than adding monitoring retrospectively. I'd start cleaning up technical debt earlier rather than letting it accumulate across sessions. And I'd write down the feedback rules more carefully when Claude gets something wrong: the correction is valuable context, and I've lost more than a few because I didn't capture them in the moment.

I'd also be more deliberate about the distinction between what I'm estimating and what I'm measuring. An earlier analysis projected these three bots would avoid an estimated 200+ support tickets a year. The Q2 figures are a different thing entirely: requests handled, not tickets deflected. Both numbers are real. They don't mean the same thing, and conflating them would give the wrong picture.

The frame that's stayed with me

A few weeks in, I noticed I'd stopped thinking about individual sessions and started thinking about the system. What needs to be true for this to keep working without me? What happens when someone new picks this up? What's written down and what's only in my head?

Those are software engineering questions. They're also just good product questions. The AI accelerates the execution. The structure determines whether any of it compounds.

If I'm honest, the biggest shift wasn't in how fast I could move. It was in how much more deliberately I had to think before I started. That's not a bad outcome for a PM.