A year of building with AI: what actually changed

Twelve months ago I was pasting code into a chat window and copying the answer back out.

Today a system I built runs the recurring work across four products, writes pull requests while I am asleep, and stops at a gate when something needs me. This site is maintained by it.

The volume is not the interesting part. The interesting part is that I was wrong about the bottleneck four times, and each correction was worth more than the year of effort around it.

Stage one: the chat window (and why it caps out)

I started where everyone starts. Describe a problem, get code, paste it back, fix what does not compile.

It works, and it caps out fast, for a reason that is not obvious at the time: you are the integration layer. Every piece of context the model needs, you carry in by hand. Every result it produces, you carry back out. The model is fast and you are the slow part, so the whole thing runs at your typing speed.

The tell is that you get good at describing your codebase. That skill feels like progress. It is actually a workaround for the model not being able to see the codebase, and the moment it can, the skill is worthless.

What I got wrong: I thought the answer was better prompting. I spent weeks getting good at something I would throw away entirely.

Stage two: give it the filesystem

Moving to Claude Code changed the shape of the problem rather than the size of it. The model reads the files. It runs the tests. It sees the actual error instead of my summary of the error.

The step change is not speed, it is being wrong less often about the wrong things. In a chat window the model invents a function that does not exist, because from where it is sitting that is a reasonable guess. Given the repository, it stops guessing.

That is when I learned the rule that has held ever since: when output is bad, the cause is almost never the model. Four things cause almost everything - missing context, a misleading tool surface, contradictory instructions, and genuine limits - and they are worth working in that order. I wrote that up properly in when the agent gets it wrong.

What I got wrong: I assumed more autonomy would mean better results. It does not, on its own. An agent with full filesystem access and no structure is just a faster way to make a mess.

Stage three: structure, or the same conversation forever

By autumn I was using it daily and hitting a different wall. I was explaining the same things constantly. How we deploy. Which directories are generated. Which conventions are real and which are aspirational. Why that one file is like that.

Every session started from nothing. The model was excellent and had amnesia.

Two things fixed it, and they are the two things I would tell anyone to do first.

Skills. A procedure written down once, with the judgement included. Not documentation - judgement. Which environment to check first, what a failing health check at 60 seconds means versus at five minutes, where the rollback command actually lives, given the README is out of date. The work that used to need a person becomes a command. The skills and plugins guide is the long version.

Memory. Plain markdown, in git, split by lifespan rather than by topic: durable facts, an append-only log, disposable working state. Not a vector database. I argued that out in the filesystem is my database, and a year on I have not wanted to change it.

The compounding starts here. Before this, every improvement was mine and temporary. After it, every correction I make is permanent and the system is better tomorrow than today.

What I got wrong: I optimised the wrong end. I spent a fortnight tuning prompts when the actual win was a project instructions file that was simply correct - how to run the tests, what is generated, what is off-limits. That one file improved every result at once without changing a single prompt.

Stage four: MCP, and the inversion that matters

Skills and memory make an agent good at working on a repository. MCP makes it able to touch everything else.

The framing that took me too long to arrive at: an MCP server is not an API with a new hat. It is your system exposed to somebody else’s AI. That inversion changes every design decision. You are building for a reader with no memory between sessions, no screen, no colleague to ask, and one shot at picking the right call from a list.

Design for that reader and it works. Design for a developer reading documentation and it does not. The whole argument is in designing an MCP server, and it is the piece of writing here I would keep if I could keep one.

What I got wrong: my first server had one tool per API endpoint. Complete, correct, unusable - the model needed five calls to answer one question and each hop was a chance to pick wrong. Wrapping the job instead of the endpoint was a rewrite I could have avoided by thinking about the reader for an hour.

What it looks like now

The system routes work to the agent that owns it, that agent has a bounded role and a scoped tool list, it loads the skill that matches, it reads memory that persists, it reaches live systems over MCP - and it stops at a gate for anything public, anything involving money, anything that cannot be undone.

That last part is not caution. It is the thing that makes the rest safe to run at all. Leverage without control is just a faster way to break things, and a rule that lives in a prompt is a preference, not a gate. It has to be in the system, enforced, where no amount of confident phrasing can talk it out.

Concretely, over the year: four products built and running, this site maintained by the system that it describes, and a working day where the recurring work is done before I look at it and what is left is the decisions. It also assists with my day job, though nothing about that work appears here.

I did not hire anyone.

The five things I would tell myself twelve months ago

1. Context beats prompting, and it is not close. The hours I spent learning to describe code were wasted. The hour spent writing a correct project instructions file improved everything permanently.

2. Write down judgement, not documentation. If a skill contains only what is in the public docs, it is costing context and adding nothing. The value is the thing you would tell a new hire on their first day that is written nowhere.

3. Memory is the part everyone gets wrong. Not because storing text is hard, but because nothing in normal operation removes anything. Six months in you are reading four thousand lines to answer something worth three. Split by lifespan and prune on a schedule.

4. Measure before you restructure. I once cut session context by 42% and then made an identical-looking change that added tokens, caught only because I measured first. That one is written up, and it is the most useful thing I learned all year.

5. Put the gate in the system on day one. Retrofitting a control is much harder than starting with one, and the day you need it is the day you find out whether it was real.

What I still get wrong

Plenty. I still occasionally rewrite a prompt when the problem is context. I still under-invest in pruning until memory is visibly rotting. I still sometimes let a long session accumulate contradictory instructions rather than starting fresh with the good context.

The difference is that these are now recognisable within minutes instead of days, and every one of them is written down somewhere so the system does not have to relearn it.

That is the whole thing, really. Not that AI made me faster. That it made mistakes cumulative - each one caught once and then encoded, instead of being made again next quarter by someone who was not in the room.

Where to start, if you are starting

Read the guides in order. Or if you want the shortest version: write a correct project instructions file this week, encode one repeated task as a skill, and put a gate on anything you cannot undo.

Everything else is detail.