The Best Solution to AI Agent Self-Learning is 70 Years Old

I want my AI agents to be more autonomous. Not just faster or cheaper, but more hands off. I've come across the idea of a self-learning loop before and it seemed like a good place to start — the basic concept being that we build up a knowledge base from mistakes the agent has made in the past so it doesn't make them again in the future.

So I went looking for how to do it well. Most of what I found was about the how: which library to use, how to structure a memory store, what vector database to reach for. Useful, but it skips the harder question: what should the agent actually be learning? The most useful thing I found was a post by @itsreallyvivek called how to be good at research. It's aimed at ML researchers, but the practices translate surprisingly well to anyone doing iterative work with AI agents. And buried inside it was an idea that was 70 years old - from Claude Shannon, a founding father of information theory - that turned out to be the most essential piece of all.

How I approached this

Before doing anything else, I ran a baseline: run an agent to produce a set of learnings across a bunch of sessions with no extra guidance, then I reviewed the output. Vivek's post has a move that paid off immediately here: pull your failures, read all of them, sort them into piles, attack the biggest pile. So that's what I did: read through the learnings in detail, listed what was right and wrong, prioritised the list, then worked through the top items to understand why those mistakes were being made. To be clear, the focus here was on what the learning agent was correctly and incorrectly identifying, not where the original agent went wrong in the first place.

That left me with a clear picture of the problems. What it didn't give me was a plan. That's where Shannon came in. His opening move in a 1952 talk on creative thinking was to shrink a problem until it's nearly trivial, crack the small version, then reintroduce the difficulty one piece at a time. When I looked at the pile of problems I was trying to solve for (autonomous, cross-session, handling assumptions, correcting for overfitting) it was obviously the hard version. Shannon's advice was to stop trying to build that and find the version I could actually solve first.

What the baseline revealed

Direct feedback gets captured well. When I explicitly corrected something, or when the agent tried an approach that failed, it picked those moments up reliably. "Verify every URL" came from a session where broken links caused real problems. These learnings are grounded and useful.

Small sample sizes produce over-fit rules. When the agent extracted a learning from a single example, it tended to encode the specifics of that example rather than the underlying principle. The rule looked generalisable but was really just a description of one case. What you actually need is for the learning to capture the family of issues — the underlying principle — so a future agent can recognise the next instance even when it looks different on the surface.

Assumptions are the hardest problem. Where there was no direct feedback, the agent filled the gap — silently, confidently, wrongly. For instance, it assumed that because one source was trusted for topic X, a different source was trusted for topic Y — without being told either of those things. The agent didn't know it had made an assumption.

Learnings retained residue of the errors. When I asked the agent to revise something, it would often include the wrong version in its explanation of the fix: "Unlike the previous solution, the revised version..." The error was still in the output, just reframed. Small thing, but exactly the kind of noise that pollutes a learning file.

The approach: make it trivially small

Shannon's method was to scope down until the problem was solvable, then build from there. So I decided for version 1, I would focus on the learnings that were most easily captured and most impactful — learnings about human intent. These are directly captured in my conversations with the agent: "you shouldn't have done that, you should have done this instead..." and over time would help teach the agent what I am actually trying to achieve with this project, which should help it make smarter decisions in the future.

In-scope: capturing direct human feedback within a single coding session that showed a misalignment of intent.

Out of scope (for now):

• Capturing learning across Claude sessions

• Capturing learning where an agent spotted its own technical mistakes

• Fixing overfitting at scale.

That left me with something concrete. A self-learning loop has three phases: data capture, data processing, and integration. I focused on making each one work at the smallest possible scale before worrying about anything else.

Data capture means having the full loop - plan, implementation, validation - visible in one session. If the loop spans multiple sessions, stitching it together is hard. That's why I opted for a single session.

Data processing is where I had to resist the temptation to capture everything. There were plenty of expensive mistakes in my initial analysis, but one category stood out: cases where the agent diverged from what I actually wanted. This is where mistakes get most expensive, and the learnings that come out of it (what the app is actually for, how it should behave, closer to user requirements than implementation detail) seem to be the most valuable thing a self-learning loop can capture. Capturing that felt more valuable than anything else, so that's where I focused.

Integration is about how to feed learnings back into the development workflow in a way that would actually benefit future tasks. Not everything learned from one task is relevant to the next, so the distinction between task-level and project-level matters. Task-level learnings (what worked, what didn't, specifics of this piece of work) stay scoped to that task. Project-level learnings, the ones that speak to what the product is, how it should behave, what good looks like, get distilled into the main agent configuration so every future session starts with that context already in place.

What I actually built

Firstly, I realised my coding agent needs to ask a lot more questions during planning and implementation, rather than making assumptions. Secondly, I needed my coding agent to extract relevant learnings at the end of a session and export these to a file in the project. Finally, I needed a separate agent to review these learnings and extract a summary of generalisable learnings to the project level CLAUDE.md, to be injected into every coding session for this project. I won't go into the detail here, but these instructions have been incorporated into my relevant skill files.

It's not the autonomous self-improving agent I originally imagined. But the Shannon instinct turns out to be right: getting the small version working properly is harder than it looks, and it teaches you things that would have been invisible in the complex version. The more ambitious version is still on the list. But I'll get there one piece at a time.