One Agent, Three Ways to Learn: How OLAV Builds a Self-Improving Agent
Learning isn't one thing. OLAV's agent improves across three layers at three speeds — memory that learns every run, weights it can consolidate into with a benchmark gate, and a harness that tunes its own knobs — unified by one loop and one governance contract so they reinforce instead of fight. Here's the story of how we built it, including the times the system correctly threw its own work away.
Small local models repeat their mistakes. Ask a 12–30B model to drive a multi-step network task twice and it will happily make the same wrong tool call the second time — unless something keeps notes for it. The obvious fix is “just fine-tune the model,” but that’s expensive, slow, and — done constantly — dangerous. So we asked a sharper question: what does it actually mean for an agent to learn?
The answer we arrived at is that learning isn’t one thing. It happens at three speeds, and a good agent needs all three — unified, but not conflated. This is the story of building OLAV’s 多位一体 (many-in-one) self-learning mechanism.
Layer 1 — memory that learns every run (and forgets the right things)
The fastest layer needs no training at all. Every run, OLAV captures what happened, distils recurring patterns into one-line lessons, and injects the relevant ones into the next similar prompt. Same model, same code — a week-old deployment is measurably smarter than a fresh install because its memory is richer. We call it the self-improving loop.
But here’s the trap we walked straight into: naive memory poisons. A task fails because a service is down. OLAV dutifully records the lesson: “NetBox is unreachable.” You fix NetBox. You re-run the task — and the agent, consulting its own notes, still refuses, because the note says NetBox is down.
Learning, it turns out, is inseparable from forgetting the right things. So memory got hygiene:
- A failure that’s environmental and transient (connection refused, permission denied, service down) is stored on a short lease with capped confidence — not as a 30-day rule.
- If the tool a lesson blamed has since succeeded, that stale lesson is suppressed the moment it would be recalled. Fixed service, unblocked instantly.
- “X works again” is recognised as a correction of “X failed” — written, and allowed to retire the old lesson, instead of being tossed out as a duplicate.
Memory is the agent’s working advice. It’s fast and cheap — which is exactly why it’s allowed to be wrong sometimes and self-correct.
Layer 2 — consolidating experience into weights (and a gate that says no)
Memory is “fast weights.” The natural next question: can the durable lessons it accumulates eventually become part of the model itself? Yes — and we ran it end to end, for real.
We exported a deployment’s experience into a clean, encrypted training dataset (transient noise filtered, secrets redacted), shipped it to a GPU box, and fine-tuned a 4-bit LoRA on a 12B model. The result worked: asked “how many devices in NetBox?”, the base model gave a generic “I can’t access your NetBox”; the fine-tuned model answered in OLAV’s own structured tool-output style. It had learned to speak OLAV.
Then the interesting part. We put the fine-tuned model through an automated benchmark against the baseline — the same task set, scored deterministically — and it scored 0.00 to the baseline’s 0.83. So the gate discarded it. Not deployed. No promotion.
That’s not a failure of the system; that’s the system working. The little adapter was overfit on a tiny dataset, and the keep/discard gate is exactly the safeguard that stops a worse model from ever reaching production. A learning system you can trust is one that reliably throws away its own bad work. When a kept model does graduate, the lessons now baked into its weights quietly step back in memory — no double-injection, and if a “learned” mistake recurs anyway, it resurfaces as a signal the training didn’t take.
Weights are the agent’s habits. They change slowly, deliberately, and only through a gate.
Layer 3 — the harness tunes its own knobs
There’s a third thing shaping behaviour that’s neither the model nor its memory: the harness — the scaffolding around the model. How much context it gets, how many memories to inject, result caps, which middleware run. These are calibrated defaults, and calibrated-by-hand usually means calibrated-once and never revisited.
So we made the harness observable: every run emits structured telemetry — which
knobs were in effect, how many memories were injected, when the budget guard
skipped, grader verdicts, tool-selection failures — all joinable to whether the
run succeeded. From that, OLAV can propose a knob change (“small-tier runs at
recall_top_k=1 fail more often — try 2”), validate it through the same
benchmark gate, and apply it as a reversible config override — but only after a
human signs off.
We ran that live too. The proposal was a hypothesis; the benchmark was the test; the test refuted it (for that model and tasks, more recall didn’t help) — so the gate discarded the change. Again: the loop’s job is to reject what doesn’t measurably help.
The harness is the agent’s constitution. It changes rarely, and only with a benchmark and a human.
The unifying idea: one loop, three speeds, escalating authority
Look at the three layers and the same shape appears every time: reflect → propose → gate → apply. What differs is the bar to change each layer, and it scales with how load-bearing the layer is:
| Layer | Analogy | Changes | Bar to change |
|---|---|---|---|
| Memory | working advice | continuously | self-edits, reversible |
| Weights | habits | periodically | benchmark gate |
| Harness | constitution | rarely | benchmark gate + human |
This is deliberately like complementary learning systems in the brain: a fast episodic store, slow-consolidating skills, and a stable rulebook. The fast layer adapts freely and can be wrong; the slow layer consolidates only what’s proven; the constitution almost never moves — and the agent can never rewrite its own constitution on its own.
We wrote the contract down (which rule belongs in which layer, who wins when they disagree, who is allowed to change what) so the three don’t fight. Conflict never comes from having three regulators — it comes from putting one rule in the wrong layer. Get the placement right and they compose into a single system that adapts continuously, consolidates deliberately, and tunes its own scaffolding carefully — each at the right speed, each behind the right safeguard.
That’s what “multi-in-one self-learning” means to us: not one clever trick, but three honest ones that know their place — and a system disciplined enough to discard its own work when the numbers say so.
Read the mechanism in depth on docs.olavai.com → Concepts → Continual Learning.