The absence this makes loud: a question nobody has written down.
Why an Obsidian vault behind an embedding index fails as an agent’s memory — not because of Obsidian, but because a document store cannot represent the thing it does not contain.
The pattern is now standard enough to have a shape. You keep a vault — Obsidian, or a docs/ folder, it makes no difference — and you point an agent at it. Notes go in. An embedding index goes over the top. The agent asks a question, the index returns the nearest few notes, and the agent reads them as though they were what it knows.
It is a good way to think and a reasonable way to start. It fails as a memory engine for a reason that has nothing to do with Obsidian, and everything to do with what a document store is able to say when it is asked a question. It has exactly one answer, and the answer is always something.
A vault cannot say “I don’t know”
Nearest-neighbour search returns the top k by similarity. That is its whole contract. Ask it about a subsystem nobody has ever written a note about and it does not return nothing — it returns the k least-unrelated notes it has, with scores that look exactly like the scores from a query it could answer well.
So two very different states arrive at the agent in identical clothing: we have recorded something about this and we have recorded nothing, here is the nearest thing anyway. The agent has no way to tell them apart, and neither do you, because the retrieval layer does not distinguish them either.
This is the failure that matters, because it is the one that compounds. An agent that receives three weakly-related notes does not conclude “there is no policy here”. It reads them, finds nothing contradicting its plan, and proceeds — and the silence gets recorded as agreement.
Everything in the vault has the same standing
A note you typed at 2am while annoyed and a finding that came out of a two-day incident are, to the index, the same object: text with an embedding. There is no field for how do we know this, no record of who agreed, and no way for retrieval to prefer the one that was paid for.
Worse, contradictions do not resolve. Write “we retry three times” in March and “retries are handled by the gateway now” in July, and both match the same query forever. A vault has a notion of similar. It has no notion of superseded, because nothing ever went back and marked the March note as wrong — you moved on, which is what people do.
It gets worse as it gets bigger
The pitch for a second brain is that it compounds. For retrieval, the opposite happens: every note added is another candidate to be confused with, and the nearest neighbours get nearer to each other rather than to the truth. The asset that was supposed to appreciate is the one thing in your stack that degrades on write.
And the important lessons were never written down
Ask what you actually want an agent to remember and it is rarely the tidy stuff. It is: this refactor looked safe and was not. That test passes for the wrong reason. The last three people who touched this file broke it the same way.
Those are learned by being wrong, and being wrong is precisely when nobody stops to write a note. The vault gets the lessons you had spare attention to record, which is a biased sample of the ones worth having — and it is silent about the bias, because a document store has no way to represent a thing it does not contain.
Zero and unknown are not the same number
Every one of these collapses into a single design error, and it is worth stating on its own because it survives long after you have stopped using Obsidian for this: a memory that cannot represent its own absences will report them as results.
| What happened | What a vault returns | What it should say |
|---|---|---|
| Nobody wrote about this | Three loosely-related notes | Not recorded |
| Two notes disagree | Both, ranked by similarity | One superseded the other |
| A lesson nobody vetted | Same as a vetted one | Unvetted — weigh accordingly |
| Never measured | 0 | NULL |
That last row is the one people argue with, so: a rule that fired zero times and a rule whose firing was never observed are different facts, and rendering both as 0 eventually retires something for being unused when nobody ever checked. Zero is a result. Unknown is not one.
What a memory engine has to do instead
None of this argues for a database over a folder. It argues for three properties a folder cannot have, and they are worth naming independently of anyone’s product.
Derive it from what happened, not from what somebody remembered to write. The record of a session already exists — the transcript. Reading episodes rather than notes removes the attention bias entirely, and it is the only source that contains the moments nobody would have written up.
Make refusal cheaper than acceptance. A candidate lesson should meet its vetoes before it meets its evidence, and a veto should not be outvoted by enthusiasm. In practice most of the work is refusing to learn from noise: in one real corpus, 19 of 42 surviving messages were harness-injected — system reminders, hook output, compaction summaries — text that looks exactly like something a person said.
Count sources, not occurrences. Recurrence is the strongest signal that a lesson is real, and the easiest to fake by accident: one confused run repeating itself three times looks identical to three independent confirmations, unless you are counting distinct origins.
The part nobody wants to hear
A memory that refuses is smaller than a vault, and it feels worse to use at first, because most of what you would have written down does not survive contact with its own vetoes. That is the trade. A hundred notes you cannot rank are not more useful than eleven you can act on — they are a search problem wearing a knowledge base’s clothes.
Keep the vault. It is a good place to think, and thinking is not the thing that broke. Just stop asking it a question it has no way to decline to answer.
Where this comes from
These are the mechanics of the learning loop in
Graphban, an agent-native work tracker whose whole premise is
that an absence should never read as a clean result. The loop reads harness transcripts
rather than authored notes, evaluates vetoes before evidence, counts distinct sources,
and renders NULL where it has not measured. The
longer write-up is memory and the learning loop; the
neighbouring argument, about why a folder cannot arbitrate between two agents, is
Markdown always says yes.