A conversation with the Action Plan team
How do I get AI memory I can trust?
My assistant forgets my preferences every week — and when it does remember something, half of it is wrong, stale, or weirdly made up. I've stopped trusting anything it "knows."
Treat memory like a ledger, not a junk drawer. Every remembered fact should have a source, a version, and an owner. Changes should be proposed and approved, not silently written. Contradictions should escalate to you. And accuracy should be measured. Memory you can't audit is just confident guessing.
Your distrust is well-placed — here's the mechanism. Most AI memory fails in one of three ways:
- No memory at all: every session starts from zero, so you re-explain yourself forever.
- Auto-written memory: the model jots unvetted notes about you as it goes. With no review, errors are stored as enthusiastically as facts — and compound.
- Silent overwrites: when new information contradicts old, the system quietly picks a winner. You never find out which version it's acting on.
And almost nobody measures whether recall is correct — so wrong memories survive until they embarrass you.
What trustworthy memory requires — in any system:
- Provenance. Every fact traces to a source: a conversation, a document, your explicit instruction.
- Versioning. Updates supersede, they don't erase — you can always see what changed and when.
- Review on writes. The agent proposes; a human (or a rule you set) approves.
- Escalated contradictions. "You said X in March and Y today — which stands?" is a question, not a coin flip.
- Measured accuracy. Test recall against known facts; track fabrication explicitly.
That's Action Plan's memory, point for point. Agents propose memories; you approve them; every read is logged; contradictions come to you; and we benchmark recall — including a fabrication rate — instead of assuming it works. Here's what a memory actually looks like:
- NEW "Prefers invoices reviewed on Fridays before sending" — source: your message, this week
- v2 Supersedes: "Reviews invoices ad hoc" (kept, versioned — not erased)
The honest trade-off: approval on writes means slightly more of your attention early on, while the team learns what you want remembered. We think that's the right price — the alternative is a system that knows a hundred things about you, a fifth of them wrong, and no way to tell which fifth.
Anatomy of trustworthy AI memory
- Provenance — every fact traces to a source.
- Versioning — updates supersede; nothing is silently erased.
- Review on writes — agents propose, you approve.
- Escalated contradictions — conflicts become questions to you, not coin flips.
- Measured accuracy — recall is benchmarked; fabrication is tracked as a number.
Why do AI assistants forget?
Either they keep no durable memory between sessions, or they auto-write unvetted notes that decay into noise. Without sources, versions, and review, memory rots.
How do you stop an AI from making things up about you?
Require provenance — cite a source or say "I don't know" — and measure fabrication against known facts. If fabrication isn't a number, it isn't controlled.
Action Plan is in private preview. When your invite opens, onboarding starts from exactly this conversation.
Got it — you're on the early-access list, and I've saved your goal: . When your invite opens, this is the first thing we'll work on together. Watch your inbox.