Open source memory layer so any AI agent can do what Claude.ai and ChatGPT do

I favor automatic recall, invisible to the agent. For memory creation, I find tool calls do a pretty good job, though I also like automatic memory creation on context compression.

I think with automatic creation you need async consolidation (calling it dreaming is a little dramatic for my taste).

My implementation is at Elroy.bot, I recently wrote about different approaches to agent memory here: https://tombedor.dev/approaches-to-agent-memory/

I really want to try this approach. I'm curious because this has not been my experience at all. I created https://github.com/flippyhead/ai-brain mostly just for myself and a few friends use it. But so far, telling the AI (via CLAUDE.md) to look for relevant memories and to think about when and how to save them has worked very well. It can create structures based on decided priorities, notes for the future, that feel like they'd be very different if it was just trying to summarize everything.

In the recent Claude Code leak, there was apparently something called "autoDream", a "background memory consolidation engine" according to this: https://kuber.studio/blog/AI/Claude-Code's-Entire-Source-Cod...

How do you benchmark that? The problem with extracting memories in the background is it's hard to make that work with the prefix cache. You can go a long way with a simple 2-stage LOG.md (detailed log of tasks & lessons) + MEMORY.md (log items that are promoted when the log itself gets truncated) + a stop hook to ensure this runs at the end of a turn.

Which is weird because if I remember correctly, there are summaries already generated by Claude on your hard drive of things you have done in the past.

Just write a hook that runs claude -p after whatever you want and update whatever memory system you want. You can use a channel to inject back what topics were update or what have you.

That's an interesting concept. So it's like if you're an agent chatting with a user, you have an army of assistants who overhear the conversation and record important facts, or search relevant facts on some database and decide on the fly when to interrupt you with "this memory X looks relevant". Sounds easy enough if tokens were free, but an interesting problem to do it efficiently.

I agree. silent agent doing agentic things async is what would be helpful, not requiring a modification to the main prompt

It's a cool website..it says memory. It shows LLM suck and this product magically just works.

In a way, if it does accomplish that, it is a vectordb needing glorification.

What's worse is how obvious the author hasn't even used it themselves. Completely unproven memory layer. No due diligence - just a fancy marketing site with outrageous claims

I've treated this as an information problem and wrote a small utility that explicitly does not store most things (https://github.com/skorokithakis/gnosis). Basically, the premise is that the things the LLM knows will always be there, so store nothing the LLM said, the code will always be there so the code-relevant things should be comments, but there are things that will be neither, and that are never captured.

When we create anything, what we ended up not doing is often more important than what we did end up doing. My utility runs at the end of the session and captures all the alternatives we rejected, and the associated rationales, and stores that as system knowledge.

Basically, I want to capture all these things that my coworkers know, but that I can't just grep the code for. So far it's worked well, but it's still early.

I have a bespoke memory system that I wrote myself and it avoids this problem entirely by making every memory a contextual search space. The “don’t use stripe” memory would only be recalled into context if the model was prompted to do something with payment processing.

I’m sorry this sounds a bit too entitled - no one is putting a gun to your head to use this project and you know you can always read the code and review it yourself and make an educated decision on whether you want to use it or not

This is a fair question, but not one I feel we can let people self answer.

I doubt many people will honestly admit they did no design, testing and that they believe the code is sub par.

It does give me an idea that maybe we need a third party system which can try and answer some of the questions you are asking… of course it too would be LLM driven and quite subjective.

There are many ways to use an LLM to generate a piece of software. I base most of my projects these days around sets of Markdown files where I use AI first to research, then plan and finally track the progress of implementation (which I do step-wise with the plan, always reviewing as I go along). If I was asked to provide documentation for my workflow those files would be it. My code is 99% generated, but I take care to ensure the LLM generates it in a way that I am happy with. I'd argue the result is often better than what I'd have managed on my own.

What's the point? You can make good or bad software, with or without LLMs. Do you ask a carpenter if they use a hammer or nail gun? Did they only use the nail gun for the roof and the deck?

If you care that much and don't have a foundation of trust, you need to either verify the construction is good, or build it yourself. Anything else is just wishful thinking.

Even as someone highly interested in memory I don’t see it as a useful tool for coding. The source of truth for what a repo does or should do is the repo itself.

What you’re describing sounds more like code review guidelines, which can be explicitly brought into context at specific times during a change. A memory system is both too complex and less accurate for this

I have a wishlist for these systems: https://zby.github.io/commonplace/notes/designing-agent-memo...

Yeah I feel the same way. Wonder when/if we'll get continual learning from these models. I feel like they are smart enough already but their lack of real memory makes them a pain to deal with.

All the memories Claude created for me fell in the category remember-to-not-forget, so I disabled it altogether.

[dead]

Interesting approach. How do you do the merging? Is it manual? Just changed files? A hybrid?

True! But this is a very naive implementation, a proper implementation could surpass these challenges.

And once you're being specific about what it needs to remember you are 0 steps away from having just told AI to write and read files with the "memory"