Stanford’s Memorilla squeezes entire conversation histories into 16 memory tokens

Every chatbot forgets. Ask it something on Monday, and by Friday it has no idea what you told it — unless you paste the whole conversation back in, at full price, token by token. An 88-second video posted to r/singularity on Friday morning claims a way out. It shows Memorilla, a system from researchers at Stanford and several partner institutions, compressing everything a user has ever told a model into 16 memory tokens — and using them to answer a question with the kind of personal context today’s chatbots can only fake by re-reading everything. The clip has 1.4K upvotes and 111 comments, and the thread is split down the middle: half of it thinks this is the missing piece of the personal assistant, and the other half is sure it will never ship.

The video is a silent animated explainer, and it is unusually concrete about what the system does. A user asks for foods that might boost their energy during the day. Instead of searching a transcript, Memorilla consults a memory graph built from past conversations — a Lisbon packing list, a sourdough starter, a half-marathon plan, a resume edit — and answers with the one detail that changes the answer: the user has iron-deficiency anemia. The point of the demo is not the recipe list. It is that the relevant fact was found without re-reading any of it.

What Memorilla actually does

The idea, as the paper presents it, is to stop treating memory as text. Most long-context tricks either stuff more transcript into the prompt — expensive, and it still forgets whatever falls outside the window — or bolt on retrieval, searching old messages for the relevant ones and hoping the search finds them. Memorilla takes a third path: it distils a user’s history into a small set of learned memory embeddings, 16 of them, that a frozen language model can read directly. Frozen matters — the underlying model is never retrained, so the memory is an add-on rather than a new model. The video’s benchmark charts push the same message, including a test built around questions about whole books, where the system has to hold on to detail across very long documents.

The paper, “Memorilla: Latent Semantic Memory for LLMs”, comes from Stanford’s SNAP group — first author Adibvafa Fallahpour, with Jure Leskovec among the co-authors — alongside collaborators at Toronto, Penn, MIT and the Vector Institute. It was first posted to arXiv in mid-October and revised in early November. The code is public on GitHub, and the project has its own site at memory.stanford.edu. The Reddit post itself was sourced from Fallahpour’s own post on X, where he shared the paper’s benchmark charts on November 5.

The numbers, and what they are not

The headline figures are the authors’ own, and they are large. Memorilla, the post’s title says, cuts document-token use by up to 4,212×, runs 27× faster on a 512-document test, and outperforms existing approaches on 5 of 8 benchmarks. Those are the numbers the video repeats, and they are the numbers the thread argues about. A 4,212× reduction is the kind of figure that either means a real change in how memory works or a benchmark that flatters the method — and nothing in an 88-second clip can tell you which. The honest reading is narrower: on the authors’ tests, a compressed latent memory can stand in for a huge pile of tokens and still answer questions well. That is a real result even if the exact multiples do not survive contact with other people’s benchmarks.

What Reddit said: the catch is compression

The top of the thread is genuinely enthusiastic. The highest-scored comment, at 287 upvotes, calls it a “big wow factor” and hopes it lands in frontier models, because memory is “one of the last weaknesses of AI systems”. The second-highest, at 119, is the mirror image: “Can’t wait to never see this implemented in any meaningful way.” A reply defends the work as a genuinely novel use of an existing idea, and another commenter notes that consolidation — deciding what to keep and what to drop — is something humans do every day, not just recall.

The sharper criticism is about what compression throws away. “This is lossy compression,” one commenter writes — the point is not that it saves space, but which details it drops. Another notes that the paper’s experiments ran on Nvidia H100 80GB hardware, and doubts the approach would be viable on a personal device yet. A third asks the question the benchmarks do not answer: can it recall the thing that matters at the time it matters, rather than in hindsight? One reply also points out that the video’s charts are hard to read, with a y-axis that labels an index score without explaining it on screen. None of the top comments claims the work is fake, misattributed or old. The scepticism is about shipping, not about whether the paper exists.

What to watch

Two things decide whether this matters beyond the thread. The first is adoption: memory is the feature every assistant maker promises and none has shipped convincingly, and a 16-token memory that plugs into a frozen model is exactly the kind of thing a lab could adopt without rebuilding its stack. The second is verification: the code is public, so independent researchers can now run the benchmarks themselves and find out whether the 4,212× figure holds up outside the authors’ lab. If it does, expect the idea to show up — quietly, as infrastructure — in the next generation of assistants. If it does not, the video will still have been a useful 88 seconds: a clear picture of what remembering should look like, even if the engineering is not there yet.

For now, the clip is the story: a research demo that makes an abstract problem — what a chatbot should remember about you — feel concrete, and a thread that immediately asked the right follow-up questions. The video is on the original post.

Sources: the original post by u/141_1337 on r/singularity; the paper on arXiv (2610.21375); the Memorilla code on GitHub; the project site at memory.stanford.edu; first author Adibvafa Fallahpour’s post on X. The editorial illustrations are AI-generated. Related: our coverage of Siri’s conversation recall in iOS 27.2 and a hands-on with Nanobot, a personal agent that runs on your own machine.