Give your chatbot a memory that outlives the chat: hands-on with MemT-1.0

Every chatbot forgets: close the session and the conversation is gone, or it comes back only as a transcript the model must re-read in full. MemT-1.0 takes a different route — memory lives inside the model as a compact latent state, so a fact you committed last week can be recalled without the original words. Here is how to run it yourself, and how to prove the memory is real.

• Who: MemTensor, the team behind the MemT-1.0 memory foundation models.
• What: MemT-1.0, a model family whose Native Memory Block keeps a dynamic local memory and a trainable hyper memory alongside backbone attention.
• When: the repository was created on 19 October 2026 and had 416 GitHub stars at the time of writing.
• Nano: 36B total parameters, 3B activated, 451M trainable memory parameters, 10M dynamic memory state — served on 2 GPUs.
• Flash: 288B total parameters, 17B activated, 3.61B trainable memory parameters — served on 8 GPUs.
• Storage: the persistent memory state stays constant in size, about 66× smaller than a full-context KV cache at 128K history length.
• Serving: SGLang 0.5.16 on Linux with NVIDIA CUDA and Python 3.11 or newer; the Nano server listens on port 30000 and stores snapshots in ./memt-state.
• Licence: repository software under the PolyForm Noncommercial License 1.0.0 — commercial use is not permitted by default.

Why this matters

Today's chatbots have two bad options for remembering you. Either they keep the whole transcript and pay for it on every reply, or they summarise it into text and lose the detail. MemT-1.0 is a research attempt at a third option: memory as part of the weights' working state. A Local Memory Block maintains a dynamic memory matrix that evolves across turns, and a Hyper Memory Block adds trainable query, key and value projections plus erase, write and forget gates. During generation, memory attention reads that state and adds it to the backbone attention output. At the end of a turn, the state updates — and the plaintext can go away.

The honest caveat comes from the project itself: across 11 memory benchmarks, the Flash and Nano models rank first and second among the evaluated parametric-memory methods, but strong textual-memory systems remain ahead overall, especially on long histories that need detailed evidence retrieval. This is a promising direction, not a solved problem. That is exactly why it is worth running: you can measure the difference yourself instead of taking anyone's word for it.

What you'll need

• A Linux server with NVIDIA CUDA and Python 3.11 or newer — this is a server deployment, not a laptop demo
• Two GPUs for MemT-1.0-Nano (eight for the larger Flash model)
• Enough disk for the Nano checkpoint: a 36B-parameter model in BF16, downloaded from Hugging Face
• uv for the installs below, as the project's own instructions use it
• About an hour, most of it download and first-load time

1. Check your hardware

Nano's backbone is Qwen3.6-35B-A3B: 36B parameters in total, 3B activated per token. The project's example deployment runs it with tensor parallelism 2 across two GPUs, and the launcher defaults to port 30000. Flash, built on DeepSeek-V4-Flash at 288B total parameters, needs eight GPUs and is a different afternoon. Start with Nano.

nvidia-smi
python3 --version  # 3.11 or newer

2. Install the server and client

Clone the repository and install from its inference directory. The server extra pins PyTorch 2.11.0, SGLang 0.5.16, Transformers 5.12.1 and the matching kernels; the client extra skips PyTorch, so install it in a second environment if your client runs on a different machine.

git clone https://github.com/MemTensor/MemT-1.0.git
cd MemT-1.0/inference
uv pip install '.[server]'

For a client-only machine, from the same directory:

uv pip install '.[client]'

3. Download the Nano checkpoint

Weights and tokenizer assets live on Hugging Face as MemTensor/MemT-1.0-Nano. Download the complete checkpoint — matching BF16 safetensors, configuration and tokenizer — into a local directory. The rest of this guide calls that directory /models/MemT-1.0-Nano; substitute your real path everywhere it appears.

huggingface-cli download MemTensor/MemT-1.0-Nano --local-dir /models/MemT-1.0-Nano

4. Start the SGLang server

The repository ships a launcher script per model. Nano's sets two GPUs, tensor parallelism 2, and a static memory fraction of 0.60, and stores memory snapshots in ./memt-state:

sh start_memt_nano.sh /models/MemT-1.0-Nano

Wait for the server to report ready on http://127.0.0.1:30000. First load takes a while: 36B parameters of BF16 weights plus the memory machinery. Keep this terminal running and open a second one for the client. The project is upfront that these scripts are one runnable starting configuration, not deeply optimised — adjust parallelism and token budgets for your hardware once it works.

5. Commit a fact to memory

Now the part that makes this more than another model server. Connect with the memt_sglang client, using the same local checkpoint directory for the tokenizer, and tell the model something specific:

import os
from transformers import AutoTokenizer
from memt_sglang.client import MemTSession, NativeClient, load_chat_format

model_path = "/models/MemT-1.0-Nano"
tokenizer = AutoTokenizer.from_pretrained(
    model_path,
    local_files_only=True,
    trust_remote_code=False,
)
chat_format, parser = load_chat_format(
    model_path,
    tokenizer=tokenizer,
    chat_template_kwargs={"enable_thinking": False},
)
client = NativeClient(
    "http://127.0.0.1:30000",
    api_key=os.environ.get("MEMT_API_KEY"),
)

Then run a session where the second turn is saved to memory and its plaintext cleared:

session = MemTSession(client, chat_format, parser)

answer = session.chat(
    "My team is building a delivery-routing app called Atlas, "
    "and we ship every Friday."
)
print(answer)

# Save this turn to latent local memory and clear its plaintext context.
answer = session.chat(
    "What day does the Atlas team ship?",
    enable_memory=True,
)
print(answer)

The key move is enable_memory=True: after the reply completes, the turn is committed to latent local memory and its plaintext conversation context is cleared. Memory reading is on by default, so committed memory is used automatically on later turns. Ask the question again in a new turn — without re-stating the fact — and the answer should come from memory, not from a transcript the model is re-reading, because there is no transcript left to re-read.

6. Save, restore, and verify

A memory you cannot carry across sessions is a party trick. Save it, close the session, and restore it into a fresh one:

# Save local memory, including any unsaved conversation turns.
snapshot = session.save_state("atlas_memory.safetensors")
session.close()

# Restore memory without the previous conversation transcript.
with MemTSession(client, chat_format, parser, state=snapshot["path"]) as restored:
    print(len(restored.messages))
    # 0 — the transcript is empty, by design

    answer = restored.chat("What is the Atlas team building, and when do they ship?")
    print(answer)

That 0 is the whole test. If the restored session answers correctly with zero messages in its transcript, the knowledge came from the saved memory state — a file in the server's ./memt-state directory — and not from context you forgot to clear. If it cannot answer, check that the turn was actually committed with enable_memory=True before the save, and that you passed the snapshot's path back as state. A standard chat session, by contrast, is just set_memory_read(False) with enable_memory=False: previous turns stay in the transcript, and nothing is remembered once it scrolls away.

What you built

A chatbot whose memory is a file, not a scrollback buffer: a SGLang server running MemT-1.0-Nano, a client that commits turns to latent memory, and a snapshot you can restore into a session with zero transcript messages. The same pattern extends to the project's other headline use — experience-guided coding agents, where procedural experience from past tasks is retrieved from native memory during a new task without pasting that experience into the context.

Honest limitations

• It is research software on serious hardware. Two GPUs for Nano, eight for Flash, and launcher scripts the authors say have not been deeply optimised. Do not expect a one-click install.
• Parametric memory is not always better. The project's own evaluation says textual-memory systems still win overall, particularly where long histories need detailed evidence retrieval. Test it on your workload before trusting it.
• The licence blocks commercial use by default. Repository software is under the PolyForm Noncommercial License 1.0.0; the paper is CC BY-NC-SA 4.0. Anything commercial needs a conversation with the authors first.
• Illustrative outputs. The example answers in this guide follow the project's documented API and the shape of its own examples; your model's exact wording will differ run to run.