Build a tool-calling agent that runs entirely on your laptop: LangGraph and a local Qwen model, no API key

Every hosted agent you have used is a loop you cannot see: your question goes to someone else's server, tokens bill by the thousand, and the trace of what the agent did lives behind their dashboard. LangGraph turns that loop inside out. An agent becomes a graph — nodes that do things, edges that decide what runs next — and once the loop is a graph, you can read it, log it, test it, and run it anywhere. This guide builds one around a model that lives on your own machine, and proves every step on CPU, with no API key and no inference-time cloud call.

• Who: LangGraph (langchain-ai/langgraph on GitHub) + Qwen2.5-0.5B-Instruct (Qwen on Hugging Face).
• What: a tool-calling agent as an explicit state graph: an agent node, a tools node, and a router, running against a model on your own CPU.
• When: versions verified in this guide are listed in Step 1; the model is Qwen2.5-0.5B-Instruct (0.49B parameters, 32,768-token context).
• Requirements: Python 3.12, pip, about 2 GB of disk for the CPU PyTorch install plus the model. No API key, no account, no GPU.
• Licence: LangGraph is MIT; Qwen2.5-0.5B-Instruct is Apache-2.0.
• Verified: every command and program in this guide was run on a 2-core Linux VM; the agent answers a single-tool question end to end, repairs its own malformed JSON, and carries state across turns — while the multi-tool and recall failures are documented, not hidden.

Why this matters now

Agent frameworks have converged on the same shape: a language model that decides, tools that act, and a loop that connects them. The differences that matter in practice are not which model you pick first — they are whether you can see the loop, bound it, and replay it. A hosted agent API gives you a transcript after the fact. A graph gives you the loop itself, as code you own: every node is a function you can unit-test, every edge is a decision you can log, and the whole run fits in a state object you can inspect at any step.

That matters more as models get smaller and cheaper. A 0.5-billion-parameter model will not out-reason a frontier model, and this guide does not pretend otherwise. What it does — follow a strict output format, call the right tool, combine a result into an answer — is exactly the mechanical core of an agent, and it is the part most tutorials skip past by renting a big model. Learn the loop on a model you own, and swapping in a bigger model later is a one-line change, not a rewrite.

The other reason is privacy and cost. A local agent sends nothing anywhere: your notes, your questions, and your tool results never leave the machine. For personal automation — a notes assistant, a file organizer, a calculator that remembers context — that is often the whole requirement, and the per-token bill is zero.

What you'll need

Python 3.12 and pip. About 2 GB of free disk: roughly 1 GB for the CPU build of PyTorch and the LangGraph stack, plus about 1 GB for the model on its first download. No GPU, no API key, no account. The commands below were run on a 2-core Linux virtual machine under heavy load from other jobs, so the timings are pessimistic — a modern laptop will be faster. Everything runs on CPU.

Versions verified in this guide, exactly as installed:

• torch 2.9.1+cpu (from the PyTorch CPU index)
• transformers 5.21.0
• langgraph 1.2.18
• langchain-core 1.7.0
• Qwen/Qwen2.5-0.5B-Instruct — 0.49 billion parameters, 24 layers, 32,768-token context, Apache-2.0

One honest warning up front: the model in this guide is small on purpose. It follows instructions well for short, structured tasks and it fumbles longer ones. That is a feature for learning — you will see the failure modes (malformed JSON, wrong tool arguments, dropped sub-tasks) and the graph mechanisms that contain them — but do not ship this exact model for work that matters. Step 7 shows how to graduate.

Step 1: Install the stack

Create a virtual environment, install the CPU build of PyTorch first (the default PyPI wheel bundles CUDA libraries you do not need and do not want on a laptop), then the rest:

python3 -m venv .venv
source .venv/bin/activate
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install transformers==5.21.0 langgraph==1.2.18 langchain-core==1.7.0

That is the whole dependency list. Two things are deliberately absent. accelerate is not needed: transformers 5.x only requires it when you pass a device_map, and the code in this guide does not — CPU is the default when no GPU is present. langchain-huggingface is not needed either: its chat model wraps a transformers pipeline but does not parse tool calls from pipeline output (tool-call parsing there only exists for structured chat APIs such as text-generation-inference servers). Since this guide parses the model's JSON itself, that package would add nothing but weight. Both facts were verified by installing, then removing, those packages and confirming the agent still runs.

Step 2: Load a local model and prove it answers

Before any graph, prove the model itself works. The first run downloads about 1 GB from the Hugging Face Hub; later runs reuse the cache. Load it with plain from_pretrained — no device_map, for the reason given in Step 1:

from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL = "Qwen/Qwen2.5-0.5B-Instruct"
tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(MODEL)

On the verification machine the weights loaded in about 3 seconds and the model reported 0.494 billion parameters, matching the model card. A plain chat completion, using the model's own chat template, came back coherent — and slow, because the machine was heavily loaded at the time:

User: Give me a short introduction to large language model.
Assistant: Certainly! A large language model (LLM) is a type of artificial
intelligence that can generate human-like text based on the input it
receives. These models are designed to be highly accurate and capable of
understanding complex natural language tasks such as translation,
summarization, question answering, and more.

That completion took 208.3 seconds for 120 tokens under a load average above 11 on two cores — quoted exactly because it is the honest number for that machine, not a promise for yours. The short generations the agent actually uses are far quicker: the JSON probe below took 25.1 seconds for 20 tokens on the same loaded machine. If your machine is idle, expect several times faster; if you have a GPU, an order of magnitude faster.

Now the probe that decides the whole design. Give the model a system prompt that allows exactly one JSON action per turn, and ask it something that needs a tool:

SYSTEM = """You are a careful assistant with two tools. Reply with EXACTLY ONE JSON object and nothing else.

Tools:
- calculator: {"expression": "<arithmetic, e.g. 96*45>"} — evaluates arithmetic.
- lookup_note: {"topic": "<topic>"} — looks up a saved note.

To call a tool: {"action": "calculator", "args": {"expression": "96*45"}}
To finish: {"action": "final", "answer": "<your answer, using the tool results>"}
Never call a tool and give a final answer in the same reply. One JSON object per turn."""

With greedy decoding (do_sample=False, so the output is deterministic), the model replied with exactly:

{"action": "calculator", "args": {"expression": "96*45"}}

That is the contract the rest of the guide is built on: a small local model, asked for one JSON object, produces a parseable tool call. Everything after this step is plumbing to run that contract in a loop.

Step 3: Define two real tools

Tools are ordinary Python functions with docstrings — the docstrings matter, because they are what the model reads about each tool. The calculator evaluates arithmetic through the ast module, never through eval on raw input: only numbers and the operators + - * / % ** are allowed, so a malicious expression cannot execute code. The notes lookup reads from a plain dictionary standing in for whatever store you already have:

"""A tool-calling agent that runs 100% locally: LangGraph + Qwen2.5-0.5B-Instruct.

No API key, no cloud call. The graph is the agent: the model proposes one
JSON action per turn, the router sends tool calls to real Python functions,
and the loop continues until the model returns a final answer.

Run:  HF_HUB_OFFLINE=1 python agent.py
"""
import ast
import json
import operator as op

from langchain_core.messages import AIMessage, HumanMessage
from langchain_core.tools import tool
from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import END, START, StateGraph
from langgraph.graph.message import MessagesState
from langgraph.prebuilt import ToolNode
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL = "Qwen/Qwen2.5-0.5B-Instruct"

# ---------------------------------------------------------------- tools
ALLOWED = {ast.Add: op.add, ast.Sub: op.sub, ast.Mult: op.mul,
           ast.Div: op.truediv, ast.Mod: op.mod, ast.Pow: op.pow,
           ast.USub: op.neg}


def _eval(node):
    if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)):
        return node.value
    if isinstance(node, ast.BinOp) and type(node.op) in ALLOWED:
        return ALLOWED[type(node.op)](_eval(node.left), _eval(node.right))
    if isinstance(node, ast.UnaryOp) and type(node.op) in ALLOWED:
        return ALLOWED[type(node.op)](_eval(node.operand))
    raise ValueError("unsupported expression")


@tool
def calculator(expression: str) -> str:
    """Evaluate a simple arithmetic expression, e.g. '96*45'. Returns the result as text."""
    return str(_eval(ast.parse(expression, mode="eval").body))


NOTES = {
    "langgraph checkpointer": "LangGraph checkpointers save the graph state after each step. "
    "MemorySaver keeps it in RAM for the session; SqliteSaver persists it to disk.",
    "qwen2.5 context": "Qwen2.5-0.5B-Instruct supports a 32,768-token context window.",
    "langgraph router": "A router is a plain Python function that reads the latest message "
    "and returns the name of the next node to run.",
}


@tool
def lookup_note(topic: str) -> str:
    """Look up a saved note by topic. Known topics: langgraph checkpointer, qwen2.5 context, langgraph router."""
    key = topic.strip().lower()
    if key in NOTES:
        return NOTES[key]
    return f"No note saved for '{topic}'. Known topics: {', '.join(sorted(NOTES))}."


TOOLS = [calculator, lookup_note]

SYSTEM = """You are a careful assistant with two tools. Reply with EXACTLY ONE JSON object and nothing else.

Tools:
- calculator: {"expression": "<arithmetic, e.g. 96*45>"} — evaluates arithmetic.
- lookup_note: {"topic": "<topic>"} — looks up a saved note.

To call a tool: {"action": "calculator", "args": {"expression": "96*45"}}
To finish: {"action": "final", "answer": "<your answer, using the tool results>"}
Never call a tool and give a final answer in the same reply. One JSON object per turn."""

Two details worth noticing. First, the @tool decorator turns each function into a LangChain tool object with a name, a description, and an argument schema — that is what the graph's tools node executes later. Second, the notes are private data: the model cannot know them without calling lookup_note. That turns out to be the key property for reliable demos, as Step 6 shows. A tool the model can route around — arithmetic it can do in its head — will sometimes be skipped, and that is rational behavior, not a bug.

Step 4: Build the graph

Here is the whole agent. Read it top to bottom before running it; every part is doing one job:

# ---------------------------------------------------------------- model
tokenizer = AutoTokenizer.from_pretrained(MODEL)
model = AutoModelForCausalLM.from_pretrained(MODEL)


def _render(state) -> list:
    """Convert graph messages to ChatML dicts the Qwen template renders natively.

    Tool calls become assistant turns holding the JSON the model emitted, and
    tool results become user turns reporting the result — so the model always
    sees a plain ChatML conversation and always replies with one JSON action.
    """
    msgs = [{"role": "system", "content": SYSTEM}]
    for m in state["messages"]:
        if isinstance(m, HumanMessage):
            msgs.append({"role": "user", "content": m.content})
        elif isinstance(m, AIMessage) and m.tool_calls:
            for tc in m.tool_calls:
                msgs.append({"role": "assistant", "content": json.dumps(
                    {"action": tc["name"], "args": tc["args"]})})
        elif isinstance(m, AIMessage):
            msgs.append({"role": "assistant", "content": m.content})
        else:  # ToolMessage
            msgs.append({"role": "user",
                         "content": f"Tool {m.name} returned: {m.content}"})
    return msgs


def agent_node(state: MessagesState):
    prompt = tokenizer.apply_chat_template(
        _render(state), tokenize=False, add_generation_prompt=True)
    inputs = tokenizer([prompt], return_tensors="pt")
    out = model.generate(**inputs, max_new_tokens=150, do_sample=False,
                          pad_token_id=tokenizer.eos_token_id)
    raw = tokenizer.decode(
        out[0][len(inputs.input_ids[0]):], skip_special_tokens=True
    ).strip()
    try:
        action = json.loads(raw[raw.index("{"):raw.rindex("}") + 1])
    except Exception:
        return {"messages": [HumanMessage(
            content="Your previous reply was not valid JSON. "
            "Reply with exactly one JSON object, nothing else.")]}
    if action.get("action") == "final":
        return {"messages": [AIMessage(content=str(action.get("answer", "")))]}
    name, args = action.get("action"), action.get("args", {})
    if name in ("calculator", "lookup_note") and isinstance(args, dict):
        return {"messages": [AIMessage(content="", tool_calls=[
            {"name": name, "args": args, "id": f"call_{name}",
             "type": "tool_call"}])]}
    return {"messages": [HumanMessage(
        content=f"Unknown action '{name}'. Use calculator, lookup_note, or final.")]}


def route(state: MessagesState):
    last = state["messages"][-1]
    if isinstance(last, AIMessage) and last.tool_calls:
        return "tools"
    if isinstance(last, HumanMessage):
        return "agent"  # repair nudge -> try again
    return END


builder = StateGraph(MessagesState)
builder.add_node("agent", agent_node)
builder.add_node("tools", ToolNode(TOOLS))
builder.add_edge(START, "agent")
builder.add_conditional_edges("agent", route,
                              {"tools": "tools", "agent": "agent", END: END})
builder.add_edge("tools", "agent")
graph = builder.compile(checkpointer=MemorySaver())

The pieces, in order. MessagesState is the graph's state: a list of messages, with new messages appended rather than replacing old ones. The agent node renders the conversation through the model's own chat template (so role markers match what the model was trained on), generates one reply with greedy decoding, and parses it. A valid tool action becomes an AIMessage carrying a tool_call; a valid final action becomes a plain answer; anything else becomes a correction message that sends the model around again. The tools node is LangGraph's prebuilt ToolNode: it executes each tool call and appends the result as a ToolMessage.

The router is custom, deliberately. LangGraph ships tools_condition, which sends the run to the tools node when the last message has tool calls and ends it otherwise — but a correction message is a HumanMessage, and tools_condition would end the run on it instead of retrying. The custom router has three exits: tool calls go to tools, corrections go back to agent, everything else ends. The recursion_limit of 12 in the demo bounds the loop: a model that never produces a valid action stops after 12 steps instead of forever. Compile with MemorySaver and every thread's state is checkpointed after each step — that is the memory in Step 6.

Step 5: Run a single-tool question — and read the trace

Run the demo. It asks three questions in one thread; this step covers the first:

if __name__ == "__main__":
    import sys

    config = {"configurable": {"thread_id": "demo-1"},
              "recursion_limit": 12}
    for question in sys.argv[1:] or [
            "What does my note say about the langgraph checkpointer?",
            "What is 173 times 284, and what does my note say about the qwen2.5 context?",
            "What did I ask you first in this conversation?"]:
        print(f"\n=== User: {question}")
        result = graph.invoke({"messages": [HumanMessage(content=question)]},
                              config=config)
        print("=== Agent:", result["messages"][-1].content)

Expected output for the first question — quoted exactly from the verification run, including the stumble:

===== User: What does my note say about the langgraph checkpointer?
[HumanMessage] What does my note say about the langgraph checkpointer?
[HumanMessage] Unknown action 'None'. Use calculator, lookup_note, or final.
[AIMessage]  tool_calls=[{'name': 'lookup_note', 'args': {'topic': 'langgraph checkpointer'}, 'id': 'call_lookup_note', 'type': 'tool_call'}]
[ToolMessage] LangGraph checkpointers save the graph state after each step. MemorySaver keeps it in RAM for the session; SqliteSaver persists it to disk.
[AIMessage] LangGraph checkpointers save the graph state after each step. MemorySaver keeps it in RAM for the session; SqliteSaver persists it to disk.
FINAL: LangGraph checkpointers save the graph state after each step. MemorySaver keeps it in RAM for the session; SqliteSaver persists it to disk.

Read that trace the way you would read a log. The model's first attempt was malformed — it produced an action the parser could not use, so the node answered with a correction instead of crashing or guessing. The model retried and produced a proper tool call with the exact argument langgraph checkpointer. The tool ran, its result entered the conversation as a ToolMessage, and the final answer quotes the note exactly, without paraphrase drift. That repair loop is not theoretical decoration: it fired three times across the full three-question demo, and without it the first question would have ended in an error.

One design choice to notice: the malformed reply itself is not stored — the node keeps only the correction. That keeps the transcript clean, at the cost of losing the exact bad output for debugging. If you want the bad output logged, append it to a file inside the node before returning the correction; the graph does not care.

Step 6: Ask for two things at once — and test memory

The second question asks for two things: hard arithmetic and a second private note. The traces below omit the messages from the first question, which repeat at the start of every later turn — that repetition is itself the checkpointer working.

===== User: What is 173 times 284, and what does my note say about the qwen2.5 context?
[AIMessage]  tool_calls=[{'name': 'calculator', 'args': {'expression': '173*284'}, 'id': 'call_calculator', 'type': 'tool_call'}]
[ToolMessage] 49132
[HumanMessage] Your previous reply was not valid JSON. Reply with exactly one JSON object, nothing else.
[AIMessage] The result of 173 times 284 is 49132.
FINAL: The result of 173 times 284 is 49132.

The calculator half is correct (173 × 284 = 49,132 — check it: 173 × 300 − 173 × 16 = 51,900 − 2,768). The note half never happened: after the tool result, the model emitted invalid JSON, took the correction, and then finalized with only the arithmetic, dropping the second sub-task entirely. The same pattern appeared in an earlier run, where a two-part question got only the note half and the arithmetic was dropped, with the note's "SqliteSaver" corrupted into "sqlitetimer" along the way. The lesson is consistent and worth stating plainly: this 0.5-billion-parameter model reliably performs one tool call per question and drops the second sub-task. The graph loops correctly; the model does not plan multi-tool sequences. If you need two tools used, ask two questions — or use a bigger model, per Step 7.

Now memory. The third question, in the same thread, asks what was asked first:

===== User: What did I ask you first in this conversation?
[AIMessage] I'm sorry, but I need more information to provide an accurate response. What question did you have?
FINAL: I'm sorry, but I need more information to provide an accurate response. What question did you have?

That is a failure, quoted exactly — and it teaches the most important distinction in this guide. The checkpointer did its job: the full transcript, both earlier questions with their tool calls and results, was present in the state the model was given (you can see it in the trace above, where every earlier message reappears). Persistence worked. Recall did not: the small model could not pick the first question out of that transcript. In LangGraph, "memory" means persisted state, and it is mechanical and reliable. Using that state well is the model's job, and it depends on the model. Do not confuse the two when you evaluate an agent: check the state object separately from the answer.

Step 7: Swap in a bigger model when you are ready

Nothing in the graph depends on which model produces the JSON. To graduate, change one line:

MODEL = "Qwen/Qwen2.5-7B-Instruct"  # 7B parameters at 2 bytes each is ~14 GB of weights alone: use a GPU machine

The tools, the node, the router, and the checkpointer all stay exactly as they are; only the brain gets bigger, and with it the reliability of multi-tool plans and recall. The same applies in the other direction: the graph also accepts a hosted model behind the same JSON protocol, but that path needs an API key and was deliberately not run in this guide, which is keyless end to end. When you do swap, re-run the three demo questions before anything else: they are a regression test for the whole loop, and any model that passes all three — tool call, repair, multi-part, recall — has earned real work.

Which approach should you use?

Three ways to build this agent, and when each earns its place:

• Prebuilt create_react_agent — fastest when your model has native tool calling: hosted APIs and text-generation-inference servers parse tool calls for you. Use it there. Do not use it with a local transformers pipeline: the pipeline returns text, and there is nothing to parse the tool call out of it.
• Hand-built StateGraph, as in this guide — the right call for local models and for learning. The loop is yours: you choose the output format, the repair behavior, the routing, and the memory. More code, total control, and the graph itself teaches how agents work.
• Hand-built graph + bigger model — the production shape. Keep the graph, upgrade the model until multi-tool plans and recall pass your own demo questions. The graph is the stable part; the model is the interchangeable part.

What this guide did not verify

• The hosted-model swap in Step 7 was not run (it needs an API key). The claim is structural — the graph does not depend on the model — not empirical.
• Quality at scale: three demo questions on one small model do not characterize it. The failure modes shown (dropped sub-tasks, paraphrase corruption, failed recall) are real observations, not an exhaustive list.
• Timings are load-bound: the verification machine was heavily loaded by other jobs, so treat every timing here as pessimistic, not typical.
• Streaming, async invocation, and persistent (disk-backed) checkpointers were not exercised; MemorySaver keeps state in RAM for the session only.

The takeaway

An agent is not a model with a prompt. It is a loop: propose an action, run it, look at the result, repeat — bounded, repairable, and inspectable. LangGraph makes that loop explicit, and once it is explicit you can test it node by node, on a laptop, for free, with a model you own. The small model in this guide will drop your second sub-task and fumble recall; that is useful information, not a reason to rent a bigger model before you understand the loop. Build the graph first, prove it with questions you can check, and let the model graduate when the loop has earned it.

Sources

• Qwen2.5-0.5B-Instruct model card (Qwen on Hugging Face) — parameter count, context length, architecture, and the chat-template quickstart used in Step 2.
• LangGraph prebuilt reference for create_react_agent (LangChain) — the agent/tools loop the hand-built graph mirrors, and the note that MessageGraph is deprecated in favor of StateGraph.
• LangChain forum discussion of ChatHuggingFace tool-call parsing — tool-call parsing exists for structured chat APIs, not for raw transformers pipelines, which is why this guide parses JSON in its own node.
• The installed packages themselves, run in this guide: torch 2.9.1+cpu, transformers 5.21.0, langgraph 1.2.18, langchain-core 1.7.0 — every version, flag, and output quoted here came from running them, not from documentation alone.