Stop parsing LLM JSON by hand: typed structured extraction with Instructor and local Ollama

Every pipeline that turns language-model output into something a program can use starts the same way: ask the model for JSON, call json.loads, and hope. Most of the time it works. Then one day the model returns "confidence": "high" instead of a number, or "priority": "ASAP" instead of one of your four allowed values, and your code crashes at 2 a.m. — or worse, it doesn't crash, and the bad value flows downstream into a dashboard, a database, or a customer-facing reply.

The usual fix is to ask for JSON more firmly: Ollama's JSON mode, a system prompt that says "reply with JSON only", a regex to strip the markdown fences. All of that helps with syntax. None of it checks meaning. JSON mode guarantees the output parses; it says nothing about whether confidence is a number between 0 and 1, whether priority is one of the values your code handles, or whether the order ID is digits instead of #48210.

Instructor closes that gap. You describe the shape you want as a Pydantic model — types, allowed values, ranges, patterns — and Instructor makes the model produce exactly that, re-asking with the validation error when the first attempt is wrong. This tutorial runs the whole loop end to end on your own machine: a local Ollama model, no API key, no GPU, no cloud bill. By the end you will have a four-ticket support inbox turned into typed, validated records written to a file — and you will know exactly where the small model you ran cuts corners, because we measured it.

What you’ll need#

• Ollama 0.40.11 (or newer) — the local model runtime. Download it for your OS; on Linux the install script puts the binary and its runner in place.
• Python 3.12 and pip.
• Instructor 1.17.0, Pydantic 2.14.3, and the OpenAI Python client 3.3.0 — installed below.
• The qwen3:0.6b model (522 MB) — small enough to run on any laptop CPU, which is the point: if the workflow holds up on a 0.6-billion-parameter model, it holds up anywhere.
• No API key, no GPU, no account. Everything here runs on localhost. Expect roughly 20–35 seconds per ticket on a CPU-only machine; the timings in this tutorial were measured on a 1.5-core VM with 6 GB of RAM, so a real laptop will be faster.

Step 1: Install Ollama and pull the model#

Install Ollama from ollama.com/download, then pull the model and start the server:

ollama pull qwen3:0.6b
ollama serve
ollama list

You should see the model listed, about 522 MB:

NAME            ID              SIZE      MODIFIED
qwen3:0.6b      7df6b6e09427    522 MB    6 weeks ago

Confirm the server answers on its local API:

curl -s http://localhost:11434/api/tags | python3 -c "import json,sys; print([m['name'] for m in json.load(sys.stdin)['models']])"
# ['nomic-embed-text:latest', 'qwen3:0.6b']

The nomic-embed-text entry is an embeddings model left over from an earlier experiment; ignore it. What matters is that qwen3:0.6b is present and the server is listening on port 11434. If ollama list shows nothing, the pull did not finish — re-run it; the model is only half a gigabyte, so it downloads in a minute or two on a normal connection.

Now install the Python side. Instructor is a thin layer over the OpenAI client, and it uses Pydantic for the schemas:

python3 -m pip install "instructor==1.17.0"
python3 -c "import instructor, pydantic, openai; print('instructor', instructor.__version__); print('pydantic', pydantic.VERSION); print('openai', openai.__version__)"
# instructor 1.17.0
# pydantic 2.14.3
# openai 3.3.0

Step 2: See the gap — raw JSON mode#

Before adding Instructor, look at what plain JSON mode actually gives you. This script sends one support ticket to the local model and asks for a JSON object with four keys:

import json
from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

TICKET = (
    "Subject: Refund for order #48210\n"
    "Hi, I was charged twice for order 48210 ($49.99 each). "
    "Please refund the duplicate charge. This is urgent, I need it before Friday. - Dana"
)

resp = client.chat.completions.create(
    model="qwen3:0.6b",
    messages=[
        {
            "role": "system",
            "content": "Reply with a JSON object only, with keys: title, priority, confidence, summary.",
        },
        {"role": "user", "content": TICKET},
    ],
    response_format={"type": "json_object"},
    temperature=0,
)
print(resp.choices[0].message.content)

The output parses as JSON. That is the whole of what JSON mode promises — and it is not enough:

{
  "title": "Refund for order #48210",
  "priority": "high",
  "confidence": "high",
  "summary": "Refund duplicate charges for order #48210 ($49.99 each). Urgent request before Friday."
}

confidence came back as the string "high", not a number between 0 and 1. json.loads accepts it happily. The first line of code that does arithmetic on it — an average, a threshold, a sort — fails, or silently does the wrong thing. This is the gap: syntax is valid, semantics are not.

Now define what the record is supposed to look like, and watch Pydantic reject the kind of payload JSON mode can hand you:

from pydantic import BaseModel, Field, ValidationError
from typing import Literal

class Ticket(BaseModel):
    title: str = Field(..., max_length=80)
    priority: Literal["low", "medium", "high", "urgent"]
    confidence: float = Field(..., ge=0.0, le=1.0)
    summary: str

bad_payload = {
    "title": "Refund for order #48210",
    "priority": "ASAP",          # not in the enum
    "confidence": 1.4,          # outside 0..1
    "summary": "Customer charged twice, wants a duplicate refund.",
}
try:
    Ticket(**bad_payload)
except ValidationError as e:
    for err in e.errors():
        print(f"  {err['loc'][0]}: {err['msg']}")
#   priority: Input should be 'low', 'medium', 'high' or 'urgent'
#   confidence: Input should be less than or equal to 1

Two precise errors, each naming the field and the rule it broke. That is the mechanism Instructor builds on: the model produces JSON, Pydantic validates it, and anything that fails gets sent back to the model with the error attached. Your code never sees an invalid record.

Step 3: Your first typed extraction#

Same ticket, same model — but now the schema does the asking. Instructor's from_provider is the shortest path for a local model: it builds an OpenAI-compatible client pointed at Ollama and picks the right mode for the model. For qwen3:0.6b it picks JSON mode; for function-calling models such as qwen2.5 it picks tool-calling mode automatically. You can force either with mode=instructor.Mode.JSON or mode=instructor.Mode.TOOLS.

import instructor
from pydantic import BaseModel, Field
from typing import Literal

class Ticket(BaseModel):
    """One support ticket, typed end to end."""

    title: str = Field(..., max_length=80, description="Short ticket title")
    priority: Literal["low", "medium", "high", "urgent"]
    confidence: float = Field(..., ge=0.0, le=1.0, description="0..1, how sure the model is")
    summary: str = Field(..., description="One-sentence summary of the request")

client = instructor.from_provider("ollama/qwen3:0.6b", mode=instructor.Mode.JSON)

ticket = client.create(
    model="qwen3:0.6b",
    messages=[
        {
            "role": "user",
            "content": "Extract the support ticket into the Ticket schema.\n\n" + TICKET,
        }
    ],
    response_model=Ticket,
    max_retries=3,
    temperature=0,
)
print(ticket.model_dump_json(indent=2))

Run it. The result is a real Ticket object, not a dict — this is the verified output from our run, in 69.7 seconds on the CPU-only VM (the first call includes loading the model into memory; later calls in the batch below ran 19–34 seconds each):

{
  "title": "Refund for order #48210",
  "priority": "urgent",
  "confidence": 1.0,
  "summary": "Refund for order #48210, duplicate charges detected, urgent request for refund before Friday."
}

Note what changed versus the raw attempt: confidence is now the float 1.0, priority is a valid enum value, and ticket.confidence is a float you can compare and average. The three arguments doing the work are response_model=Ticket (the schema), max_retries=3 (how many times Instructor may re-ask after a validation failure), and temperature=0 (deterministic output, which matters when you want the same ticket to extract the same way twice).

Step 4: Batch it — four tickets to a file#

One ticket proves the mechanism. A batch proves the workflow. The script below loops over four tickets — a refund, a damaged parcel, a lockout, and a shipping question — and appends each validated record to tickets.jsonl:

import json
import instructor
from openai import OpenAI
from pydantic import BaseModel, Field
from typing import Literal

client = instructor.from_openai(
    OpenAI(base_url="http://localhost:11434/v1", api_key="ollama"),
    mode=instructor.Mode.JSON,
)

class Ticket(BaseModel):
    title: str = Field(..., max_length=80)
    priority: Literal["low", "medium", "high", "urgent"]
    confidence: float = Field(..., ge=0.0, le=1.0)
    summary: str

TICKETS = [
    ("t1", "Subject: Refund for order #48210\nHi, I was charged twice for order 48210 "
           "($49.99 each). Please refund the duplicate charge. This is urgent, "
           "I need it before Friday. - Dana"),
    ("t2", "Subject: Damaged delivery\nHello, the vase in order #99173 arrived broken — "
           "the box was crushed on one side. Can you send a replacement? No rush, "
           "next week is fine. - Marc"),
    ("t3", "Subject: Can't log in\nI requested a password reset three times and the email "
           "never arrives, not even in spam. I need to access my account today to "
           "download an invoice. - Priya"),
    ("t4", "Subject: Shipping question\nHi! Do you ship to Canada, and if so how long does "
           "delivery usually take? Thanks! - Alex"),
]

with open("tickets.jsonl", "w") as f:
    for tid, text in TICKETS:
        ticket = client.create(
            model="qwen3:0.6b",
            messages=[{"role": "user", "content": "Extract the support ticket into the Ticket schema.\n\n" + text}],
            response_model=Ticket,
            max_retries=3,
            temperature=0,
        )
        f.write(json.dumps({"id": tid, **ticket.model_dump()}) + "\n")

Measured on the same CPU-only VM: all four tickets extracted, one model call each, 114.4 seconds total, 1,165 prompt tokens and 1,250 completion tokens, cost $0. Per-ticket times were 19.5, 29.3, 34.2, and 31.4 seconds.

And here is the honest part. The records are all valid — every field type-checks — but the model's judgment is visibly that of a 0.6-billion-parameter model. Ticket t2, the damaged vase, says "No rush, next week is fine" and still came back priority: "urgent" with confidence 0.5: the model reacted to "broken" and "replacement" and ignored the explicit "no rush". Ticket t4's summary ends with a stray /think — a fragment of qwen3's reasoning trace leaking into the output. And the confidence values cluster at exactly 1.0 or 0.5, which is not a calibrated probability, it is a small model hedging in coarse steps.

None of that is a schema failure. The schema guarantees shape; it cannot guarantee judgment. That distinction is the whole point of running this on a tiny model first: you see exactly which errors validation catches (types, enums, ranges) and which it cannot (a wrong priority that is still a valid enum value). If priority accuracy matters to you, the fix is a bigger model or better field descriptions — not a stricter schema.

Step 5: Make the schema strict — and watch retries work#

Add two fields that a sloppy model gets wrong: an order ID that must be digits only, and an amount that must be a number:

class StrictTicket(BaseModel):
    title: str = Field(..., max_length=80)
    priority: Literal["low", "medium", "high", "urgent"]
    confidence: float = Field(..., ge=0.0, le=1.0)
    order_id: str = Field(..., pattern=r"^\d+$", description="digits only, no # prefix")
    amount_usd: float = Field(..., ge=0.0, description="amount in USD, number only")
    summary: str

ticket = client.create(
    model="qwen3:0.6b",
    messages=[{"role": "user", "content": "Extract the support ticket into the StrictTicket schema.\n\n" + TICKET}],
    response_model=StrictTicket,
    max_retries=2,
    temperature=0,
)
print(ticket.model_dump_json(indent=2))
# {
#   "title": "Refund for order #48210",
#   "priority": "urgent",
#   "confidence": 1.0,
#   "order_id": "48210",
#   "amount_usd": 49.99,
#   "summary": "Refund the duplicate charge for order 48210 ($49.99 each)."
# }

It succeeded on the first call, in 40.6 seconds: order_id came back as "48210" without the #, and amount_usd as the number 49.99. The pattern and the description did real work here — the description tells the model what the field wants, and the pattern is the rule that rejects anything else.

But what happens when the model does get it wrong? In our live runs it never did, so we tested the retry loop on demand instead: a tiny stub server that answers the first request with an invalid payload ("confidence": "high", the exact mistake from Step 2) and the second with a valid one. Instructor made two requests and returned the valid record — the first failure was re-prompted with the validation error attached, exactly as designed. The mechanism is deterministic; whether it triggers on a given day depends on the model, not on your code. Two practical consequences follow. First, always set max_retries explicitly — the default is low, and a retry budget of 2–3 is cheap insurance. Second, if the retries are exhausted, Instructor raises the last validation error instead of returning bad data. That is a feature: a loud failure you can log and re-queue beats a silent bad record in your database.

One more parameter worth knowing: timeout. With Ollama, Instructor's docs recommend setting it, because it bounds the total time across all retry attempts rather than per attempt — a 60-second timeout stays 60 seconds even if two retries fire. On a slow CPU model, set it generously (a few minutes) or the retries will be cut off before they can help.

Step 6: One call or many?#

Instructor can also extract a list in a single call, streaming results as they arrive:

from typing import Iterable

tickets = list(
    client.create(
        model="qwen3:0.6b",
        messages=[{"role": "user", "content": PROMPT_WITH_ALL_FOUR_TICKETS}],
        response_model=Iterable[Ticket],
        max_retries=3,
        temperature=0,
        stream=True,
    )
)

We ran it: four tickets back in 109.1 seconds, one call. Slightly faster than four separate calls (114.4 seconds), and streaming means you can start processing the first record before the last one is generated. But there is a catch, and it is visible in the output: every ticket came back priority: "urgent" with confidence 1.0 — including the shipping question and the "no rush" parcel. Asked to produce a list, the small model collapsed to a single label for all four records. The per-ticket loop in Step 4 did better: three urgent, one medium, with confidences split between 1.0 and 0.5.

The rule of thumb that falls out of the measurements: with a small local model, one call per record is the reliable shape, and the list form is for when you have a model big enough to keep four judgments distinct. With a hosted frontier model the list form is usually fine — the constraint here is the 0.6B model, not Instructor.

Which approach should you use?#

The last row is the reason to learn the pattern locally: from_provider takes the same call for OpenAI, Anthropic, Gemini, or a local Ollama model. You develop and test against the free local model, and when the extraction quality matters — a real support queue, a real customer — you swap the provider string and keep the schema, the retries, and the validation. Nothing else in the code changes.

And if you stay local, the next step up is a bigger model. Qwen2.5:3B is the documented sweet spot for this workflow — it supports tool-calling mode, so Instructor can use the structured path natively — at roughly 2 GB of download and noticeably better judgment than the 0.6B model used here. Swap qwen3:0.6b for qwen2.5:3b in the pull and the model= argument; the rest of the tutorial is unchanged.

Honest limitations#

• Small models sometimes wrap the object. On a toy schema (Character with name and age), qwen3:0.6b returned {"char": {...}} — the right data nested one level down under the class name — and Instructor's retries could not fix it, because the wrapper is valid JSON and the error message does not tell the model to unwrap it. If you hit this, simplify the prompt, rename the class, or use a bigger model.
• The schema guarantees shape, not judgment. A wrong-but-valid priority sails through validation. Measure judgment separately — a small labeled set, scored by hand — before trusting the output.
• Small-model quirks leak into fields. We saw a /think fragment in a summary and confidence values that only ever read 0.5 or 1.0. Strip reasoning traces from text fields, and never treat a small model's confidence as a probability.
• CPU speed is the bottleneck. Twenty to thirty-five seconds per ticket on a weak VM is fine for a nightly batch of a few hundred tickets; it is not a live API. Parallelize with more Ollama workers, or move to a bigger machine, if latency matters.
• Retries cost tokens and time. Every retry is another full generation. A schema the model cannot satisfy — too many pattern constraints, an enum it keeps missing — will burn your retry budget on every record. Simplify the schema first, then raise max_retries.
• JSON mode is not free-form. With response_format={"type": "json_object"} the model must output JSON, which is what you want here — but it also means the model cannot explain itself, so debugging a bad extraction means reading the validation error, not the model's reasoning.

The takeaway#

The pipeline that breaks in production is never the one that returns invalid JSON — that fails loudly and gets fixed. It is the one that returns valid JSON with wrong values inside it. Instructor moves the check to where it belongs: the schema is the contract, Pydantic enforces it, and the model re-tries until the record it returns actually satisfies it. Run it locally first, on a model that costs nothing, with tickets you can read. When the records look right, the same code runs against any provider. The checklist:

1. Write the schema first. Types, enums, ranges, patterns — the Pydantic model is the specification, not an afterthought.
2. Use from_provider and response_model. One call pattern, local or hosted.
3. Set max_retries and temperature=0. Retries turn validation failures into corrections; temperature 0 makes the same ticket extract the same way twice.
4. Measure judgment, not just validity. A record can pass every check and still be wrong. Spot-check priorities on real data.
5. One call per record on small models. List extraction is faster, but a 0.6B model collapses to one label across the list.
6. Fail loudly. Exhausted retries should raise, log, and re-queue — never silently write a bad record.

The one-line version: JSON mode makes output that parses; Instructor makes output you can trust — and on a local model, that trust costs nothing but a few dozen seconds per record.

Sources#

• Instructor Docs — Structured outputs with Ollama
• Instructor — Structured outputs for LLMs
• Instructor on GitHub (jxnl/instructor)
• Ollama — Run large language models locally
• Pydantic Documentation