One endpoint for every free model: hands-on with FreeLLMAPI, the self-hosted LLM router

Every serious AI lab now offers a free tier, but each one alone is too small to build on. FreeLLMAPI stacks 34 providers' free tiers behind one OpenAI-compatible endpoint on your own machine, with failover and quota tracking. This guide installs it from source, registers a local Ollama model, and sends a real request through the gateway — no account, no provider key.

• Who: FreeLLMAPI (tashfeenahmed/freellmapi on GitHub), MIT license, about 44,000 stars and about 4,400 stars gained in the seven days to October 29, 2026, per an independent trending tracker; checkout verified here dated October 29, 2026.
• What: a self-hosted LLM router: one OpenAI-compatible endpoint on your own machine that routes across providers' free tiers, with failover, quota tracking, encrypted local key storage, and a dashboard.
• When: verified October 29, 2026, on Node.js 24.22.1, against the cloned repository (commit 36826e7) plus Ollama 0.40.13 and qwen3:0.6b (522 MB) on CPU.
• Requirement: Git and Node.js 20+ for everything verified here. The cloud free tiers need free provider signups (no credit card); no GPU, no paid key for the verified steps.

Why one endpoint for free models, and why now

Every serious AI lab now offers a free tier: a few million tokens a month here, a few thousand requests a day there. On its own, each tier is a toy — enough to try a model, not enough to build on. Stacked together, the project's own README puts the total at roughly 7.4 billion tokens a month across 34 providers, 474 model families and 635 free endpoints. The catch is that stacking them by hand means thirty-four different SDKs, thirty-four rate limits and thirty-four places a request can fail. FreeLLMAPI exists to collapse that stack into one OpenAI-compatible endpoint on your own machine: point any OpenAI client at http://localhost:3001/v1 with one unified key, and the router picks a provider that still has quota, fails over when one is rate-limited, and tracks usage so you stay under every cap.

The numbers explain the attention. FreeLLMAPI (tashfeenahmed/freellmapi on GitHub, MIT license) sits at about 44,000 stars with roughly 4,400 stars gained in the seven days to October 29, 2026, on an independent trending tracker — the kind of slope a genuinely useful piece of plumbing gets, not a demo that went viral and stalled. The repository is actively maintained, with the checkout verified here dated October 29, 2026. What makes it worth a tutorial rather than a mention is that the whole thing is self-hosted: provider keys are encrypted in a local SQLite database and decrypted in memory per request, so the only credential your apps ever see is a single unified key. There is no hosted middleman taking a cut of your prompts.

This guide verifies the part that matters most and is hardest to fake: a real request travelling through the gateway to a real model running on the same machine, with no provider account and no API key anywhere in the chain. It also names exactly what it did not run — the cloud free tiers themselves, which need free provider keys this sandbox does not hold — and gives you the documented path for adding them.

What you'll need

• Everything verified in this guide: Git, Node.js 20 or newer, and Ollama for the local model. The verification environment ran Node.js 24.22.1. No account, no API key, no GPU for any step verified here — the local model is a 522 MB download that runs on CPU.
• For the cloud free tiers (documented, not run here): free-tier keys from any providers you want in the pool — Google AI Studio, Groq, Cerebras, NVIDIA, Mistral, OpenRouter, Cohere, Cloudflare Workers AI and others. No credit card is needed for those tiers, but each one does need its own free signup, and this guide does not create accounts for you.
• Optional, for the easiest install: Docker, for the one-line installer. The sandbox used for verification has no Docker, so this guide runs the gateway from source instead — the documented path for machines without Docker — and says so wherever the two paths differ.
• One honest limitation, up front: the free-tier catalog on a free install tracks the monthly snapshot, so a model joins your catalog about 30 days after it lands in the live feed. The same-day catalog is the paid Premium tier ($19 a year). If you need a model the week it launches, the free install will lag; plan around that rather than discovering it mid-project.

Step 1 — Install the gateway from source

Clone the repository and install its dependencies. The monorepo holds the server, the dashboard client, the shared code and the agent-setup CLI, so the install pulls 840 packages; on the verification machine it took about two minutes:

git clone https://github.com/tashfeenahmed/freellmapi.git
cd freellmapi
npm install

Build the server. This compiles the TypeScript to server/dist/ and must exit cleanly before you go further — it did here, with no errors:

npm run build -w server

If you have Docker and would rather not build anything, the project's one-line installer is the documented alternative: it sets up a ~/freellmapi directory, generates an encryption key, pulls the image and starts the container. It was not run here because the verification environment has no Docker; everything below uses the from-source server, which serves the same API and dashboard on the same port.

curl -fsSL https://freellmapi.co/install.sh | bash

Step 2 — Configure it before the first boot

FreeLLMAPI can apply a JSON config on every boot, which is the repeatable way to set it up — and, for this guide, the way to register a local model without touching the dashboard. Create two files next to each other. The first holds the encryption key that protects stored provider keys at rest; generate a fresh one, never reuse one from a blog post:

ENCRYPTION_KEY="$(node -e 'console.log(require("crypto").randomBytes(32).toString("hex"))')"
printf "ENCRYPTION_KEY=%s\nPORT=3001\n" "$ENCRYPTION_KEY" > .env
printf "FREEAPI_CONFIG_PATH=%s/freellmapi.config.json\n" "$PWD" >> .env

The second file is the declarative config itself. It creates the first dashboard account, registers Ollama as a custom provider, and sets the routing strategy. The admin entry only works while no account exists, so it can never take over a claimed install later:

{
  "admin": { "email": "[email protected]", "password": "choose-a-long-password" },
  "customProviders": [
    {
      "baseUrl": "http://127.0.0.1:11434/v1",
      "label": "Ollama",
      "apiKey": "no-key",
      "models": [
        { "model": "qwen3:0.6b", "displayName": "Local Qwen3 0.6B", "supportsTools": false }
      ]
    }
  ],
  "routing": { "strategy": "balanced" }
}

The "apiKey": "no-key" line deserves an explanation, because the project's own install doc shows this same custom-provider block without it — and without it, the server does not start. The first boot here failed with key is required for custom: the code that stores a custom endpoint insists on a non-empty key even when the endpoint needs none. Ollama on localhost ignores the placeholder — it only checks an Authorization header if you have set an Ollama API key yourself — so no-key is a harmless stand-in, and with it in place the boot log reads applied …: 0 keys, 1 custom models, 0 model edits, 0 fallback rows, routing, admin account created. If you register a custom endpoint through the dashboard instead, the dashboard handles the keyless case for you; the placeholder is only needed in the declarative file. This is the kind of sharp edge a tutorial should surface rather than smooth over: the docs example, as written, bricks a keyless custom provider at startup.

Start the server in production mode, which runs database migrations automatically:

NODE_ENV=production node server/dist/index.js

Expect Server running on http://[::]:3001 and Proxy endpoint: http://[::]:3001/v1/chat/completions. Confirm the liveness probe answers before going further:

curl -s http://127.0.0.1:3001/livez
# {"status":"ok","version":"0.4.1","uptime_s":10}

Step 3 — Put a real model behind it with Ollama

Install Ollama from its official installer, then start it and pull the small model this guide routes to. The model is 522 MB and was already in the local cache from an earlier project on the verification machine; a first pull downloads it from Ollama's registry:

curl -fsSL https://ollama.com/install.sh | sh
ollama serve &
ollama pull qwen3:0.6b
ollama list
# NAME                ID            SIZE      MODIFIED
# qwen3:0.6b          7df6b6e09427  522 MB    4 weeks ago

Two sandbox notes, stated plainly because they affect reproducibility. First, the verification machine's first Ollama download was cut off by its egress proxy at about 82%, which left the runner binary missing; a resumable re-download completed the install, and the version that finally served the model is Ollama 0.40.13. If your own download stalls, resume it rather than trusting a partial install — the symptom of a partial install is a precise error, llama-server binary not found, which looks like a gateway problem but is not. Second, Ollama serves on 127.0.0.1:11434, which is exactly the base URL the gateway config points at; if you run Ollama on another host, change the config's baseUrl to match.

Step 4 — Send the first request through the gateway

The gateway mints one unified key at first boot and deliberately does not print it to the log. Take it from the dashboard's Keys page header — the path every reader should use — or, for a headless machine, read the unified_api_key setting from the gateway's SQLite database at server/data/freeapi.db. With that key, list what the gateway is serving. The custom model appears under an endpoint-scoped id, local-qwen3-0.6b, alongside the catalog's 285 models:

curl -s http://127.0.0.1:3001/v1/models \
  -H "Authorization: Bearer $FREEAPI_KEY" | python3 -c "
import json, sys
d = json.load(sys.stdin)
ids = [m['id'] for m in d['data']]
print('model count:', len(ids))
print('local model present:', 'local-qwen3-0.6b' in ids)
"
# model count: 285
# local model present: True

Now send a completion through the gateway, pinned to the local model. The gateway forwards it to Ollama, and the reply comes back through the same OpenAI-compatible shape any OpenAI client already understands:

curl -s http://127.0.0.1:3001/v1/chat/completions \
  -H "Authorization: Bearer $FREEAPI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "local-qwen3-0.6b",
       "messages": [{"role": "user", "content": "What is the capital of France?"}],
       "temperature": 0, "max_tokens": 120, "stream": false}'

The response, trimmed to the fields that matter (the gateway also returns an execution_id, a system_fingerprint, and the model's thinking in a separate reasoning field):

{
  "model": "qwen3:0.6b",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "The capital of France is **Paris**."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 17,
    "completion_tokens": 119,
    "total_tokens": 136
  }
}

Two details worth knowing. First, the model in the reply is the upstream id (qwen3:0.6b), not the gateway's endpoint-scoped id — that is how you confirm which model actually answered. Second, budget generously: the gateway counts the model's reasoning tokens toward max_tokens, so a 20-token cap cut this same reply off mid-thought; 120 tokens let the reasoning finish and the clean answer come through. A bigger local model, or a cloud free tier from Step 6, answers with less preamble, but the plumbing is identical.

That round trip is the whole product in miniature: one key, one endpoint, a local model answering through a router that also knows about 284 other models. The gateway's error reporting earns a mention too. Before the Ollama runner was fixed, the same call returned a structured error naming the attempt trail — custom/qwen3:0.6b key1: model_not_found — and the upstream cause, rather than a bare 500. When a provider in your pool goes wrong at 2 a.m., that trail is the difference between debugging and guessing.

Step 5 — Point a real client at it, without touching config files by hand

The repository ships a CLI that writes the config each coding agent expects. Build it, then ask it to show — not write — what it would do for a generic OpenAI-compatible client, pinned to the local model:

npm run build -w cli
node cli/dist/index.js setup-generic --url http://127.0.0.1:3001 \
  --api-key "$FREEAPI_KEY" --model local-qwen3-0.6b --dry-run

The output, quoted from the actual run, is three lines and a curl command — the entire integration surface for any tool that speaks OpenAI's API:

OPENAI_BASE_URL=http://127.0.0.1:3001/v1
OPENAI_API_KEY=freellmapi-…
OPENAI_MODEL=local-qwen3-0.6b
curl http://127.0.0.1:3001/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"local-qwen3-0.6b","messages":[{"role":"user","content":"Hello"}]}'

Drop --dry-run and the same command writes the config for a named tool instead of printing it: setup-claude for Claude Code, setup-codex for Codex CLI, plus Cline, Continue, Aider, Goose, Cursor and others. Every generator backs up an existing config file before merging into it, and the two launch commands inject credentials into the child process only, writing nothing to disk at all. That last detail matters: a router that centralises your keys should not spray them into a dozen tool configs, and this one, by default, does not.

Step 6 — Add a cloud free tier (documented path, not run here)

The local model proves the plumbing; the cloud tiers are where the 7.4-billion-token headline comes from. In the dashboard, open the Keys page and add a free-tier key — Groq and Google AI Studio are the usual first two, because their free tiers are generous and their signup is a form, not a sales call. The router validates a key when you save it, and the keys test CLI command re-checks stored keys later. Then reorder the Fallback Chain to taste: the routing strategies are priority, balanced, smartest, fastest, reliable, custom and decision, and a request that hits a 429 or 5xx cools that key down and tries the next model in the chain. This step is described from the project's docs and CLI help, not executed here — the verification environment holds no provider accounts, and this guide does not create any — so treat the free-tier counts as the project's claims, not as re-measured figures: they assume you hold a key for every provider, and no single reader will.

Two operational details are worth setting on day one. First, request analytics are kept for 90 days or 100,000 rows by default; set REQUEST_ANALYTICS_RETENTION_DAYS=0 or REQUEST_ANALYTICS_MAX_ROWS=0 in .env if you want either limit off. Second, the gateway binds to localhost only by default. That is the right default for a proxy that holds your keys — only open it to your LAN on a network you trust, since the single-user proxy is guarded only by the unified key.

When to use this vs alternatives

Use FreeLLMAPI when you prototype, learn, or run a coding agent on your own machine and you want every free tier behind one endpoint with failover and quota tracking — especially if you already juggle half a dozen provider keys in as many .env files. Its distinctive value is the catalog: it tracks which models are actually free this week, so you stop hand-maintaining a list that rots every time a provider changes a quota.

Use a general proxy like LiteLLM when your traffic is mostly paid keys across providers and you need the broadest protocol coverage, load balancing and spend tracking for production workloads. FreeLLMAPI's centre of gravity is the free tier; a paid-first setup gets more from a tool built for paid-first traffic.

Use a hosted aggregator when you want zero operations — no server to keep up, no catalog to sync — and you accept that your prompts flow through someone else's infrastructure and billing. The price of FreeLLMAPI's privacy is a process on your machine that you own, update and back up; the project's own backup page exists because a SQLite file with all your keys in it deserves one.

Do not use FreeLLMAPI as a way to avoid paying for production inference. Free tiers are rate-limited, change without notice, and owe you no uptime. The project's own docs are blunt about this: the router is for prototyping, side projects and learning, not for shipping to customers on an SLA. The 7.4-billion-token figure is a ceiling across every provider at once, not a budget any one reader can spend.

The takeaway

FreeLLMAPI does one job well: it turns thirty-four free tiers, each too small to build on, into a single OpenAI-compatible endpoint you control, with failover, quota tracking and one key to guard. Verified here, with no account and no provider key: the from-source install, the declarative config that registers a local Ollama model, the gateway listing 285 models including the local one, and a real completion travelling through the gateway to that local model. Not verified here, and stated as such: the cloud free tiers themselves, which need free provider signups this environment does not hold, and the Docker one-liner, which needs Docker this environment does not have. Start from source, register one local model, prove the round trip with curl, and only then add the free-tier keys you actually use — one endpoint, one key, and a fallback chain that keeps working when any single free tier does not.