gputaskman hands-on: find out what is really eating your GPU memory

How it works: three measurements, one table

Every GPU monitor starts from the same number — total memory used — and stops there. gputaskman keeps going by answering three questions in order: how much is the card using, which process holds it, and what is inside that process. Each question has its own data source, and the tool is unusually honest about which answers are measurements and which are estimates.

First, the device totals. On NVIDIA cards it reads NVML, the same library nvidia-smi uses, which ships with the driver — no CUDA Toolkit needed. On Windows it reads WDDM counters; on native Linux AMD and Intel cards it reads the kernel's DRM interface, per-process file descriptors in /proc/<pid>/fdinfo. That gives it VRAM used, GPU activity, memory activity, PCIe traffic, temperature, power, and video-engine clocks, drawn as eight history charts.

Second, per-process attribution. The tool walks each process's mapped files in /proc/<pid>/maps and its command line, and matches the memory against the device counters. A process running a model shows up with its runtime — CUDA, HIP, Vulkan, Direct3D, ggml, ONNX Runtime — instead of a bare PID and a byte count. On Windows the same job is done with WDDM's per-process counters, which is how it can also show spill: GPU memory that lives in system RAM, and paged: memory committed but not resident.

Third, the model itself. For local LLMs it parses the GGUF file the process mapped, and reads the server's load log when one exists, to split the process's memory into weights, KV cache, and compute buffers. The inspector shows the model name, architecture, quantization, layer count, and the estimated KV cost — the part that grows as your conversation gets longer.

The concrete test is what sold it. On a bare-metal Linux laptop with an RTX 4050, a llama-server running Qwen2.5-1.5B Q4_K_M at -ngl 99 -c 4096 read 1.19 GB in gputaskman against 1218 MiB from nvidia-smi — the same memory, within a rounding error. The diff command put the device change at +1.20 GB against +1224 MiB from nvidia-smi, and the load log gave the exact split: 935 MB of weights, 112 MB of KV cache. While the server generated, the TUI showed 96% GPU load against 95% from nvidia-smi pmon. An Ollama runner with qwen3:0.6b was detected and named the same way, and a Qwen3-4B-Instruct-2507 at -ngl 99 -c 8192 on AMD showed 3.95 GB: 2.49 GB of weights plus 1.21 GB of f16 KV cache plus compute buffers, generating at 103 tokens per second.

What you'll need

• A Windows 10/11 PC with an NVIDIA or AMD GPU and a current driver, a Linux machine with NVIDIA, AMD, or Intel graphics, or a Mac on Apple Silicon (run from a clone; there is no portable macOS build yet).
• Python 3.11 or newer if you install from PyPI — or nothing at all if you take the portable Windows ZIP.
• A terminal about 140 columns wide; the full process table is cramped in anything narrower.
• Something using the GPU to look at: a local LLM server, a training run, or even just a browser with hardware acceleration.

Step 1 — Install it

The fastest route is uv, which keeps the app in an isolated environment. The shorter gtm command comes with the install:

uv tool install gputaskman
gputaskman

With pipx instead:

pipx install gputaskman
gputaskman

On Windows without Python, download the portable ZIP from the Releases page — gputaskman-0.3.4-windows-x64.zip for Intel/AMD PCs — extract the whole ZIP, keeping _internal beside the exe, and run .\gputaskman.exe. The builds are currently unsigned, and each ZIP ships with a .zip.sha256 file for integrity checks. Linux users get the same treatment as a tarball: gputaskman-0.3.4-linux-x64.tar.gz, needing glibc 2.35 or newer (Ubuntu 22.04 or Debian 12 and up). Upgrades are uv tool upgrade gputaskman or pipx upgrade gputaskman.

Step 2 — Open the live view and read a row

Run gputaskman (or gtm) and you get a live terminal UI: a device header with VRAM, load, temperature, power and clocks, eight history charts, and a process table. Each row shows the process, its local GPU memory, spill, paged, engine activity, and detected runtime. The keys are single letters:

Up / Down   select a process and inspect it
s           cycle sort: VRAM, GPU%, spill, paged, name
c           cycle all charts or a single metric
a           switch between top and all collected processes
m           toggle full diagnostics / slim monitoring
p           pause / resume collection
?           explain the counters and their limits
q           quit

The m key matters on a small screen. Full diagnostics shows driver and CUDA versions, PCI bus address, PCIe generation and width, and power and clock limits in the header; slim monitoring hides those and keeps the identity, memory bars, utilization, temperature, power, and clocks. Newly loaded models can take about 15 seconds to appear, because model discovery is throttled.

Step 3 — Inspect a model: weights vs KV cache

Select a row with Up/Down and the inspector opens. For an LLM process it names the model, its architecture and quantization, and breaks the memory into weights, estimated KV cost, and buffer sizes from the load log. This is the view that earns the tool its keep: a 7 GB row stops being a mystery and becomes, say, 4 GB of weights and 3 GB of KV cache from a long context window — which tells you whether to shrink the model or shorten the context.

Try it with a live server. Start an Ollama model or a llama-server, then watch the row appear and the inspector fill in. The README's own scope list names decision models, vision, speech, LLM inference, and training and fine-tuning — including Unsloth — so if you are running a QLoRA fine-tune with Unsloth, this is the view that shows whether the training run or the base model is holding the memory. The same inspector works for speech and vision workloads, so a local voice-cloning run with IndexTTS 2.5 shows up as a named model row too, not an anonymous Python process.

Step 4 — Snapshot, diff, and record

The TUI is for watching; the flags are for proving. One-shot text and JSON use the same collector as the live view:

gputaskman --once                # one text snapshot
gputaskman --json > before.json  # one JSON snapshot; implies --once
gputaskman --json > after.json   # after loading a model
gputaskman diff before.json after.json

diff is the workflow the README's spot checks use: snapshot, start the workload, snapshot again, and read exactly what changed — the Qwen2.5-1.5B check above came out as +1.20 GB against nvidia-smi's +1224 MiB. For a whole session, record appends one JSON snapshot per second:

gputaskman record -o session.jsonl --duration 300

That gives you five minutes of per-second data to graph or grep later — useful when a training run OOMs at 2 a.m. and you want to know what the memory curve looked like before it died. Two more flags cover special setups: --no-host stops WSL from reading Windows host counters, and --gpu N pins one card on a multi-GPU machine (list them first with --list-gpus).

Why not just nvidia-smi?

nvidia-smi is accurate and already installed, so the comparison is the right question. Here is what each tool actually tells you:

The honest answer is that the tools agree on the totals — the spot checks above match nvidia-smi within a few megabytes — and gputaskman earns its place on the attribution: which process, which model, and how the bytes split. One reading deserves care: the Spill column is where a buffer currently sits, not proof it was pushed out of VRAM, and the README says to read the DWM, Spill, and Paged section carefully before concluding the compositor is eating your card. A mapped file also does not prove a model is resident on the GPU — the tool says so itself.

The catch

The README opens with a warning, and it is worth repeating: this is early, experimental software. Device and process counters are measurements; classifications and GGUF-derived breakdowns are estimates. A PyTorch or CUDA DLL in a process does not prove training, and a mapped file does not prove a model is resident or being exercised.

Coverage is uneven by platform. The Windows portable builds are unsigned. On macOS there is no VRAM to split — Apple Silicon shares one memory pool — so the tool shows a footprint instead, and other users' processes need sudo. Multi-GPU on Windows and two discrete GPUs are not yet validated. In WSL, per-process VRAM is matched by inference from the Windows host, marked * for matched, ? for a guess, and n/a when no context could be assigned — a guess is recomputed every refresh and never counts as evidence for itself. And if you run a model big enough to matter — say, a 744-billion-parameter model on your own machine — expect the inspector to show you the split, but do not expect it to make the model fit.

What you built

You installed a 30-second answer to the question every local-AI user asks weekly: what is actually using my GPU? The live view tells you in real time, the inspector names the model and splits weights from KV cache, and diff and record turn a hunch into a file you can point at. The next time a load fails with an out-of-memory error, the first command is gputaskman --once — before you blame the model, check whether the browser, the compositor, or yesterday's forgotten server is holding the gigabytes.

It is built for people who run models on their own hardware: local inference, fine-tuning, speech and vision work, and anyone who has ever watched a training run die and wondered what else was on the card. It is not a cloud tool, not a benchmark, and not a profiler — it measures what is there and labels its guesses as guesses, which is exactly the right honesty for a tool you consult before spending money on a bigger card.