Consistent characters in AI images: a reference-image workflow

Ask any image model to draw the same character twice and you'll get two cousins, not a twin. Every diffusion model starts from random noise, so it has no memory of the character it drew five minutes ago. Text prompts — even long ones — can't encode the exact geometry of a face, the precise shade of hair, or the cut of a jacket. That gap between "a red-haired woman" and your red-haired woman is the character-consistency problem, and solving it is what separates one-off pretty pictures from usable storyboards, comics, brand mascots, and product imagery.

The fix is a reference-image workflow: lock the identity once, then condition every later generation on that reference instead of on text alone. Here's how, from the fastest no-code path to the strongest DIY setup.

Step 1: Build one rock-solid reference (the anchor)#

Everything downstream depends on your starting image. Generate your character in a clean, neutral pose — front-facing or three-quarter view, plain background, even lighting — and upscale your favorite result. Flat, neutral lighting prevents the model from baking a dramatic background or harsh shadow into your character's identity.

Build a small reference package:

• One anchor portrait: the canonical image of your character, the single source of truth.
• A turnaround or expression strip (optional but powerful): front, side, and three-quarter views, or a row of expressions.
• A written identity spec: 5–8 distinctive, stable traits (face shape, eye color and spacing, hair style and exact shade, signature garment, accessories, scars or tattoos). You'll paste this into prompts to reinforce what the reference shows.

A useful rule from character-design practice: keep the anchor's background clean and the lighting flat, and annotate distinctive features. The less incidental noise in the reference, the more the model learns the character rather than the scene.

Step 2: The hosted route — Midjourney's reference system#

If you'd rather not run models yourself, Midjourney's built-in reference tools are the fastest path to a consistent character.

Character Reference (--cref, V6). Introduced in March 2024 for Midjourney V6, it takes an image URL and steers the character toward it:

/imagine prompt: a detective in a rain-soaked alley at night --cref <image-url> --cw 100 --v 6.0

The --cw (character weight) parameter runs from 0 to 100: --cw 100 (the default) copies face, hair, and clothing; lower it to around 50 to keep face and hair while freeing up the outfit, or drop it to 0 to preserve the face alone — handy when you want the same person in a new costume or hairstyle. You can pass more than one reference URL to blend characters, and it combines with style references (--sref). Note the limits: it captures the essence of the character, not exact freckles, dimples, or logos on a t-shirt.

Omni Reference (--oref, V7). Midjourney's newer approach generalizes reference conditioning beyond people to characters, objects, vehicles, and creatures. Usage is --oref <image-url> with --ow (omni weight) between 1 and 1,000, defaulting to 100; the official docs advise staying under 400 unless pairing with very high stylize values. Differences from the older system: Omni Reference accepts only a single image and costs roughly double the GPU time of a regular V7 generation. On V8.x the docs point to the Edit Model's reference support — as of 2026, check the current docs before locking in a workflow, since the parameter set has shifted across versions.

Practical tips that hold across versions:

• Describe the new scene fully. The reference anchors identity; your text carries the pose, action, and environment. Don't let the prompt restate traits that contradict the reference.
• Re-anchor, don't chain. For every new scene, go back to your original anchor as the reference — not the previous generation. Chaining outputs into outputs lets tiny deviations compound until your character becomes someone else.

Step 3: The open-source identity stack (IP-Adapter, InstantID, ControlNet)#

If you run your own pipeline (Stable Diffusion XL, Flux, ComfyUI), you get finer control through dedicated identity adapters.

IP-Adapter encodes your reference image into features the diffusion model reads alongside the text prompt. For faces, use the face-specific variant (e.g., ip-adapter-plus-face for SDXL) on a cropped face reference, with the adapter weight around 0.6–0.8 for face similarity and 0.4–0.6 for overall style matching. Start the adapter at 0% of the diffusion process and end around 80%, so it steers composition while the model finishes fine details freely.

Identity specialists. Face-focused adapters such as InstantID and FaceID use a dedicated face encoder rather than a general image encoder. Practitioners commonly combine an identity adapter with ControlNet OpenPose, which locks body position via a skeletal map while the identity adapter holds the face — a widely used setup is OpenPose plus IP-Adapter FaceID at 20–30 steps, CFG around 5–9, adapter strength near 0.6–0.7. Newer adapters (notably PuLID) are reported by users to preserve art style better than earlier ones.

Best practices: feed 3–5 reference views, not one; run one adapter instance for the face at a higher weight and another for costume/body lower; and never let the prompt contradict the reference (a prompt saying "blonde" against a brunette reference ruins both).

Step 4: Seeds, re-rolls, and drift control#

A fixed seed does not reproduce your character when anything else changes — it only fixes the starting noise, and even small prompt tweaks push the result elsewhere. Use seeds for what they're good at: lock the seed while you adjust lighting or framing in one scene, so changes are attributable to your edits; then re-roll the seed a few times and keep the best-matching face.

The real drift control is editorial: compare every output against the anchor, regenerate the failures, and always re-anchor to the original reference rather than the last output. That habit is the single highest-leverage trick in this tutorial.

Step 5: The gold standard — a character LoRA#

When you need a character to survive hundreds of images, ads, or a whole comic, fine-tuning beats per-image conditioning. A LoRA (low-rank adaptation) is a small add-on — typically tens to a couple hundred megabytes — trained on images of your specific character so the base model learns their identity directly.

The workflow in brief:

1. Collect 15–50 training images spanning angles, poses, expressions, backgrounds, and close-ups. Diverse, consistent-quality data beats near-duplicates; returns diminish past roughly 100–150 images.
2. Caption each image with a trigger word plus stable identity traits, then per-image variables: alchemist_elena, purple asymmetric hair, golden eyes, freckles, green leather apron, standing with arms crossed, confident smile, in a cluttered workshop, warm candlelight
3. Train sensibly (for SDXL-class models: rank ~32, learning rate ~1e-4, 15–25 epochs at 1024px). Watch for underfitting (features not preserved — train longer or raise rank) versus overfitting (outputs clone your training photos — train less or add more diverse data). Expect 30–60 minutes on a high-end consumer GPU.
4. Deploy with the trigger word at a LoRA weight around 0.7–0.9, stacking a style LoRA on top if you need a fixed art direction.

The trade-off is honest: LoRA training costs you an afternoon of data prep and GPU time, but it's the only method where you can type alchemist_elena into any prompt and get her, in any scene, without fussing over reference weights.

Step 6: Editing chains that preserve identity#

Sometimes you don't need a new scene — you need to fix the one you have:

• Edit-model passes. Instruction-based editing models let you submit an image plus an instruction like "change the background to a night market, keep the character identical." For small changes, edits preserve identity far better than regenerating, because most pixels stay anchored.
• Inpainting for spot fixes. Mask only the problem region — a mangled hand, a wrong accessory — and regenerate that patch with the anchor description in the prompt. Keep masks tight; the smaller the repainted area, the less identity is at risk.

The rule of thumb: generate with references, fix with edits, and re-anchor to the original reference whenever a chain starts drifting.

Which approach should you use?#

The takeaway#

Character consistency is a pipeline, not a parameter. Build one excellent anchor, re-anchor every generation to the original reference, match the tool to the scale (native reference features for speed, identity adapters for control, LoRA for permanence), and fix small errors with edits instead of full regenerations. Do that, and the same face shows up in every scene — which is the entire job.