LoRA (Low-Rank Adaptation)
LoRA (Low-Rank Adaptation) is a fine-tuning method for image-generation models that teaches the model a specific face or style using 20-50 reference images. The result is a lightweight file that makes the model generate a consistent character every time.
What is a LoRA?
LoRA (Low-Rank Adaptation) is a fine-tuning technique for image-generation models like Stable Diffusion. In practice: you provide 20-50 reference images of a specific face or character, run a training process (typically 4 hours on a cloud GPU), and get back a small file (50-200MB). When you attach that file to your Stable Diffusion workflow, the model generates images that consistently show the same face — your character — across different outfits, poses, and settings.
Why this matters for virtual influencers: character consistency is the hardest problem in building a fictional Instagram persona. Without a LoRA, standard Stable Diffusion produces a different-looking face every generation. With a custom LoRA, you can generate 200 images of the same character — what Aitana López’s team does at scale.
How LoRA training works
- Collect reference images — 20-50 images of your character (you likely generate these first in Midjourney or a base SD run)
- Caption the images — label each with a text description of the character’s unique identifier (e.g., “portrait of [character name], brown eyes, sharp jawline”)
- Train the LoRA — run the training job on a cloud GPU (RunPod costs ~£0.50-3 per training run; about 4 hours at £0.002/min)
- Test and refine — generate 20 test images, check for face drift, adjust training steps if needed
- Deploy — load the LoRA in ComfyUI or Automatic1111 for all future character image generation
Total cost for first LoRA
- RunPod GPU rental: ~£3-6 per training run
- Midjourney (for reference images): £24/mo
- Total month 1: ~£30-45 (one-time LoRA cost + ongoing Midjourney)
Where to find pre-trained LoRAs
Civitai is the main LoRA marketplace. Note: Civitai hosts LoRAs across a wide content range including explicit material; navigate with care and read each LoRA’s licence before commercial use.
Related
- Stable Diffusion — the base model LoRAs run on
- Character consistency — why this is the hard problem
- ComfyUI — the node-based editor used to deploy LoRAs in workflows
- LoRA training walkthrough