Diffusers documentation
快速入门
You are viewing main version, which requires installation from source. If you'd like
regular pip install, checkout the latest stable version (v0.42.1).
快速入门
Diffusers 是一个面向开发者和研究人员的库，提供了用于生成图像、视频和音频的便捷推理 API，以及用于实现新工作流的构建模块。
Diffusers 提供了许多开箱即用的优化功能，使得在内存有限的设备上加载和运行大型模型成为可能，或加速推理过程。
本快速入门将带你概览 Diffusers，并让你快速上手生成内容。
在开始之前，请确保您拥有一个 Hugging Face 账户，以便使用 Flux 等受控模型。如果您打算构建自定义流水线，请参阅 Modular Diffusers 快速入门 和 概述。
如果尚未安装 Diffusers，请按照安装指南进行安装。
Agent 提示
将此内容粘贴到您的编码代理中，以完成 Diffusers 的推理环境配置。
Help me get set up with Hugging Face Diffusers for inference. 1. Install Diffusers for my environment with `uv pip install "diffusers[torch]"`. 2. If I need gated Hub models, help me authenticate to the Hugging Face Hub. 3. Install Diffusers coding agent skills with `diffusers-cli skills add diffusers-cli`. Pass `--cursor`, `--claude`, or `--codex` if auto-detect fails. Optionally pass `--all` to install every skill in the registry. 4. Run a first text-to-image with DiffusionPipeline or `diffusers-cli run`, using a small or current Quickstart model and the right `device_map` for my machine. 5. Ask what I want next and point me at the matching docs.
DiffusionPipeline
DiffusionPipeline packages the pieces of a diffusion model (text encoder, scheduler, UNet or DiT, and VAE) into one class for inference. Load with from_pretrained(), then call the pipeline.
__call__() 中的参数（例如 num_inference_steps）会影响生成质量和速度。有关加载详情以及组件的混合搭配，请参阅 加载流水线。如需更换调度器，请参阅 调度器。
以下示例使用默认参数值。
文本到图像
文本到视频
使用 .images[0] 访问生成的图像输出。
import torch
from diffusers import DiffusionPipeline
pipeline = DiffusionPipeline.from_pretrained(
"Qwen/Qwen-Image", dtype=torch.bfloat16, device_map="cuda" # or "mps", "xpu", "cpu"
)
prompt = """
cinematic film still of a cat sipping a margarita in a pool in Palm Springs, California
highly detailed, high budget hollywood movie, cinemascope, moody, epic, gorgeous, film grain
"""
pipeline(prompt).images[0]使用 .frames[0] 访问生成的视频输出，并使用 export_to_video() 保存视频。
import torch
from diffusers import AutoencoderKLWan, DiffusionPipeline
from diffusers.utils import export_to_video
vae = AutoencoderKLWan.from_pretrained(
"Wan-AI/Wan2.2-T2V-A14B-Diffusers",
subfolder="vae",
dtype=torch.float32
)
pipeline = DiffusionPipeline.from_pretrained(
"Wan-AI/Wan2.2-T2V-A14B-Diffusers",
vae=vae,
dtype=torch.bfloat16,
device_map="cuda" # or "mps", "xpu", "cpu"
)
prompt = """
Cinematic video of a sleek cat lounging on a colorful inflatable in a crystal-clear turquoise pool in Palm Springs,
sipping a salt-rimmed margarita through a straw. Golden-hour sunlight glows over mid-century modern homes and swaying palms.
Shot in rich Sony a7S III: with moody, glamorous color grading, subtle lens flares, and soft vintage film grain.
Ripples shimmer as a warm desert breeze stirs the water, blending luxury and playful charm in an epic, gorgeously composed frame.
"""
video = pipeline(prompt=prompt, num_frames=81, num_inference_steps=40).frames[0]
export_to_video(video, "output.mp4", fps=16)LoRA
LoRA 适配器会在基础流水线之上添加一个小型的风格或主题检查点。使用 load_lora_weights() 加载其中一个。某些 LoRA 需要触发短语。请查看该 LoRA 的模型卡。
import torch
from diffusers import DiffusionPipeline
pipeline = DiffusionPipeline.from_pretrained(
"Qwen/Qwen-Image", dtype=torch.bfloat16, device_map="cuda" # or "mps", "xpu", "cpu"
)
pipeline.load_lora_weights(
"threecrowco/VolkClipartQwen",
weight_name="pytorch_lora_weights.safetensors",
)
prompt = """
Volk clipart style drawing of a cat sipping a margarita in a pool in Palm Springs, California, flat colors, bold outlines, simple shapes
"""
pipeline(prompt).images[0]量化与优化
大型模型通常需要更少的内存或更高的速度。使用 量化 来缩小内存中的权重，并使用 compile_repeated_blocks() 来加速后续的生成过程。有关 模型卸载 及其他选项，请参阅 优化与扩展。
要减少内存占用，请使用 bitsandbytes 以 4 位精度加载。
import torch
from diffusers import DiffusionPipeline
from diffusers.quantizers import PipelineQuantizationConfig
quant_config = PipelineQuantizationConfig(
quant_backend="bitsandbytes_4bit",
quant_kwargs={"load_in_4bit": True, "bnb_4bit_quant_type": "nf4", "bnb_4bit_compute_dtype": torch.bfloat16},
components_to_quantize=["transformer", "text_encoder"],
)
pipeline = DiffusionPipeline.from_pretrained(
"Qwen/Qwen-Image",
dtype=torch.bfloat16,
quantization_config=quant_config,
device_map="cuda" # or "mps", "xpu", "cpu"
)
prompt = """
cinematic film still of a cat sipping a margarita in a pool in Palm Springs, California
highly detailed, high budget hollywood movie, cinemascope, moody, epic, gorgeous, film grain
"""
pipeline(prompt).images[0]为了加快后续运行速度，请对 transformer 中重复出现的块进行编译。编译后的首次 generate 属于冷启动，速度较慢。之后的 generate 会更快。
import torch
from diffusers import DiffusionPipeline
pipeline = DiffusionPipeline.from_pretrained(
"Qwen/Qwen-Image", dtype=torch.bfloat16, device_map="cuda" # or "mps", "xpu", "cpu"
)
pipeline.transformer.compile_repeated_blocks(fullgraph=True)
prompt = """
cinematic film still of a cat sipping a margarita in a pool in Palm Springs, California
highly detailed, high budget hollywood movie, cinemascope, moody, epic, gorgeous, film grain
"""
pipeline(prompt).images[0]后续步骤
- 推理 — 流水线、提示词和适配器
- 优化与扩展 — 内存、速度、量化和推理服务
- 模块化 Diffusers — 可组合的模块和自定义流水线
- 训练与微调 — 训练脚本和适配器
- CLI — 从命令行生成