Stable Diffusion - 扩展 Roop 换脸 (Face Swapping) 插件的配置与使用_stable diffusion 换脸

欢迎关注我的CSDN：https://spike.blog.csdn.net/
本文地址：https://spike.blog.csdn.net/article/details/131856141

官网：GitHub - roop，参考论文：RobustSwap: A Simple yet Robust Face Swapping Model against Attribute Leakage

面部替换的目标是将源图像的身份（即，面部特征）注入目标图像，同时严格保留与身份无关的目标属性。然而，以前的方法仍然受到源属性泄露的困扰，其中源图像的属性会干扰目标图像的属性。通过分析 StyleGAN 的潜在空间，并且找到适合面部替换任务的潜在组合。因此，开发出简单但稳健的面部替换模型，RobustSwap，对于可能的源属性泄露有抵抗力。此外，利用 3DMM 的隐式和显式信息的协调作为指导，以融入源图像的结构和目标图像的精确姿势。尽管方法仅使用没有身份标签的图像数据集进行训练，但是模型具有生成高保真度和时间一致性视频的能力。通过大量的定性和定量评估，在合成图像和视频方面，与以前的面部替换模型相比，有显著的改进。

---

1. 测试 SD 模型

模型：墨幽人造人_v1010，使用 LiblibAI 墨幽人造人 官网下载，同时包括 麦橘写实，参考：

wget https://liblibai-online.liblibai.com/web/model/6a226dd292a983b3ed4987402453ad4d954b77825c1cf99d39d8746909791761.safetensors?attname=%E5%A2%A8%E5%B9%BD%E4%BA%BA%E9%80%A0%E4%BA%BA_v1010.safetensors -O 墨幽人造人_v1010.safetensors

wget https://liblibai-online.liblibai.com/web/model/e4a30e4607faeb06b5d590b2ed8e092690c631da0b2becb6224d4bb5327104b7.safetensors?attname=majicMIX%20realistic%20%E9%BA%A6%E6%A9%98%E5%86%99%E5%AE%9E_v6.safetensors -O 麦橘写实_MajicMIX_Realistic_v6.safetensors
1
2
3

LiblibAI 下载速度约 10M / s，非常快。

其他真实系模型包括：MajicMIX、DreamShaper 等。

Embeddings 模型：

• Age Slider，控制年龄，由不同的 Embeddings 控制。
• Gender Slider，控制性别，由不同的 Embeddings 控制。

LoRA 模型：

• Fashion Girl，LoRA，提升真实系模型的人物表现。

---

2. 安装 Roop 插件

Roop 插件：https://github.com/s0md3v/sd-webui-roop

下载插件：

git clone https://ghproxy.com/https://github.com/s0md3v/sd-webui-roop.git
1

安装包：

pip install insightface==0.7.3
pip install --upgrade protobuf
1
2

下载： inswapper_128.onnx，放入 <webui_dir>/models/roop/

cd stable_diffusion_webui_docker/models/roop

bypy downfile /stable_diffusion/extensions/roop/inswapper_128.onnx inswapper_128.onnx
1
2
3

下载其他模型文件：

bypy downfile /stable_diffusion/extensions/roop/roop_other_models.zip roop_other_models.zip
1

模型位置：

cd ~
mkdir .ifnude
cd .ifnude
# detector.onnx 和 classes
mv roop_other_models/.ifnude/* .

cd ~
mkdir .insightface
cd .insightface
# .insightface/models/buffalo_l
# 1k3d68.onnx\2d106det.onnx\det_10g.onnx\genderage.onnx\w600k_r50.onnx
mv roop_other_models/.insightface/* .
1
2
3
4
5
6
7
8
9
10
11
12

inswapper_128.onnx: 下载地址 henryruhs/roop at main (huggingface.co)

安装 Roop 插件成功，版本 v0.0.2，在下面的插件区域显示，如下：

---

3. 生成测试图像

正向提示词：

1girl,black footwear,black hair,boots,curtains,dress,floral print,full body,handbag,lips,long hair,long sleeves,looking at viewer,polka dot legwear,shoes,smile,solo,standing,tile floor,tile wall,
a woman standing in front of a building wearing a gray dress and purple socks and a gray jacket with polka dots,Ayami Kojima,long black hair,a hologram,private press,
best quality,masterpiece,ultra high res,(photorealistic:1.4),

负向提示词：

(ng_deepnegative_v1_75t,badhandv4,bad_prompt_version2-neg,EasyNegative:0.9),(worst quality, low quality:1.3),(depth of field, blurry:1.2),(greyscale, monochrome:1.1),croped,lowres,text,jpeg artifacts,(logo,signature,watermark,username,artist name,title:1.3),

注意：提示词来源真实图像反推

After Detailer：

• After Detailer 的脸部提示词，推荐：detailed face, close-up, portrait,
• After Detailer 的手部提示词，推荐：detailed hand, perfect hand,
• After Detailer 相关的 ControlNet 模型，推荐 Inpaint 模型，Module 选择 Inpaint_global_harmonious。 测试效果不好。

其他配置：

• Upscale: 8x_NMKD-Superscale_150000_G 算法，适合真人风格。
• 固定随机种子：2086435253

墨幽人造人的信息汇总：

1girl,black footwear,black hair,boots,curtains,dress,floral print,full body,handbag,lips,long hair,long sleeves,looking at viewer,polka dot legwear,shoes,smile,solo,standing,tile floor,tile wall,
a woman standing in front of a building wearing a gray dress and purple socks and a gray jacket with polka dots,Ayami Kojima,long black hair,a hologram,private press,
best quality,masterpiece,ultra high res,(photorealistic:1.4),
Negative prompt: (ng_deepnegative_v1_75t,badhandv4,bad_prompt_version2-neg,EasyNegative:0.9),(worst quality, low quality:1.3),(depth of field, blurry:1.2),(greyscale, monochrome:1.1),croped,lowres,text,jpeg artifacts,(logo,signature,watermark,username,artist name,title:1.3),
Steps: 35, Sampler: DPM++ 2M SDE Karras, CFG scale: 7, Seed: 2086435253, Size: 768x1024, Model hash: 6a226dd292, Model: 墨幽人造人_v1010, Denoising strength: 0.3, Clip skip: 2, ADetailer model: face_yolov8n.pt, ADetailer prompt: "detailed face, close-up, portrait, ", ADetailer confidence: 0.3, ADetailer dilate/erode: 4, ADetailer mask blur: 4, ADetailer denoising strength: 0.4, ADetailer inpaint only masked: True, ADetailer inpaint padding: 32, ADetailer version: 23.7.6, Hires upscale: 2, Hires steps: 10, Hires upscaler: 8x_NMKD-Superscale_150000_G, Version: v1.4.0

麦橘写实的信息汇总：

1girl,black footwear,black hair,boots,curtains,dress,floral print,full body,handbag,lips,long hair,long sleeves,looking at viewer,polka dot legwear,shoes,smile,solo,standing,tile floor,tile wall,
a woman standing in front of a building wearing a gray dress and purple socks and a gray jacket with polka dots,Ayami Kojima,long black hair,a hologram,private press,
best quality,masterpiece,ultra high res,(photorealistic:1.4),
lora:add_detail:0.5
Negative prompt: (ng_deepnegative_v1_75t,badhandv4,bad_prompt_version2-neg,EasyNegative:0.9),(worst quality, low quality:1.3),(depth of field, blurry:1.2),(greyscale, monochrome:1.1),croped,lowres,text,jpeg artifacts,(logo,signature,watermark,username,artist name,title:1.3),
Steps: 35, Sampler: DPM++ 2M SDE Karras, CFG scale: 7, Seed: 2086435253, Size: 768x1024, Model hash: e4a30e4607, Model: 麦橘写实_MajicMIX_Realistic_v6, Denoising strength: 0.3, Clip skip: 2, ADetailer model: face_yolov8n.pt, ADetailer prompt: "detailed face, close-up, portrait, ", ADetailer confidence: 0.3, ADetailer dilate/erode: 4, ADetailer mask blur: 4, ADetailer denoising strength: 0.4, ADetailer inpaint only masked: True, ADetailer inpaint padding: 32, ADetailer model 2nd: hand_yolov8s.pt, ADetailer prompt 2nd: “detailed hand, perfect hand,”, ADetailer confidence 2nd: 0.3, ADetailer dilate/erode 2nd: 4, ADetailer mask blur 2nd: 4, ADetailer denoising strength 2nd: 0.4, ADetailer inpaint only masked 2nd: True, ADetailer inpaint padding 2nd: 32, ADetailer version: 23.7.6, Hires upscale: 2, Hires steps: 10, Hires upscaler: 4x-UltraSharp, Lora hashes: “add_detail: 7c6bad76eb54”, Version: v1.4.0

左侧是墨幽人造人（侧脸） ，右侧是麦橘写实（正脸），图像：

4. 配置与使用 Roop 插件

脸型参考：英国籍亚裔女演员 克拉拉 的图像，头像清晰，即：

调用 Roop 插件生成换脸图像，配置如下：

1. 选择 启用
2. 调低 面部修复强度 至 0.5
3. 放大算法，使用现实风格的 8x_NMKD-Superscale；
4. 模型使用默认的 inswapper_128.onnx，即可
5. 注意：放大倍率是2，会导致图像放大 2 倍，即由 1536x2048 放大至 3072x4096。

即：

需要重新运行之前的配置（或者使用 图生图 功能），再次生成图像，之前的脸部由 AI 生成，现在转换成 克拉拉 图像中的脸型。

运行日志如下：

Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /root/.insightface/models/buffalo_l/1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /root/.insightface/models/buffalo_l/2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /root/.insightface/models/buffalo_l/det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /root/.insightface/models/buffalo_l/genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /root/.insightface/models/buffalo_l/w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (640, 640)
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /root/.insightface/models/buffalo_l/1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /root/.insightface/models/buffalo_l/2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /root/.insightface/models/buffalo_l/det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /root/.insightface/models/buffalo_l/genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /root/.insightface/models/buffalo_l/w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (640, 640)
2023-07-21 16:32:20,145 - roop - INFO - Upscale with 8x_NMKD-Superscale_150000_G scale = 2
2023-07-21 16:32:43,872 - roop - INFO - Restore face with CodeFormer
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

墨幽人造人，侧脸对比，如图：

麦橘写实，正脸对比，如图：

参考：站酷 - lulusmile服装街拍