WithEveryone: Unified Planning and Identity Grounding for Group Image Generation

arXiv:2608.203362026-08-19

让AI在画多人合照前先规划好谁站在哪里,结果脸部混淆问题大幅减少

根据多张参考人脸生成合影的AI,一旦人数增加到五到十人,就容易出现认错人、脸部混淆,或明显的直接拼贴痕迹。WithEveryone的做法是先规划好画面中有哪些人、他们分别在什么位置和姿势,再据此渲染图像,并利用规划好的人脸区域直接监督每个人的身份,不再依赖训练时不稳定的人脸匹配。结果显示,其在目标图像中的人脸相似度达到0.499,高于GPT-Image 2的0.462,同时拼贴痕迹得分从0.169降到0.055。

他们做了什么

现有方法在人数增多后,训练时很难判断生成的哪张脸对应哪个参考身份,容易导致身份互相混淆
WithEveryone为每个人分配一个身份标记令牌,并在生成图像前先自动预测一份结构化的布局方案,包含每个人的位置、姿势和身份绑定,再将方案渲染成视觉条件图
核心方法Layout-Grounded ID Loss利用规划好的人脸区域标注直接监督每个身份,避免了传统基于人脸嵌入匹配的不稳定问题
在五到十人的基准测试中,该方法取得了最高的目标图像人脸相似度0.499,覆盖了97.3%的所要求身份,重复率仅2.8%
拼贴痕迹得分为0.055,是所有对比系统中最低的,说明其相似度并非靠直接复制参考人脸得来

Figure 1: Overview of WithEveryone. WithEveryone generates coherent group images from five to ten reference identities. A single model reasons about which references participate, plans identity–layout bindings, person regions, and poses, and generates the image conditioned on that plan, so identity and composition are decided in one context rather than by separate modules.

Figure 2: Group images generated by WithEveryone. The samples cover different group sizes, scene types, and visual styles.

Table 1: Quantitative comparison on our 5–10-person benchmark. All models are scored on the same 210 examples, and the identity similarities are averaged over the three face encoders of the main protocol. Coverage is the fraction of references whose best-matching generated face reaches a similarity of 0.20, and Dup the fraction that collapse onto a face already claimed by another identity. , , and mark the first-, second-, and third-best results; for Copy-Paste only the three methods with the highest Sim(Ref) are ranked, since lower similarity means less copying naturally.
Method	Identity Similarity	Identity Coverage	Generation Quality
Sim(Tgt) ↑	Sim(Ref) ↑	Copy-Paste ↓	Coverage ↑	Dup ↓	CLIP-I ↑	DINO-I ↑	CLIP-T ↑
Academic identity-preserving methods
WithAnyone	0.405	0.483	0.096	0.957	0.045	0.807	0.695	0.281
UMO	0.371	0.484	0.112	0.630	0.258	0.780	0.663	0.286
UniPortrait	0.339	0.464	0.115	0.635	0.187	0.679	0.415	0.301
DreamO	0.297	0.331	0.027	0.298	0.299	0.724	0.611	0.282
ID-Patch	0.225	0.259	0.032	0.247	0.224	0.609	0.320	0.331
Open-source general-purpose models
FLUX.2 Klein	0.264	0.265	0.002	0.314	0.265	0.787	0.685	0.291
Qwen-Image-Edit	0.253	0.289	0.018	0.176	0.291	0.614	0.312	0.227
LongCat-Image-Edit	0.287	0.288	−0.001	0.381	0.264	0.802	0.657	0.288
OmniGen2	0.267	0.275	−0.002	0.418	0.267	0.774	0.633	0.288
SenseNova U1	0.253	0.243	−0.009	0.213	0.256	0.797	0.686	0.289
BAGEL	0.223	0.225	0.003	0.136	0.276	0.752	0.635	0.282
HiDream-O1	0.353	0.376	0.026	0.780	0.190	0.806	0.692	0.286
Proprietary systems
Nano Banana Pro	0.453	0.478	0.041	0.674	0.148	0.839	0.712	0.275
Nano Banana 2	0.451	0.480	0.045	0.884	0.099	0.860	0.731	0.276
GPT-Image 2	0.462	0.583	0.169	0.905	0.075	0.853	0.719	0.270
Seedream 4.5	0.407	0.495	0.114	0.859	0.175	0.829	0.698	0.286
Seedream 5.0 Pro	0.436	0.522	0.114	0.913	0.065	0.850	0.715	0.275
WithEveryone	0.499	0.540	0.055	0.973	0.028	0.861	0.716	0.273

Figure 3: Overview of WithEveryone. The model loads selected reference identities as ID tokens, predicts structured Layout CoT, renders a visual layout condition, and predicts identity representations before generating the target group image. Modality-switch tokens that mark the boundaries between autoregressive reasoning and flow-based image generation are used in the sequence but omitted from the figure.

Figure 4: Data statistics, pipeline, and example. We report the distributions of identity occurrence frequency, reference count, and real photographs versus collage and poster images; the data-construction pipeline; and an example training sample. The corpus contains 400K group-image samples, and the reference-count distribution covers the five-to-ten-person range used at evaluation time.

Table 3: Dispersion of the main comparison. Per-example means on the 210 benchmark examples with their standard errors. The last column is the single-encoder ArcFace similarity to the references used in Section 4.2.
Method	Sim(Tgt) ↑	Sim(Ref) ↑	CLIP-I ↑	ArcFace Sim(Ref) ↑
WithEveryone	0.499±0.004	0.540±0.004	0.861±0.004	0.614±0.005
GPT-Image 2	0.462±0.004	0.583±0.008	0.853±0.004	0.566±0.009
Nano Banana 2	0.451±0.005	0.480±0.007	0.860±0.004	0.471±0.008
Seedream 5.0 Pro	0.436±0.005	0.522±0.008	0.850±0.004	0.506±0.009

Figure 6: Qualitative comparison with proprietary models. , , and mark faces that reproduce a person already generated elsewhere in the image, that are not recognizable as their bound reference, and that appear transplanted without adapting to the pose, lighting, or viewpoint of the scene. These author annotations are illustrative; their quantitative counterparts are the duplicate rate, Sim(Ref), and Copy-Paste.

Figure 7: Identity similarity against face size. Curves are local means with 95% confidence bands, annotated with the fitted slope. The lower panel of (a) shows the distribution of relative face size for each group size, and the lower panel of (b) shows the face-size distribution of each system.

Table 4: High-resolution comparison. The first two rows use the same internal pipeline; the last row uses the main 2K benchmark pipeline.
Train / test	Sim(Ref) ↑	Sim(Tgt) ↑	Layout∗ ↑	Plan IoU ↑
1K / 1K	0.546	0.460	0.740	0.773
2K / 1K	0.555	0.461	0.759	0.814
2K / 2K†	0.614	0.511	–	–

(b) Absolute face size in pixels; the dashed line marks 112 px, the input resolution of the ArcFace encoder, below which a face has to be upsampled before it can be compared.

Figure 10: Additional group images generated by WithEveryone.

Table 5: Effect of the LG-ID Loss. It gives the largest individual identity gain while preserving or improving all reported layout measures.
Variant	Sim(Ref) ↑	Sim(Tgt) ↑	Layout ↑	Count ↑	Coverage ↑
Default	0.339	0.304	0.690	0.771	0.741
+ ID Loss	0.506	0.435	0.700	0.845	0.947
Full	0.555	0.461	0.759	0.869	0.960

Figure 11: Additional group images generated by WithEveryone.

Figure 12: High-resolution training. We compare the Full model trained at 1K and 2K while evaluating both at 1K. The horizontal references in (a) show the final 2K benchmark scores from a different evaluation pipeline.

为什么重要

多人合影、海报、拼图类图像的AI生成需求正在增加,但现有工具在人数变多时表现急剧下降。这项研究表明,先规划构图再生成图像的思路真正能解决大规模人数下的身份保持问题,而不只是缓解症状。

(b) Planning quality and plan execution.

Figure 13: Representation Forcing convergence. The supervised identity prediction is steadily optimized during training and becomes increasingly aligned with both the target and reference identity embeddings.

本文术语

身份保持生成(Identity-preserving generation) · 根据某人的参考照片生成新图像,并让生成结果依然像本人的技术
拼贴痕迹(Copy-Paste artifact) · 生成的人脸看起来像是直接从参考照片剪贴过来,与场景的光线、姿势、角度不协调
Layout CoT · 在生成图像前,以文字形式逐步预测每个人位置、姿势和身份分配的规划过程
ArcFace · 一种将人脸照片转换为数值向量、用来计算两张脸相似度的人脸识别模型
身份令牌(ID token) · 代表某个人身份信息的紧凑信号,供模型在生成过程中参考

论文原文摘要（英文）

Identity-preserving image generation becomes increasingly unreliable when a scene must contain many specified people. Beyond retaining each identity, the model must bind every reference to a distinct person and location, while training-time identity losses must establish correspondence among several noisy predicted faces. We introduce WithEveryone, a unified framework for generating group images up to ten reference identities. WithEveryone injects each selected identity as an addressed token, predicts a structured identity--layout plan, and renders the plan as a visual condition. Its key objective, Layout-Grounded ID Loss, uses annotated face regions to supervise the intended identities directly, avoiding unstable embedding-based face matching; ID Representation Forcing additionally trains a prediction for each identity before image synthesis. On an identity-disjoint benchmark, WithEveryone achieves the highest target-context identity similarity, improving face similarity from 0.462 for GPT-Image-2 to 0.499, while reducing copy-paste artifacts from 0.169 to 0.055. It further covers 97.3\% of the requested identities with a duplicate rate of only 2.8\%. These results show that explicit identity--layout grounding enables identity-preserving generation to scale to larger groups without relying on direct reference-face copying.

作者 · Hengyuan Xu

在 arXiv 阅读

METAL LAB 最新报道

图片来源： Hengyuan Xu et al., arXiv:2608.20336, cc-by-nc-sa-4.0

WithEveryone: Unified Planning and Identity Grounding for Group Image Generation

他们做了什么

为什么重要

本文术语

最新论文

METAL LAB 最新报道