工作日早上 7 点读 AI，周日早上 8 点读周报订阅邮件

METAL LAB

Seeing or Knowing? Visual Context Sensitivity in Multimodal Large Language Models

arXiv:2607.263262026-07-27

多模态AI不是看不到图像,而是看到了却控制不住要不要信它

这篇论文把多模态大语言模型(MLLM)在图像信息和预训练知识冲突时为何出错,拆成两种可能原因分别验证。研究者只用模型最后一层的图像token重建图片,发现反事实(与常识相悖)的视觉信息其实还保留着,说明问题不是没看到,而是看到了却不会控制要不要用。他们构建了WhatIfVis基准,并通过微调、激活patching和一个学习出的引导向量证明这种控制能力可以被测量和部分恢复。

METAL LAB 解读图

感知失败(H1)与利用失败(H2),以及控制干预流程

证据状态已报告实测结果

  1. 1. 重建诊断仅用冻结MLLM最后一层的图像token重建图片,检验反事实属性是否仍然存在,以此排除H1
  2. 2. WhatIfVis评测用配对的图像/文字反事实样本,在'只依据图像'与'忽略图像'两种指令下测试基础模型,以此暴露H2
  3. 3. 有监督微调仅用空间-时间属性数据微调,检验这种可控性能否迁移到其他属性类型
  4. 4. 激活patching替换微调后模型内部某层的激活值,定位视觉与先验权衡发生的狭窄层段
  5. 5. 引导向量将定位层的一个学习方向固定为常数值,不加任何提示指令即可控制模型对图像与先验的依赖倾向
这是 METAL LAB 制作的解读图，并非论文作者提供的原图。

他们做了什么

  1. 研究者把MLLM无法按指示决定该信任图像还是该信任先验知识这一问题,拆成两个假设:感知失败(H1,图像信息根本没传进来)和利用失败(H2,信息传进来了但模型无法可控地决定是否使用)。
  2. 他们只用冻结的(未针对该任务训练的)MLLM最后一层图像token来重建图片,再让人工标注员检查诸如动物腿数异常等反事实属性是否在重建图中依然存在。
  3. 他们构建了WhatIfVis基准,包含3049个样本,涵盖五种属性类型,每个反事实图像都配有对应的文字陈述,用于测试模型能否按“只依据图像回答”或“忽略图像依靠常识回答”的指令来行动。
  4. 他们在六个模型上对比了三种干预方式:有监督微调(SFT)、激活patching(替换模型内部某层的计算结果以追踪因果影响的技术),以及一个无需任何指令、只需固定某个内部方向数值就能改变模型行为的引导向量(steering vector)。
Figure 2: The counterfactual attribute survives to the final layer and is visible in the reconstruction. Species identity, pose, structure, and dominant colors are preserved, including the counterfactual attribute (e.g. the anomalous limb count), while high-frequency texture softens.
Figure 2: The counterfactual attribute survives to the final layer and is visible in the reconstruction. Species identity, pose, structure, and dominant colors are preserved, including the counterfactual attribute (e.g. the anomalous limb count), while high-frequency texture softens.
Table 1: Human evaluation of reconstructed images. Across three LLM backbones, the counterfactual attribute (count) is preserved in the vast majority of generated samples. CF denotes counterfactual.
BackboneCF(%)Real(%)
Gemma-4-E2B-IT97.82.2
Qwen2.5VL-3B93.46.6
Qwen3.5-4B97.82.2
Figure 3: Evaluation pipeline for controllable multimodal context sensitivity. We probe a model’s sensitivity to visual and textual contexts separately through targeted instruction following. Given a visual (left) or textual (right) context, the model must answer a question by either strictly grounding its reasoning in the explicit context ("Consider only…") or suppressing the context to recall standard world knowledge ("Ignore…").
Figure 3: Evaluation pipeline for controllable multimodal context sensitivity. We probe a model’s sensitivity to visual and textual contexts separately through targeted instruction following. Given a visual (left) or textual (right) context, the model must answer a question by either strictly grounding its reasoning in the explicit context ("Consider only…") or suppressing the context to recall standard world knowledge ("Ignore…").
Table 3: WhatIfVis composition. The benchmark spans five conflict types across two families, incorporating data from VLindBench (16), ViLP (31), Pixel-vs-Priors (10), VLMsAreBiased (45), ROME (57), and a Generated set (49). Perception tasks require reading directly observable attributes; Perception+Reasoning tasks additionally require inference beyond direct visual features.
FamilyTaskDescriptionSources# Samples
PerceptionSpatial & TemporalIdentify what surrounds, feeds, or characterizes an entity.VLindBench, ViLP, GeneratedVLindBench,ViLP,Generated1,537
VLindBench,
ViLP,
Generated
ColorReport the color of an object shown in a non-canonical hue.Pixel-vs-Priors, ViLP, VLindBenchPixel-vs-Priors,ViLP,VLindBench282
Pixel-vs-Priors,
ViLP,
VLindBench
ReasoningCountCount parts or instances when the depicted number is anomalous.VLMsAreBiased, ViLP, VLind-BenchVLMsAreBiased,ViLP,VLind-Bench390
VLMsAreBiased,
ViLP,
VLind-Bench
SizeJudge relative size when the visual cue inverts the usual ordering.ROME, Pixel-vs-Priors, VLindBenchROME,Pixel-vs-Priors,VLindBench773
ROME,
Pixel-vs-Priors,
VLindBench
WeightJudge relative weight from a balance-scale image with counterfactuals.Generated67
Total3,049
Figure 4: Illustration of the five WhatIfVis tasks across text and vision modalities. Each WhatIfVis example pairs a counterfactual image with a matched text statement asserting the same content. Contextual answers (CTX) are inferred from the visual evidence in the input image, whereas prior answers (PRI) rely on the model’s internal world knowledge. This distinction can reveal the visual context preference of MLLMs.
Figure 4: Illustration of the five WhatIfVis tasks across text and vision modalities. Each WhatIfVis example pairs a counterfactual image with a matched text statement asserting the same content. Contextual answers (CTX) are inferred from the visual evidence in the input image, whereas prior answers (PRI) rely on the model’s internal world knowledge. This distinction can reveal the visual context preference of MLLMs.
Table 4: Reconstruction fidelity extends beyond count to the other coarse attributes. Pixel-level agreement between each reconstruction and its counterfactual reference image on the Color, Size, and Weight subsets.
AttributeBackboneCounterfactual accuracy%SSIM↑PSNR↑LPIPS↓
ColorQwen2.5VL-3B940.68814.390.253
Qwen3.5-4B980.71816.730.179
Gemma-4-E2B980.74918.290.146
SizeQwen2.5VL-3B920.64213.400.308
Qwen3.5-4B960.66615.140.245
Gemma-4-E2B980.72017.290.197
WeightQwen2.5VL-3B920.65616.200.246
Qwen3.5-4B920.64316.040.245
Gemma-4-E2B940.68718.000.189
Figure 5: SFT lifts visual controllability across every model and task. Top row: pair-accuracy per task. Bottom row: the same runs as context-accuracy (x) against prior-accuracy (y); points above the diagonal follow the prior over the image. SFT lifts controllability consistently, with the largest gains on Color and Spatial-Temporal and the smallest on Weight and Count. All pair-accuracies are probabilities unless noted.
Figure 5: SFT lifts visual controllability across every model and task. Top row: pair-accuracy per task. Bottom row: the same runs as context-accuracy (x) against prior-accuracy (y); points above the diagonal follow the prior over the image. SFT lifts controllability consistently, with the largest gains on Color and Spatial-Temporal and the smallest on Weight and Count. All pair-accuracies are probabilities unless noted.
Table 5: The modality gap is a failure to follow visual evidence, not to override it. Per-intent accuracy (probability scorer, macro-averaged over six models) on the image and matched-text channels. PairAcc credits satisfying both intents on the same example. Here we separate its two halves. Following the evidence (context intent) is far harder from an image than from a matched sentence (+19.0 to +30.8 pp), whereas overriding the evidence (prior intent) is nearly identical across channels (≤4.5 pp, and negative under steering). The gap is thus concentrated in the context intent, the model’s ability to follow visual evidence on demand.
Context intent (follow evidence)Prior intent (override evidence)
ConditionImage (%)Text (%)Gap (%)Image (%)Text (%)Gap (%)
Base45.773.9+28.268.969.5+0.6
SFT62.593.3+30.888.492.9+4.5
Steering54.773.7+19.077.674.7−2.9
Figure 6: Activation patching localizes the vision-versus-prior trade-off to a narrow layer band. The trade-off initiates within the patched window (marked) and amplifies sharply in downstream layers. Δ(Answer probability)=S​o​u​r​c​ep​r​o​b​a​b​i​l​i​t​y−T​a​r​g​e​tp​r​o​b​a​b​i​l​i​t​y, which is the larger the better patched.
Figure 6: Activation patching localizes the vision-versus-prior trade-off to a narrow layer band. The trade-off initiates within the patched window (marked) and amplifies sharply in downstream layers. Δ(Answer probability)=S​o​u​r​c​ep​r​o​b​a​b​i​l​i​t​y−T​a​r​g​e​tp​r​o​b​a​b​i​l​i​t​y, which is the larger the better patched.
Table 6: Pair-accuracy (probability scorer), averaged over six models The text control is an instruction-following upper bound, since the statement asserts the counterfactual outright. The Random rank-1 projection row applies a randomly-initialized projection at the same image-channel layers with the same multipliers (no DAS; three seeds, per-model std ≤3.5 pp): it falls below the base (−8.8 pp), confirming the steering lift (+23.0 pp) comes from the learned vector, not the freedom of a rank-1 edit.
ConditionImage (%)Δ vs. base (pp)Text (%)Text−Image (pp)
Baseline14.7—43.428.6
Random rank-1 projection (image)5.9−8.8——
SFT52.7+38.086.333.6
Steering37.7+23.053.315.6
(b) Qwen3.5-9B, pri→ctx, L14–21
(b) Qwen3.5-9B, pri→ctx, L14–21

研究结果

  • 在三个不同系列的模型(Qwen2.5VL-3B、Qwen3.5-4B、Gemma-4-E2B-IT)上,重建图像正确还原反事实腿数的比例分别为93.4%、97.8%、97.8%。
  • 未经任务微调的基础模型在同一样本上同时满足'依据图像'和'忽略图像'两个指令的配对准确率(PairAcc)平均只有14.7%,处于很低水平。
  • 仅用空间-时间这一种属性的数据做微调后,六个模型的平均配对准确率从14.7%提升到52.7%,其中颜色和空间-时间属性接近80%,且这种提升能迁移到其他未训练过的属性类型上。
  • 激活patching在全部六个模型中都将视觉与先验的权衡定位到一小段狭窄的网络层,但具体所在深度因模型架构不同而不同。
  • 在完全不加任何指令的情况下,仅施加一个学习出的引导向量就能把平均配对准确率从14.7%提升到37.7%;而用同样方式施加一个随机初始化的向量反而把准确率降到5.9%,低于基线,证明效果来自这个特定学习到的方向,而非随机干预本身。
(c) Gemma-4-E4B, ctx→pri, L20–25
(c) Gemma-4-E4B, ctx→pri, L20–25

可应用场景

  • 需要按需决定信任图像证据还是背景知识的多模态应用(例如异常检测、事实核查聊天机器人)可以借鉴类似的诊断方法来检查模型的可控性。
  • 在涉及需要推理而非直接读图的属性(如数量统计、重量比较)时,提示设计或结果校验应考虑到模型更容易依赖常识而非图像证据这一倾向。
  • 仅用少量数据(1136个样本)对单一属性做微调就能让控制能力部分迁移到其他属性,这一发现可为在标注资源有限时提升视觉指令遵循能力提供参考。
(d) Gemma-4-E4B, pri→ctx, L20–27
(d) Gemma-4-E4B, pri→ctx, L20–27

局限与待验证事项

  • 图像重建实验和基准测试仅限于颜色、数量、大小、重量、空间-时间等较粗粒度的属性,不涉及纹理等细粒度细节或小物体。
  • 像数量和重量这类需要超越直接视觉信息进行推理的属性,无论是微调还是引导向量带来的提升都比较有限。
  • 引导向量的位置是先在微调后的模型上定位、再应用回原始基础模型的,这不代表基础模型本身就具备可被引导的天然状态。
  • 文本渠道的控制效果始终比图像渠道更可靠,且这一差距随模型规模增大而扩大,但背后原因尚未得到解释,留待未来研究。
  • 研究只覆盖三个模型系列、总共六个模型,这些发现是否适用于其他架构或更大规模的模型尚未得到验证。
Figure 7: An instruction-free steering knob recovers controllability over the vanilla model, and the visual channel stays harder to control than text. Per-model macro pair-accuracy (probability scorer); left: image, right: text. Steering lifts every model; text is uniformly higher (Appendix E).
Figure 7: An instruction-free steering knob recovers controllability over the vanilla model, and the visual channel stays harder to control than text. Per-model macro pair-accuracy (probability scorer); left: image, right: text. Steering lifts every model; text is uniformly higher (Appendix E).

为什么重要

此前人们普遍认为MLLM在这类任务上出错是因为视觉编码器丢失了细节,但这项研究表明视觉信息往往已经保留在模型内部,真正的瓶颈是模型无法可靠地决定何时该信图像、何时该信先验知识。这意味着未来改进的重点应从单纯提升视觉编码器,转向模型内部决定信图像还是信先验的这套决策机制。

Figure 8: Metaquery Framework. Without learnable queries, we adapt the Metaquery to only use the image tokens of the final layers of a MLLM.
Figure 8: Metaquery Framework. Without learnable queries, we adapt the Metaquery to only use the image tokens of the final layers of a MLLM.

本文术语

  • MLLM(多模态大语言模型) · 同时接收图像和文本输入并生成回答的AI模型
  • 激活patching · 将模型某一层的计算结果替换成来自另一次输入的结果,以此测试该层对最终输出的影响程度的分析技术
  • 引导向量(steering vector) · 在模型内部表示中学习出的一个方向,将其固定为某个数值就能在不加任何提示指令的情况下改变模型的行为(比如更信图像还是更信先验知识)
  • PairAcc(配对准确率) · 一种严格的评价指标,要求同一个例子上'依据图像回答'和'忽略图像回答'两个指令都同时被满足才算正确
  • 反事实(counterfactual)图像 · 内容与常识相悖的图像,例如一只有六条腿的动物
Figure 9: The SFT lift reproduces under exact-match at lower absolute values. Per-task pair-accuracy (exact-match scorer, image channel). Orange: baseline (with intent instructions); blue: SFT (LoRA, trained on Spatial-Temporal only; all other tasks are out-of-distribution). The qualitative pattern, with large gains on the perceivable attributes and marginal gains on the reasoning-heavy ones, matches the probability scorer (Figure 5).
Figure 9: The SFT lift reproduces under exact-match at lower absolute values. Per-task pair-accuracy (exact-match scorer, image channel). Orange: baseline (with intent instructions); blue: SFT (LoRA, trained on Spatial-Temporal only; all other tasks are out-of-distribution). The qualitative pattern, with large gains on the perceivable attributes and marginal gains on the reasoning-heavy ones, matches the probability scorer (Figure 5).

无法转载的图表

  • Figure 1: One MLLM, two opposite failures. A single model lacks a stable setpoint on the visual context-sensitivity axis between its parametric prior and the image, giving rise to two opposite failure modes: (Left) Prior-dominant failures, where strong parametric priors overwhelm anomalous visual input; and (Right) Context-dominant failures, where the model over-indexes on visual features even when instructed to ignore them..
在原文中查看图表 →

论文原文摘要（英文）

Multimodal Large Language Models (MLLMs) achieve strong performance by integrating visual inputs with the rich priors of pretrained language models. However, they often fail on vision-centric tasks, especially when visual evidence conflicts with pretrained knowledge. We explore these failures separately using two diagnostic paradigms: (1) probing whether visual information is available, via image reconstruction, and (2) measuring multimodal context sensitivity, the extent to which the model foll

作者 · Jiaang Li

在 arXiv 阅读

最新论文

全部论文 →

METAL LAB 最新报道

图片来源： Jiaang Li et al., arXiv:2607.26326, CC BY 4.0