每天早上一封邮件，把昨天的 AI 梳理好订阅邮件

METAL LAB

Time-Series Retrieval for Grounding Multimodal Language Models in Remaining Useful Life

arXiv:2608.192182026-08-21

给AI预测设备寿命时，先展示相似的历史故障案例，预测效果就会更好

这项研究探讨了用能同时理解图像和文字的多模态大语言模型(MLLM)来预测设备（比如飞机发动机）还能用多久才会损坏，也就是剩余使用寿命(RUL)的估计问题。核心思路是先检索出与待测设备当前退化状态相似的历史案例，把它们做成对比图表一起交给AI参考。在C-MAPSS航空发动机退化数据集的FD001子集上的实验显示，比起随机挑选历史案例，用检索方式挑选相似案例能让预测误差更低、结果更稳定。

他们做了什么

  1. 用基于LSTM的编码器把传感器时间序列片段转换成向量，再通过向量检索找出与测试设备最近状态最相似的历史退化片段。
  2. 把检索到的历史轨迹和测试设备的轨迹绘制成同一张对比图，嵌入结构化提示词中，交给三款Gemini系列多模态模型（Flash-Lite、Flash、Pro），让模型同时输出数值化的RUL预测和文字解释。
  3. 在提示词结构保持一致的前提下，重复10次实验，对比'随机选择历史案例'(MLLM+Random)和'检索式选择历史案例'(MLLM+RAG)两种方式。
  4. 三款模型在检索方式下的预测误差都低于随机方式，结果波动也更小，其中能力更强的模型（Pro）从检索中获益更明显，说明模型越强越能有效利用检索到的证据。
  5. 不过即便是表现最好的检索增强方案，准确度依然比不上专门为该任务设计的深度学习模型，且实验只在C-MAPSS里最简单的FD001子集上进行。
Figure 1: Preprocessing workflow applied to the FD001 partition of the C-MAPSS dataset.
Figure 1: Preprocessing workflow applied to the FD001 partition of the C-MAPSS dataset.
Table 1: Overview of the main pipeline modules according to their inputs, transformations, and outputs.
ModuleInputTransformationOutput
Segment encoderNormalized sliding-window segmentEncode with an LSTM autoencoder.Fixed-dimensional embedding.
Retrieval memory builderTraining segments and RUL labelsSample by RUL bin and index embeddings.Balanced vector database.
Query retrieverLast observed test segmentEncode query and perform k-nearest-neighbor search.Top-k historical references.
Trajectory comparison builderQuery trajectory and retrieved referencesAlign trajectories and generate a multisensor plot.Trajectory-comparison image.
Prompt composerComparison image and task instructionsInsert visual evidence into a structured prompt.Multimodal prompt for RUL estimation.
Figure 2: Overview of the time‑series RAG mechanism: (a) offline indexing of encoded training segments; (b) online retrieval using the encoded test segment.
Figure 2: Overview of the time‑series RAG mechanism: (a) offline indexing of encoded training segments; (b) online retrieval using the encoded test segment.
Table 2: Configuration of the learned LSTM encoder used to generate retrieval embeddings.
AspectConfiguration
Input window lengthL=30 cycles
Input variablesSelected sensor features
Encoder typeLSTM encoder
Number of recurrent layers1
Embedding layerLinear projection
Embedding dimensiond=64
Hidden dimension128
Decoder typeFeed-forward head
Training objectiveFinal-step reconstruction
Loss functionMean squared error
OptimizerAdamW
Training epochs200
Batch size128
Learning rate10−3
Weight decay10−5
Embedding normalizationL2 normalization
(b) Query retriever. The last segment of the test trajectory is encoded and used to retrieve the nearest indexed train segments.
(b) Query retriever. The last segment of the test trajectory is encoded and used to retrieve the nearest indexed train segments.
Table 3: FD001 test-set comparison between baseline methods and Gemini-based RAG variants.
BaselinesGemini RAG
MetricMeanRandom predictorLSTM (?, ?)Flash-LiteFlashPro
RMSE41.98 ± 0.0057.19 ± 2.4914.5443.12 ± 2.7232.85 ± 3.3022.69 ± 1.44
MAE36.07 ± 0.0047.53 ± 2.66–32.91 ± 2.1922.47 ± 1.9917.91 ± 1.24
R2-0.021 ± 0.000-0.897 ± 0.165–-0.080 ± 0.1350.370 ± 0.1250.701 ± 0.037
PHM’08 Score20302 ± 0181531 ± 74903322.44127441 ± 6661027712 ± 158301488 ± 323
Figure 3: Multimodal prompt generation and example comparison artifact used for MLLM inference.
Figure 3: Multimodal prompt generation and example comparison artifact used for MLLM inference.

为什么重要

在设备维护场景中，除了数字预测，能否给出可理解的解释同样重要，这项工作展示了多模态AI可以在给出数值预测的同时提供文字说明，具备一定的实用价值。它也说明通用语言模型不需要重新训练，只需检索相关历史案例辅助，就能在专业工业任务上获得提升，这为将此类方法用于其他设备监测问题提供了一个可参考的方向。

(b) Example of the trajectory-comparison image.
(b) Example of the trajectory-comparison image.

本文术语

  • RUL（剩余使用寿命） · 设备或零件在发生故障前还能继续运行的时间估计，用于制定维护计划
  • MLLM（多模态大语言模型） · 能够同时处理图像、文字等多种输入形式并进行推理的AI模型
  • RAG（检索增强生成） · AI在生成答案前先检索相关参考资料，再据此作答的技术
  • C-MAPSS / FD001 · 广泛使用的模拟涡扇发动机退化基准数据集，FD001是其中条件最简单的子集
  • LSTM编码器 · 一种能把随时间变化的传感器序列压缩成单个固定长度数值向量的神经网络结构
Figure 4: Predictive performance over 10 repetitions for the three evaluated Gemini MLLMs under the two inference settings, MLLM+Random and MLLM+RAG.
Figure 4: Predictive performance over 10 repetitions for the three evaluated Gemini MLLMs under the two inference settings, MLLM+Random and MLLM+RAG.

论文原文摘要（英文）

Large language models (LLMs) and agentic AI systems are increasingly being explored for domain-specific maintenance and prognostics tasks, raising the question of whether they can effectively support prognostics and health management (PHM). In this paper, we investigate remaining useful life (RUL) estimation with multimodal large language models (MLLMs) grounded through time-series retrieval. We propose a framework in which historically similar degradation segments are retrieved from the training set and, together with the test trajectory, transformed into a visual comparison artifact that is processed by the MLLM through a structured multimodal prompt. The approach is evaluated on the FD001 partition of the C-MAPSS benchmark under repeated experiments comparing retrieval-based inference against a non-retrieval baseline based on random reference selection. The results show that time-series retrieval consistently improves MLLM-based RUL prediction across the evaluated models, yielding lower error and more stable performance. At the same time, the magnitude of the benefit depends on model capacity, indicating that retrieval is most effective when the underlying MLLM is able to exploit the retrieved evidence. Overall, the study shows that time-series RAG is a promising mechanism for improving multimodal prognostic reasoning, while also highlighting the current limitations of MLLM-based RUL estimation in practical PHM settings.

作者 · Valeriu Dimidov, Rapha\"el Frank

在 arXiv 阅读

最新论文

全部论文 →

METAL LAB 最新报道

图片来源： Valeriu Dimidov et al., arXiv:2608.19218, arxiv-nonexclusive