每天早上一封邮件，把昨天的 AI 梳理好订阅邮件

METAL LAB

Towards Real-Time and Adaptable LiDAR Scene Completion

arXiv:2608.164902026-08-16

让自动驾驶汽车激光雷达的盲区在0.1秒内被补全

自动驾驶车上的激光雷达在物体被遮挡或距离较远时只能采集到稀疏且有缺口的点云。RapidLiDAR不再像以往方法那样用随机噪声或固定幅度的扰动来填补缺口,而是直接从数据中学习该把点挪到哪里,从而更好地覆盖大片遮挡区域。在SemanticKITTI和KITTI-360上的实验显示,该方法在补全质量上与现有最好方法相当,同时0.1秒即可完成一整个场景,比此前最快的方法快2.3倍。

他们做了什么

  1. 提出RapidLiDAR,输入稀疏激光雷达点云,通过一次前向传播直接输出补全后的密集三维场景。
  2. 设计自适应初始化模块,为每个输入点预测应该移动的方向和距离,在遮挡或稀疏区域让点移动得更远,在已经密集的区域只做小幅调整。
  3. 增加多尺度重建模块,通过查询多个分辨率的三维体素特征和二维鸟瞰图(BEV)特征来进一步微调点的位置,而无需搜索点的邻域。
  4. 用基于体素和BEV的特征提取替代最远点采样、k近邻搜索等逐点邻域搜索操作,使方法更快,并能天然适应不同的输入分辨率。
  5. 在SemanticKITTI和KITTI-360上,补全质量与现有最好方法相当,同时0.1秒完成一个场景,恰好匹配车载激光雷达每秒10次的采集频率。
Figure 1: Initialization matters. Top row: each method’s initialization; bottom row: the corresponding refined output. The highlighted box marks a large unobserved region. (a) LiDiff [19] starts from Gaussian noise that carries no information about the scene; (b) LiNeXt [6] perturbs the input with a fixed noise variance, so its points stay near the observed surface and never reach across the gap; (c) Our adaptive module learns data-dependent displacements that populate the region from the surrounding geometry, and this coverage is preserved in both the coarse initialization and the final refined result.
Figure 1: Initialization matters. Top row: each method’s initialization; bottom row: the corresponding refined output. The highlighted box marks a large unobserved region. (a) LiDiff [19] starts from Gaussian noise that carries no information about the scene; (b) LiNeXt [6] perturbs the input with a fixed noise variance, so its points stay near the observed surface and never reach across the gap; (c) Our adaptive module learns data-dependent displacements that populate the region from the surrounding geometry, and this coverage is preserved in both the coarse initialization and the final refined result.
Table 1: Scene Completion on SemanticKITTI and KITTI-360. Quantitative comparison with prior methods. † denotes methods with an additional refinement. Best results in each group are highlighted in bold.
SemanticKITTIKITTI-360
MethodCD ↓JSD 3D ↓JSD BEV ↓CD ↓JSD 3D ↓JSD BEV ↓
LMSCNet0.641–0.4310.979–0.496
LODE1.029–0.4511.565–0.483
MID0.503–0.4700.637–0.476
PVD1.256–0.498–––
LiDiff0.4340.5640.4440.564–0.459
LiDPM0.4460.5320.440–––
ScoreLiDAR0.406–0.4250.472–0.444
LiFlow0.309–0.416–––
LiNeXt0.2140.4940.3360.2170.5080.355
Ours0.2060.4750.3320.2110.4920.338
LiDiff†0.3760.5730.4160.517–0.446
ScoreLiDAR†0.342–0.3990.452–0.437
LiDPM†0.3760.5420.403–––
LiNeXt†0.1490.4810.3310.1490.4990.339
Ours†0.1380.4780.3300.1400.4900.336
Figure 2: Overview of RapidLiDAR. The Multi-Scale Feature Extraction module (center) voxelizes the input point cloud X∈ℝM×3 and extracts multi-scale 3D voxel features (F1,F2,…,Fn) and a dense 2D BEV feature map Bdense. The Adaptive Initialization Module (top) predicts a spatially varying displacement Δ for each point in P~ to obtain the initialized scene Pinit. The Multi-Scale Reconstruction Module (bottom) projects voxel features into BEV feature maps and cross-attends per-point features from Pinit with the multi-scale BEV feature maps using multi-scale deformable attention. A final MLP predicts a residual displacement that aligns each point with the underlying target surfaces, producing the completed scene.
Figure 2: Overview of RapidLiDAR. The Multi-Scale Feature Extraction module (center) voxelizes the input point cloud X∈ℝM×3 and extracts multi-scale 3D voxel features (F1,F2,…,Fn) and a dense 2D BEV feature map Bdense. The Adaptive Initialization Module (top) predicts a spatially varying displacement Δ for each point in P~ to obtain the initialized scene Pinit. The Multi-Scale Reconstruction Module (bottom) projects voxel features into BEV feature maps and cross-attends per-point features from Pinit with the multi-scale BEV feature maps using multi-scale deformable attention. A final MLP predicts a residual displacement that aligns each point with the underlying target surfaces, producing the completed scene.
Table 2: Computational Efficiency Comparison. We report Chamfer distance, number of learnable parameters, and inference time per scan.
MethodCD ↓Param (M) ↓Time (s) ↓
LiDiff0.43432.6730.1
ScoreLiDAR0.40632.677.1
LiNeXt0.2141.990.23
Ours0.20611.80.10
Figure 3: Illustration of Dense BEV Head. Converts sparse 3D volumetric features into a dense BEV map. The channel and depth dimensions are merged and projected to Cout using a 2D convolution. Multi-head self-attention over BEV tokens captures global scene context, followed by residual 2D convolutions for feature refinement.
Figure 3: Illustration of Dense BEV Head. Converts sparse 3D volumetric features into a dense BEV map. The channel and depth dimensions are merged and projected to Cout using a 2D convolution. Multi-head self-attention over BEV tokens captures global scene context, followed by residual 2D convolutions for feature refinement.
Table 3: Architectural Ablation Study. We evaluate the impact of our core modules on the SemanticKITTI validation set. Best results are in bold.
MethodCD ↓JSD 3D ↓JSD BEV ↓
Ours0.2060.4750.332
Ours w/o AIM0.2180.4880.345
Ours w/o MSRM0.2150.4940.342
Figure 4: Qualitative Comparison on SemanticKITTI. Our method produces more complete geometry in large occluded regions compared to prior methods.
Figure 4: Qualitative Comparison on SemanticKITTI. Our method produces more complete geometry in large occluded regions compared to prior methods.
Table 4: Effect of Maximum Displacement Bound. Reconstruction performance for different values of Smax, evaluated on a downsampled validation set of 180,000 output points.
SmaxCD ↓
500.2594
700.2592
1000.2589
Table 5: Voxel Resolution Ablation. Impact of voxel resolution η on reconstruction performance, parameter count, and inference time.
η (m)CD ↓Param (M)Time (s) ↓
0.50.21410.00.07
0.40.21011.60.09
0.30.20611.80.10
0.20.20811.90.14

为什么重要

激光雷达场景补全必须实时运行才能被自动驾驶下游任务实际使用,但现有的扩散模型方法太慢,固定噪声方法又需要针对每种新传感器手动重新调参。这项工作同时缓解了这两个问题,让场景补全更接近可以真正装车使用的速度。

本文术语

  • 激光雷达场景补全(LiDAR scene completion) · 把传感器没能观测到的三维场景部分推断补全出来
  • 扩散模型(diffusion model) · 通过多步逐渐去噪把随机噪声变成结果的生成模型,步骤多所以速度慢
  • 鸟瞰图(BEV, Bird's-Eye-View) · 从正上方俯视三维场景得到的二维网格表示
  • 体素(voxel) · 把三维空间划分成的小立方体单元,相当于三维版的像素
  • 倒角距离(Chamfer Distance) · 衡量两组点之间接近程度的指标,用于评估重建的准确度

论文原文摘要（英文）

LiDAR scene completion is a key component of 3D perception in autonomous driving, where the scene must be completed in real time to be usable in downstream tasks. Existing approaches typically follow an initialize-and-refine paradigm, in which a coarse initialization of the scene is first constructed, then refined into complete 3D geometry. Generative models are slower because they iteratively refine random Gaussian noise into the scene, while non-generative methods perturb the partial scene with a fixed noise scale, which limits coverage of large gaps and occluded regions and requires manual recalibration for each new sensor configuration. We present RapidLiDAR, a LiDAR scene completion method that treats the initialization itself as a learned, data-driven component. We propose an adaptive initialization module that predicts a spatially varying displacement for each partial input point, expanding the partial observations into a coarse scene initialization adapted to the local geometry, without requiring manual noise tuning. To refine this coarse initialization into a complete and coherent scene, we additionally propose a multi-scale reconstruction module that further refines point positions by querying multi-scale 3D voxel and 2D BEV feature maps constructed from the input scan. By replacing point-neighborhood operators such as farthest point sampling and k-nearest neighbor search with voxel- and BEV-based feature extraction, our architecture is faster and can handle different input resolutions by design. Experiments on SemanticKITTI and KITTI-360 show that our method achieves completion performance on par with the state of the art while completing a full scene in 0.1 seconds, which is 2.3 times faster than the fastest prior method. This matches the 10 Hz acquisition rate of typical automotive LiDAR sensors, taking a step toward real-time LiDAR scene completion.

作者 · Azhar Hussian

在 arXiv 阅读

最新论文

全部论文 →

METAL LAB 最新报道

图片来源： Azhar Hussian et al., arXiv:2608.16490, cc-by-nc-sa-4.0