매일 아침, 어제의 AI를 한 통으로 정리해 보내드립니다메일로 받아보기

METAL LAB

Towards Real-Time and Adaptable LiDAR Scene Completion

arXiv:2608.164902026-08-16

자율주행차 라이다가 못 본 빈 공간을 0.1초 만에 채우는 방법

자율주행차의 라이다 센서는 가려지거나 멀리 있는 물체를 성글게 찍기 때문에 주변 3D 장면에 구멍이 생긴다. RapidLiDAR는 이 빈 공간을 메우는 시작점 자체를 데이터로부터 학습해, 무작위 노이즈나 고정된 흔들림에 의존하던 기존 방식보다 크게 뚫린 영역도 잘 채운다. SemanticKITTI와 KITTI-360 실험에서 기존 최고 성능과 맞먹는 품질을 유지하면서 한 장면을 0.1초에 완성해 가장 빠른 기존 방법보다 2.3배 빨랐다.

무엇을 했나

  1. 라이다로 찍은 듬성듬성한 점들을 입력받아 빈틈까지 채운 촘촘한 3D 장면을 한 번의 순전파로 만들어내는 RapidLiDAR를 제안했다.
  2. 점마다 얼마나, 어느 방향으로 이동시킬지를 예측하는 적응형 초기화 모듈을 도입해 가려진 영역 주변에서는 더 멀리 퍼지고 이미 촘촘한 영역에서는 조금만 움직이게 했다.
  3. 3D 복셀 특징과 2D 조감도(BEV) 특징을 여러 해상도로 뽑아, 점들을 격자 탐색 없이 참조해 위치를 다시 다듬는 다중 스케일 재구성 모듈을 추가했다.
  4. 점 주변 이웃을 일일이 찾는 기존 연산(최원점 샘플링, k-최근접 이웃 탐색)을 복셀·BEV 기반 특징 추출로 대체해 속도를 높이고 입력 해상도가 달라져도 그대로 동작하게 했다.
  5. SemanticKITTI와 KITTI-360 실험에서 기존 최고 방법과 대등한 완성 품질을 보이면서, 자동차용 라이다의 초당 10회 촬영 속도에 맞춰 한 장면을 0.1초에 완성했다.
Figure 1: Initialization matters. Top row: each method’s initialization; bottom row: the corresponding refined output. The highlighted box marks a large unobserved region. (a) LiDiff [19] starts from Gaussian noise that carries no information about the scene; (b) LiNeXt [6] perturbs the input with a fixed noise variance, so its points stay near the observed surface and never reach across the gap; (c) Our adaptive module learns data-dependent displacements that populate the region from the surrounding geometry, and this coverage is preserved in both the coarse initialization and the final refined result.
Figure 1: Initialization matters. Top row: each method’s initialization; bottom row: the corresponding refined output. The highlighted box marks a large unobserved region. (a) LiDiff [19] starts from Gaussian noise that carries no information about the scene; (b) LiNeXt [6] perturbs the input with a fixed noise variance, so its points stay near the observed surface and never reach across the gap; (c) Our adaptive module learns data-dependent displacements that populate the region from the surrounding geometry, and this coverage is preserved in both the coarse initialization and the final refined result.
Table 1: Scene Completion on SemanticKITTI and KITTI-360. Quantitative comparison with prior methods. † denotes methods with an additional refinement. Best results in each group are highlighted in bold.
SemanticKITTIKITTI-360
MethodCD ↓JSD 3D ↓JSD BEV ↓CD ↓JSD 3D ↓JSD BEV ↓
LMSCNet0.641–0.4310.979–0.496
LODE1.029–0.4511.565–0.483
MID0.503–0.4700.637–0.476
PVD1.256–0.498–––
LiDiff0.4340.5640.4440.564–0.459
LiDPM0.4460.5320.440–––
ScoreLiDAR0.406–0.4250.472–0.444
LiFlow0.309–0.416–––
LiNeXt0.2140.4940.3360.2170.5080.355
Ours0.2060.4750.3320.2110.4920.338
LiDiff†0.3760.5730.4160.517–0.446
ScoreLiDAR†0.342–0.3990.452–0.437
LiDPM†0.3760.5420.403–––
LiNeXt†0.1490.4810.3310.1490.4990.339
Ours†0.1380.4780.3300.1400.4900.336
Figure 2: Overview of RapidLiDAR. The Multi-Scale Feature Extraction module (center) voxelizes the input point cloud X∈ℝM×3 and extracts multi-scale 3D voxel features (F1,F2,…,Fn) and a dense 2D BEV feature map Bdense. The Adaptive Initialization Module (top) predicts a spatially varying displacement Δ for each point in P~ to obtain the initialized scene Pinit. The Multi-Scale Reconstruction Module (bottom) projects voxel features into BEV feature maps and cross-attends per-point features from Pinit with the multi-scale BEV feature maps using multi-scale deformable attention. A final MLP predicts a residual displacement that aligns each point with the underlying target surfaces, producing the completed scene.
Figure 2: Overview of RapidLiDAR. The Multi-Scale Feature Extraction module (center) voxelizes the input point cloud X∈ℝM×3 and extracts multi-scale 3D voxel features (F1,F2,…,Fn) and a dense 2D BEV feature map Bdense. The Adaptive Initialization Module (top) predicts a spatially varying displacement Δ for each point in P~ to obtain the initialized scene Pinit. The Multi-Scale Reconstruction Module (bottom) projects voxel features into BEV feature maps and cross-attends per-point features from Pinit with the multi-scale BEV feature maps using multi-scale deformable attention. A final MLP predicts a residual displacement that aligns each point with the underlying target surfaces, producing the completed scene.
Table 2: Computational Efficiency Comparison. We report Chamfer distance, number of learnable parameters, and inference time per scan.
MethodCD ↓Param (M) ↓Time (s) ↓
LiDiff0.43432.6730.1
ScoreLiDAR0.40632.677.1
LiNeXt0.2141.990.23
Ours0.20611.80.10
Figure 3: Illustration of Dense BEV Head. Converts sparse 3D volumetric features into a dense BEV map. The channel and depth dimensions are merged and projected to Cout using a 2D convolution. Multi-head self-attention over BEV tokens captures global scene context, followed by residual 2D convolutions for feature refinement.
Figure 3: Illustration of Dense BEV Head. Converts sparse 3D volumetric features into a dense BEV map. The channel and depth dimensions are merged and projected to Cout using a 2D convolution. Multi-head self-attention over BEV tokens captures global scene context, followed by residual 2D convolutions for feature refinement.
Table 3: Architectural Ablation Study. We evaluate the impact of our core modules on the SemanticKITTI validation set. Best results are in bold.
MethodCD ↓JSD 3D ↓JSD BEV ↓
Ours0.2060.4750.332
Ours w/o AIM0.2180.4880.345
Ours w/o MSRM0.2150.4940.342
Figure 4: Qualitative Comparison on SemanticKITTI. Our method produces more complete geometry in large occluded regions compared to prior methods.
Figure 4: Qualitative Comparison on SemanticKITTI. Our method produces more complete geometry in large occluded regions compared to prior methods.
Table 4: Effect of Maximum Displacement Bound. Reconstruction performance for different values of Smax, evaluated on a downsampled validation set of 180,000 output points.
SmaxCD ↓
500.2594
700.2592
1000.2589
Table 5: Voxel Resolution Ablation. Impact of voxel resolution η on reconstruction performance, parameter count, and inference time.
η (m)CD ↓Param (M)Time (s) ↓
0.50.21410.00.07
0.40.21011.60.09
0.30.20611.80.10
0.20.20811.90.14

왜 중요한가

라이다 장면 완성이 실시간으로 돌아가야 자율주행차가 실제로 활용할 수 있는데, 기존 확산 모델 기반 방법은 느리고 고정 노이즈 방식은 센서가 바뀔 때마다 손으로 다시 조정해야 했다. 이 연구는 그 두 문제를 동시에 줄여 실제 차량에 탑재 가능한 속도의 장면 완성에 한 걸음 다가섰다.

이 논문의 용어

  • 라이다 장면 완성(LiDAR scene completion) · 센서가 못 본 빈 공간까지 채워서 완전한 3D 장면을 만드는 작업
  • 확산 모델(diffusion model) · 무작위 노이즈를 여러 단계에 걸쳐 서서히 원하는 결과로 바꾸는 생성 모델, 단계가 많아 느리다
  • 조감도(BEV, Bird's-Eye-View) · 3D 장면을 위에서 내려다본 2D 격자 형태로 표현한 것
  • 복셀(voxel) · 3D 공간을 작은 정육면체 칸으로 나눈 단위, 픽셀의 3차원 버전
  • 챔퍼 거리(Chamfer Distance) · 두 점 집합 사이의 거리를 재는 지표로, 예측한 점들과 정답 점들이 서로 얼마나 가까운지 측정한다

논문 원문 초록 (영문)

LiDAR scene completion is a key component of 3D perception in autonomous driving, where the scene must be completed in real time to be usable in downstream tasks. Existing approaches typically follow an initialize-and-refine paradigm, in which a coarse initialization of the scene is first constructed, then refined into complete 3D geometry. Generative models are slower because they iteratively refine random Gaussian noise into the scene, while non-generative methods perturb the partial scene with a fixed noise scale, which limits coverage of large gaps and occluded regions and requires manual recalibration for each new sensor configuration. We present RapidLiDAR, a LiDAR scene completion method that treats the initialization itself as a learned, data-driven component. We propose an adaptive initialization module that predicts a spatially varying displacement for each partial input point, expanding the partial observations into a coarse scene initialization adapted to the local geometry, without requiring manual noise tuning. To refine this coarse initialization into a complete and coherent scene, we additionally propose a multi-scale reconstruction module that further refines point positions by querying multi-scale 3D voxel and 2D BEV feature maps constructed from the input scan. By replacing point-neighborhood operators such as farthest point sampling and k-nearest neighbor search with voxel- and BEV-based feature extraction, our architecture is faster and can handle different input resolutions by design. Experiments on SemanticKITTI and KITTI-360 show that our method achieves completion performance on par with the state of the art while completing a full scene in 0.1 seconds, which is 2.3 times faster than the fastest prior method. This matches the 10 Hz acquisition rate of typical automotive LiDAR sensors, taking a step toward real-time LiDAR scene completion.

저자 · Azhar Hussian

arXiv에서 원문 보기

최신 논문

논문 전체 보기 →

METAL LAB 최신 기사

그림 출처: Azhar Hussian et al., arXiv:2608.16490, cc-by-nc-sa-4.0