The research team from the Innovation Center for Intelligent Connected Electric Vehicles at Shanghai Jiao Tong University has made a significant contribution to the field by publishing a review paper, "Learning-based 3D Reconstruction in Autonomous Driving: A Comprehensive Survey", in the IEEE Transactions on Intelligent Transportation Systems (T-ITS), a premier journal in the intelligent transportation domain. This paper zeroes in on learning-based 3D reconstruction technologies within autonomous driving contexts, offering a systematic overview of their development history and practical significance.

As autonomous driving technology progresses towards achieving Level 4 and Level 5 autonomy, the necessity for highly accurate and reliable environmental perception becomes paramount. Nonetheless, the acquisition of extensive, multimodal training data in real-world settings is fraught with difficulties, including high expenses and considerable safety hazards, especially the scarcity of data for infrequent scenarios. Learning-based 3D reconstruction technologies emerge as a solution to these data limitations by generating highly realistic digital replicas of the physical environment. This approach provides a cost-effective and safe platform for generating training data and testing autonomous driving algorithms.

The paper introduces a novel classification system, dissecting static background reconstruction, traffic agent reconstruction, and dynamic scene spatiotemporal modeling at various levels. It also explores the utilization of 3D reconstruction technologies in pivotal tasks such as data augmentation, multimodal label generation, localization and mapping, scene understanding, and simulation.

Moreover, the study identifies critical challenges, including ensuring the physical realism of weather and lighting modifications, addressing computational limitations for onboard deployment, and conducting quantitative safety evaluations of generated data. It also forecasts future advancements, envisioning a synergy between world models and generative AI to enable controllable scenario generation and interactive closed-loop simulation.