Researchers use AI to enable real-time car imaging around corners

Researchers used deep learning to create a new laser-based system that can image around corners in real time. The systems might one day let self-driving cars 'look' around parked cars or busy intersections to not only see cars but also read licence plates. Credit: Felix Heide, Princeton University

Researchers have reportedly harnessed the power of a type of artificial intelligence (AI) known as deep learning to create a new laser-based system that can image around corners in real time. With further development, the system looks to let self-driving cars ‘look’ around parked cars or busy intersections to see hazards or pedestrians. 

“Compared to other approaches, our non-line-of-sight imaging system provides uniquely high resolutions and imaging speeds,” said research team leader Christopher A. Metzler from Stanford University and Rice University. “These attributes enable applications that wouldn’t otherwise be possible, such as reading the licence plate of a hidden car as it is driving, or reading a badge worn by someone walking on the other side of a corner.”

Metzler reported that the new system can distinguish submillimetre details of a hidden object from one metre away. The system is designed to image small objects at very high resolutions but can be combined with other imaging systems that produce low-resolution room-sized reconstructions.

“Non-line-of-sight imaging has important applications in medical imaging, navigation, robotics and defence,” said co-author Felix Heide from Princeton University. “Our work takes a step toward enabling its use in a variety of such applications.”

“Compared to other approaches for non-line-of-sight imaging, our deep learning algorithm is far more robust to noise and thus can operate with much shorter exposure times,” said co-author Prasanna Rangarajan from Southern Methodist University. “By accurately characterising the noise, we were able to synthesise data to train the algorithm to solve the reconstruction problem using deep learning without having to capture costly experimental training data.”

The researchers tested the new technique by reconstructing images of one-centimetre-tall letters and numbers hidden behind a corner using an imaging setup about one metre from the wall. Using an exposure length of a quarter of a second, the approach produced reconstructions with a resolution of 300 microns.

The research is part of DARPA’s Revolutionary Enhancement of Visibility by Exploiting Active Light-fields (REVEAL) programme, which is developing a variety of different techniques to image hidden objects around corners. The researchers are now working to make the system practical for more applications by extending the field of view so that it can reconstruct larger objects.