How artificial intelligence sharpens blurry thermal vision images

The fuzzy, ghostly figures typically seen in thermal images might become a thing of the past. By pairing artificial intelligence and thermal vision, scientists can make crisp, detailed images — even in the dark. The technique could one day help improve self-driving vehicles’ ability to navigate at night.

Thermal imaging, which is often used in night vision systems, works by detecting heat sources. The infrared images are blurry because of a phenomenon called ghosting. Heat from an object overwhelms any details about the object’s texture, much like turning on a light makes it difficult to decipher any etching on a lightbulb.

Theoretical physicist Fanglin Bao of Purdue University in West Lafayette, Ind., and colleagues used a thermal camera that can distinguish between different wavelengths of infrared light. The researchers paired that camera with a computer program that uses AI to untangle information from the device to reveal the temperature, texture and type of material of objects in an image. This technique painted bright, detailed images out of dark, nighttime scenes, the team reports July 26 in Nature.

“There is no restriction of harsh weather conditions or nighttime scenarios,” says electrical engineer Muhammad Ali Farooq of the University of Galway in Ireland, who was not involved with the study. “You can get very good and very crisp data even in low-lighting conditions.”

The technique can also measure distance with about the same accuracy as current camera-based methods. This means it could be used in self-driving vehicles, which have to know when to brake to avoid causing a wreck (SN: 10/12/18).

Current self-driving vehicles often gauge distance by bouncing signals off objects, similar to how sonar works. Many autonomous cars sending out signals could confuse one another. Because the new technique doesn’t need to send out a signal, it could be safer to scale up in a world with more self-driving cars, the researchers say.  

Even so, the technology won’t be cruising through busy streets any time soon. The camera is hefty, about half a meter on each side and expensive — it costs upward of $1 million, Bao says. And capturing each image takes about a second, which is too slow for a self-driving vehicle that needs to respond to situations in real time.

Still, Bao looks forward to seeing versions of this technology might do for self-driving vehicles or robots in the future. “We humans have an evolutionary bias towards the light,” he says. “But it turns out AI can overcome this long-standing dichotomy between day and night.”