City Thermal Image Datasets Enable Researchers and Developers to Accelerate Localized Testing of Thermal Sensors on Self-Driving Systems

FLIR Systems, Inc., last week announced the creation of the FLIR Thermal Imaging Regional Dataset program for machine learning advanced driver assistance development (ADAS) and autonomous vehicle (AV) systems.

Specific to major cities, FLIR also announced San Francisco as the first available dataset, enabling developers to evolve convolutional neural networks with FLIR’s Autonomous Developer Kit (ADK), a cost-effective, weatherproof thermal camera developed for ADAS and AV testing. Building on a free dataset program FLIR launched in 2018 of more than 14,000 annotated thermal images of day and night scenes, the San Francisco thermal dataset features nearly 10,000 annotated thermal images with 181,000 annotations in thermal and the corresponding visible camera images.

It includes new variations in weather including fog and rain plus additional driving scenes at different hours of the day. With the introduction of city-specific datasets, FLIR also increased the number of annotation classes to include car, sign, light, people, truck, bus, hydrant, bike, rider, motorcycle, and train. FLIR is currently collecting thermal data in other major metropolitan areas, covering common seasonal driving conditions at all hours. Future datasets include several large U.S. and international cities with regional customization to offer accelerated thermal testing.

“Creating datasets takes time and resources, and the datasets FLIR has created empowers the automotive community to more quickly evaluate thermal sensors on next-generation algorithms,” said Frank Pennisi, president of the Industrial Business Unit at FLIR. “When combined with visible light cameras in an Automatic Emergency Braking (AEB) system, the thermal data will create a more comprehensive, redundant, and safer system in cars today. Then in the near future, when thermal is fused with visible, LIDAR, and radar, thermal sensor data, paired with machine learning, this will create a more comprehensive, redundant, and safer system for identifying and classifying roadway objects, especially pedestrians and other living things in an autonomous driving mode. San Francisco is a significant autonomous vehicle development hub, and this data will ultimately allow developers and researchers to create safer vehicle systems.”

With more than a decade of experience in the automotive industry, FLIR thermal sensors are in driver warning systems in vehicles from General Motors, Volkswagen, Audi, Peugeot, BMW and Mercedes-Benz through tier-one auto supplier Veoneer. FLIR thermal cameras have proven reliable in the classification of pedestrians, bicycles, and vehicles in total darkness at nearly four times the distance of a typical vehicle’s headlight range, along with other challenging lighting conditions such as fog, smoke, shadows, inclement weather and sun glare, according to the compoany.

To purchase the San Francisco thermal dataset or to receive information about other city dataset development, visit www.flir.com/dataset.