A day of ferry trips between Finland and Estonia became some of the most documented voyages in maritime history. Cameras, sensors, radio and satellite navigation receivers and even microphones recorded every instant of the crossings over the Baltic, gathering raw data for a new European Space Agency (ESA) project applying artificial intelligence (AI) to the situational awareness of shipping — as an important step to full autonomy.

The Tallink shipping company’s new 212.2 meter-long Megastar passenger and car ferry was fitted with data-gathering devices for its sailings on the busy stretch of sea between Helsinki and Tallinn. The testing was overseen by a team from the Finnish Geospatial Research Institute (FGI) for an ESA project called Artificial Intelligence / Machine Learning Sensor Fusion for Autonomous Vessel Navigation, or Maritime AI-NAV for short.

“Our aim is to show how AI can be applied to achieve autonomous situational awareness, so that a ship can reliably sense its own environment,” noted Sarang Thombre of FGI. “Such autonomous systems would initially be deployed in support of human crews, for enhanced safety and efficiency, with crewless ships a much longer-term goal. The most experienced human ship captains will have the least trust in any single navigational device but will rather continuously cross-reference between them. Similarly, our autonomous functionality will not be overly reliant on a single data source but combine and verify data from multiple sensors. Having gathered many gigabytes of data during our initial August field campaign, then again in October with more days planned in December, we are applying the results to train and test our data-fusing algorithms. A follow-up seagoing test will then verify their performance in practice.”

The Maritime AI-NAV team plans to employ a variety of sensor types, including satellite navigation receivers — also utilizing Europe’s Galileo system — monocular and stereo cameras, standard radar, LiDAR and an array of microphones, along with Automatic Identification System (AIS) radio signals. These AIS signals transmit position, size and routing information of all vessels above a certain class, as well as fixed infrastructure such as oil rigs or wind turbines.

Dr Thombre adds, “Satellite navigation lets the ship know where it is in the sea, while the other sensors let it know what is around it, which is essential for identifying and avoiding any obstacles. The different data sources operate across a variety of ranges, so radar and AIS provide longer range detection out to the horizon, while cameras and LiDARs come into their own at shorter distances. Plus we had a trio of microphones aboard the Megastar, determining the angle of arrival of sound from other ships. The challenge now is to fully integrate all these sources using machine learning, to build up a holistic picture.”

Maritime AI-NAV is supported through ESA’s Navigation Innovation and Support Programme (NAVISP), working with European industry and academia to develop navigation technology.

FGI is joined in the Maritime AI-NAV consortium by Aalto University’s Sensor Informatics and Medical Technology group and maritime IT startup Fleetrange Ltd.