For the first time in history an image was processed in space using artificial intelligence. The image was processed by the tailored artificial intelligence hardware of HyperScout 2, a miniaturized Earth observation instrument that is developed under the leadership of cosine. The deep neural network algorithm identified the clouds in an image of part of the Earth’s surface. The capability to process images using artificial intelligence on a satellite opens up possibilities for a large number of applications.
The premier on artificial intelligence in space was announced today by Josef Aschbacher, director of he Earth Observation program of the European Space Agency, at the opening of the Phi-Week 2020 symposium. The first HyperScout 2 instrument, carrying the Ф-sat-1 artificial intelligence (AI) experiment, was launched into space on 3 September 2020 from the Guiana Space Center in Kourou. The HyperScout 2 instrument is on board one of the two nanosatellites of the FSSCat mission that monitors sea ice and soil moisture in support of the Copernicus Land and Marine Environment Services, and was made possible by ESA’s InCubed program.
Integrated into the HyperScout 2 instrument is a Myriad 2 Vision Processing Unit (VPU) from Intel. This allows the instrument to process the images with machine learning algorithms, without requiring more power than available on a nanosatellite. The identification of clouds in an image from Earth is a first demonstration of the capabilities of an AI system on board an Earth observation satellite. The AI algorithm is trained on ground using machine learning on synthetic as well as HyperScout data, which includes properties of the image that are invisible to the human eye due to the 45 different color bands in the visible and infrared spectrum. The resulting algorithms are implemented on the dedicated AI hardware to analyze the images extremely efficiently on board.
Applied to Earth – fire and water
The capability to process spectral images in space using AI makes a large range of applications possible. One of the potential applications is wildfire management. High risk fire zones can be mapped, alerts can be triggered and the development and spreading of an eventual wildfire can be monitored.
There are also many possible benefits for agriculture, such as crop yield prediction, indicating how much crop could be harvested and when to harvest. Problem areas can be identified, alerts can be triggered and the situation analyzed. This makes it possible to take action, such as optimizing irrigation, adjusting levels of fertilization and targeted use of minimal levels of pesticides.
Being a state-of-the-art miniaturized instrument, HyperScout 2 can be flown on a small satellite, which can be launched in larger numbers even on a single rocket. By using a constellation consisting of multiple HyperScout instruments, observations can be made several times per week or even per day, so that alerts can be sent and action can be taken in time. This can also be of great benefit for environmental monitoring, water quality, air quality, deforestation and change detection.
cosine is currently organizing the development of several of these applications with partners and clients. ‘’Data that is acquired by the HyperScout 2 instrument can be combined with data from the HyperScout 1 instrument, which has been in orbit for almost 3 years on the GOMX-4B satellite. In addition, a version of the AI algorithms developed for HyperScout 2 can be uploaded to the GPU, the graphic processing unit on HyperScout 1. This presents a unique opportunity for partners to work with cosine to develop applications using distributed AI on both HyperScout 1 and HyperScout 2’’, explains Marco Esposito, business unit manager Remote Sensing at cosine.
cosine is also looking for partners and clients to extend the use of on-board data processing using AI to other instrument lines cosine is developing. These include bathimetry, imaging polarimetry for cloud and aerosol assessment, spectroscopic imaging for air quality, infrared imaging for agricultural monitoring and LIDAR for height mapping of land, vegetation, buildings and water levels.
This world’s first was achieved in cooperation with and support from many partners.
HyperScout 1 was developed by cosine (NL) with consortium partners S&T (NO), TU Delft (NL), VDL (NL) and VITO (BE), with funding through the ESA GSTP program, with contributions from the Dutch, Belgian and Norwegian national space organizations: Netherlands Space Office, BELSPO and Norsk Romsenter.
HyperScout 2 was developed by cosine Remote Sensing (NL) supported by the InCubed and Earth Observation programs of ESA.
The FSSCat consortium consists of DEIMOS Engenharia (PT), Universitat Politècnica de Catalunya (ES), Golbriak Space (ET), Tyvak International (IT) and cosine Remote Sensing (NL).
The Ф-sat-1 consortium comprises cosine Remote Sensing (NL), University of Pisa (IT), Sinergise (SI) and Ubotica (IE), supported by ESA’s Earth Observation Ф-lab and GSTP fly element.
Read more about HyperScout 2 on our website.