Super-resolved Land Surface Temperature maps.

What follows is an example of the output of a small neural network that combines the observations of two satellite-borne instruments to obtain a high resolution Land Surface Temperature map with short revisit times.

The ECOSTRESS TIR sensor on board the ISS carries 5 bands in the 8–12 μm region with an effective resolution of 70 m at nadir, providing science objectives such as evapotranspiration, water stress, fire and urban heat.

The MSI (MultiSpectral Instrument) on board Sentinel-2A and Sentinel-2B is a multi-band imaging spectrometer optimized for land and coastal monitoring carrying 13 bands, spanning visible (VIS), near-infrared (NIR), and shortwave infrared (SWIR) with resolutions between 10 m and 60 m.

The following applet shows the original 70 m resolution signal against its reconstructed 10 m resolution inference.

Some years ago, I processed some data for the Metropolitan Adelaide urban heat mapping project 2022 carried out by Airborne Research Australia. My duties consisted mostly in coregistration, geometric and radiometric harmonization, quality control and validation of the data acquired by an airborne microbolometer array instrument producing a set of Land Surface Temperature maps, daytime and nighttime, with an effective resolution between 1 m and 2 m.

Such maps are extremely valuable in the emerging context of climate change. Besides, urban heat islands exhibit strong interactions with the smog and pollution-related atmospheric chemistry, prevalence of respiratory diseases or use of non-renewable energy sources.

However, I don't expect these matters will ever be taken seriously in my region.

This started as a personal project to construct and train a neural network model that feeds on freely available sources to generate comparable maps in terms of resolution with much higher availability and it was, as far as I know, the first of its kind in its day.

It consists of a neural network with separate branches to process local radiometric, high frequency, low frequency and temperature data and fill the gaps based on the observed radiometric and structural properties of the land covers. What follows is its diagram.

The following results correspond to an old version of the algorithm, with fewer training iterations, not to its current state.

The original Metropolitan Adelaide urban heat mapping project 2022 data was used as ground truth.

It is worth noting that its accuracy roughly fell within the margins of most airborne instruments (units are °C).

An improved version of this model was evaluated under the scope of the Green gap Project and is available for the autonomous community of Galicia in Spain.

Now, why does it work?

The training process

I won't go into much detail. This architecture was trained by providing patches of low resolution temperatures and high resolution multispectral imagery that correspond to short times between the passes of both instruments, so the inference of the final high-resolution temperature is informed by the low resolution temperature of the area and the trend observed in the different types of materials.

As the data from the Adelaide UHI 2022 was available, I used a downsampled (70 m) version of the original 1 m data as input and a downsampled version (10 m) as target. The UHI 2022 is constrained to eight different flights (four daytime and four nighttime) and the Adelaide Metropolitan area, so here comes the interesting part.

Once initialized with these very high quality data, I enriched the training dataset with ECOSTRESS' data downsampled to 490 m resolution and a target of the original 70 m resolution, so it learned from the same data, the same 49x increase in information, but at a completely different scale! I have worked with compressive sensing algorithms and observed and read about similar phenomena in other models. The fact that a very simple neural network was able to effectively transfer learning through a curriculum from one scale to another poses an interesting question, which in this case was a lucky guess.

Nature repeats itself, largely consisting, in many domains, of a limited set of scale-invariant features. I find that fascinating.