Harnessing Technology to Monitor Urban Changes in Conflict Zones

In recent years, the convergence of remote sensing technology and machine and deep learning has opened up new possibilities for monitoring urban changes, especially in conflict-affected regions. A groundbreaking study applied this synergy to the Mariupol area of Ukraine, a city severely impacted by the Russo-Ukrainian War that began in February 2022. The study's findings shed light on the potential of continuous high-temporal resolution change detection using inexpensive remote sensing data and the challenges of utilizing these cutting-edge techniques in such scenarios.

Georg Zitzlsberger, the study's main author, emphasized, "One of the challenges of this work was the embargo of data for transfer learning during the war. The work demonstrates that data of older time frames are sufficient for transfer learning, which reduces costs for the data as free Google Earth historical data was used, yet demonstrates its robustness."

Covering the city of Mariupol and its surrounding area of over 536 square kilometers, the study used multi-modal level 1 Sentinel-1 (Synthetic Aperture Radar) and Sentinel-2 (multispectral optical) satellite data to monitor changes across various landscapes, including urban, suburban, rural, farming, mining, and coastal regions. The combination of multiple sensors and the use of minimally processed level 1 data helped to increase the temporal resolution at the cost of more complex processing and increased noise in the data, which imposes more challenges on the applied deep neural network. Georg Zitzlsberger and Michal Podhoranyi preprocessed this data to a half-year windowed time series with their recently published library rsdtlib to enable a fully automatic processing pipeline. This formed the foundation to employ continuous monitoring without manual intervention.

To prepare the deep neural network model for monitoring of the selected region, it had to be transferred once. This required a set of ground truth labels to describe the change patterns of interest to highlight in the subsequent monitoring process. The study leveraged older but freely available Very High-resolution (VHR) historic satellite and aerial imagery from Google Earth to create an approximate ground truth of changes in the Mariupol area between 2017 and 2020. This labeling process only required a few person days, which demonstrates further cost efficiency by using only a small amount of samples. A novel transfer learning approach was applied to retain the localization of changes at a high temporal resolution using a sliding half-year window despite the coarse four-year approximation represented in the ground truth.

The study had two main outcomes. First, it was demonstrated that the novel transfer learning method enables training with an older time frame, like 2017-2020, and yet applies the transferred model to later time frames, i.e., to monitor changes in 2022/23. A classical machine learning strategy was also employed to utilize better the limited amount of ground truth samples known as bagging ensemble. Second, the trained model shows good resiliency to data loss. Even when the frequency of available observations was reduced, the model performed effectively. This finding has important implications for limited data availability, such as satellite outages or even the malfunction of Sentinel 1B in December 2021.

The research article also acknowledged some limitations. Detecting changes smaller than the sensor resolution and the influence of training data selection on model performance were among the challenges highlighted. Also, the localization of changes over time is limited by the window size (half-year). Further research and optimizations are required to reduce the latency to detect changes earlier for other stricter real-time applications than for slower but long-ranging urban change monitoring purposes.

This research demonstrates the power of combining remote sensing data, machine and deep learning, and High-Performance Computing to monitor urban changes. The ability to constantly monitor urban changes holds substantial socio-economic interest and promises to become an invaluable and inexpensive tool to monitor urban changes even in conflict zones.

For more detailed information, you can read the full article, "Monitoring Urban Changes in Mariupol/Ukraine in 2022/2023," which is currently under review, at https://arxiv.org/pdf/2309.08607.pdf. 

You can find more information on IT4Innovations GitHub.