Natural Disaster Management (NDM) is a complex task, requiring a plethora of different methods to facilitate it. Artificial Intelligence (AI) can be instrumental in this cause, by providing quick and reliable solutions to many problems related to NDM, such as fire detection, social media analysis, etc., and by reducing the workload required to be done by humans. For this purpose, AUTH presents the AI for NDM challenge.
AI for NDM challenge: The goal is to develop AI algorithms that demonstrate state-of-the-art performance in NDM-related tasks. To this end, a series of datasets are provided, each created for dealing with a different aspect of NDM in the context of the TEMA project. Besides the datasets, performance goals are also provided. MSc/PhD students/researchers are challenged to develop their own algorithms that surpass the performance goals in any of the provided datasets, as shown in the tables below.
Multilabel Text Emotion Detection
The Mastodon posts dataset comprises of social media posts in Greek from the platform “Mastodon”, spanning the 2023 wildfires in Greece. Each post was annotated internally with Plutchik-8 emotions. Using state-of-the-art methods the following results in multilabel text emotion detection were achieved:
| Sentiment/Metric | Accuracy | ROC-AUC | Macro F1 | Micro F1 |
| Anger | 0.77 | 0.79 | 0.76 | 0.77 |
| Anticipation | 0.71 | 0.75 | 0.69 | 0.69 |
| Disgust | 0.77 | 0.80 | 0.76 | 0.77 |
| Fear | 0.73 | 0.74 | 0.72 | 0.73 |
| Neutral | 0.84 | 0.75 | 0.68 | 0.84 |
| Sadness | 0.74 | 0.73 | 0.64 | 0.74 |
| Surprise | 0.79 | 0.76 | 0.71 | 0.79 |
| Trust | 0.79 | 0.86 | 0.72 | 0.79 |
To get the Mastodon dataset click here
Fire Classification – Segmentation
The Blaze dataset was created to train models on wildfire image classification and burnt area segmentation tasks, with the aim to be used by Unmanned Aerial Vehicles.
For the classification task, there are a total of 5 classes, ‘Burnt’, ‘Half-Burnt’, ’Non-Burnt’, ‘Fire’, ‘Smoke’. A multitude of state-of-the-art models was used to classify the dataset. The classification was done in three episodes where different classes were used:
- EPI, used only the classes ‘Burnt’, ‘Half-Burnt’, ’Non-Burnt’.
- EPII, had the four classes ‘Burnt’, ‘Half-Burnt’, ’Non-Burnt’ and ‘Fire’.
- EPIII, used all the classes ‘Burnt’, ‘Half-Burnt’, ’Non-Burnt’, ‘Fire’ and ‘Smoke’. It is noted that the union of the ‘Fire’ and ‘Smoke’ classes here constitutes the ‘Fire’ class on EPII.
Since the trained models are to be used for real time classification, their speed is of paramount importance. The performance of the various models are shown in the table below:
| Model | EPI (%) | EPII (%) | EPIII (%) | FPS EPI | FPS EPII | FPS EPIII |
| InceptionV3 | 54.36 | 65.92 | 59.22 | 109.086 | 106.55 | 106.99 |
| ResNet101 | 65.55 | 68.92 | 61.01 | 99.99 | 101.03 | 101.42 |
| ResNet50 | 68.01 | 76.90 | 69.43 | 195.97 | 196.91 | 187.54 |
| EficientNetB0 | 77.74 | 82.32 | 78.94 | 140.26 | 143.95 | 139.81 |
| EfficientNetB1 | 77.41 | 85.32 | 82.71 | 103.29 | 105.20 | 99.67 |
| EfficientNetB2 | 78.97 | 84.49 | 80.03 | 106.25 | 101.02 | 100.92 |
| EfficientNetB3 | 81.21 | 83.85 | 80.03 | 91.185 | 87.88 | 88.01 |
For the segmentation task the dataset provides segmentation masks that assign each pixel of an image to the class ‘Burnt’ or ‘Non-Burnt’. The use of state-of-the-art models achieved the following results:
| Model | Mean loU (%) | Non-Burnt area IoU (%) | Burnt area IoU (%) |
| CNN-121 | 74.82 | 74.19 | 75.45 |
| CABiNet | 74.58 | 74.02 | 75.14 |
| PIDNet-S | 73.76 | 72.45 | 75.07 |
| PIDNet-M | 71.27 | 70.29 | 72.24 |
| PIDNet-L | 68.23 | 67.12 | 69.33 |
| UNet++ | 63.51 | 60.07 | 66.96 |
To get the Blaze dataset click here
Flooded Area Segmentation
The Flood Master Database consists of flood images picked from different publicly available datasets, with each image having a normalized segmentation mask. These images where used for the training of models with a train-validation split ratio of 3:1. For the test set, video frames from real flooding scenarios in Greece and Italy were gathered and annotated, in order to test models on real life data.
State-of-the-art methods were employed for this task. Because the models trained are to be used for real time image segmentation, speed is a critical factor of their performance. The results are shown below:
| Greek video (mIoU) | Italian Video (mIoU) | Speed (ms) | FPS | |
| CNN-121 | 81.48% | 83.07% | 9.82 | 101.85 |
| PSPnet | 82.94% | 81.84% | 12 | 79.5 |
To get access to the Flood Master Database click here.