Smoke Segmentation Improvement Based on Fast Segment Anything Model with YOLOv11 for a Wildfire Monitoring System

Abstract

Forests and wildlife are crucial parts of our ecosystem. Wildfires occurring in dry and hot regions represent a significant threat to these areas, particularly in ASEAN countries during the dry season. While human observers are often employed to detect wildfires, their scarcity and limited availability highlight the need for automated solutions. This study explores the use of machine learning, specifically computer vision, to enhance wildfire detection by segmenting smoke, an approach which potentially gives information regarding the size and the direction of the spread of the smoke, aiding mitigation efforts. We extend prior work by proposing a model to predict the errors and performance of segmentation masks without access to the ground truth, with the aim of facilitating iterative self-improvement of segmentation models. The FireSpot dataset is used to fine-tune a YOLOv11 model to predict bounding boxes of smoke successfully; subsequently, the outputs of this model are used as a prompt to refine a FastSAM model designed to segment the image into a proposed mask containing the smoke. The proposed mask and the corresponding original image are then used to train a machine learning model where the targets are metrics regarding the error rates of the masks. The results show that a gradient boosting model achieves good prediction performance in predicting some error metrics like the IoU (denoted TPP in this paper) between the proposed and actual segmentation masks with an MSE of 0.03 and R2 of 0.46, as well as the proportion of false positives over the union of the proposed and actual masks (denoted FPP in our paper) with an MSE of 0.0002 and R2 of 0.95, while a pre-trained deep learning model fails to learn the distribution, achieving considerably lower performance for IoU with an MSE of 0.05 and R2 of 0.06 and FPP with an MSE of 0.0002 and R2 of -1.15. These findings open the way to future work where the results of the error prediction model can be used as feedback to improve the prompts and hyperparameters of the segmentation model. Copyright © 2025 by SCITEPRESS - Science and Technology Publications, Lda.