Loading...
Development of artificial-intelligence based autonomous roof fall hazard detection system
Isleyen, Ergin
Isleyen, Ergin
Citations
Altmetric:
Editor
Date
Date Issued
2021
Date Submitted
Keywords
Collections
Research Projects
Organizational Units
Journal Issue
Embargo Expires
2022-06-25
Abstract
Autonomous technologies have the potential to transform mining by improving efficiency, safety, and productivity. The transition to fully autonomous mining requires an interoperable system that is autonomous at all stages of a mining operation including geotechnical hazard management. Deep learning is a viable option to develop autonomous technologies based on the improvements in computing power and sensor technology. However, the success of deep learning development depends on having a large and diverse training set. Geotechnical hazard management requires on-site data collection. The restrictions on on-site data collection such as accessibility, time, and financial constraints cause small and unbalanced datasets that limit the performance of deep learning systems and thereby prevent the increased use of autonomous technologies. This thesis presents a methodology that includes transfer learning, synthetic data generation, and an image-feature based data sampling technique to train a convolutional neural network (CNN) using a small dataset collected on-site. The methodology is implemented in a case study to develop an autonomous roof fall hazard detection system. The case study is a large-opening limestone mine in the Midwestern United States that has frequent roof fall problems caused by high horizontal stresses. The typical hazard management approach for this type of roof fall hazard relies heavily on visual inspections and expert knowledge. Therefore, images depicting hazardous and non-hazardous roof conditions are collected based on expert labeling. Using the transfer learning approach, a pre-trained network provided the suitable low-level features to train the final fully connected layer of a CNN. The network classification accuracy was improved by expanding the training set with synthetic images rendered from a 3-D model generated with digital photogrammetry. Also, a data sampling technique that uses Gabor magnitude responses was introduced. The objective of this technique is to improve the quality of the training set by removing non-informative samples, and it is shown that implementing this technique improves network performance. Besides, accuracy alone is not enough to ensure a reliable hazard management system. System constraints and reliability are improved when the features being used by the network are understood. Therefore, a deep learning interpretation technique called integrated gradients is used to identify the important features in each image for prediction. The analysis of integrated gradients shows that the system the same visual features as the expert on roof fall hazard detection. The final network performance is verified with a test dataset that had not been used during training. The original contribution of this research is a methodology to develop artificial intelligence-based, autonomous, and interpretable geotechnical hazard detection systems by implementing the techniques to overcome limited training data problem. The findings of this thesis demonstrate the potential of deep learning even with small and unbalanced datasets. The results provide a foundation for the increased use of autonomous technologies in situations in which the absence of large and diverse data previously prevented the development of deep learning systems.
Associated Publications
Rights
Copyright of the original work is retained by the author.