DMF-Net: A Dual-Encoding Multi-Scale Fusion Network for Pavement Crack Detection

Bai, Suli; Yang, Lei; Liu, Yanhong; Yu, Hongnian

Abstract

Currently, cracks are the most common defect in pavement diseases. Long-term non-maintenance can lead to crack lengthening and expansion, causing serious traffic accidents, as well as shortening the service life of pavement cracks. Therefore, it is of utmost importance to maintain cracks at an early stage. Due to the effect of some challenging factors, such as various shape information of the cracks, complex textured backgrounds, light shadows, similar texture objects, micro cracks and other factors, accurate crack detection still faces a certain challenges. To solve the above problems, a dual-encoding multi-scale fusion network based on the combination of convolutional neural network (CNN) and transformer network is proposed, named DMF-Net. To obtain stronger feature representations, a dual-encoding path is built to acquire global context features and local detail information simultaneously, where global context features are extracted based on the transformer branch, and the local detail features are extracted based on the CNN branch to detect tiny details of the cracks. Meanwhile, an interactive attention learning (IAL) module is introduced to effectively fuse the global features from the transformer branch and the local detail information from the CNN branch, achieving mutual communication and learning of different feature information. In addition, to enrich the feature representation ability, an attention-based feature enhancement (AFE) module is introduced to acquire more global contexts. Furthermore, faced with the crack detection task with class imbalance issue, a triple attention module (TAM) is built to emphasize the micro cracks. Finally, in the segmentation prediction stage, the deep supervision mechanism is also introduced to accelerate the convergence speed of the model, and serve effective multi-scale feature fusion. Compared with the current mainstream segmentation models, excellent performance has been obtained, which could provide a feasible scheme ...

Citation

Bai, S., Yang, L., Liu, Y., & Yu, H. (in press). DMF-Net: A Dual-Encoding Multi-Scale Fusion Network for Pavement Crack Detection. IEEE Transactions on Intelligent Transportation Systems, https://doi.org/10.1109/tits.2023.3331769

• Journal Article Type: Article
• Acceptance Date: Nov 7, 2023
• Online Publication Date: Dec 1, 2023
• Deposit Date: Jan 12, 2024
• Journal: IEEE Transactions on Intelligent Transportation Systems
• Print ISSN: 1524-9050
• Electronic ISSN: 1558-0016
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• DOI: https://doi.org/10.1109/tits.2023.3331769
• Keywords: Crack segmentation, transformer, CNN, attention mechanism, multi-scale feature learning
• Public URL: http://researchrepository.napier.ac.uk/Output/3407541