Multi-Scale Marine Object Detection in Side-Scan Sonar Images Based on BES-YOLO
Abstract
:1. Introduction
- (1)
- Aiming at the characteristics of underwater images with severe noise, the data set is preprocessed using the wavelet transform method to improve the generalization and robustness of the model.
- (2)
- The feature fusion structure of the original model is replaced with Bifpn, which can capture feature information at different scales more efficiently and thus improve detection accuracy.
- (3)
- To enhance the model’s ability to focus on target features, the EMA module is used, which is effective in obtaining clearer multi-scale features.
- (4)
- Finally, we introduce the Shape_IoU loss function, which takes into account the characteristics of the bounding box, such as shape and scale, to make the bounding box regression more accurate, thus improving the accuracy of target detection.
2. Materials and Methods
2.1. Preprocessing
2.2. BES-YOLO
2.2.1. Bifpn
2.2.2. EMA
2.2.3. Shape_IoU
3. Results and Discussion
3.1. Experimental Environment and Datasets
3.2. Evaluation Metrics
3.3. Performance Evaluation
3.4. Ablation Study
3.4.1. Attention Mechanisms
3.4.2. Loss Function
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Zhao, J.; Li, J.; Li, M. Progress and Future Trend of Hydrographic Surveying and Charting. J. Geomat. 2009, 34, 25–27. [Google Scholar]
- Wang, J.; Cao, J.; Lu, B.; He, B. Underwater Target Detection Project Equipment Application and Development Trend. China Water Transport 2016, 11, 43–44. [Google Scholar] [CrossRef]
- Chen, Z.; Wang, Z. Research on Underwater Target Detection Using Side-scan Sonar and Multibeam Sounding System. Hydrogr. Survering Chart. 2013, 33, 51–54. [Google Scholar]
- Wang, J.; Zhou, J. Comprehensive Application of Side-scan Sonar and Multi-beam System in Shipwreck Survey. China Water Transp. 2010, 10, 35–37. [Google Scholar]
- Liu, C. The Comparative Analysis of Multi-beam Sounding System, Side-Scan Sonar and Magnetometer in the Wreck Detection. Master’s Thesis, China University of Geosciences (Beijing), Beijing, China, 2015. [Google Scholar]
- Barngrovver, C.; Althoff, A.; DeGuzman, P.; Kastner, R. A brain computer interface (BCI) for the detection of mine-like objects in sidescan sonar imagery. IEEE J. Ocean. Eng. 2016, 41, 123–138. [Google Scholar] [CrossRef]
- Neupane, D.; Seok, J. A Review on Deep Learning-Based Approaches for Automatic Sonar Target Recognition. Elecronics 2020, 9, 1972. [Google Scholar] [CrossRef]
- Ioup, J.W.; Gendron, M.L.; Bourgeois, B.S.; Ioup, G.E. Wavelet denoising of sidescan sonar images. J. Acoust. Soc. Am. 2001, 110 (Suppl. S5), 2741. [Google Scholar] [CrossRef]
- Yang, D.; Wang, C.; Cheng, C.; Pan, G.; Zhang, F. Semantic Segmentation of Side-Scan Sonar Images with Few Samples. Electronics 2022, 11, 3002. [Google Scholar] [CrossRef]
- Xie, Y.; Bore, N.; Folkesson, J. Bathymetric Reconstruction from Sidescan Sonar with Deep Neural Networks. IEEE J. Ocean. Eng. 2023, 48, 372–383. [Google Scholar] [CrossRef]
- Yang, Z.; Zhao, J.; Zhang, H.; Yu, Y.; Huang, C. A Side-Scan Sonar Image Synthesis Method Based on a Diffusion Model. J. Mar. Sci. Eng. 2023, 11, 1103. [Google Scholar] [CrossRef]
- Tang, Y.; Wang, L.; Jin, S.; Zhao, J.; Huang, C.; Yu, Y. AUV-Based Side-Scan Sonar Real-Time Method for Underwater-Target Detection. J. Mar. Sci. Eng. 2023, 11, 690. [Google Scholar] [CrossRef]
- Ge, Q.; Ruan, F.; Qiao, B.; Zhang, Q.; Zuo, X.; Dang, L. Side-Scan Sonar Image Classification Based on Style Transfer and Pre-Trained Convolutional Neural Networks. Electronics 2021, 10, 1823. [Google Scholar] [CrossRef]
- Peng, C.; Jin, S.; Bian, G.; Cui, Y.; Wang, M. Sample Augmentation Method for Side-Scan Sonar Underwater Target Images Based on CBL-sinGAN. J. Mar. Sci. Eng. 2024, 12, 467. [Google Scholar] [CrossRef]
- Wang, Z.; Guo, J.; Huang, W.; Zhang, S. Side-Scan Sonar Image Segmentation Based on Multi-Channel Fusion Convolution Neural Networks. IEEE Sens. J. 2022, 22, 5911–5928. [Google Scholar] [CrossRef]
- Acosta, G.G.; Villar, S.A. Accumulated CA–CFAR Process in 2-D for Online Object Detection From Sidescan Sonar Data. IEEE J. Ocean. Eng. 2015, 40, 558–569. [Google Scholar] [CrossRef]
- Liu, J.; Pang, Y.; Yan, L.; Zhu, H. An Image Quality Improvement Method in Side-Scan Sonar Based on Deconvolution. Remote Sens. 2023, 15, 4908. [Google Scholar] [CrossRef]
- Tang, Y.; Wang, L.; Li, H.; Bian, H. Side-scan sonar underwater target segmentation using the BHP-UNet. EURASIP J. Adv. Signal Process. 2023, 2023, 76. [Google Scholar] [CrossRef]
- Yu, Y.; Zhao, J.; Gong, Q.; Huang, C.; Zheng, G.; Ma, J. Real-Time Underwater Maritime Object Detection in Side-Scan Sonar Images Based on Transformer-YOLOv5. Remote Sens. 2021, 13, 3555. [Google Scholar] [CrossRef]
- Ge, X.; Wei, N.; Zhou, H. Research on Small Underwater Target Detection Technology Based on Side-scan Sonar. Digit. Ocen Underw. Warf. 2023, 6, 155–161. [Google Scholar] [CrossRef]
- Priyadharsini, R.; Sharmila, T.S.; Rajendran, V. A wavelet transform based contrast enhancement method for underwater acoustic images. Multidimens. Syst. Signal Process. 2018, 29, 1845–1859. [Google Scholar] [CrossRef]
- Koukos, I.; Mavroidis, T.; Vardoulias, G. Two Dimensional Wavelet Coefficient Statistics for Sea Bottom Classification. J. Acoust. Soc. Am. 2008, 123 (Suppl. S5), 3748. [Google Scholar] [CrossRef]
- Atallah, L.; Probert Smith, P.J. Using wavelet analysis to classify and segment sonar signals scattered from underwater sea beds. Int. J. Remote Sens. 2003, 24, 4113–4128. [Google Scholar] [CrossRef]
- Tao, W.-L.; Liu, Y. Edge preserving filter of side scan sonar images with wavelet modulus maxima shift-correlative technique. Int. J. Imaging Syst. Technol. 2011, 21, 349–355. [Google Scholar] [CrossRef]
- Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 23–28 June 2014; pp. 580–587. [Google Scholar]
- Girshick, R. Fast R-CNN. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 7–13 December 2015; pp. 1440–1448. [Google Scholar]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 1137–1149. [Google Scholar] [CrossRef]
- Tang, Y.; Jin, S.; Bian, G.; Zhang, Y.; Li, F. Wreckage Target Recognition in Side-scan Sonar Images Based on an Improved Faster R-CNN Model. In Proceedings of the International Conference on Big Data & Artificial Intelligence & Software Engineering 2020, Bangkok, Thailand, 30 October–1 November 2020; pp. 348–354. [Google Scholar]
- Redmon, J.; Divvala, S.; Girshick, R.; Farhad, A. You Only Look Once: Unified, Real-Time Object Detection. In Proceedings of the Computer Vision & Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016. [Google Scholar]
- Redmon, J.; Farhadi, A. YOLOv3: An Incremental Improvement. arXiv 2018, arXiv:1804.02767. [Google Scholar]
- Yulin, T.; Jin, S.; Bian, G.; Zhang, Y. Shipwreck Target Recognition in Side-Scan Sonar Images by Improved YOLOv3 Model Based on Transfer Learning. IEEE Access 2020, 8, 173450–173460. [Google Scholar] [CrossRef]
- Wang, J.; Wang, Q.; Gao, G.; Qin, P.; He, B. Improving Yolo5 for Real-Time Detection of Small Targets in Side Scan Sonar Images. J. Ocean Univ. China 2023, 22, 1551–1562. [Google Scholar] [CrossRef]
- Tan, M.; Pang, R.; Le, Q.V. EfficientDet: Scalable and Efficient Object Detection. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 10778–10787. [Google Scholar]
- Hu, J.; Shen, L.; Albanie, S.; Sun, G.; Wu, E. Squeeze-and-Excitation Networks. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.; Kweon, I. CBAM: Convolutional Block Attention Module. arXiv 2018, arXiv:1807.06521. [Google Scholar]
- Ouyang, D.; He, S.; Zhan, J.; Guo, H.; Huang, Z.; Luo, M.; Zhang, G. Efficient Multi-Scale Attention Module with Cross-Spatial Learning. In Proceedings of the ICASSP 2023—2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece, 4–10 June 2023; pp. 1–5. [Google Scholar]
- Zhang, H.; Zhang, S. Shape-IoU: More Accurate Metric considering Bounding Box Shape and Scale. arXiv 2023, arXiv:2312.17663. [Google Scholar]
Name | Parameters |
---|---|
Operating system | Windows11 |
CPU | Intel Core i7-14650HX |
GPU | NVIDIA GeForce RTX 4060 Laptop |
CUDA | 11.7 |
Pytorch | 1.13.1 |
Python | 3.8 |
Parameter Name | Parameter Information |
---|---|
lr0 | 0.01 |
momentum | 0.937 |
warm_up | 3 |
batch_size | 16 |
imgsz | 640 |
Detection Network | Human | Boat | Plane | [email protected] |
---|---|---|---|---|
AP | AP | AP | ||
Faster-RCNN | 68.4 | 93.3 | 95.5 | 85.7 |
SSD | 66.2 | 87.6 | 89.7 | 81.2 |
YOLOv5s | 57.7 | 77.3 | 79.1 | 71.4 |
YOLOv7 | 58.1 | 75.2 | 77 | 70.1 |
YOLOv8n | 69.5 | 94.8 | 97 | 87.1 |
BES-YOLO | 82.6 | 95.9 | 98.5 | 92.4 |
Bifpn | EMA | Shape_IoU | [email protected] | [email protected] |
---|---|---|---|---|
87.1 | 63.3 | |||
√ | 91 | 64.2 | ||
√ | 90.6 | 64.4 | ||
√ | 89.8 | 65.8 | ||
√ | √ | 91.5 | 64.5 | |
√ | √ | 91.3 | 65.8 | |
√ | √ | 91.6 | 66.1 | |
√ | √ | √ | 92.4 | 67.7 |
Detection Network | Human | Boat | Plane | [email protected] |
---|---|---|---|---|
AP | AP | AP | ||
YOLOv8 | 69.5 | 94.8 | 97 | 87.1 |
YOLOv8+SE | 72.4 | 93.4 | 97.4 | 87.7 |
YOLOv8+CBAM | 77.8 | 92.5 | 96.8 | 89.1 |
YOLOv8+EMA | 79.6 | 94 | 98.1 | 90.6 |
Detection Network | Human | Boat | Plane | [email protected] |
---|---|---|---|---|
AP | AP | AP | ||
CIoU | 69.5 | 94.8 | 97 | 87.1 |
GIoU | 69.6 | 93.9 | 98.4 | 87.3 |
DIoU | 72.7 | 95.4 | 95 | 88.7 |
Shape_IoU | 76.7 | 94.6 | 98 | 89.8 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ma, Q.; Jin, S.; Bian, G.; Cui, Y. Multi-Scale Marine Object Detection in Side-Scan Sonar Images Based on BES-YOLO. Sensors 2024, 24, 4428. https://doi.org/10.3390/s24144428
Ma Q, Jin S, Bian G, Cui Y. Multi-Scale Marine Object Detection in Side-Scan Sonar Images Based on BES-YOLO. Sensors. 2024; 24(14):4428. https://doi.org/10.3390/s24144428
Chicago/Turabian StyleMa, Quanhong, Shaohua Jin, Gang Bian, and Yang Cui. 2024. "Multi-Scale Marine Object Detection in Side-Scan Sonar Images Based on BES-YOLO" Sensors 24, no. 14: 4428. https://doi.org/10.3390/s24144428