Comparison of Semantic Convolution Neural Networks on the Example of Crack Segmentation in Asphalt Images

Authors

  • Svetlana Mustafina
  • Andrey Akimov
  • Sofia Mustafina
  • Alexandra Plotnikova

DOI:

https://doi.org/10.47839/ijc.20.3.2288

Keywords:

cracks, defect pavement, Dice, image classification, IoU, LinkNet, PSPNet, road pavement, segmentation, U-Net

Abstract

The article is devoted to a comparative analysis of the effectiveness of convolutional neural networks for semantic segmentation of road surface damage marking. Currently, photo and video surveillance methods are used to control the condition of the road surface. Assessing and analyzing new manual data can take too long. Thus, a completely different procedure is required to inspect and assess the state of controlled objects using technical vision. The authors compared 3 neural networks (Unet, Linknet, PSPNet) used in semantic segmentation using the example of the Crack500 dataset. The proposed architectures have been implemented in the Keras and TensorFlow frameworks. The compared models of neural convolutional networks effectively solve the assigned tasks even with a limited amount of training data. High accuracy is observed. The considered models can be used in various segmentation tasks. The results obtained can be used in the process of modeling, monitoring, and predicting the wear of the road surface.

References

K. Gopalakrishnan, “Deep learning in data-driven pavement image analysis and automated distress detection: A review,” Data, vol. 3, issue 3, article 28, 2018. https://doi.org/10.3390/data3030028.

W. Li, H. Ju, S. Tighe, Q. Ren, Z. Sun, “Three-dimensional pavement crack detection algorithm based on two-dimensional empirical mode decomposition,” J. Transp. Eng., Part B: Pavements, vol. 143, issue 2, 04017005, 2017. https://doi.org/10.1061/JPEODX.0000006.

D. Hwang, D.E. Kim, “Special features on intelligent imaging and analysis,” Appl. Sci, vol. 9, article 4804, 2019. https://doi.org/10.3390/app9224804.

R. Augustauskas, A. Lipnickas, “Improved pixel-level pavement-defect segmentation using a deep autoencoder,” Sensors, vol. 20, issue 9, article 2557, 2020. https://doi.org/10.3390/s20092557.

H. Oliveira, P.L. Correia, “Automatic road crack segmentation using entropy and image dynamic thresholding,” In Proceedings of the European Signal Processing Conference, Glasgow, UK, 2009, pp. 622–626.

M.-R. Jahanshahi, et al., “An innovative methodology for detection and quantification of cracks through incorporation of depth perception,” Machine Vision and Applications, vol. 24, pp. 227–241, 2013. https://doi.org/10.1007/s00138-011-0394-0.

H. Oliveira, P. L. Correia, “CrackIT – An image processing toolbox for crack detection and characterization,” in Proceedings of the IEEE International Conference on Image Processing, Paris, 2014, pp. 798–802, 2014. https://doi.org/10.1109/ICIP.2014.7025160.

R. J. Medina, et al., “Enhanced automatic detection of road surface cracks by combining 2D/3D image processing techniques,” in Proceedings of the IEEE International Conference on Image Processing (ICIP-2014), Paris, pp. 778–782, 2014. https://doi.org/10.1109/ICIP.2014.7025156.

S. Chanda, et al., “Automatic bridge crack detection – A texture analysis-based approach,” Artificial Neural Networks in Pattern Recognition (ANNPR), Lecture Notes in Computer Science, Springer, vol. 8774, 2014, pp. 193–203. https://doi.org/10.1007/978-3-319-11656-3_18.

E. Salari, G. Bao, “Automated pavement distress inspection based on 2D and 3D information,” In Proceedings of the 2011 IEEE International Conference on Electro/Information Technology, Mankato, MN, USA, 2011, pp. 2–5. https://doi.org/10.1109/EIT.2011.5978575.

Y. Hu and C.-X. Zhao, “A local binary pattern based methods for pavement crack detection,” Journal of Pattern Recognition Research, vol. 1, no. 2010, pp. 140–147, 2010. https://doi.org/10.13176/11.167.

V. Baltazart, P. Nicolle, L. Yang, “Ongoing tests and improvements of the MPS algorithm for the automatic crack detection within grey level pavement images,” In Proceedings of the 25th European Signal Processing Conference (EUSIPCO), Kos, Greece, 2017, pp. 2016–2020. https://doi.org/10.23919/EUSIPCO.2017.8081563.

D. Zhang, Q. Zou, H. Lin, X. Xu, L. He, R. Gui, Q. Li, “Automatic pavement defect detection using 3D laser profiling technology,” Automation in Construction, vol. 96, pp. 350-365, 2018. https://doi.org/10.1016/j.autcon.2018.09.019.

Q. Zou, Z. Zhang, Q. Li, X. Qi, Q. Wang, and S. Wang, “Deep crack: Learning hierarchical convolutional features for crack detection,” IEEE Transactions on Image Processing, vol. 28, no. 3, pp. 1498–1512, 2018. https://doi.org/10.1109/TIP.2018.2878966.

B. Varona, A. Monteserin, A. Teyseyre, “A deep learning approach to automatic road surface monitoring and pothole detection,” Pers Ubiquit Comput, vol. 24, pp. 519–534, 2020. https://doi.org/10.1007/s00779-019-01234-z.

O. Ronneberger, P. Fischer, T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” In Proceedings of the Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Munich, Germany, 5–9 October 2015; Springer: Cham, Switzerland, 2015, vol. 9351, pp. 234–241. https://doi.org/10.1007/978-3-319-24574-4_28.

R. Augustauskas, A. Lipnickas, “Improved pixel-level pavement-defect segmentation using a deep autoencoder,” Sensors, vol. 20, 2557, 2020. https://doi.org/10.3390/s20092557.

C. Baur, B. Wiestler, S. Albarqouni, and N. Navab, “Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images,” arXiv preprint arXiv:1804.04488, 2018. https://doi.org/10.1007/978-3-030-11723-8_16.

D. P. Kingma and M. Welling, “Auto-encoding variational Bayes,” Proceedings of the International Conference on Learning Representations, 2014.

N. Navab, J. Hornegger, W. Wells, A. Frangi, “Medical image computing and computer-assisted intervention,” MICCAI 2015, Lecture Notes in Computer Science, vol. 9351, Springer, Cham, 2015. https://doi.org/10.1007/978-3-319-24571-3.

A. Chaurasia and E. Culurciello, “LinkNet: Exploiting encoder representations for efficient semantic segmentation,” Proceedings of the 2017 IEEE, Visual Communications and Image Processing (VCIP), St. Petersburg, FL, USA, 2017, pp. 1-4, 2017. https://doi.org/10.1109/VCIP.2017.8305148.

H. Zhao, J. Shi, X. Qi, X. Wang and J. Jia, “Pyramid scene parsing network,” Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 2017, pp. 6230-6239. https://doi.org/10.1109/CVPR.2017.660

R. Zhang, L. Du, Q. Xiao, J. Liu, “Comparison of backbones for semantic segmentation network,” Journal of Physics: Conference Series, 1544, 2020. https://doi.org/10.1088/1742-6596/1544/1/012196.

F. Yang, L. Zhang, S. Yu, D. Prokhorov, X. Mei, H. Ling, “Feature pyramid and hierarchical boosting network for pavement crack detection,” IEEE Trans. Intell. Transp. Syst., vol. 21, pp. 1525–1535, 2020. https://doi.org/10.1109/TITS.2019.2910595.

D.M.W. Powers, “Ailab evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation,” Inf. Markedness Correl., vol. 2, pp. 37–63, 2011.

H. Rezatofighi, N. Tsoi, J.-Y. Gwak, A. Sadeghian, I. Reid, S. Savarese, “Generalized intersection over union: A metric and a loss for bounding box regression,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 2019, pp. 658-666. https://doi.org/10.1109/CVPR.2019.00075.

ImageNet large scale visual recognition competition 2014 [Online]. Available at: URL: http://imagenet.org/challenges/LSVRC/2014/.

Downloads

Published

2021-09-30

How to Cite

Mustafina, S., Akimov, A., Mustafina, S., & Plotnikova, A. (2021). Comparison of Semantic Convolution Neural Networks on the Example of Crack Segmentation in Asphalt Images. International Journal of Computing, 20(3), 415-423. https://doi.org/10.47839/ijc.20.3.2288

Issue

Section

Articles