Transfer learning and deep convolutional neural networks for safety guardrail detection in 2D images

Automation in Construction

Volumn 89, Issue , 2018, Pages 58-70

  Kolar, Zdenek ^a   Chen, Hainan ^a   Luo, Xiaowei ^a  

^a CITY UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Computer vision; Construction safety; Convolutional neural networks; Guardrail detection; Transfer learning; VGG 16

Indexed keywords

ACCIDENT PREVENTION; COMPUTER VISION; CONSTRUCTION INDUSTRY; CONVOLUTION; GUARD RAILS; IMAGE PROCESSING; INSPECTION; NEURAL NETWORKS; SAFETY ENGINEERING;

CONSTRUCTION ACCIDENTS; CONSTRUCTION SAFETY; CONVOLUTIONAL NEURAL NETWORK; ENVIRONMENT MONITORING; INTELLIGENT MONITORING; LOCALIZATION TECHNIQUE; TRANSFER LEARNING; VGG-16;

DEEP NEURAL NETWORKS;

EID: 85041454603     PISSN: 09265805     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.autcon.2018.01.003     Document Type: Article

References (46)

1. BLS, Fatal occupational injuries for selected events or exposures. https://www.bls.gov/news.release/cfoi.t02.htm, 2016. (Accessed 10 October 2017)
2. Occupational Safety and Health Administration, Top 10 Most Frequently Cited Standards. https://www.osha.gov/Top_Ten_Standards.html, 2016. (Accessed 10 October 2017)
3. Li, H., Lu, M., Hsu, S.-C., Gray, M., Huang, T., Proactive behavior-based safety management for construction safety improvement. Saf. Sci. 75 (2015), 107–117, 10.1016/j.ssci.2015.01.013.
4. Li, H., Lu, M., Chan, G., Skitmore, M., Proactive training system for safe and efficient precast installation. Autom. Constr. 49 ((2015), 163–174, 10.1016/j.autcon.2014.10.010.
5. Hecker, S., Gambatese, J.A., Safety in design: a proactive approach to construction worker safety and health. Appl. Occup. Environ. Hyg. 18 (2003), 339–342, 10.1080/10473220301369.
6. Garrett, J.W., Teizer, J., Human factors analysis classification system relating to human error awareness taxonomy in construction safety. J. Constr. Eng. Manag. 135 (2009), 754–763, 10.1061/(ASCE)CO.1943-7862.0000034.
7. Hung, Y.H., Winchester, W.W., Smith-Jackson, T.L., Kleiner, B.M., Babski-Reeves, K.L., Mills, T.H., Identifying fall-protection training needs for residential roofing subcontractors. Appl. Ergon. 44 (2013), 372–380, 10.1016/j.apergo.2012.09.007.
8. Chow, Y.Y., Case studies of accidents involving work at height, Singapore. https://www.wshc.sg/files/wshc/upload/infostop/attachments/2017/IS201704210000000417/Case_Studies_of_Accidents_Involving_Working_at_Heights.pdf, 2017. (Accessed 10 October 2017)
9. Carbonari, A., Giretti, A., Naticchia, B., A proactive system for real-time safety management in construction sites. Autom. Constr. 20 (2011), 686–698, 10.1016/j.autcon.2011.04.019.
10. Cheng, T., Teizer, J., Migliaccio, G.C., Gatti, U.C., Automated task-level activity analysis through fusion of real time location sensors and worker's thoracic posture data. Autom. Constr. 29 (2013), 24–39, 10.1016/j.autcon.2012.08.003.
11. Seo, J., Han, S., Lee, S., Kim, H., Computer vision techniques for construction safety and health monitoring. Adv. Eng. Inform. 29 (2015), 239–251, 10.1016/j.aei.2015.02.001.
12. Cheng, T., Teizer, J., Real-time resource location data collection and visualization technology for construction safety and activity monitoring applications. Autom. Constr. 34 (2013), 3–15, 10.1016/j.autcon.2012.10.017.
13. Campbell, R.J., Flynn, P.J., A survey of free-form object representation and recognition techniques. Comput. Vis. Image Underst. 81 (2001), 166–210, 10.1006/cviu.2000.0889.
14. Cheng, G., Han, J., A survey on object detection in optical remote sensing images. ISPRS J. Photogramm. Remote Sens. 117 (2016), 11–28, 10.1016/j.isprsjprs.2016.03.014.
15. Mohan, A., Papageorgiou, C., Poggio, T., Example-based object detection in images by components. IEEE Trans. Pattern Anal. Mach. Intell. 23 (2001), 349–361, 10.1109/34.917571.
16. Park, M., Palinginis, E., Brilakis, I., Detection of construction workers in video frames for automatic initialization of vision trackers. Construction Research Congress, 2012, 10.1061/9780784412329.095.
17. Brilakis, I., Park, M.W., Jog, G., Automated vision tracking of project related entities. Adv. Eng. Inform., 25, 2011, 10.1016/j.aei.2011.01.003.
18. Pətrəucean, V., Armeni, I., Nahangi, M., Yeung, J., Brilakis, I., Haas, C., State of research in automatic as-built modelling. Adv. Eng. Inform. 29 (2015), 162–171, 10.1016/j.aei.2015.01.001.
19. Park, M.W., Brilakis, I., Construction worker detection in video frames for initializing vision trackers. Autom. Constr. 28 (2012), 15–25, 10.1016/j.autcon.2012.06.001.
20. Ham, Y., Han, K.K., Lin, J.J., Golparvar-Fard, M., Visual monitoring of civil infrastructure systems via camera-equipped unmanned aerial vehicles (UAVs): a review of related works. Vis. Eng., 4(1), 2016, 10.1186/s40327-015-0029-z.
21. Teizer, J., Status quo and open challenges in vision-based sensing and tracking of temporary resources on infrastructure construction sites. Adv. Eng. Inform. 29 (2015), 225–238, 10.1016/j.aei.2015.03.006.
22. Yang, J., Shi, Z., Wu, Z., Vision-based action recognition of construction workers using dense trajectories. Adv. Eng. Inform. 30 (2016), 327–336, 10.1016/j.aei.2016.04.009.
23. Viola, P., Jones, M.J., Robust real-time face detection. Int. J. Comput. Vis. 57 (2004), 137–154, 10.1023/B:VISI.0000013087.49260.fb.
24. Du, S., Shehata, M., Badawy, W., Hard hat detection in video sequences based on face features, motion and color information, ICCRD2011–2011 3rd International Conference on Computer Research and Development, 2011, 25–29, 10.1109/ICCRD.2011.5763846.
25. Kim, H., Kim, K., Kim, H., Data-driven scene parsing method for recognizing construction site objects in the whole image. Autom. Constr. 71 (2016), 271–282, 10.1016/j.autcon.2016.08.018.
26. Lowe, D.G., Distinctive image features from scale invariant keypoints. Int. J. Comput. Vis. 60 (2004), 91–11020042, 10.1023/B:VISI.0000029664.99615.94.
27. Li, F.-F., Pietro, P., A Bayesian hierarchical model for learning natural scene categories, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), 2, 2005, 524–531, 10.1109/CVPR.2005.16.
28. Veres, G., Grabner, H., Middleton, L., Van Gool, L., Automatic workflow monitoring in industrial environments. Asian Conference on Computer Vision, 2010, 200–213, 10.1007/978-3-642-19315-6_16.
29. Zhu, Z., Ren, X., Chen, Z., Integrated detection and tracking of workforce and equipment from construction jobsite videos. Autom. Constr. 81 (2017), 161–171, 10.1016/j.autcon.2017.05.005.
30. Park, M.-W., Brilakis, I., Continuous localization of construction workers via integration of detection and tracking. Autom. Constr. 72 (2016), 129–142, 10.1016/j.autcon.2016.08.039.
31. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P., Gradient-based learning applied to document recognition. IEEE, 1998, 10.1109/5.726791.
32. LeCun, Y., Cortes, C., Burges, C.J.C., The MNIST Database of Handwritten Digits. 1998 http://yann.lecun.com/exdb/mnist/. (Accessed 10 January 2018)
33. Ghayoumi, M., A quick review of deep learning in facial expression. J. Commun. Comput., 14, 2017, 10.17265/1548-7709/2017.01.004.
34. John, V., Umetsu, M., Boyali, A., Mita, S., Imanishi, M., Sanma, N., Shibata, S., Real-time hand posture and gesture-based touchless automotive user interface using deep learning. 2017 IEEE Intelligent Vehicles Symposium (IV), IEEE, 2017, 869–874, 10.1109/IVS.2017.7995825.
35. Salehi, S.S.M., Erdogmus, D., Gholipour, A., Auto-context convolutional neural network (Auto-Net) for brain extraction in magnetic resonance imaging. IEEE Trans. Med. Imaging, 2017, 1, 10.1109/TMI.2017.2721362.
36. Zhang, Y., Chan, W., Jaitly, N., Very deep convolutional networks for end-to-end speech recognition, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2017, 4845–4849, 10.1109/ICASSP.2017.7953077.
37. Yeung, H.W.F., Li, J., Chung, Y.Y., Improved performance of face recognition using CNN with constrained triplet loss layer, 2017 International Joint Conference on Neural Networks (IJCNN), IEEE, 2017, 1948–1955, 10.1109/IJCNN.2017.7966089.
38. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L., ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115 (2015), 211–252, 10.1007/s11263-015-0816-y.
39. Siddula, M., Dai, F., Ye, Y., Fan, J., Unsupervised feature learning for objects of interest detection in cluttered construction roof site images. Procedia Eng. 145 (2016), 428–435, 10.1016/j.proeng.2016.04.010.
40. Li, F.-F., Rod, F., Pietro, P., Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories, Conference on Computer Vision and Pattern Recognition Workshop (CVPR 2004), 178, 2004, 10.1109/CVPR.2004.109.
41. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A., The Pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88 (2010), 303–338, 10.1007/s11263-009-0275-4.
42. Liu, X., Liang, W., Wang, Y., Li, S., Pei, M., 3D head pose estimation with convolutional neural network trained on synthetic images. 2016 IEEE International Conference on Image Processing (ICIP), 2016, 1289–1293, 10.1109/ICIP.2016.7532566.
43. Soltani, M.M., Zhu, Z., Hammad, A., Automated annotation for visual recognition of construction resources using synthetic images. Autom. Constr. 62 (2016), 14–23, 10.1016/j.autcon.2015.10.002.
44. Pan, S.J., Yang, Q., A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22 (2010), 1345–1359, 10.1109/TKDE.2009.191.
45. Simonyan, K., Zisserman, A., Very deep convolutional networks for large-scale image recognition, International Conference on Learning Representations (ICRL), 2015, 1–14, 10.1016/j.infsof.2008.09.005.
46. Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R., Improving neural networks by preventing co-adaptation of feature detectors. ArXiv E-Prints, 2012, 1–18 doi:arXiv:1207.0580.