DEEP LEARNING-BASED RACING BIB NUMBER DETECTION AND RECOGNITION

(Received: 2019-07-10, Revised: 2019-08-27 , Accepted: 2019-08-28)

Authors Yan Chiew Wong, Li Jia Choi, Ranjit Singh Sarban Singh, Haoyu Zhang, A. R., Syafeeza,

Keywords #Racing Bib Number #You Only Look Once Version 3 #Convolutional Recurrent Neural Network.

Abstract Healthy lifestyle trends are getting more prominent around the world. There are numerous numbers of marathon running race events that have been held and inspired interest among peoples of different ages, genders and countries. Such diversified truths increase more difficulties to comprehending large numbers of marathon images, since such process is often done manually. Therefore, a new approach for racing bib number (RBN) localization and recognition for marathon running races using deep learning is proposed in this paper. Previously, all RBN application systems have been developed by using image processing techniques only, which limits the performance achieved. There are two phases in the proposed system that are phase 1: RBN detection and phase 2: RBN recognition. In phase 1, You Only Look Once version 3 (YOLOv3) which consists of a single convolutional network is used to predict the runner and RBN by multiple bounding boxes and class probabilities of those boxes.YOLOv3 is a new classifier network that outperforms other state-of-art networks. In phase 2, Convolutional Recurrent Neural Network (CRNN) is used to generate a label sequence for each input image and then select the label sequence that has the highest probability. CRNN can be straight trained from sequence labels such as words without any annotation of characters. Therefore, CRNN recognizes the contents of RBN detected. The experimental results based on mean average precision (mAP) and edit distance have been analyzed. The developed system is suitable for marathon or distance running race events and automates the localization and recognition of racers, thereby increasing efficiency in event control and monitoring as well as post-processing the event data.

References

[1] S. Roy, P. Shivakumara, P. Mondal, R. Raghavendra, U. Pal and T. Lu, "A New Multi-modal Technique for Bib Number/Text Detection in Natural Images," Proceedings of Pacific Rim Conference on Multimedia, pp. 483-494, 2015.

[2] K. S. Younis, "Arabic Handwritten Character Recognition Based on Deep Convolutional Neural Networks," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 3, no. 3, pp. 186-200, 2017.

[3] P. Shivakumara, R. Raghavendra, L. Qin, K. B. Raja and T. Lu, "A New Multi-modal Approach to Bib Number / Text Detection and Recognition in Marathon Images," Pattern Recognition, vol. 61, pp. 479– 491, 2017.

[4] T. Basha, S. Avidan and I. Ben-Ami, "Racing Bib Number Recognition," Br. Mach. Vis. Conf. (BMVC), pp. 1–10, 2012.

[5] B. Epshtein, "Detecting Text in Natural Scenes with Stroke Width Transform," Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2963–2970, 2010.

[6] F. Su and H. Xu, "Robust Seed-based Stroke Width Transform for Text Detection in Natural Images," Proc. of the 13th Int. Conf. Doc. Anal. Recognit., pp. 916–920, 2015.

[7] R. Smith, "An Overview of the Tesseract OCR Engine," Proc. of the 9th International Conference on Document Analysis and Recognition, pp. 629-633, 2005

[8] N. Boonsim, "Racing Bib Number Localization on Complex Backgrounds," WSEAS Transactions on Systems and Control, vol. 13, pp. 226–231, 2018.

[9] Q. Ye and D. Doermann, "Text Detection and Recognition in Imagery : A Survey," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 7, pp. 1480–1500, 2015.

[10] Y. Zheng and A. R. O. Theory, "Edge Detection Methods in Digital Image Processing," Proc. of the 5th Int. Conf. Comput. Sci. Educ., pp. 471-473, 2010.

[11] Celsius, "Delaware Running Festival 2018,"[Online], Available: https://pic2go.nascent- works.com/c0ab85474c3597a8dcd1a3d95e917516,[Accessed: 15-Apr-2019].

[12] J. Redmon, S. Divvala, R. Girshick and A. Farhadi, "You Only Look Once: Unified, Real-Time Object Detection," IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 779-788, 2016.

[13] J. Redmon, A. Farhadi and C. Ap, "YOLOv3 : An Incremental Improvement,"[Online], Available: https://arxiv.org/abs/1804.02767, 2018.

[14] OpenCV, "AI Courses by OpenCV,"[Online], Available: https://opencv.org/[Accessed: 15-Apr-2019]

[15] B. Shi, X. Bai and C. Yao, "An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 11, pp. 2298–2304, 2016

[16] N. Aloysius, "A Review on Deep Convolutional Neural Networks," Proc. of Int. Conf. Commun. Signal Process., pp. 588–592, 2017.

[17] H. Li, P. Wang and C. Shen, "Towards End-to-end Text Spotting with Convolutional Recurrent Neural Networks," IEEE Int. Conf. Comput. Vis., no. 2, pp. 5248–5256, 2017.

[18] A. Graves, M. Liwicki, S. Ferna, R. Bertolami and H. Bunke, "A Novel Connectionist System for Unconstrained Handwriting Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 5, pp. 855–868, 2009.

[19] A. Graves and S. Fern, "Connectionist Temporal Classification : Labelling Unsegmented Sequence Data with Recurrent Neural Networks," Proceedings of the 23rd International Conference on Machine Learning, pp. 369-376, 2006.

[20] Y. C. Wong and Y. Q. Lee, "Design and Development of Deep Learning Convolutional Neural Network on a Field Programmable Gate Array," Journal of Telecommunication, Electronic and Computer Engineering, vol. 10, no. 4, pp. 25-29, 2018.

[21] T. Vo, T. Nguyen and C. T. Le, "Race Recognition Using Deep Convolutional Neural Network," Symmetry, vol. 10, no. 11, 564, 2018.

,abstract={Healthy lifestyle trends are getting more prominent around the world. There are numerous numbers of marathon running race events that have been held and inspired interest among peoples of different ages, genders and countries. Such diversified truths increase more difficulties to comprehending large numbers of marathon images, since such process is often done manually. Therefore, a new approach for racing bib number (RBN) localization and recognition for marathon running races using deep learning is proposed in this paper. Previously, all RBN application systems have been developed by using image processing techniques only, which limits the performance achieved. There are two phases in the proposed system that are phase 1: RBN detection and phase 2: RBN recognition. In phase 1, You Only Look Once version 3 (YOLOv3) which consists of a single convolutional network is used to predict the runner and RBN by multiple bounding boxes and class probabilities of those boxes.YOLOv3 is a new classifier network that outperforms other state-of-art networks. In phase 2, Convolutional Recurrent Neural Network (CRNN) is used to generate a label sequence for each input image and then select the label sequence that has the highest probability. CRNN can be straight trained from sequence labels such as words without any annotation of characters. Therefore, CRNN recognizes the contents of RBN detected. The experimental results based on mean average precision (mAP) and edit distance have been analyzed. The developed system is suitable for marathon or distance running race events and automates the localization and recognition of racers, thereby increasing efficiency in event control and monitoring as well as post-processing the event data.},
keywords={Racing Bib Number,You Only Look Once Version 3,Convolutional Recurrent Neural Network.},
ISSN={2413-9351},
month={December}}