ARABIC SIGN LANGUAGE CHARACTERS RECOGNITION BASED ON A DEEP LEARNING APPROACH AND A SIMPLE LINEAR CLASSIFIER

(Received: 27-Apr .- 2020, Revised: 22 -Jun. -2020 , Accepted: 1 8-Jul. -2020)

Authors Ahmad Hasasneh,

Keywords #Arabic sign language #S ign language recognition #D eep belief network #Softmax regression #C lassification

Abstract One of the best ways of communication between deaf people and hearing people is based on sign language or so-called hand gestures. In the Arab society, only deaf people and specialists could deal with Arabic sign language, which makes the deaf community narrow and thus communicating with normal people dif ficult. In addition to that, studying the problem of Arabic sign language recognition (ArSLR) has been paid attention recently, which emphasizes the necessity of investigating other approaches for such a problem. This paper proposes a novel ArSLR scheme based on an unsupervised deep learning algorithm, a deep belief network (DBN) coupled with a direct use of tiny images , which has been used to recognize and classify Arabic alphabetical letters. The use of deep learning contributed to extracting the most important features that are sparsely represented and played an important role in simplifying the overall recognition task. In total, around 6,000 samples of the 28 Arabic alphabetic signs have been used after resizing and normalization for feature extraction. The classification process was investigated using a softmax regression and achieved an overall accuracy of 83. 32%, showing high reliability of the DBN-based Arabic alphabetical character recognition model. This model also achieved a sensitivity and a specificity of 70.5% and 96.2% , respectively.

References

[1] M. Mohandes, M. Deriche and J. Liu, "Image-based and Sensor-based Approaches to Arabic Sign Language Recognition," IEEE Trans. Human-Machine Syst., vol. 44, no. 4, pp. 551–557, 2014.

[2] "Palestinian Central Bureau of Statistics," [Online], Available: http://www.pcbs.gov.ps/site/lang__ar/507/default.aspx#.

[3] M. A. Abdel-Fattah, "Arabic Sign Language: A Perspective," Journal of Deaf Studies and Deaf Education, vol. 10, no. 2, pp. 212–221, 2005.

[4] A. Tiwari, A. B. Narayan and O. Dikshit, "Deep Larning Networks for Selection of Persistent Scatterer Pixels in Multi-temporal SAR Interferometric Processing," arXiv Prepr. arXiv …, 2019.

[5] R. Naoum, H. Owaied and S. Joudeh, "Development of A New Arabic Sign Language Recognition Using k-nearest Neighbor Algorithm," J. Emerg. Trends Comput. Inf. Sci., vol. 3, no. 8, pp. 1173–1178, 2012.

[6] N. Saliza et al., "Sign Language to Voice Recognition: Hand Detection Techniques for Vision-based Approach, " Current Developments in Technology-assisted Education, vol. 422, pp. 967-972, 2006.

[7] M. A. Mohandes, "Recognition of Two-Handed Arabic Signs Using the CyberGlove," Arab. J. Sci. Eng., vol. 38, no. 3, pp. 669–677, 2013.

[8] A. Hasasneh, E. Frenoux and P. Tarroux, "Semantic Place Recognition Based on Deep Belief Networks and Tiny Images," Proceedings of the 9th International Conference on Informatics in Control, Automation and Robotics (ICINCO-2012), vol. 2, pp. 236–241, 2012.

[9] A. Hasasneh, N. Kampel, P. Sripad and N. J. Shah, "Deep Learning Approach for Automatic Classification of Ocular and Cardiac Artifacts in MEG Data," vol. 2018, pp. 15–17, 2018.

[10] A. Hasasneh, Y.-A. Daraghmi and N. M. Hasasneh, "Towards Accurate Real-Time Traffic Sign Recognition Based on Unsupervised Deep Learning of Spatial Sparse Features: A Perspective," Int. J. Comput. Inf. Technol., vol. 06, no. 01, pp. 32–36, 2017.

[11] L. Liu et al., "Deep Learning for Generic Object Detection: A Survey," International Journal of Computer Vision, vol. 128, no. 2, pp. 261–318, Feb. 2020.

[12] J. Cornejo and H. Pedrini, "Bimodal Emotion Recognition Based on Audio and Facial Parts Using Deep Convolutional Neural Networks," Proceedings of the 18th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 111–117, Dec. 2019.

[13] Y. C. Wong, L. J. Choi, R. S. S. Singh, H. Zhang and A. R. Syafeeza, "Deep Learning-based Racing Bib Number Detection and Recognition," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 5, no. 3, pp. 181–194, Dec. 2019.

[14] M. Elleuch, N. Tagougui and M. Kherallah, "Optimization of DBN Using Regularization Methods Applied for Recognizing Arabic Handwritten Scripts," Procedia Computer Science, vol. 108, pp. 2292–2297, Jan. 2017.

[15] C. Wan, "Research on Computer Information Retrieval Based on Deep Learning," IOP Conference Series: Materials Science and Engineering, vol. 677, no. 3, DOI: 10.1088/1757-899X/677/3/032101 2019.

[16] Z. Zhao, S. Wang, C. Jia, X. Zhang, S. Ma and J. Yang, "Light Field Image Compression Based on Deep Learning," Proceedings of IEEE International Conference on Multimedia and Expo (ICME), vol. 2018-July, San Diego, CA, USA, Oct. 2018.

[17] A. M. Riad, H. K. Elmonier, S. M. Shohieb and A. S. Asem, "SignsWorld: Deeping into the Silence World and Hearing Its Signs (State-of-the-Art), " Int. J. Comput. Sci. Inf. Technol., vol. 4, no. 1, pp. 189–208, Feb. 2012.

[18] R. Anderson, F. Wiryana et al., "Sign Language Recognition Application Systems for Deaf-Mute People: A Review Based on Input-Process-Output," Procedia Computer Science, vol. 116, pp. 441–448, 2017.

[19] M. Mohandes, M. Deriche and J. Liu, "Image-based and Sensor-based Approaches to Arabic Sign Language Recognition," IEEE Trans. Human-Machine Syst., vol. 44, no. 4, pp. 551–557, 2014.

[20] P. K. Vijay, N. Suhas, C. S. Chandrashekhar and D. K. Dhananjay, "Recent Developments in Sign Language Recognition: A Review," Int. J. Adv. Comput. Eng. Commun. Technol., pp. 21–26, 2012.

[21] N. Tubaiz, T. Shanableh and K. Assaleh, "Glove-based Continuous Arabic Sign Language Recognition in User-Dependent Mode," IEEE Trans. Human-Machine Syst., vol. 45, no. 4, pp. 526–533, Aug. 2015.

[22] P. Kumar, H. Gauba and P. Roy, "A Multimodal Framework for Sensor Based Sign Language Recognition," Neurocomputing, vol. 259, pp. 21–38, 2017.

[23] H. M. Zawbaa, A. Ella Hassanien, K. Nakamatsu, N. El-Bendary and M. S. Daoud, "ArSLAT: Arabic Sign Language Alphabet Translator, " Int. J. Comput. Inf. Syst. Ind. Manag. Appl., vol. 3, pp. 498–506, 2011.

[24] M. Deriche, S. Aliyu and M. Mohandes, "An Intelligent Arabic Sign Language Recognition System Using a Pair of LMCs with GMM Based Classification," IEEE Sens. J., vol. 19, no. 18, pp. 1–12, 2019.

[25] A. A. I. Sidig, H. Luqman and S. A. Mahmoud, "Transform-based Arabic Sign Language Recognition," Procedia Computer Science, vol. 117, pp. 2–9, Jan. 2017.

[26] N. B. Ibrahim, M. M. Selim and H. H. Zayed, "An Automatic Arabic Sign Language Recognition System (ArSLRS)," J. King Saud Univ. - Comput. Inf. Sci., vol. 30, no. 4, pp. 470–477, Oct. 2018.

[27] S. Hayani, M. Benaddy, O. El Meslouhi and M. Kardouchi, "Arab Sign Language Recognition with Convolutional Neural Networks," Proceedings of International Conference of Computer Science and Renewable Energies (ICCSRE 2019), DOI: 10.1109/ICCSRE.2019.8807586, Agadir, Morocco, Jul. 2019.

[28] S. Shohieb and A. M. Riad, "Dynamic Hand Gesture Recognition: Real Time vs. Offline Recognition Using Kinect," Advances in Computer Science and Engineering, vol. 11, pp. 39–47, 2013.

[29] R. Alzohairi, R. Alghonaim, W. Alshehri, S. Aloqeely, M. Alzaidan and O. Bchir, "Image-based Arabic Sign Language Recognition System," Int. J. Adv. Comput. Sci. Appl., vol. 9, no. 3, pp. 185–194, 2018.

[30] A. Addin I. Sidig, H. Luqman and S. A. Mahmoud, "Arabic Sign Language Recognition Using Optical Flow-based Features and HMM," Proc. of International Conference of Reliable Information and Communication Technology (IRICT), pp. 297–305, 2018.

[31] M. Elbadawy, A. S. Elons, H. A. Shedeed and M. F. Tolba, "Arabic Sign Language Recognition with 3D Convolutional Neural Networks," Proc. of the 8th IEEE International Conference on Intelligent Computing and Information Systems (ICICIS 2017), vol. 2018-January, pp. 66–71, Cairo, Egypt, 2017.

[32] M. Mohandes, "Arabic Sign Language Recognition," Proc. of Int. Conf. Imaging Sci. Syst. Technol., vol. 1, pp. 753–759, Las Vegas, Nevada, USA, 2001.

[33] M. Maraqa, F. Al-Zboun, M. Dhyabat and R. A. Zitar, "Recognition of Arabic Sign Language (ArSL) Using Recurrent Neural Networks," J. Intell. Learn. Syst. Appl., vol. 04, no. 01, pp. 41–52, 2012.

[34] M. Maraqa and R. Abu-Zaiter, "Recognition of Arabic Sign Language (ArSL) Using Recurrent Neural Networks," Proc. of the 1st International Conference on the Applications of Digital Information and Web Technologies (ICADIWT 2008), pp. 478–481, Ostrava, Czech Republic, 2008.

[35] A. Hasasneh and S. Taqatqa, "Towards Arabic Alphabet and Number Sign Language Recognition," Global Journal of Computer Science and Technology, vol. 17, no. 2, 2017.

[36] G. E. Hinton and R. R. Salakhutdinov, "Reducing the Dimensionality of Data with Neural Networks," Science, vol. 313, no. 5786, pp. 504–507, Jul. 2006.

[37] A. Hasasneh, Robot Semantic Place Recognition Based on Deep Belief Networks and a Direct Use of Tiny Images, Ph.D. Diss., Paris SUD Univ., vol. 2, 2012.

[38] A. Tharwat et al., "SIFT-based Arabic Sign Language Recognition System," Proc. of Afro-European Conference for Industrial Advancement, Springer, Cham, vol. 334, pp. 359–370, 2015.