AN ENHANCED WORD LEVEL ARABIC OCR BASED ON DUAL ENCODER TRANSFORMER ARCHITECTURE

(Received: 16-Jun.-2025, Revised: 12-Jul.-2025 and 12-Aug.-2025 , Accepted: 13-Sep.-2025)

Authors Khulood Gaashan, Maram Bani Younes,

Keywords #Arabic OCR #Multi-batch size #Transformer #Dual encoder transformer #Decoder #Feature extraction #Self-attention mechanism

Abstract Arabic script is one of the most sophisticated and difficult scripts. It uses different shapes of characters with complex diacritical marks that are difficult to distinguish from the dots of characters. This script’s distinctive features make the Optical Character Recognition (OCR) procedure more challenging and result in low-accuracy recognition. Different studies have aimed to introduce high-accuracy Arabic OCR in the literature. However, enhancing the accuracy of reading the words has been an open issue that depends on the used dataset and the developed recognition model. Besides, considering diacritics has been limited and not sufficiently addressed. Experimental tests on words with diacritics in prior models have shown bad accuracy that does not exceed 60%. Consequently, this work aims to introduce a new, accurate deep-learning model for Arabic OCR that considers words with and without diacritical marks. It utilizes a dual encoder transformer (DTrOCR), a deep-learning architecture that has demonstrated superior performance in identification and classification tasks. The proposed DTrOCR creates multi-batch sizes. It has been trained using a comprehensive, generated Arabic word-based dataset named MFSRHRD and tested on unseen datasets. The accuracy of configuring Arabic words without diacritics reaches 98.5%. However, for words with diacritics, it achieved an accuracy of 89.9%.

References

[1] A. Wasfy et al., "QARI-OCR: High-fidelity Arabic Text Recognition through Multimodal Large Language Model Adaptation," arXiv preprint, arXiv: 2506.02295, 2025.

[2] A. Waly, B. Tarek, A. Feteha, R. Yehia, G. Amr, W. Gomaa and A. Fares, "Invizo: Arabic HandwrittenDocument Optical Character Recognition Solution," arXiv preprint, arXiv: 2502.05277, 2025.

[3] A. Chan, A. Mijar, M. Saeed, C.-W. Wong and A. Khater, "HATFormer: Historic Handwritten ArabicText Recognition with Transformers," arXiv preprint, arXiv: 2410.02179, 2025.

[4] M. Dahbali, N. Aboutabit and N. Lamghari, "A Hybrid Model for Arabic Script Recognition Based onCNN-CBAM and BLSTM," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 10, no. 3, pp. 294–305, DOI: 10.5455/jjcit.71- 1709571516, Sep. 2024.

[5] S. Raminedi, S. Shridevi and D. Won, "Multi-modal Transformer Architecture for Medical ImageAnalysis and Automated Report Generation," Scientific Reports, vol. 14, no. 1, p. 19281, 2024.

[6] K. Gaashan and M. B. Younes, "Deep Learning-based Arabic Optical Character Recognition: A NewComprehensive Dataset at Character and Word Levels," Proc. of the 2024 15th IEEE Int. Conf. on Information and Communication Systems (ICICS), pp. 1–6, Irbid, Jordan, 2024.

[7] R. Najam and S. Faizullah, "Analysis of Recent Deep Learning Techniques for Arabic Handwritten-textOCR and Post-OCR Correction," Applied Sciences, vol. 13, no. 13, p. 7568, 2023.

[8] A. Mortadi et al., "ALNASIKH: An Arabic OCR System Based on Transformers," Proc. of 2023 IEEEInt. Mobile, Intelligent and Ubiquitous Computing Conf. (MIUCC), pp. 74–81, Cairo, Egypt, 2023.

[9] S. Faizullah, M. S. Ayub, S. Hussain and M. A. Khan, "A Survey of OCR in Arabic Language:Applications, Techniques and Challenges," Applied Sciences, vol. 13, no. 7, p. 4584, 2023.

[10] S. Alghyaline, "Arabic Optical Character Recognition: A Review," Computer Modeling in Engineering& Sciences, vol. 135, no. 3, pp. 1825–1861, 2023.

[11] R. E. Shtaiwi, G. A. Abandah and S. A. Sawalhah, "End-to-End Machine Learning Solution forRecognizing Handwritten Arabic Documents," Proc. of the 2022 13th IEEE Int. Conf. on Information and Communication Systems (ICICS), pp. 180–185, Irbid, Jordan, 2022.

[12] S. Momeni et al., "Arabic Offline Handwritten Text Recognition with Transformers," Research Square,DOI: 10.21203/rs.3.rs-2300065/v1, 2022.

[13] S. K. Jemni et al., "Enhance to Read Better: A Multitask Adversarial Network for HandwrittenDocument Image Enhancement," Pattern Recognition, vol. 123, p. 108370, 2022.

[14] M. Eltay et al., "Generative Adversarial Network-based Adaptive Data Augmentation for HandwrittenArabic Text Recognition," PeerJ Computer Science, vol. 8, p. e861, 2022.

[15] M. Boualam et al., "Arabic Handwriting Word Recognition Based on Convolutional Recurrent NeuralNetwork," Proc. of the 6th Int. Conf. on Wireless Technologies, Embedded and Intelligent Systems (WITS 2020), pp. 877–885, Springer, 2022.

[16] Y. M. Alwaqfi, M. Mohamad and A. T. Al-Taani, "Generative Adversarial Network for an ImprovedArabic Handwritten Characters Recognition," Int. Journal of Advances in Soft Computing & Its Applications, vol. 14, no. 1, pp. 176-195, 2022.

[17] A. Mostafa et al., "OCFormer: A Transformer-based Model for Arabic Handwritten Text Recognition,"Proc. of the 2021 IEEE Int. Mobile, Intelligent and Ubiquitous Computing Conference (MIUCC), pp. 182–186, Cairo, Egypt, 2021.

[18] A. T. Al-Taani and S. T. Ahmad, "Recognition of Arabic Handwritten Characters Using ResidualNeural Networks," JJCIT, vol. 7, no. 2, pp. 192–205, DOI: 10.5455/jjcit.71-1615204606, Jun. 2021.

[19] R. S. Alkhawaldeh, "Arabic (Indian) Digit Handwritten Recognition Using Recurrent Transfer DeepArchitecture," Soft Computing, vol. 25, no. 4, pp. 3131–3141, 2021.

[20] R. Yacouby and D. Axman, "Probabilistic Extension of Precision, Recall and F1-score for Morethorough Evaluation of Classification Models," Proc. of the 1st Workshop on Evaluation and Comparison of NLP Systems, pp. 79–91, DOI: 10.18653/v1/2020.eval4nlp-1.9, 2020.

[21] Z. Noubigh et al., "Transfer Learning to Improve Arabic Handwriting Text Recognition," Proc. of the2020 21st IEEE Int. Arab Conf. on Information Technology (ACIT), IEEE, pp. 1–6, Giza, Egypt, 2020.

[22] M. Fasha, B. Hammo, N. Obeid and J. AlWidian, "A Hybrid Deep Learning Model for Arabic TextRecognition," Int. Journal of Advanced Computer Science and Applications, vol. 11, no. 8, 2020.

[23] A. Vaswani et al., "Attention Is All You Need," Advances in Neural Information Processing Systems,vol. 30, 2017.

[24] G. A. Abandah et al., "Challenges and Pre-processing Recommendations for MADCAT Dataset ofHandwritten Arabic Documents," Proc. of the 2018 IEEE 11th Int. Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pp. 1–9, Beijing, China, 2018.

[25] S. Faizullah et al., "A Survey of OCR in Arabic Language: Applications, Techniques and Challenges," Proc. of the 2015 IEEE Int. Conf. on Communication, Networks and Satellite (COMNESTAT), vol. 13, pp. 111–114, 2015.

[26] S. A. Mahmoud et al., "KHATT: Arabic Offline Handwritten Text Database," Int. Workshop onFrontiers in Handwriting Recognition (IWFHR), Bari, Italy, pp. 449–454, 2012.

[27] A. AbdelRaouf, C. A. Higgins, T. Pridmore and M. Khalil, "Building a Multi-modal Arabic Corpus(MMAC)," Int. Journal on Document Analysis and Recognition, vol. 13, no. 4, pp. 285–302, Dec. 2010.

[28] F. Slimane, R. Ingold, S. Kanoun, A. M. Alimi and J. Hennebert, "A New Arabic Printed Text ImageDatabase and Evaluation Protocols," Proc. of the 2009 IEEE 10th Int. Conf. on Document Analysis and Recognition, pp. 946–950, Barcelona, Spain, 2009.

[29] N. Ben Amara et al., "ARABASE: A Relational Database for Arabic OCR Systems," International ArabJournal of Information Technology, vol. 2, pp. 259–266, 2005.

[30] M. Pechwitz et al., "IFN/ENIT-database of Handwritten Arabic Words," Proc. of Francophone Int.Conf. on Writing and Document (CIFED), vol. 2, Citeseer, pp. 127–136, Hammamet, Tunisia, 2002.