CAN THE COMBINATION OF FACIAL FEATURES ENHANCE THE PERFORMANCE OF FACE RECOGNITION?

(Received: 18-Jul.-2023, Revised: 7-Sep.-2023 and 3-Oct.-2023 , Accepted: 3-Oct.-2023)

Authors Djellab Issam, Laimeche Lakhdar, Redjimi Mohamed,

Keywords #Classifier combination #Deep learning #Ensemble CNN #Face recognition #Machine learning

Abstract In recent years, researchers have investigated into various approaches of data combination for face recognition, opening up a novel path of exploration aimed at enhancing recognition reliability by capitalizing on the synergy inherent in diverse data sources. This paper implements a comprehensive comparison between two combination methods based on the score-level and feature-level combination, to determine which method highly improves the overall system performance. In the initial method called Fusion-based Classifier Combination (FCC), we introduce a new fusion rule based on score-level combination. This novel model comprises three classifiers; each trained utilizing well-established feature extraction techniques: Local Binary Patterns (LBP), Histogram of Oriented Gradients (HOG) and Compact Binary Facial Descriptors (CBFD). Instead of adhering to conventional combination rules, such as majority vote or maximum scores, the derived scores from each classifier are merged and then trained using a Multi-Layer Perceptron (MLP) classifier to reach the final decision. In the subsequent method, named Sequential CNN deep learning-based face recognition (S-CNN), we extract high-level features from multiple image regions considered as sequential data, employing an ensemble of Convolutional Neural Networks (CNNs). In this scheme, the fully connected layers of each CNN-based image region are combined and fed into a Deep Neural Network (DNN) tailored for facial recognition. The experimental results obtained from well-known face datasets, including Labeled Faces in the Wild (LFW), Olivetti Research Laboratory (ORL) and IARPA Janus Benchmark-C (IJB-C) highlight the competitive performance of both the proposed multi-classifier combination model and the S-CNN deep-learning model when compared to state-of-the-art methods.

References

[1] I. Bendib, A. Meraoumia, M. Y. Haouam et al., "A New Cancelable Deep Biometric Feature Using Chaotic Maps," Pattern Recognition and Image Analysis, vol. 32, no. 1, pp. 109–128, 2022.

[2] M. Y. Haouam, A. Meraoumia, L. Laimeche and I. Bendib, "S-DCTNet: Security-oriented Biometric Feature Extraction Technique: An Effective Pathway to Secure and Reliable Biometric Systems," Multimedia Tools and Applications, vol. 80, pp. 36059–36091, 2021.

[3] R. Rameswari, K. S. Naveen, A. M. Abishek and C. Deepak, "Automated Access Control System Using Face Recognition," Materials Today: Proceedings, vol. 45, Part 2, pp. 1251-1256, 2021.

[4] N. Singhal, V.Ganganwar, M. Yadav, A. Chauhan, M. Jakhar and K. Sharma, "Comparative Study of Machine Learning and Deep Learning Algorithm for Face Recognition," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 07, no. 03, pp. 313-325, September 2021.

[5] Z. Wanling and X .Shijun, "Face Anti-spoofing Detection Based on DWT-LBP-DCT Features," Signal Processing: Image Communication, vol. 89, p. 115990, DOI: 10.1016/j.image.2020.115990, 2020.

[6] S. Liangliang, W. Xia and S. Yongliang, "Research on 3D Face Recognition Method Based on LBP and SVM," Optik, vol. 220, p. 165157, DOI: 10.1016/j.ijleo.2020.165157, 2020.

[7] K. Weiyi., Y. Zhisheng and L. Xuebin, "3D Face Recognition Algorithm Based on Deep Laplacian Pyramid under the Normalization of Epidemic Control," Computer Communications, vol. 199, pp. 30-41, 2023.

[8] D. Mamieva, A.B. Abdusalomov, M. Mukhiddinov and T. K. Whangbo, "Improved Face Detection Method via Learning Small Faces on Hard Images Based on a Deep Learning Approach," Sensors, vol. 23, no. 1, p. 502, DOI: 10.3390/s23010502, 2023.

[9] G.-S. J. Hsu, H. -Y. Wu, C.-H. Tsai, S. Yanushkevich and M. L. Gavrilova, "Masked Face Recognition from Synthesis to Reality," IEEE Access, vol. 10, pp. 37938-37952, 2022.

[10] A. S. Sanchez-Moreno et al., "Efficient Face Recognition System for Operating in Unconstrained Environments," Journal of Imaging, vol. 7, no. 9, p. 161, 2021.

[11] L. Zhou, H. Wang, S. Lin. et al., "Face Recognition Based on Local Binary Pattern and Improved Pairwise-constrained Multiple Metric Learning," Multimedia Tools Application, vol. 79, pp. 675–691, DOI: 10.1007/s11042-019-08157-0, 2020.

[12] J. Tang, Q. Su, B. Su, S. Fong, W. Cao and X. Gong, "Parallel Ensemble Learning of Convolutional Neural Networks and Local Binary Patterns for Face Recognition," Computer Methods and Programs in Biomedicine, vol. 197, p. 105622, DOI: 10.1016/j.cmpb.2020.105622, 2020.

[13] J. Deng, J. Guo, N. Xue and S. Zafeiriou, "ArcFace: Additive Angular Margin Loss for Deep Face Recognition," Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 4690-4699, Long Beach, USA, 2019.

[14] Z. Mortezaie and H. Hassanpour, "A Survey on Age-invariant Face Recognition Methods," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 05, no. 02, pp. 87 - 96, August 2019.

[15] M.A. Muqeet and R. S. Holambe, "Local Binary Patterns Based on Directional Wavelet Transform for Expression and Pose-invariant Face Recognition," Applied Computing and Informatics, vol. 15, no. 2, pp. 163-171, 2019.

[16] H. Yu, J. Zhao and Y. Zhu, "Research on Face Recognition Method Based on Deep Learning," Proc. of the 12th Int. Congress on Image and Signal Processing, Biomedical Engineering and Informatics (CISP-BMEI), Suzhou, China, 2019, pp. 1-5, 2019.

[17] G. Storey, R. Jiang, S. Keogh, A. Bouridane and C. Li, "DPalsyNet: A Facial Palsy Grading and Motion Recognition Framework Using Fully 3D Convolutional Neural Networks," IEEE Access, vol. 7, pp. 121655-121664, 2019.

[18] J. Kong, M. Chen, M. Jiang, J. Sun and J. Hou, "Face Recognition Based on CSGF (2D) 2PCANet," IEEE Access, vol. 6, pp. 45153-45165, 2018.

[19] Z. H. Zhou and J. Feng, "Deep Forest: Towards an Alternative to Deep Neural Networks," Proc. of the 26th Int. Joint Conf. on Artificial Intel. (IJCAI-17), pp. 3553-3559, DOI: 10.24963/ijcai.2017/497, 2017.

[20] M. Ramgopal et al., "Masked Facial Recognition in Security Systems Using Transfer Learning," SN Computer Science, vol. 4, Article no. 27, DOI: 10.1007/s42979-022-01400-w, 2023.

[21] D. Samai, A. Meraoumia, H. Bendjenna and L. Laimeche, "Oriented Local Binary Pattern (LBP????): A New Scheme for an Efficient Feature Extraction Technique," Proc. of the IEEE Int. Conf. on Mathematics and Inf. Techn. (ICMIT), DOI: 10.1109/MATHIT.2017.8259710, Adrar, Algeria, 2017.

[22] B. Berkant, "Implementation of Hog Edge Detection Algorithm Onfpga’s," Procedia - Social and Behavioral Sciences, Vol. 174, pp. 1567-1575, DOI: 10.1016/j.sbspro.2015.01.806, 2015.

[23] L. Jiwen et al., "Learning Compact Binary Face Descriptor for Face Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), vol. 37, no. 10, pp. 2041-2056, 2015.

[24] T. Devries, K. Biswaranjan and G. W. Taylor, "Age Estimation and Face Verification across Aging Using Landmarks," Proc. of the IEEE Canadian Conf. on Computer and Robot Vision, pp. 98–103, Montreal, Canada, 2014.

[25] S. Liu., X. Li, C. Hu et al., "Spammer Detection Using Multi-classifier Information Fusion Based on Evidential Reasoning Rule," Scientific Reports, vol. 12, Article no. 12458, DOI: 10.1038/s41598-022-16576-7, 2022.

[26] G. B Huang et al., "Labeled Faces in the Wild: A Dataset for Studying Face Recognition in Unconstrained Environments," Proc. of Workshop on Faces in ‘Real-Life’ Images: Detection, Alignment and Recognition, [Online], Available: http://vis-www.cs.umass.edu/lfw/, 2008.

[27] F. S. Samaria and A. C. Harter, "Parameterization of a Stochastic Model for Human Face Identification," Proc. of IEEE Workshop on Applications of Computer Vision, pp. 138-142, Sarasota, USA, 1994.

[28] B. Maze, J. Adams, J. A. Duncan et al., "IARPA Janus Benchmark-C: Face Dataset and Protocol," Proc. of the 2018 IEEE Int. Conf. on Biometrics (ICB), pp. 158-165, Gold Coast, Australia, 2018.

[29] H. Wang, Y. Wang, Z. Zhou, X. Ji, D. Gong, J. Zhou, Z. Li and W. Liu, "CosFace: Large Margin Cosine Loss for Deep Face Recognition," Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 5265–5274, DOI 10.1109/CVPR.2018.00552, 2018.

[30] W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj and L. Song, "SphereFace: Deep Hypersphere Embedding for Face Recognition," Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 212–220, arXiv: 1704.08063, 2017.

[31] M. Kim, A. K. Jain and X. Liu, "AdaFace: Quality Adaptive Margin for Face Recognition," Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR2022), pp.18729-18738, [Online], Available: https://openaccess.thecvf.com/content/CVPR2022/papers/Kim_AdaFace_Quality_Adaptive_Margin_for_Face_Recognition_CVPR_2022_paper.pdf, 2022.

Abstract: In recent years, researchers have investigated into various approaches of data combination for face recognition, opening up a novel path of exploration aimed at enhancing recognition reliability by capitalizing on the synergy inherent in diverse data sources. This paper implements a comprehensive comparison between two combination methods based on the score-level and feature-level combination, to determine which method highly improves the overall system performance. In the initial method called Fusion-based Classifier Combination (FCC), we introduce a new fusion rule based on score-level combination. This novel model comprises three classifiers; each trained utilizing well-established feature extraction techniques: Local Binary Patterns (LBP), Histogram of Oriented Gradients (HOG) and Compact Binary Facial Descriptors (CBFD). Instead of adhering to conventional combination rules, such as majority vote or maximum scores, the derived scores from each classifier are merged and then trained using a Multi-Layer Perceptron (MLP) classifier to reach the final decision. In the subsequent method, named Sequential CNN deep learning-based face recognition (S-CNN), we extract high-level features from multiple image regions considered as sequential data, employing an ensemble of Convolutional Neural Networks (CNNs). In this scheme, the fully connected layers of each CNN-based image region are combined and fed into a Deep Neural Network (DNN) tailored for facial recognition. The experimental results obtained from well-known face datasets, including Labeled Faces in the Wild (LFW), Olivetti Research Laboratory (ORL) and IARPA Janus Benchmark-C (IJB-C) highlight the competitive performance of both the proposed multi-classifier combination model and the S-CNN deep-learning model when compared to state-of-the-art methods.
URL: https://jjcit.org/paper/206

,abstract={In recent years, researchers have investigated into various approaches of data combination for face recognition, opening up a novel path of exploration aimed at enhancing recognition reliability by capitalizing on the synergy inherent in diverse data sources. This paper implements a comprehensive comparison between two combination methods based on the score-level and feature-level combination, to determine which method highly improves the overall system performance. In the initial method called Fusion-based Classifier Combination (FCC), we introduce a new fusion rule based on score-level combination. This novel model comprises three classifiers; each trained utilizing well-established feature extraction techniques: Local Binary Patterns (LBP), Histogram of Oriented Gradients (HOG) and Compact Binary Facial Descriptors (CBFD). Instead of adhering to conventional combination rules, such as majority vote or maximum scores, the derived scores from each classifier are merged and then trained using a Multi-Layer Perceptron (MLP) classifier to reach the final decision. In the subsequent method, named Sequential CNN deep learning-based face recognition (S-CNN), we extract high-level features from multiple image regions considered as sequential data, employing an ensemble of Convolutional Neural Networks (CNNs). In this scheme, the fully connected layers of each CNN-based image region are combined and fed into a Deep Neural Network (DNN) tailored for facial recognition. The experimental results obtained from well-known face datasets, including Labeled Faces in the Wild (LFW), Olivetti Research Laboratory (ORL) and IARPA Janus Benchmark-C (IJB-C) highlight the competitive performance of both the proposed multi-classifier combination model and the S-CNN deep-learning model when compared to state-of-the-art methods.},
keywords={Classifier combination,Deep learning,Ensemble CNN,Face recognition,Machine learning},
ISSN={2413-9351},
month={December}}