NEWS

ASSOCIATIVE CLASSIFICATION IN MULTI-LABEL CLASSIFICATION: AN INVESTIGATIVE STUDY

(Received: 9-Mar.-2021, Revised: 24-Apr.-2021 , Accepted: 5-May-2021)
Multi-label classification (MLC) is a very interesting and important domain that has attracted many researchers in the last two decades. Several single-label classification algorithms that belong to different learning strategies have been adapted to handle the problem of MLC. Surprisingly, no Associative Classification (AC) algorithm has been adapted to handle the MLC problem, where AC algorithms have shown a high predictive performance compared with other learning strategies in single-label classification. In this paper, a deep investigation regarding utilizing AC in MLC is presented. An evaluation of several AC algorithms on three multi-label datasets with respect to five discretization techniques revealed that utilizing AC algorithms in MLC is very promising compared with other algorithms from different learning strategies.

[1] L. Al Qadi, H. El Rifai, S. Obaid and A. Elnagar, "A Scalable Shallow Learning Approach for Tagging Arabic News," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 06, no. 03, pp. 263-280, September 2020.

[2] L. T. Nguyen, B. Vo, T. Mai and T. L. Nguyen, "A Weighted Approach for Class Association Rules," Proc. of Modern Approaches for Intelligent Information and Database Systems, pp. 213-222, Part of the Studies in Computational Intelligence Book Series, vol. 769, Springer, Cham, 2018.

[3] E. Gibaja and S. Ventura, "A Tutorial on Multilabel Learning," ACM Computing Surveys, vol. 47, no. 3, pp. 1-39, 2015.

[4] R. Sousa and J. Gama, "Multi-label Classification from High-speed Data Streams with Adaptive Model Rules and Random Rules," Progress in Artificial Intelligence, vol. 7, pp. 177-187, 2018.

[5] R. Alazaidah, F. K. Ahmad and M. F. M. Mohsen, "A Comparative Analysis between the Three Main Approaches that are Being Used to Solve the Problem of Multi Label Classification," International Journal of Soft Computing, vol. 12, no. 4, pp. 218-223, 2017.

[6] J. Huang, G. Li, S. Wang, Z. Xue and Q. Huang, "Multi-label Classification by Exploiting Local Positive and Negative Pairwise Label Correlation," Neurocomputing, vol. 257, pp. 164-174, 2017.

[7] R. Alazaidah, F. K. Ahmad, M. F. M. Mohsen and A. K. Junoh, "Evaluating Conditional and Unconditional Correlations Capturing Strategies in Multi Label Classification," Journal of Telecommunication, Electronic and Computer Engineering (JTEC), vol. 10, no. 2-4, pp. 47-51, 2018.

[8] Y. Qu, G. Yue, C. Shang, L.Yang, R. Zwiggelaar and Q. Shen, "Multi-criterion Mammographic Risk Analysis Supported with Multi-label Fuzzy-rough Feature Selection," Artificial Intelligence in Medicine, vol. 100, Article ID: 101722, DOI: 10.1016/j.artmed.2019.101722, 2019.

[9] R. Alazaidah, F. K. Ahmad, M. F. M. Mohsin and W. A. AlZoubi, "Multi-label Ranking Method Based on Positive Class Correlations," Jordanian Journal of Computers and Information Technology (JJCIT), vol. 6, no. 4, pp. 377-391, Dec. 2020.

[10] R. Alazaidah, F. Thabtah and Q. Al-Radaideh, "A Multi-label Classification Approach Based on Correlations among Labels," International Journal of Advanced Computer Science and Applications, vol. 6, no. 2, pp. 52-59, 2015.

[11] R. Alazaidah, F. Ahmad and M. Mohsin, "Multi Label Ranking Based on Positive Pairwise Correlations among Labels," The International Arab Journal of Information Technology, vol. 17, no. 4, DOI: 10.34028/iajit/17/4/2, 2019.

[12] M. Atzmueller, N. Hayat, M. Trojahn and D. Kroll, "Explicative Human Activity Recognition Using Adaptive Association Rule-based Classification," Proc. of the IEEE International Conference on Future IoT Technologies (Future IoT), pp. 1-6, DOI: 10.1109/FIOT.2018.8325603, Eger, Hungary, 2018.

[13] Y. Chengxin, G. Yan, Y. Jianguo and R. Xiaoting, "A New Recommendation System on the Basis of Consumer Initiative Decision Based on an Associative Classification Approach," Industrial Management & Data Systems, vol. 118, no. 1, pp.188-203, 2018.

[14] N. Abdelhamid, A. A. Jabbar and F. Thabtah, "Associative Classification Common Research Challenges," Proceedings of the 45th IEEE International Conference on Parallel Processing Workshops, pp.432-437, Philadelphia, USA, 2016.

[15] G. Corani and M. Scanagatta, "Air Pollution Prediction via Multi-label Classification," Environmental Modelling & Software, vol. 80, pp. 259-264, 2016.

[16] E. C. Gonçalves, A. Plastino and A. A. Freitas, "A Genetic Algorithm for Optimizing the Label Ordering in Multi-label Classifier Chains," Proceedings of the 25th IEEE International Conference on Tools with Artificial Intelligence, pp. 469-476, Herndon, USA, 2013.

[17] X. Li, J. Ouyang and X. Zhou, "Centroid Prior Topic Model for Multi-label Classification," Pattern Recognition Letters, vol. 62, pp. 8-13, 2015.

[18] S. Ali and A. Majid, "Can–Evo–Ens: Classifier Stacking Based Evolutionary Ensemble System for Prediction of Human Breast Cancer Using Amino Acid Sequences," Journal of Biomedical Informatics, vol. 54, pp. 256-269, 2015.

[19] E. A. Tanaka, S. R. Nozawa, A. A. Macedo and J. A. Baranauskas, "A Multi-label Approach Using Binary Relevance and Decision Trees Applied to Functional Genomics," Journal of Biomedical Informatics, vol. 54, pp. 85-95, 2015.

[20] K. Trohidis, G. Tsoumakas, G. Kalliris and I. P. Vlahavas, "Multi-label Classification of Music into Emotions," Proceedings of the 9th International Conference on Music Informative Retrieval, pp. 325-330, Philadelphia, PN: Drexel University, 2008.

[21] A. Dimou, G. Tsoumakas, V. Mezaris, I. Kompatsiaris and I. Vlahavas, "An Empirical Study of Multi-label Learning Methods for Video Annotation," Proceedings of the 7th IEEE International Workshop on Content-based Multimedia Indexing, pp. 19-24, Chania, Greece, 2009.

[22] T. Li, C. Zhang and S. Zhu, "Empirical Studies on Multi-label Classification," Proc. of the 18th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'06), DOI: 10.1109/ICTAI.2006.55, Arlington, USA, 2006.

[23] Z. Barutcuoglu, R. E. Schapire and O. G. Troyanskaya, "Hierarchical Multi-label Prediction of Gene Function," Bioinformatics, vol. 22, no. 7, pp. 830-836, 2006.

[24] J. Lee, H. Kim, N. R. Kim and J. H. Lee, "An Approach for Multi-label Classification by Directed Acyclic Graph with Label Correlation Maximization," Information Sciences, vol. 351, pp. 101-114, 2016.

[25] M. Alluwaici, A. K. Junoh, F. K. Ahmad, M. F. M. Mohsen and R. Alazaidah, "Open Research Directions for Multi Label Learning," Proc. of the IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE), pp. 125-128, DOI: 10.1109/ISCAIE.2018.8405456, Penang, 2018.

[26] S. Xu, X. Yang, H. Yu, D. J. Yu, J. Yang and E. C. Tsang, "Multi-label Learning with Label-specific Feature Reduction," Knowledge-based Systems, vol. 104, pp. 52-61, 2016.

[27] J. Read, B. Pfahringer, G. Holmes and E. Frank, "Classifier Chains for Multi-label Classification," Machine Learning, vol. 85, no. 3, Article no. 333, 2011.

[28] F. Thabtah, "Challenges and Interesting Research Directions in Associative Classification," Proceedings of the 6th IEEE Int. Conference on Data Mining Workshops, pp. 785-792, Hong Kong, China, 2006.

[29] F. A. Thabtah, P. Cowling and Y. Peng, "MMAC: A New Multi-class, Multi-label Associative Classification Approach," Proceedings of the 4th IEEE International Conference on Data Mining, pp. 217- 224, Brighton, UK, 2004.

[30] N. Abdelhamid and F. Thabtah, "Associative Classification Approaches: Review and Comparison," Journal of Information & Knowledge Management, vol. 13, no. 3, pp. 1-30, 2014.

[31] Y. Yang, A. Stein, V. A. Tolpekin and Y. Zhang, "High-resolution Remote Sensing Image Classification Using Associative Hierarchical CRF Considering Segmentation Quality," IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 5, pp. 754-758, 2018.

[32] P. B. Shekhawat and S. S. Dhande, "A Classification Technique Using Associative Classification," International Journal of Computer Applications, vol. 20, no. 5, pp. 20-28, 2011.

[33] Y. W. C. Chien and Y. L. Chen, "Mining Associative Classification Rules with Stock Trading Data: A GA-based Method," Knowledge-based Systems, vol. 23, no. 6, pp. 605-614, 2010.

[34] F. Thabtah, Q. Mahmood, L. McCluskey and H. Abdel-Jaber, "A New Classification Based on Association Algorithm," J. of Information & Knowledge Management, vol. 9, no. 1, pp. 55-64, 2010.

[35] R. AlShboul, F. Thabtah, N. Abdelhamid and M. Al-Diabat, "A Visualization Cybersecurity Method Based on Features' Dissimilarity," Computers & Security, vol. 77, pp. 289-303, 2018.

[36] B. Liu, W. Hsu and Y. Ma, "Integrating Classification and Association Rule Mining," Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining, pp.80-86, American Association for Artificial Intelligence, 1998.

[37] R. Agrawal and R. Srikant, "Fast Algorithms for Mining Association Rules," Proceedings of the 20th International Conference on Very Large Databases, pp. 487-499, Santiago, Chile, 1994.

[38] B. Liu, Y. Ma and C. K. Wong, "Improving an Association Rule Based Classifier," Proceedings of the 4th European Conference on Principles of Data Mining and Knowledge Discovery, pp. 504-509, Berlin, Heidelberg: Springer-Verlag, 2009.

[39] S. J. Huang, Z. H, Zhou and Z. Zhou, "Multi-label Learning by Exploiting Label Correlations Locally," Proceedings of the 26th AAAI Conf. on Artificial Intelligence, pp. 949-955, Palo Alto, CA: AAAI, 2012.

[40] X. Yin and J. Han, "CPAR: Classification Based on Predictive Association Rules," Proceedings of the 2003 SIAM International Conference on Data Mining, pp. 331-335, Society for Industrial and Applied Mathematics, Philadelphia, PN, USA, 2003.

[41] W. Li, J. Han and J. Pei, "CMAR: Accurate and Efficient Classification Based on Multiple Class-association Rules," Proc. of the IEEE Int. Conf. on Data Mining, pp. 369-376, San Jose, USA, 2001.

[42] Y. C. Hu, R. S. Chen and G. H. Tzeng, "Finding Fuzzy Classification Rules Using Data Mining Techniques," Pattern Recognition Letters, vol. 24, no. 1-3, pp. 509-519, 2003.

[43] Z. Chen and G. Chen, "Building an Associative Classifier Based on Fuzzy Association Rules," International Journal of Computational Intelligence Systems, vol. 1, no. 3, pp. 262-273, 2008.

[44] N. Abdelhamid, A. Ayesh and W. Hadi, "Multi-label Rules Algorithm Based Associative Classification," Parallel Processing Letters, vol. 24, no. 1, pp. 1-21, 2014.

[45] A. Veloso, W. Meira, M. Gonçalves and M. Zaki, "Multi-label Lazy Associative Classification," Proc. of the European Conference on Principles of Data Mining and Knowledge Discovery, pp. 605-612, Berlin, Heidelberg: Springer-Verlag, 2007.

[46] R. E. Schapire and Y. Singer, "BoosTexter: A Boosting-based System for Text Categorization," Machine Learning, vol. 39, no. 2-3, pp. 135-168, 2000.

[47] X. Li, D. Qin and C. Yu, "ACCF: Associative Classification Based on Closed Frequent Itemsets," Proceeding of the 5th IEEE International Conference on Fuzzy Systems and Knowledge Discovery, pp. 380-384, Jinan, China, 2008.

[48] J. Han, J. Pei, Y. Yin and R. Mao, "Mining Frequent Patterns without Candidate Generation: A Frequent- pattern Tree Approach," Data Mining and Knowledge Discovery, vol. 8, no. 1, pp. 53-87, 2004.

[49] G. Tsoumakas, E. Spyromitros-Xioufis, J. Vilcek, and I. Vlahavas, "Mulan: A Java Library for Multi-label Learning," Journal of Machine Learning Research, vol. 12, no. July, pp. 2411-2414, 2011.

[50] I. Triguero, S. González, J. M. Moyano, S. García, J. Alcalá-Fdez, J. Luengo et al., "KEEL 3.0: An Open Source Software for Multi-stage Analysis in Data Mining," International Journal of Computational Intelligence Systems, vol. 10, no. 1, pp. 1238-1249, 2017.

[51] H. Liu and R. Setiono, "Feature Selection via Discretization," IEEE Transactions on Knowledge and Data Engineering, vol. 9, no. 4, pp. 642-645, 1997.

[52] X. Wu, "A Bayesian Discretizer for Real-valued Attributes," The Computer Journal, vol. 39, no. 8, pp. 688-691, 1996.

[53] L. Gonzalez-Abril, F. J. Cuberos, F. Velasco and J. A. Ortega, "Ameva: An Autonomous Discretization Algorithm," Expert Systems with Applications, vol. 36, no. 3, pp. 5327-5332, 2009.

[54] R. C. Holte, "Very Simple Classification Rules Perform Well on Most Commonly Used Datasets," Machine Learning, vol. 11, no. 1, pp. 63-90, 1993.

[55] C. T. Su and J. H. Hsu, "An Extended Chi2 Algorithm for Discretization of Real Value Attributes," IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 3, pp. 437-441, 2005.

[56] J. R. Quinlan, "Combining Instance-based and Model-based Learning," Proceedings of the 10th International Conference on Machine Learning, pp. 236-243, Burlington, MA: Morgan Kauffman, 1993.

[57] D. R. Carvalho and A. A. Freitas, "A Hybrid Decision Tree/Genetic Algorithm Method for Data Mining," Information Sciences, vol. 163, no. 1-3, pp. 13-35, 2004.

[58] G. F. Miller, P. M. Todd and S. U. Hegde, "Designing Neural Networks Using Genetic Algorithms," Proceedings of the 3rd International Conference on Genetic Algorithms, pp. 379-384, Hillsdale, IL: Morgan Kaufmann, 1989.

[59] F. J. Martínez-Estudillo, C. Hervás-Martínez, P. A. Gutiérrez and A. C. Martínez-Estudillo, "Evolutionary Product-unit Neural Network Classifiers," Neurocomputing, vol. 72, no. 1-3, pp. 548-561, 2008.

[60] J. Hühn and E. Hüllermeier, "FURIA: An Algorithm for Unordered Fuzzy Rule Induction," Data Mining and Knowledge Discovery, vol. 19, no. 3, pp. 293-319, 2009.

[61] T. Nakashima, G. Schaefer, Y. Yokota and H. Ishibuchi, "A Weighted Fuzzy Classifier and Its Application to Image Processing Tasks," Fuzzy Sets and Systems, vol. 158, no. 3, pp. 284-294, 2007.

[62] T. Cover and P. Hart, "Nearest Neighbor Pattern Classification," IEEE Transactions on Information Theory, vol. 13, no. 1, pp. 21-27, 1967.

[63] J. Wang, P. Neskovic and L. N. Cooper, "Improving Nearest Neighbor Rule with a Simple Adaptive Distance Measure," Pattern Recognition Letters, vol. 28, no. 2, pp. 207-213, 2007.

[64] S. Le Cessie and J. C. Van Houwelingen, "Ridge Estimators in Logistic Regression," Applied Statistics, vol. 41, no. 1, pp. 191-201, 1992.

[65] G. J. McLachlan, Discriminant Analysis and Statistical Pattern Recognition, vol. 544. John Wiley & Sons, ISBN: 978-0-471-69115-0, 2004.

[66] C. Cortes and V. Vapnik, "Support-vector Networks," Machine Learning, vol. 20, no. 3, pp. 273-297, 1995.

[67] S. S. Keerthi, S. K. Shevade, C. Bhattacharyya and K. R. K. Murthy, "Improvements to Platt's SMO Algorithm for SVM Classifier Design," Neural Computation, vol. 13, no. 3, pp. 637-649, 2001.

[68] R. Alazaidah and F. K. Ahmad, "Trending Challenges in Multi Label Classification," International Journal of Advanced Computer Science and Applications, vol. 7, no. 10, pp. 127-131, 2016.