CONSENSUAL BASED CLASSIFICATION AS EMERGENT DECISIONS IN A COMPLEX SYSTEM

(Received: 9-Dec.-2021, Revised: 30-Jan.-2022 , Accepted: 5-Feb.-2022)

Authors Rabah Mazouzi, Malanga Kennedy, Cyril De Runz, Herman Akdag,

Keywords #Collaborative classification #Complex system #Emergence #Multi-agent system

Abstract In massive multi-agent systems that are used to model some complex systems, emergence is a key feature that allows to model high-level states of such systems. According to this perspective, the work we introduce in this paper entails the handling of emergence in massive multi-classifiers that we consider as complex systems. We aim to build a collaborative system for supervised data classification that we expect to provide better performance, compared to conventional classifiers. Modeled as a multi-agent system, the massive multi-classifier is composed of a high number of agents that are interconnected according to a given neighborhood. Each agent plays the role of a weak classifier. At the micro-level, the elementary interaction between agents consists of combining their respective classification results. Every agent, according to the majority vote rule, combines its result with those of its neighbors by taking into account their respective performances. This process is iterated continuously in a cyclic manner within the neighborhood of each agent. Therefore, a complex dynamic will be created within the system. After a certain time, this complex dynamic stabilizes, allowing the exhibition of an emergent structure that will be observed at the macro-level and is considered as a consensual class prediction for the data we want to classify. Obtained experimental results and the comparison with conventional classifiers show the potential of the approach to enhance classification and to be an alternative for classifier combination and aggregation.

References

[1] S. Umadevi and K. S. J. Marseline, "A Survey on Data Mining Classification Algorithms," Proc. of the IEEE International Conference on Signal Processing and Communication (ICSPC), pp. 264-268, Coimbatore, India, 2017.

[2] S. Minaee, N. Kalchbrenner, E. Cambria et al., "Deep Learning-based Text Classification: A Comprehensive Review," ACM Computing Surveys, vol. 54, pp. 1-40, 2021.

[3] A. H. Mohd Aman, E. Yadegaridehkordi, Z. S. Attarbashi, R. Hassan and Y. Park, "A Survey on Trend and Classification of Internet of Things Reviews," IEEE Access, vol. 8, pp. 111763- 111782, 2020.

[4] L. F. Cordeiro Robson, C. Faloutsos and C. Traina Jnior, Data Mining in Large Sets of Complex Data, ISBN-13: 978-1447148890, Springer Publishing Company, Incorporated, 2013.

[5] L. I. Kuncheva, "Combining Pattern Classifiers: Methods and Algorithms", Wiley-Interscience, 2004.

[6] X. Dong, Z. Yu, W. Cao, Y. Shi and Q. Ma, "A Survey on Ensemble Learning," Frontiers of Computer Science, vol. 14, no. 2, pp. 241-258, 2020.

[7] O. Sagi and L. Rokach, "Ensemble Learning: A Survey," Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 8. No. 4, DOI: 10.1002/widm.1249, 2018.

[[8] F. Yucalar, A. Ozcift, E. Borandağ and D. Kilinç, "Multiple-classifiers in Software Quality Engineering: Combining Predictors to Improve Software Fault Prediction Ability," Engineering Science and Technology: An International Journal, vol. 23, no. 4, pp. 938-950, 2020.

[9] S. Tulyakov, S. Jaeger, V. Govindaraju and D. Doermann, "Review of Classifier Combination Methods," Proc. of Machine Learning in Document Analysis and Recognition, Part of the Studies in Computational Intelligence Book Series, vol. 90, pp. 361-386, 2008.

[10] L. I. Kuncheva and C. J. Whitaker, "Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy," Machine Learning, vol. 51, no. 2, pp.181-207, 2003.

[11] D. Gacquer, On the Usability of Diversity When Training Multiple Classifier Systems. Application to the Detection of Hazardous Smokes on Industrial Facilities, Thesis at University of Valenciennes and the Hainaut-Cambresis, France, 2008.

[12] R. Roelofs, V. Shankar, B. Recht, S. Fridovich-Keil, M. Hardt, J. Miller and L. Schmidt, "A Meta- analysis of Overfitting in Machine Learning," NeurIPS Proceedings, [Online], Available: https://papers.nips.cc/paper/9117-a-meta-analysis-of-overfitting-in-machine-learning, 2019.

[13] D. Che, M. S. Safran and Z. Peng, "From Big Data to Big Data Mining: Challenges, Issues and Opportunities," Proc. of the 18th International Conference DASFAA, Part of Lecture Notes in Computer Science, vol. 7827, pp. 1-15, Springer, 2013.

[14] M. A. Niazi, "Introduction to the Modeling and Analysis of Complex Systems: A Review," Complex Adaptive Systems Modeling, vol. 4, Article no. 3, DOI: 10.1186/s40294-016-0015-x, 2016.

[15] A. Dogan and D. Birant, "A Weighted Majority Voting Ensemble Approach for Classification," Proc. of the 4th IEEE International Conference on Computer Science and Engineering (UBMK), pp. 1-6, Samsun, Turkey, 2019.

[16] R. Olfati-Saber, J. A. Fax and R. M. Murray, "Consensus and Cooperation in Networked Multi-agent Systems," Proceedings of the IEEE, vol. 95, no. 1, pp. 215-233, 2007.

[17] Y. Chen, J. Liu, H. Zhao and H. Su, "Social Structure Emergence: A Multi-agent Reinforcement Learning Framework for Relationship Building," Proc. of the 19th International Conference on Autonomous Agents and Multi Agent Systems (AAMAS '20), pp. 1807-1809, 2020.

[18] C. Yang and L. Jiamou, "Social Capital Games As a Framework for Social Structural Pattern Emergence," Proc. of the 12th IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pp. 309-316, The Hague, Netherlands, 2020.

[19] O. Paunovski, G. Eleftherakis and T. Cowling, "Disciplined Exploration of Emergence Using Multi- agent Simulation Framework," Computing and Informatics, vol. 28, pp. 369-391, 2009.

[20] O. Paunovski, G. Eleftherakis and A. J. Cowling, "A Fuzzy Reasoning Approach to Automated Detection of Emergent Herd Formations in Computer Aided Simulation," Proc. of the 3rd South East European Doctoral Student Conference, Thessaloniki, Greece: South-East European Re- search Centre (SEERC), vol. 2, pp. 150-165, 2008.

[21] V. Robu, H. Halpin and H. Shepherd, "Emergence of Consensus and Shared Vocabularies in Collaborative Tagging Systems," ACM Transactions on the Web, vol. 3, no. 4, pp. 1-14, 2009.

[22] X. Wang, D.-W. Bi, L. Ding and S. Wang, "Agent Collaborative Target Localization and Classification in Wireless Sensor Networks," Sensors, vol. 8, pp. 1359-1386, 2007.

[23] R. Mazouzi, L. Seddiki, C. de Runz and H. Akdag, "Towards a Collective and Distributed System for Consensual Data Classification," HAL-EGC 2014, hal-03251347, version 1, 2014.

[24] L. Maystre, N. Kumarappan, J. Bütepage and M. Lalmas, "Collaborative Classification from Noisy Labels," Proc. of the 24th International Conference on Artificial Intelligence and Statistics, Proc. of Machine Learning Research, vol. 130, pp. 1639-1647, 2021.

[25] W. Sun, Z. Wang, G. Zhang and B. Dong, "MACCA: A SDN Based Collaborative Classification Algorithm for QoS Guaranteed Transmission on IoT," Proc. of the 15th International Conference on Advanced Data Mining and Applications (ADMA 2019), pp. 815-827, Springer, 2019.

[26] C. D. Stefano, A. D. Cioppa and A. Marcelli, "An Adaptive Weighted Majority Vote Rule for Combining Multiple Classifiers," Proc. of the IEEE International Conference on Pattern Recognition (ICPR), pp. 192-195, Quebec City, Canada, 2002.

[27] K. P. Murphy, "Naive Bayes Classifiers," University of British Columbia, vol. 18, no. 60, pp. 1-8, 2006.

[28] R. Hickey, "A Note on the Measurement of Randomness," Journal of Applied Probability, vol. 19, no. 1, pp. 229-232, 1982.

[29] J. Lin, "Divergence Measures Based on the Shannon Entropy," IEEE Transactions on Information Theory, vol. 37, no. 1, pp. 145-151, 1991.

[30] J. E. Hurtado and A. H. Barbat, "Monte Carlo Techniques in Computational Stochastic Mechanics," Archives of Computational Methods in Engineering, vol. 5, no. 1, pp. 3-29, 1998.

[31] M. Tavallaee. E. Bagheri, W. Lu and A. Ghorbani, "A Detailed Analysis of the KDD CUP 99 Data Set," Proc. of the IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA 2009), pp. 1-6, Ottawa, Canada, 2009.

[32] M. Lichman, "Connect-4", UCI Machine Learning Repository, 2013.

[33] J. A. Blackard, "Comparison of Neural Networks and Discriminant Analysis in Predicting Forest Cover Types," PhD Thesis, Computer Science Dept. Fort Collins, Colorado State University, USA, 1998.