Unleashing the Power of Ensemble Learning: Predicting National Ranks in Iran’s University Entrance Examination

Document Type : Research Paper

Authors

1 Prof., Department of Industrial Management, Faculty of Industrial Management and Technology, College of Management, University of Tehran, Tehran, Iran.

2 PhD., Department of Industrial Management, Faculty of Industrial Management and Technology, College of Management, University of Tehran, Tehran, Iran.

Abstract

Objective
This study seeks to explore and compare ensemble learning models for more accurate predictions of students’ national ranks in Iran’s nationwide university entrance examination, commonly known as the Konkur. The primary aim is to identify optimal models capable of predicting students’ ranks with the highest precision by analyzing data from preparatory and simulated exams conducted before the Konkur. These identified models can then empower academic counselors to offer data-driven recommendations, assisting students in making informed decisions about their educational paths and academic planning.
 
Methods
Initially, Octoparse software was utilized to collect data related to the preparatory tests conducted by the Kanoon Farhangi Amoozesh (Cultural Center of Education, also known as Ghalamchi) to facilitate a precise analysis of students' performance. The dataset contained key information such as the average test scores, students' national and regional ranks in the exams, as well as the academic disciplines and universities where the students were admitted. Additionally, four advanced ensemble learning models—XGBoost, LightGBM, CatBoost, and Random Forest—were selected for comparison and evaluation based on their performance in predicting students' national ranks. To assess the accuracy and efficiency of these models, various metrics were used, including Mean Squared Error (MSE), Root Mean Squared Error (RMSE), coefficient of determination (R²), and training and prediction times. Afterward, The collected data were split into training and testing sets to ensure optimal model training and evaluation. To achieve the best possible performance, the grid search method was applied, enabling precise tuning of the model parameters to obtain optimal results.
 
Results
The study findings underscored the exceptional performance of the XGBoost and LightGBM models in predicting students' national ranks. XGBoost emerged as the most accurate model, delivering predictions that are closely aligned with the actual values and exhibiting the lowest error rates, positioning it as the top performer in this research. Similarly, LightGBM, with results closely mirroring those of XGBoost, was also recognized as a strong contender for predicting national ranks. Its blend of speed and precision made it a highly effective tool in this context. The Random Forest model also demonstrated superior accuracy compared to CatBoost, albeit requiring more time for training and prediction. In contrast, CatBoost was identified as the weakest option, displaying lower accuracy and higher error rates than the other models, despite its faster prediction times. This suggests that CatBoost may require further refinement to compete with the more successful models.
 
Conclusion
The research findings highlight the effectiveness of XGBoost and LightGBM models in ensemble learning, showcasing their exceptional performance in predicting students' ranks in nationwide examinations. With their high accuracy and optimal performance, these models can serve as valuable tools within educational systems, contributing to the improvement of learning processes. They have the potential to identify students who may encounter challenges in their educational journeys and aid in the development of more effective learning programs. Furthermore, the outcomes of this study can assist school leaders and educational policymakers in making well-informed decisions to promote educational equity and ensure equal learning opportunities for all students. Looking ahead, leveraging more advanced deep learning models and integrating additional data, such as social and economic factors and school types, could lead to enhanced prediction accuracy. Additionally, the integration of diverse machine learning models to create hybrid systems may enhance precision and minimize prediction errors. Overall, this research marks a significant milestone in the advancement of predictive and decision-making systems within the education sector, laying the groundwork for substantial progress in this field.

Keywords

Main Subjects

References

اسدی، علی اکبر و جوادزاده، محمدعلی (1400). پیش‌بینی نمرات دانش‌آموزان با استفاده از روش یادگیری ماشین (مقایسه سه الگوریتم درخت تصمیم، جنگل تصادفی و رگرسیون خطی). ششمین کنفرانس بین‌المللی پژوهش‌های کاربردی در کامپیوتر، برق و فناوری اطلاعات.
بابایی، مرضیه و شویدی نوش آبادی، مهدی (1399). مروری بر روش‌های پیش‌بینی عملکرد دانشجویان با استفاده از الگوریتم‌های یادگیری ماشین. ششمین کنفرانس بین‌المللی نوآوری و تحقیق در علوم مهندسی.
خانی، امیر محمد؛ کزازی، ابولفضل و بیرامی، ثریا (1401). مدل‌سازی ساختاری بر مبنای مدیریت زنجیره‌تأمین در رابطه با مدیریت کیفیت جامع، نگهداری‌و تعمیرات بهره‌ور فراگیر، سازمان یادگیرنده و عملکرد عملیاتی. مطالعات مدیریت صنعتی، 20(65)، 39-84.
زنگوئی، حمید و فاطمی، سید امید (1400). پیش‌بینی دانش‌آموزان در معرض خطر افت تحصیلی با استفاده از تجزیه‏وتحلیل یادگیری در سیستم مدیریت یادگیری. مجله ایرانی آموزش از دور، 3(2)، 32-44.
سالاری، مژده؛ رادفر، رضا و فقیهی، مهدی (1403). پیش‌بینی عملکرد دانشجویان با استفاده از الگوریتم‌های یادگیری ماشین و داده کاوی آموزشی (مطالعه موردی دانشگاه شاهد). مطالعات مدیریت کسب‌وکار هوشمند، 12(47)، 315-366.
قدوسی، محمد، میرسعیدی، فاطمه و کوشا، حمیدرضا (1399). پیش‌بینی و تحلیل عملکرد دانشجویان به کمک تکنیک‌های داده‌کاوی به‏منظور بهبود عملکرد تحصیلی. فناوری آموزش، 14(4)، 821-834.
کاظمیان، مینا؛ افشارکاظمی، محمدعلی؛ فتحی هفشجانی، کیامرث و معتدل، محمدرضا (1402). ارائه مدل هوشمند تعیین قیمت فولاد با رویکرد ‌‌‌‌‌ترکیبی نظریه بازی‌ها و الگوریتم‌های ‌‌‌یادگیری ماشین. مدیریت صنعتی، 15(3)، 478- 507.
 
References
Adejo, O. W. & Connolly, T. (2018). Predicting student academic performance using multi-model heterogeneous ensemble approach. Journal of Applied Research in Higher Education, 10(1), 61–75. https://doi.org/10.1108/jarhe-09-2017-0113
Asadi, A.A. & Javadzadeh, M.A. (2021). Prediction of students' grades using machine learning method (comparison of three algorithms of decision tree, random forest and linear regression). The 6th International Conference on Applied Research in Computer, Electricity and Information Technology. (in Persian)
Babaei, M. & Shovidi Noushabadi, M. (2019). A review of student performance prediction methods using machine learning algorithms. The 6th International Conference on Innovation and Research in Engineering Sciences, https://civilica.com/doc/1033467
(in Persian)
Breiman, L. (2001). Random forests. Machine Learning, 45, 5-32. https://doi.org/10.1023/A:1010933404324
Butt, N. A., Mahmood, Z., Shakeel, K., Alfarhood, S., Safran, M. & Ashraf, I. (2023). Performance Prediction of students in Higher Education using Multi-Model Ensemble Approach. IEEE Access, 11, 136091–136108. https://doi.org/10.1109/access.2023.3336987
Chen, T. & Guestrin, C. (2016, August). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785-794). https://doi.org/10.1145/2939672.2939785
Daniele, V. (2021). Socioeconomic inequality and regional disparities in educational achievement: The role of relative poverty. Intelligence, 84, 101515. https://doi.org/10.1016/j.intell.2020.101515
Frost, J. (2019). Regression Analysis: An Intuitive Guide for Using and Interpreting Linear Models. Statistics by Jim Publishing. https://doi.org/10.1007/978-3-030-36077-8
Ghodoosi, M., Mirsaeedi, F. & Koosha, H. (2020). Predicting and analyzing the performance of students through data mining techniques to improve academic performance. Technology of Education Journal (TEJ), 14(4), 821-834. doi: 10.22061/jte.2019.4902.2134 (in Persian)
Gibson, D. C. & Webb, M. E. (2015). Data science in educational assessment. Education and Information Technologies, 20, 697-713. https://doi.org/10.1007/s10639-015-9411-7
Goodfellow, I., Bengio, Y. & Courville, A. (2016). Deep Learning. MIT Press. https://doi.org/10.5555/3086952
Han, J., Pei, J. & Kamber, M. (2011). Data mining: Concepts and techniques (3rd ed.). Morgan Kaufmann. https://doi.org/10.1016/C2009-0-61819-5
Han, M., Tong, M., Chen, M., Liu, J. & Liu, C. (2017). Application of Ensemble Algorithm in Students’ Performance Prediction. 2017 6th IIAI International Congress on Advanced Applied Informatics (IIAI-AAI), 735-740.
Hodson, T. O. (2022). Root-mean-square error (RMSE) or mean absolute error (MAE): when to use them or not. Geoscientific Model Development, 15(14), 5481–5487. https://doi.org/10.5194/gmd-15-5481-2022
Hyndman, R. J. & Koehler, A. B. (2006). Another look at measures of forecast accuracy. International Journal of Forecasting, 22(4), 679-688. https://doi.org/10.1016/j.ijforecast.2006.03.001
Injadat, M., Moubayed, A., Nassif, A. B. & Shami, A. (2020). Systematic ensemble model selection approach for educational data mining. Knowledge-Based Systems, 200, 105992. https://doi.org/10.1016/j.knosys.2020.105992
James, G., Witten, D., Hastie, T. & Tibshirani, R. (2013). An Introduction to Statistical Learning with Applications in R. Springer. https://doi.org/10.1007/978-1-4614-7138-7
Kazemian, M., Afsharkazemi, M. A., Fathi Hafashjani, K. & Motadel, M. (2023). Presenting Smart Steel Pricing Model: An Integration of Game Theory and Machine Learning Algorithms. Industrial Management Journal, 15(3), 478-507. doi: 10.22059/imj.2023.356697.1008039 (in Persian)
Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., ... & Liu, T. Y. (2017). LightGBM: A highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems, 30. https://doi.org/10.5555/3294996.3295074
Khani, A. M., Kazzazi, A. & Birami, S. (2022). Structural modeling based on supply chain management in relation to total quality management, maintenance and comprehensive productivity, learning organization and operational performance. Industrial Management Studies, 20(65), 39-84. doi: 10.22054/jims.2022.62763.2688 (in Persian)
Navarro, C. L. A., Damen, J. A., Takada, T., Nijman, S. W., Dhiman, P., Ma, J. , ... & Hooft, L. (2021). Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review. BMJ, 375. https://doi.org/10.1136/bmj.n2281
Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A. V. & Gulin, A. (2018). CatBoost: Unbiased boosting with categorical features. Advances in Neural Information Processing Systems, 31. https://doi.org/10.48550/arXiv.1706.09516
Sakri, S. & Saleh, A. (2020). RHEM: a robust hybrid ensemble model for students’ performance assessment on cloud computing course. International Journal of Advanced Computer Science and Applications, 11(11). https://doi.org/10.14569/ijacsa.2020.0111150
Salari, M., Radfar, R. & Faghihi, M. (2024). Predicting students' performance using machine learning algorithms and educational data mining (a case study of Shahed University). Business Intelligence Management Studies, 12(47), 315-366. doi: 10.22054/ims.2023.75523.2375 (in Persian)
Wang, N. Z. & Shi, N. Y. (2016). Prediction of the admission lines of college entrance examination based on machine learning. 2016 2nd IEEE International Conference on Computer and Communications (ICCC), 332-335. https://doi.org/10.1109/compcomm.2016.7924718
Yağcı, M. (2022). Educational data mining: prediction of students' academic performance using machine learning algorithms. Smart Learning Environments, 9(1), 11. https://doi.org/10.1186/s40561-022-00192-z
Yan, L. & Liu, Y. (2020). An ensemble prediction model for potential student recommendation using machine learning. Symmetry, 12(5), 728. https://doi.org/10.3390/sym12050728
Zangooei, H. & Fatemi, O. (2021). Predicting students at risk of academic failure using learning analytics in the learning management system. Quarterly of Iranian Distance Education Journal, 3(2), 32-44. doi: 10.30473/idej.2022.63913.1104 (in Persian)
Zhang, C., Bengio, S., Hardt, M., Recht, B. & Vinyals, O. (2017). Understanding deep learning requires rethinking generalization. https://doi.org/10.48550/arXiv.1611.03530
Industrial Management Journal
Volume 16, Issue 3
2024
Pages 457-481

Files

Share

How to cite

Statistics