STAP Journal of Security Risk Management

ISSN: 3080-9444 (Online)

Optimizing Intrusion Detection Systems through Benchmarking of Ensemble Classifiers on Diverse Network Attacks

by 

Dena Abu Laila ;

Mahmoud Aljawarneh ;

Qais Al-Na’amneh ;

Rejwan Bin Sulaiman

PDF logoPDF

Published: 2025/11/29

Abstract

The escalating sophistication of cyber threats requires transparent and reproducible benchmarks for intelligent security paradigms. This study presents a comprehensive benchmark analysis of a machine learning pipeline for network intrusion detection, addressing critical deployment oriented challenges such as class imbalance, feature optimization, and cross-environment generalization. Trained rigorously on the NF-CSE-CIC-IDS2018-v2 dataset and validated on the distinct UNSW-NB15 dataset, this work tackles the complexities of identifying diverse network threats through the systematic integration of data preprocessing, advanced class-imbalance handling with SMOTE, and an embedded feature selection methodology. A comparative evaluation is conducted between state-of-the-art ensemble models (Random Forest and XGBoost), recent deep learning approaches, and a logistic regression baseline, examining predictive accuracy, computational trade-offs, and per-class performance across stealthy and volumetric attack types. The optimized Random Forest model achieves 99.95% accuracy and a 0.9837 F1-score on the primary dataset, while demonstrating strong generalization performance with a 94.8% F1-score on cross-validation, supported by thorough overfitting analysis and model validation procedures.

Keywords

Network securityIntrusion Detection SystemsMachine learning CybersecurityEnsemble methodsCross-dataset validationRisk assessment

Optimizing Intrusion Detection Systems through Benchmarking of Ensemble Classifiers on Diverse Network Attacks is licensed under CC BY 4.0CCBY

References

  1. Ahmad, Z., Shahid Khan, A., Wai Shiang, C., Abdullah, J., & Ahmad, F. (2021). Network intrusion detection system: A systematic study of machine learning and deep learning approaches. Transactions on Emerging Telecommunications Technologies, 32(1), e4150.
  2. Al-Hwaiti, Y., Al-Haj, A., & Moustafa, N. (2023). A survey of dimensionality reduction and feature selection methods for cyber security. IEEE Access, 11, 72314–72338.
  3. Allasasmh, O., Laila, D. A., Aljaidi, M., Alsarhan, A., & Samara, G. (2024, December). Integrated approaches to steganography: Embedding static information across audio, visual, and textual formats. In 2024 International Jordanian Cybersecurity Conference (IJCC) (pp. 33–39). IEEE.
  4. Al-Mousa, M. R., Albilasi, S. M., Al-mashagbeh, M. H., Asassfeh, M., Odeh, M., AlQawasmi, K., & Laila, D. A. (2025, April). Review of the challenges associated with steganography using artificial intelligence techniques. In 2025 1st International Conference on Computational Intelligence Approaches and Applications (ICCIAA) (pp. 1–6). IEEE.
  5. Al-Na’amneh, Q., Aljawarneh, M., & Hazaymih, R. (2025). A framework for insider threat detection using role-based profile assessment and threshold. In Utilizing AI in Network and Mobile Security for Threat Detection and Prevention (pp. 97–114). IGI Global.
  6. Al-Na’amneh, Q., Aljawarneh, M., Hazaymih, R., Alzboon, L., Laila, D. A., & Albawaneh, S. (2025). Trust evaluation enhancing security in the cloud market based on trust framework using metric parameter selection. In Utilizing AI in Network and Mobile Security for Threat Detection and Prevention (pp. 233–254). IGI Global.
  7. Alsarhan, A., Al-Aiash, I., Al-Fraihat, D., Aljaidi, M., & Laila, D. A. A. H. A. (2024, July). Expert phishing detection system. In 2024 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT) (pp. 54–59). IEEE.
  8. Issa, W., Moustafa, N., Turnbull, B., Sohrabi, N., & Tari, Z. (2023). Blockchain-based federated learning for securing Internet of Things: A comprehensive survey. ACM Computing Surveys, 55(9), 1–43.
  9. Khalaf, Y., Aljaidi, M., Laila, D. A., Alsarhan, A., Alkhawaldeh, A. K., Alsuwaylimi, A. A., & Kharabsheh, M. (2025). An effective encryption algorithm based on RSA and DES. International Journal of Communication Networks and Information Security, 17(4), 10–19.
  10. Kim, J., Shin, N., Kim, K., & Kim, H. (2023). A survey on network intrusion detection systems: From the perspective of the grand challenge of security and privacy. Applied Sciences, 13(13), 7540.
  11. Laila, D. A., Aljaidi, M., Almaiah, M. A., AlBourini, M., Al-Na’amneh, Q., Samara, G., & Momani, K. (2025). A novel scheme to optimize LSB steganography based on a logistic chaotic map and genetic algorithm. Iraqi Journal for Computer Science and Mathematics, 6(2), 24.
  12. Aljumaiah, O., Jiang, W., Addula, S. R., & Almaiah, M. A. (2025). Analyzing cybersecurity risks and threats in IT infrastructure based on NIST framework. J. Cyber Secur. Risk Audit, 2025(2), 12-26.
  13. Laila, D. A., Al-Na’amneh, Q., Aljaidi, M., Nasayreh, A. N., Gharaibeh, H., Al Mamlook, R., & Alshammari, M. (2024b). Simulation of routing protocols for jamming attacks in mobile ad-hoc network. In Risk Assessment and Countermeasures for Cybersecurity (pp. 235–252). IGI Global.
  14. Moustafa, N., & Slay, J. (2015). UNSW-NB15: A comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set). In 2015 Military Communications and Information Systems Conference (MilCIS) (pp. 1–6). IEEE.
  15. Mughaid, A., Obaidat, I., Aljammal, A., AlZu’bi, S., Quiam, F., Laila, D. A., & Abualigah, L. (2023). Simulation and analysis performance of ad-hoc routing protocols under DDoS attack and proposed solution. International Journal of Data & Network Science, 7(2).
  16. Nasayreh, A., Jaradat, A. S., Gharaibeh, H., Dawaghreh, W., Al Mamlook, R. M., Alqudah, Y., & Abualigah, L. (2024). Jordanian banknote data recognition: A CNN-based approach with attention mechanism. Journal of King Saud University – Computer and Information Sciences, 36(4), 102038.
  17. Sarhan, M., Layeghy, S., & Portmann, M. (2022). Towards a standard feature set for network intrusion detection system datasets. Mobile Networks and Applications, 1–14.
  18. Sharafaldin, I., Lashkari, A. H., & Ghorbani, A. A. (2018). Toward generating a new intrusion detection dataset and intrusion traffic characterization. In Proceedings of the 4th International Conference on Information Systems Security and Privacy (ICISSP) (pp. 108–116).
  19. Ullah, I., & Mahmoud, Q. H. (2023). A survey on deep learning and its applications for cybersecurity. Journal of Cybersecurity and Privacy, 3(3), 400–426.
  20. Vinayakumar, R., Alazab, M., Soman, K. P., Poornachandran, P., Al-Nemrat, A., & Venkatraman, S. (2019). Deep learning approach for intelligent intrusion detection system. IEEE Access, 7, 41565–41587.