An Improved Genetic Algorithm for Hyperparameter Optimization in Predictive Modeling

Authors

  • Dmitriy Kravtsov
  • Nikolay Poletaev

DOI:

https://doi.org/10.47839/ijc.25.3.4839

Keywords:

genetic algorithm, global optimization, hyperparameter optimization, machine learning

Abstract

This paper presents an improved genetic algorithm (GenAlg) for hyperparameter optimization in predictive modelling. It addresses common limitations of traditional genetic algorithms (GAs), including slow convergence, high computational cost, and premature convergence. The proposed method incorporates early stopping, flexible mutation, intermediate result caching, parallel computation, and advanced crossover. These enhancements accelerate convergence to the global optimum and improve resilience against local minima.

The algorithm’s effectiveness is demonstrated through experiments on benchmark functions such as Rastrigin and Schwefel, as well as a real-world task—predicting real estate rental prices. Comparative analysis shows that the improved GenAlg outperforms PyGAD, DEAP, and Optuna in both accuracy and stability, while also reducing computational demands.

This approach is particularly useful for automatic hyperparameter tuning and can be applied to a wide range of optimization problems in machine learning, financial modelling, and engineering.

References

J. H. Holland, Adaptation in Natural and Artificial Systems, Ann Arbor, MI, USA: University of Michigan Press, 1975.

G. Wang, S. Xiao, X. Chen, and X. Li, “Application of genetic algorithm in automatic train operation,” Wireless Personal Communications, vol. 102, no. 1, 2018, https://doi.org/10.1007/s11277-017-5228-6.

F. Pezzella, G. Morganti, and G. Ciaschetti, “A genetic algorithm for the flexible job-shop scheduling problem,” Computers & Operations Research, vol. 35, no. 10, pp. 3202–3212, 2008, https://doi.org/10.1016/j.cor.2007.02.014.

D. Beasley, D. R. Bull, and R. R. Martin, “An overview of genetic algorithms: Part 1, fundamentals,” University Computing, vol. 15, no. 2, pp. 58–69, 1993.

D. Whitley, “A genetic algorithm tutorial,” Statistics and Computing, vol. 4, no. 2, pp. 65–85, 1994, https://doi.org/10.1007/BF00175354.

A. Lambora, K. Gupta, and K. Chopra, “Genetic algorithm—A literature review,” in 2019 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), Faridabad, India, 2019, pp. 380–384, https://doi.org/10.1109/COMITCon.2019.8862255.

M. Lozano and C. García-Martínez, “Hybrid metaheuristics with evolutionary algorithms specializing in intensification and diversification: Overview and progress report,” Computers & Operations Research, vol. 37, no. 3, pp. 481–497, 2010, https://doi.org/10.1016/j.cor.2009.02.010.

S. Katoch, S. S. Chauhan, and V. Kumar, “A review on genetic algorithm: Past, present, and future,” Multimedia Tools and Applications, vol. 80, no. 5, pp. 8091–8126, 2021, https://doi.org/10.1007/s11042-020-10139-6.

Y. Wang and Q. Zhu, “A hybrid genetic algorithm for flexible job shop scheduling problem with sequence-dependent setup times and job lag times,” IEEE Access, 2021, https://doi.org/10.1109/ACCESS.2021.3096007.

M. Črepinšek, S.-H. Liu, and M. Mernik, “Exploration and exploitation in evolutionary algorithms: A survey,” ACM Computing Surveys, vol. 45, no. 3, Art. no. 35, 2013, https://doi.org/10.1145/2480741.2480752.

J. J. Grefenstette, “Optimization of control parameters for genetic algorithms,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 16, no. 1, pp. 122–128, 1986, https://doi.org/10.1109/TSMC.1986.289288.

A. E. Eiben, R. Hinterding, and Z. Michalewicz, “Parameter control in evolutionary algorithms,” IEEE Transactions on Evolutionary Computation, vol. 3, no. 2, pp. 124–141, 1999, https://doi.org/10.1109/4235.771166.

Y. Jin, “A comprehensive survey of fitness approximation in evolutionary computation,” Soft Computing, vol. 9, no. 1, pp. 3–12, 2005, https://doi.org/10.1007/s00500-003-0328-5.

G. Squillero and A. Tonda, “Divergence of character and premature convergence: A survey of methodologies for promoting diversity in evolutionary optimization,” Information Sciences, vol. 329, pp. 782–799, 2016, https://doi.org/10.1016/j.ins.2015.09.056.

T. H. Nguyen, X. H. Nguyen, B. McKay, and Q. U. Nguyen, “Where should we stop? An investigation on early stopping for GP learning,” in Simulated Evolution and Learning (SEAL 2012), Lecture Notes in Computer Science, vol. 7673. Berlin, Heidelberg, Germany: Springer, 2012, pp. 216–227, https://doi.org/10.1007/978-3-642-34859-4_39.

X. Yao, Y. Liu, and G. Lin, “Evolutionary programming made faster,” IEEE Transactions on Evolutionary Computation, vol. 3, no. 2, pp. 82–102, 1999, https://doi.org/10.1109/4235.771163.

I. De Falco, A. Della Cioppa, and E. Tarantino, “Mutation-based genetic algorithm: Performance evaluation,” Applied Soft Computing, vol. 1, no. 4, pp. 285–299, 2002, https://doi.org/10.1016/S1568-4946(02)00021-2.

M. Schubotz, A. Satpute, A. Greiner-Petter, A. Aizawa, and B. Gipp, “Caching and reproducibility: Making data science experiments faster and FAIRer,” Frontiers in Research Metrics and Analytics, vol. 7, 2022, https://doi.org/10.3389/frma.2022.861944.

I. G. Tsoulos and V. Charilogis, “Introducing a parallel genetic algorithm for global optimization problems,” AppliedMath, vol. 4, no. 2, pp. 709–730, 2024, https://doi.org/10.3390/appliedmath4020038.

M. Furqan, E. Hartono, and M. I. Ongko, “Performance of arithmetic crossover and heuristic crossover in genetic algorithm based on alpha parameter,” IOSR Journal of Computer Engineering, vol. 19, no. 5, pp. 31–36, 2017.

S. Yang, “Genetic algorithms with memory- and elitism-based immigrants in dynamic environments,” Evolutionary Computation, vol. 16, no. 3, pp. 385–416, 2008, https://doi.org/10.1162/evco.2008.16.3.385.

B. Alhijawi and A. Awajan, “Genetic algorithms: Theory, genetic operators, solutions, and applications,” Evolutionary Intelligence, vol. 17, no. 3, pp. 1245–1256, 2024, https://doi.org/10.1007/s12065-023-00822-6.

M. Ayob and F. Al-Dhief, “Genetic algorithm based on natural selection theory for optimization problems,” Symmetry, vol. 12, no. 11, p. 1758, 2020, https://doi.org/10.3390/sym12111758.

A. Jenkins, V. Gupta, A. Myrick, and M. Lenoir, “Variations of genetic algorithms,” arXiv preprint, arXiv:1911.00490, 2019, doi: https://doi.org/10.48550/arXiv.1911.00490.

M. Jamil and X.-S. Yang, “A literature survey of benchmark functions for global optimization problems,” International Journal of Mathematical Modelling and Numerical Optimisation, vol. 4, no. 2, pp. 150–194, 2013, https://doi.org/10.1504/IJMMNO.2013.055204.

A. Borisenko, M. Haidl, and S. Gorlatch, “A GPU parallelization of branch-and-bound for multiproduct batch plants optimization,” The Journal of Supercomputing, vol. 73, no. 2, pp. 639-651, 2017. https://doi.org/10.1007/s11227-016-1784-x.

A. Borisenko and S. Gorlatch, “Efficient GPU-parallelization of batch plants design using metaheuristics with parameter tuning,” Journal of Parallel and Distributed Computing, vol. 154, pp. 74-81, 2021. https://doi.org/10.1016/j.jpdc.2021.03.012.

Downloads

Published

2026-09-30

How to Cite

Kravtsov, D., & Poletaev, N. (2026). An Improved Genetic Algorithm for Hyperparameter Optimization in Predictive Modeling. International Journal of Computing, 25(3), 438-445. https://doi.org/10.47839/ijc.25.3.4839

Issue

Section

Articles