An Improved Genetic Algorithm for Hyperparameter Optimization in Predictive Modeling
DOI:
https://doi.org/10.47839/ijc.25.3.4839Keywords:
genetic algorithm, global optimization, hyperparameter optimization, machine learningAbstract
This paper presents an improved genetic algorithm (GenAlg) for hyperparameter optimization in predictive modelling. It addresses common limitations of traditional genetic algorithms (GAs), including slow convergence, high computational cost, and premature convergence. The proposed method incorporates early stopping, flexible mutation, intermediate result caching, parallel computation, and advanced crossover. These enhancements accelerate convergence to the global optimum and improve resilience against local minima.
The algorithm’s effectiveness is demonstrated through experiments on benchmark functions such as Rastrigin and Schwefel, as well as a real-world task—predicting real estate rental prices. Comparative analysis shows that the improved GenAlg outperforms PyGAD, DEAP, and Optuna in both accuracy and stability, while also reducing computational demands.
This approach is particularly useful for automatic hyperparameter tuning and can be applied to a wide range of optimization problems in machine learning, financial modelling, and engineering.
References
J. H. Holland, Adaptation in Natural and Artificial Systems, Ann Arbor, MI, USA: University of Michigan Press, 1975.
G. Wang, S. Xiao, X. Chen, and X. Li, “Application of genetic algorithm in automatic train operation,” Wireless Personal Communications, vol. 102, no. 1, 2018, https://doi.org/10.1007/s11277-017-5228-6.
F. Pezzella, G. Morganti, and G. Ciaschetti, “A genetic algorithm for the flexible job-shop scheduling problem,” Computers & Operations Research, vol. 35, no. 10, pp. 3202–3212, 2008, https://doi.org/10.1016/j.cor.2007.02.014.
D. Beasley, D. R. Bull, and R. R. Martin, “An overview of genetic algorithms: Part 1, fundamentals,” University Computing, vol. 15, no. 2, pp. 58–69, 1993.
D. Whitley, “A genetic algorithm tutorial,” Statistics and Computing, vol. 4, no. 2, pp. 65–85, 1994, https://doi.org/10.1007/BF00175354.
A. Lambora, K. Gupta, and K. Chopra, “Genetic algorithm—A literature review,” in 2019 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), Faridabad, India, 2019, pp. 380–384, https://doi.org/10.1109/COMITCon.2019.8862255.
M. Lozano and C. García-Martínez, “Hybrid metaheuristics with evolutionary algorithms specializing in intensification and diversification: Overview and progress report,” Computers & Operations Research, vol. 37, no. 3, pp. 481–497, 2010, https://doi.org/10.1016/j.cor.2009.02.010.
S. Katoch, S. S. Chauhan, and V. Kumar, “A review on genetic algorithm: Past, present, and future,” Multimedia Tools and Applications, vol. 80, no. 5, pp. 8091–8126, 2021, https://doi.org/10.1007/s11042-020-10139-6.
Y. Wang and Q. Zhu, “A hybrid genetic algorithm for flexible job shop scheduling problem with sequence-dependent setup times and job lag times,” IEEE Access, 2021, https://doi.org/10.1109/ACCESS.2021.3096007.
M. Črepinšek, S.-H. Liu, and M. Mernik, “Exploration and exploitation in evolutionary algorithms: A survey,” ACM Computing Surveys, vol. 45, no. 3, Art. no. 35, 2013, https://doi.org/10.1145/2480741.2480752.
J. J. Grefenstette, “Optimization of control parameters for genetic algorithms,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 16, no. 1, pp. 122–128, 1986, https://doi.org/10.1109/TSMC.1986.289288.
A. E. Eiben, R. Hinterding, and Z. Michalewicz, “Parameter control in evolutionary algorithms,” IEEE Transactions on Evolutionary Computation, vol. 3, no. 2, pp. 124–141, 1999, https://doi.org/10.1109/4235.771166.
Y. Jin, “A comprehensive survey of fitness approximation in evolutionary computation,” Soft Computing, vol. 9, no. 1, pp. 3–12, 2005, https://doi.org/10.1007/s00500-003-0328-5.
G. Squillero and A. Tonda, “Divergence of character and premature convergence: A survey of methodologies for promoting diversity in evolutionary optimization,” Information Sciences, vol. 329, pp. 782–799, 2016, https://doi.org/10.1016/j.ins.2015.09.056.
T. H. Nguyen, X. H. Nguyen, B. McKay, and Q. U. Nguyen, “Where should we stop? An investigation on early stopping for GP learning,” in Simulated Evolution and Learning (SEAL 2012), Lecture Notes in Computer Science, vol. 7673. Berlin, Heidelberg, Germany: Springer, 2012, pp. 216–227, https://doi.org/10.1007/978-3-642-34859-4_39.
X. Yao, Y. Liu, and G. Lin, “Evolutionary programming made faster,” IEEE Transactions on Evolutionary Computation, vol. 3, no. 2, pp. 82–102, 1999, https://doi.org/10.1109/4235.771163.
I. De Falco, A. Della Cioppa, and E. Tarantino, “Mutation-based genetic algorithm: Performance evaluation,” Applied Soft Computing, vol. 1, no. 4, pp. 285–299, 2002, https://doi.org/10.1016/S1568-4946(02)00021-2.
M. Schubotz, A. Satpute, A. Greiner-Petter, A. Aizawa, and B. Gipp, “Caching and reproducibility: Making data science experiments faster and FAIRer,” Frontiers in Research Metrics and Analytics, vol. 7, 2022, https://doi.org/10.3389/frma.2022.861944.
I. G. Tsoulos and V. Charilogis, “Introducing a parallel genetic algorithm for global optimization problems,” AppliedMath, vol. 4, no. 2, pp. 709–730, 2024, https://doi.org/10.3390/appliedmath4020038.
M. Furqan, E. Hartono, and M. I. Ongko, “Performance of arithmetic crossover and heuristic crossover in genetic algorithm based on alpha parameter,” IOSR Journal of Computer Engineering, vol. 19, no. 5, pp. 31–36, 2017.
S. Yang, “Genetic algorithms with memory- and elitism-based immigrants in dynamic environments,” Evolutionary Computation, vol. 16, no. 3, pp. 385–416, 2008, https://doi.org/10.1162/evco.2008.16.3.385.
B. Alhijawi and A. Awajan, “Genetic algorithms: Theory, genetic operators, solutions, and applications,” Evolutionary Intelligence, vol. 17, no. 3, pp. 1245–1256, 2024, https://doi.org/10.1007/s12065-023-00822-6.
M. Ayob and F. Al-Dhief, “Genetic algorithm based on natural selection theory for optimization problems,” Symmetry, vol. 12, no. 11, p. 1758, 2020, https://doi.org/10.3390/sym12111758.
A. Jenkins, V. Gupta, A. Myrick, and M. Lenoir, “Variations of genetic algorithms,” arXiv preprint, arXiv:1911.00490, 2019, doi: https://doi.org/10.48550/arXiv.1911.00490.
M. Jamil and X.-S. Yang, “A literature survey of benchmark functions for global optimization problems,” International Journal of Mathematical Modelling and Numerical Optimisation, vol. 4, no. 2, pp. 150–194, 2013, https://doi.org/10.1504/IJMMNO.2013.055204.
A. Borisenko, M. Haidl, and S. Gorlatch, “A GPU parallelization of branch-and-bound for multiproduct batch plants optimization,” The Journal of Supercomputing, vol. 73, no. 2, pp. 639-651, 2017. https://doi.org/10.1007/s11227-016-1784-x.
A. Borisenko and S. Gorlatch, “Efficient GPU-parallelization of batch plants design using metaheuristics with parameter tuning,” Journal of Parallel and Distributed Computing, vol. 154, pp. 74-81, 2021. https://doi.org/10.1016/j.jpdc.2021.03.012.
Downloads
Published
How to Cite
Issue
Section
License
International Journal of Computing is an open access journal. Authors who publish with this journal agree to the following terms:• Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
• Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
• Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.