Improving Continuous Hopfield Network Stability Using Runge-Kutta Method
DOI:
https://doi.org/10.47839/ijc.22.1.2876Keywords:
Continuous Hopfield network, Runge-Kutta method, Euler methodAbstract
Continuous Hopfield network is a recurring network that has shown its ability to solve important optimization problems. Continuous Hopfield network dynamic system is characterized by a differential equation. This equation is difficult to solve, especially for large problems. This led researchers to discretize the differential equation using Euler’s method. However, this method generally does not converge to a good solution because it is sensitive to the step size decision and initial conditions. In this work, we discretize the dynamic system of continuous Hopfield network by a new method of Runge-Kutta. This method is strong in terms of stability and performance in order to converge to a better solution. This new method introduces two phases for better network stability. The first phase targets to solve the dynamic equation by the Euler method, while the second phase allows refining the solution found in the first phase. Experimental results on benchmarks show that the proposed approach can effectively improve Hopfield neural network performance.
References
J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Feynman Comput., vol. 79, no. 4, pp. 7–19, 2018, https://doi.org/10.1201/9780429500459.
D. Calabuig, S. Gimenez, J. E. Roman, and J. F. Monserrat, “Fast hopfield neural networks using subspace projections,” Neurocomputing, vol. 73, no. 10–12, pp. 1794–1800, 2010, https://doi.org/10.1016/j.neucom.2009.12.031.
P. M. Talaván and J. Yáñez, “A continuous Hopfield network equilibrium points algorithm,” Comput. Oper. Res., vol. 32, no. 8, pp. 2179–2196, 2005, https://doi.org/10.1016/j.cor.2004.02.008.
R. D’Ambrosio and S. Di Giovacchino, “Nonlinear stability issues for stochastic Runge-Kutta methods,” Commun. Nonlinear Sci. Numer. Simul., vol. 94, p. 105549, 2021, https://doi.org/10.1016/j.cnsns.2020.105549.
M. Fitzsimmons and H. Kunze, “Combining Hopfield neural networks, with applications to grid-based mathematics puzzles,” Neural Networks, vol. 118, pp. 81–89, 2019, https://doi.org/10.1016/j.neunet.2019.06.005.
S. G. Hu et al., “Associative memory realized by a reconfigurable memristive Hopfield neural network,” Nat. Commun., vol. 6, no. May, pp. 1–5, 2015, https://doi.org/10.1038/ncomms8522.
D. Stauffer, A. Aharony, L. Da Fontoura Costa, and J. Adler, “Efficient Hopfield pattern recognition on a scale-free neural network,” Eur. Phys. J. B, vol. 32, no. 3, pp. 395–399, 2003, https://doi.org/10.1140/epjb/e2003-00114-7.
J. Li, J. Yang, and W. Wu, “Stability and periodicity of discrete Hopfield neural networks with column arbitrary-magnitude-dominant weight matrix,” Neurocomputing, vol. 82, pp. 52–61, 2012, https://doi.org/10.1016/j.neucom.2011.10.025.
S. Kumar and M. P. Singh, “Pattern recall analysis of the Hopfield neural network with a genetic algorithm,” Comput. Math. with Appl., vol. 60, no. 4, pp. 1049–1057, 2010, https://doi.org/10.1016/j.camwa.2010.03.061.
X. Y. Wang and Z. M. Li, “A color image encryption algorithm based on Hopfield chaotic neural network,” Opt. Lasers Eng., vol. 115, no. July 2018, pp. 107–118, 2019, https://doi.org/10.1016/j.optlaseng.2018.11.010.
S. Liu, W. Jiang, X. Li, and X. F. Zhou, “Lyapunov stability analysis of fractional nonlinear systems,” Appl. Math. Lett., vol. 51, pp. 13–19, 2016, https://doi.org/10.1016/j.aml.2015.06.018.
J. Tsinias, N. Kalouptsidis, and A. Bacciotti, “Lyapunov functions and stability of dynamical polysystems,” Math. Syst. Theory, vol. 19, no. 1, pp. 333–354, 1986, https://doi.org/10.1007/BF01704919.
X. S. Yang, “Optimization and metaheuristic algorithms in engineering,” Metaheuristics Water, Geotech. Transp. Eng., pp. 1–23, 2013, https://doi.org/10.1016/B978-0-12-398296-4.00001-5.
M. Campelo and R. Correa, “A Lagrangian relaxation for the maximum stable set problem,” CoRR, vol. abs/0903.1, pp. 1–12, 2009.
L. Zuo, B. Zhang, and S. Zhang, “The k-path vertex cover in product graphs of stars and complete graphs,” Int. J. Appl. Math., vol. 46, no. 1, 2016.
H. Jiang, C. M. Li, and F. Manyà, “An exact algorithm for the maximum weight clique problem in large graphs,” Proceedings of the 31st AAAI Conf. Artif. Intell. AAAI 2017, vol. 9, November 2017, pp. 830–838, https://doi.org/10.1609/aaai.v31i1.10648.
G. Gruber and F. Rendl, “Computational experience with stable set relaxations,” SIAM J. Optim., vol. 13, no. 4, pp. 1014–1028, 2003, https://doi.org/10.1137/S1052623401394092.
C. Friden, A. Hertz, and D. de Werra, “STABULUS: A technique for finding stable sets in large graphs with tabu search,” Computing, vol. 42, no. 1, pp. 35–44, 1989, https://doi.org/10.1007/BF02243141.
I. Miguel and Q. Shen, “Solution techniques for constraint satisfaction problems: Advanced approaches,” Artif. Intell. Rev., vol. 15, no. 4, pp. 269–293, 2000, https://doi.org/10.1023/A:1011096320004.
A. Haselböck, “Exploiting interchangeabilities in constraint satisfaction problems,” Ijcai, pp. 282–287, 1993, [Online]. Available at: http://www.cse.unl.edu/~choueiry/Documents/Haselboeck.pdf.
M. Jarke and J. Koch, “Query optimization in database systems,” ACM Comput. Surv., vol. 16, no. 2, pp. 111–152, 1984, https://doi.org/10.1145/356924.356928.
N. Zealand, “History of Runge-Kutta methods,” vol. 20, pp. 247–260, 1996, https://doi.org/10.1016/0168-9274(95)00108-5.
Downloads
Published
How to Cite
Issue
Section
License
International Journal of Computing is an open access journal. Authors who publish with this journal agree to the following terms:• Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
• Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
• Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work.
