Improving Continuous Hopfield Network Stability Using Runge-Kutta Method

Authors

  • Mohammed El Alaoui
  • Mohamed Ettaouil

DOI:

https://doi.org/10.47839/ijc.22.1.2876

Keywords:

Continuous Hopfield network, Runge-Kutta method, Euler method

Abstract

Continuous Hopfield network is a recurring network that has shown its ability to solve important optimization problems. Continuous Hopfield network dynamic system is characterized by a differential equation. This equation is difficult to solve, especially for large problems. This led researchers to discretize the differential equation using Euler’s method. However, this method generally does not converge to a good solution because it is sensitive to the step size decision and initial conditions. In this work, we discretize the dynamic system of continuous Hopfield network by a new method of Runge-Kutta. This method is strong in terms of stability and performance in order to converge to a better solution. This new method introduces two phases for better network stability. The first phase targets to solve the dynamic equation by the Euler method, while the second phase allows refining the solution found in the first phase. Experimental results on benchmarks show that the proposed approach can effectively improve Hopfield neural network performance.

References

J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Feynman Comput., vol. 79, no. 4, pp. 7–19, 2018, https://doi.org/10.1201/9780429500459.

D. Calabuig, S. Gimenez, J. E. Roman, and J. F. Monserrat, “Fast hopfield neural networks using subspace projections,” Neurocomputing, vol. 73, no. 10–12, pp. 1794–1800, 2010, https://doi.org/10.1016/j.neucom.2009.12.031.

P. M. Talaván and J. Yáñez, “A continuous Hopfield network equilibrium points algorithm,” Comput. Oper. Res., vol. 32, no. 8, pp. 2179–2196, 2005, https://doi.org/10.1016/j.cor.2004.02.008.

R. D’Ambrosio and S. Di Giovacchino, “Nonlinear stability issues for stochastic Runge-Kutta methods,” Commun. Nonlinear Sci. Numer. Simul., vol. 94, p. 105549, 2021, https://doi.org/10.1016/j.cnsns.2020.105549.

M. Fitzsimmons and H. Kunze, “Combining Hopfield neural networks, with applications to grid-based mathematics puzzles,” Neural Networks, vol. 118, pp. 81–89, 2019, https://doi.org/10.1016/j.neunet.2019.06.005.

S. G. Hu et al., “Associative memory realized by a reconfigurable memristive Hopfield neural network,” Nat. Commun., vol. 6, no. May, pp. 1–5, 2015, https://doi.org/10.1038/ncomms8522.

D. Stauffer, A. Aharony, L. Da Fontoura Costa, and J. Adler, “Efficient Hopfield pattern recognition on a scale-free neural network,” Eur. Phys. J. B, vol. 32, no. 3, pp. 395–399, 2003, https://doi.org/10.1140/epjb/e2003-00114-7.

J. Li, J. Yang, and W. Wu, “Stability and periodicity of discrete Hopfield neural networks with column arbitrary-magnitude-dominant weight matrix,” Neurocomputing, vol. 82, pp. 52–61, 2012, https://doi.org/10.1016/j.neucom.2011.10.025.

S. Kumar and M. P. Singh, “Pattern recall analysis of the Hopfield neural network with a genetic algorithm,” Comput. Math. with Appl., vol. 60, no. 4, pp. 1049–1057, 2010, https://doi.org/10.1016/j.camwa.2010.03.061.

X. Y. Wang and Z. M. Li, “A color image encryption algorithm based on Hopfield chaotic neural network,” Opt. Lasers Eng., vol. 115, no. July 2018, pp. 107–118, 2019, https://doi.org/10.1016/j.optlaseng.2018.11.010.

S. Liu, W. Jiang, X. Li, and X. F. Zhou, “Lyapunov stability analysis of fractional nonlinear systems,” Appl. Math. Lett., vol. 51, pp. 13–19, 2016, https://doi.org/10.1016/j.aml.2015.06.018.

J. Tsinias, N. Kalouptsidis, and A. Bacciotti, “Lyapunov functions and stability of dynamical polysystems,” Math. Syst. Theory, vol. 19, no. 1, pp. 333–354, 1986, https://doi.org/10.1007/BF01704919.

X. S. Yang, “Optimization and metaheuristic algorithms in engineering,” Metaheuristics Water, Geotech. Transp. Eng., pp. 1–23, 2013, https://doi.org/10.1016/B978-0-12-398296-4.00001-5.

M. Campelo and R. Correa, “A Lagrangian relaxation for the maximum stable set problem,” CoRR, vol. abs/0903.1, pp. 1–12, 2009.

L. Zuo, B. Zhang, and S. Zhang, “The k-path vertex cover in product graphs of stars and complete graphs,” Int. J. Appl. Math., vol. 46, no. 1, 2016.

H. Jiang, C. M. Li, and F. Manyà, “An exact algorithm for the maximum weight clique problem in large graphs,” Proceedings of the 31st AAAI Conf. Artif. Intell. AAAI 2017, vol. 9, November 2017, pp. 830–838, https://doi.org/10.1609/aaai.v31i1.10648.

G. Gruber and F. Rendl, “Computational experience with stable set relaxations,” SIAM J. Optim., vol. 13, no. 4, pp. 1014–1028, 2003, https://doi.org/10.1137/S1052623401394092.

C. Friden, A. Hertz, and D. de Werra, “STABULUS: A technique for finding stable sets in large graphs with tabu search,” Computing, vol. 42, no. 1, pp. 35–44, 1989, https://doi.org/10.1007/BF02243141.

I. Miguel and Q. Shen, “Solution techniques for constraint satisfaction problems: Advanced approaches,” Artif. Intell. Rev., vol. 15, no. 4, pp. 269–293, 2000, https://doi.org/10.1023/A:1011096320004.

A. Haselböck, “Exploiting interchangeabilities in constraint satisfaction problems,” Ijcai, pp. 282–287, 1993, [Online]. Available at: http://www.cse.unl.edu/~choueiry/Documents/Haselboeck.pdf.

M. Jarke and J. Koch, “Query optimization in database systems,” ACM Comput. Surv., vol. 16, no. 2, pp. 111–152, 1984, https://doi.org/10.1145/356924.356928.

N. Zealand, “History of Runge-Kutta methods,” vol. 20, pp. 247–260, 1996, https://doi.org/10.1016/0168-9274(95)00108-5.

Downloads

Published

2023-03-29

How to Cite

El Alaoui, M., & Ettaouil, M. (2023). Improving Continuous Hopfield Network Stability Using Runge-Kutta Method. International Journal of Computing, 22(1), 29-34. https://doi.org/10.47839/ijc.22.1.2876

Issue

Section

Articles