Stochastic Gradient Descent-like Relaxation Is Equivalent To Metropolis Dynamics In Discrete Optimization And Inference Problems

This article discusses the similarities between Stochastic Gradient Descent (SGD) and Metropolis Monte Carlo dynamics, two commonly used algorithms in Machine Learning. The authors demonstrate that the dynamics of an SGD-like algorithm closely resemble those of Metropolis Monte Carlo with a properly chosen temperature, making it efficient for solving hard inference problems.