Grokfast: A Machine Learning Approach That Accelerates Grokking By Amplifying Slow Gradients

Grokking is a newly discovered phenomenon where overparameterized neural networks can generalize and reason beyond just memorizing the dataset. Researchers have introduced GROKFAST, an algorithm that accelerates grokking by amplifying slow gradients, making it a practical tool for a wide variety of tasks.