In machine learning, grokking, or delayed generalization, is a transition to generalization that occurs many training iterations after the interpolation threshold, after many iterations of seemingly little progress, as opposed to the usual process where generalization occurs slowly and progressively once the interpolation threshold has been reached.
Questions tagged [grokking]
1 questions
2
votes
1 answer
How to train a neural network to generalize out-of-distribution (e.g., sin function extrapolation) without data leakage?
I am trying to train a neural network to approximate the sin(x) function, but I want it to generalize outside the range of the training data. Specifically, I train the network on x values within [-π, π] and test it on a disjoint range, such as [π,…
Amirhossein Rezaei
- 623
- 3
- 14