bibkey: “xuvardisafran2026grok” authors: “Mingyue Xu; Gal Vardi; Itay Safran” year: 2026 title: “To Grok Grokking: Provable Grokking in Ridge Regression” doi: null claim: “Gradient descent with weight decay on over-parameterized linear regression provably overfits early, generalizes poorly for a long time, and then reaches arbitrarily small population error, with the delay governed by weight decay and initialization variance.” strata_touched: [] license: “citation-only” triage: “anchor” url: “https://arxiv.org/abs/2601.19791”

To Grok Grokking: Provable Grokking in Ridge Regression

Verified locator

ICML 2026; arXiv:2601.19791. Theorem A.2’s proof gives the null-component recursion theta_perp^{t+1} = (1 - eta lambda) theta_perp^t; Theorems 4.1-4.2 separate the fast empirical-error convergence from the slow population-error convergence and Eq. (8) lower-bounds the population threshold time by a term of order log(initialization variance)/(eta lambda). Direct antecedent of the escaped-readout decay and the logarithmic shift in the fiber-law volume.

Declared identifiers: https://arxiv.org/abs/2601.19791.