bibkey: “levibeckbarsinai2023grokking” authors: “Noam Levi; Alon Beck; Yohai Bar-Sinai” year: 2023 title: “Grokking in Linear Estimators – A Solvable Model that Groks without Understanding” doi: null claim: “Linear teacher-student regression with weight decay exhibits delayed generalization with explicitly solvable training and generalization dynamics.” strata_touched: [] license: “citation-only” triage: “anchor” url: “https://arxiv.org/abs/2310.16441”

Grokking in Linear Estimators – A Solvable Model that Groks without Understanding

Verified locator

ICLR 2024; arXiv:2310.16441. Solvable linear estimator model of delayed generalization under weight decay with Gaussian inputs. Their Eq. (12) also gives a delay without weight decay, produced by near-zero spectral modes of the input covariance (the aspect ratio there is written lambda); the fiber-law volume cites it as a high-dimensional instance of slow row-space modes. The fiber-law volume uses the same regularized least-squares gradient flow as an attributed intermediate step in its invisibility theorem and records the escaped-readout laws only as attributed remarks; its sole increment there is the escape-set formulation with an explicit witness.

Declared identifiers: https://arxiv.org/abs/2310.16441.