bibkey: “chanin2024absorption” authors: “David Chanin; James Wilken-Smith; Tomas Dulka; Hardik Bhatnagar; Satvik Golechha; Joseph Bloom” year: 2024 title: “A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders” doi: null claim: “Hierarchical features cause sparse-autoencoder latents for the parent to stop firing on child instances (absorption); varying width or sparsity does not remove it; Appendix A.2 of v6 gives a loss-decreasing family for strictly hierarchical orthogonal binary features.” strata_touched: [] license: “citation-only” triage: “anchor” url: “https://arxiv.org/abs/2409.14507”

A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders

Verified locator

NeurIPS 2025; arXiv:2409.14507, v6 dated 17 November 2025. Appendix A.2, Propositions 1-2: decoder columns f_1 and f_2 + delta f_1 (the second not unit norm) preserve exact reconstruction while the expected l1 activation falls from 2 p_11 + p_10 to (2 - delta) p_11 + p_10. Used as the empirical report and as the prior descent-family mechanism; not a global-minimizer characterization and no threshold for a nonzero child-alone probability.

Declared identifiers: https://arxiv.org/abs/2409.14507.