What is the relationship between memorization and generalization in AI? Is there a fundamental tradeoff? In infinitefaculty.substack.com/p/memorizati... I’ve reviewed some of the evolving perspectives on memorization & generalization in machine learning, from classic perspectives through LLMs.
Memorization vs. generalization in deep learning: implicit biases, benign overfitting, and moreOr: how I learned to stop worrying and love the memorizationinfinitefaculty.substack.com