28
个编辑
更改
跳到导航
跳到搜索
第131行：
第131行： − + − + + + + + + + + + + + + + + +
→大语言模型涌现的机制解释
=大语言模型涌现的机制解释=
=大语言模型涌现的机制解释=
=参考文献=
=== 渗流模型 ===
<references />
Lubana, E. S., Kawaguchi, K., Dick, R. P., & Tanaka, H. (2024). A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language (arXiv:2408.12578). arXiv. <nowiki>https://doi.org/10.48550/arXiv.2408.12578</nowiki>
=== Quantization Model ===
The Quantization Model of Neural Scaling. <nowiki>https://arxiv.org/abs/2303.13506</nowiki>
=== 大模型的顿悟现象（Grokking） ===
* Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets，<nowiki>https://arxiv.org/pdf/2201.02177</nowiki>
* Towards Understanding Grokking: An Effective Theory of Representation Learning. <nowiki>https://proceedings.neurips.cc/paper_files/paper/2022/hash/dfc310e81992d2e4cedc09ac47eff13e-Abstract-Conference.html</nowiki>
=== Double Descent ===
Deep Double Descent: Where Bigger Models and More Data Hurt. <nowiki>https://arxiv.org/abs/1912.02292</nowiki>
参考文献<references />