2016研读营之统计物理，网络与机器学习

Statistical Physics that will be used in my lectures

1. The Ising model and the maximum entropy distribution
2. From dynamics to equilibrium
3. Mean-field approximation and the Curie-Weiss model
4. Variational approaches
5. Random graphs
6. Cavity method and belief propagation for the Ising model
7. Thouless-Anderson-Palmer equations and the Plefka expansions

统计物理，优化与推断

1. 自旋玻璃与组合优化
2. Boltzmann分布与贝叶斯统计

• Roudi, Y.; Tyrcha J.& Hertz J., Ising model for neural data: Model quality and approximate methods for extracting functional connectivity Phys. Rev. E, 2009, 79, 051915
• Yedidia J. ; Freeman W. & Weiss Y. ,Understanding belief propagation and its generalizations International Joint Conference on Artificial Intelligence (IJCAI), 2001
• Mezard, M.; Parisi, G. & Zecchina, R. Analytic and algorithmic solution of random satisfiability problems Science, 297, 812 ,2002.
• Mezard, M. & Montanari, A. Information, Physics and Computation Oxford University press, ,2009.
• 周海军，自旋玻璃与消息传递，科学出版社，2015
• Zdeborova, L. & Krzakala, F. Statistical physics of inference: Thresholds and algorithms arXiv preprint arXiv:1511.02476, ,2015.

一些网络中问题的统计物理描述

1. 网络中的流行病传播，网络的鲁棒性与Percolation相变
2. 从四色地图问题到社区结构探测: Modularity, Stochastic Block Model及可探测相变

• Karrer, B.; Newman, M. E. J. & Zdeborova, L. Percolation on Sparse Networks Phys. Rev. Lett., American Physical Society, 113, 208702 ,2014.
• Zdeborov\'a L. & Krzakala F. Phase transitions in the coloring of random graphs Phys. Rev. E, 2007, 76, 031131
• Fortunato, S. Community detection in graphs Physics Reports, 486, 75 - 174 ,2010.
• Decelle, A.; Krzakala, F.; Moore, C. & Zdeborova, L. Inference and Phase Transitions in the Detection of Modules in Sparse Networks Phys. Rev. Lett., American Physical Society, 107, 065701 ,2011.

网络与随机矩阵

1. 邻接矩阵, 随机行走矩阵，Laplacian矩阵及它们的简单谱性质
2. Gaussian orthogonal ensemble, 谱密度，Wigner's Semi-cycle, ...
3. 统计推断，消息传递与谱方法

• Dan Spielman在Yale开的课程Spectral Graph Theory
• Chung, F. R. Spectral graph theory American Mathematical Soc., 92,1997.
• Luxburg, U. V.; Belkin, M.; Bousquet, O. & Pertinence A tutorial on spectral clustering Stat. Comput, ,2007.
• F. Krzakala et al, Spectral redemption in clustering sparse networks Proc. Natl. Acad. Sci. USA, 2013, 110, 20935-20940

从Ising模型到神经网络

1. Ising自旋玻璃模型，平均场方法和副本对称破缺
2. Ising模型反问题, Boltzmann Machine及Restricted Boltzmann Machine
3. 深度神经网络与重整化群

• Ackley, D. H.; Hinton, G. E. & Sejnowski, T. J. A learning algorithm for boltzmann machines Cognitive Science, 9, 147 - 169 ,1985.
• Kappen, H. & Rodriguez, F. B. Efficient Learning in Boltzmann Machines Using Linear Response Theory Neural Computation, 10, 1137-1156 ,1998.
• Roudi, Y.; Tyrcha, J. & Hertz, J. Ising model for neural data: Model quality and approximate methods for extracting functional connectivity Phys. Rev. E, 79, 051915 ,2009.
• Hinton, G. A practical guide to training restricted Boltzmann machines Momentum, 9, 926 ,2010
• Nielsen, MA. Neural Networks and Deep Learning - URL: http://neuralnetworksanddeeplearning. com
• Gabrie M.;Tramel E. W. & Krzakala F. , Training Restricted Boltzmann Machine via the Thouless-Anderson-Palmer free energy Advances in Neural Information Processing Systems 28, Curran Associates, Inc., 2015, 640-648