更改

=引言=

=简介=

[[深度神经网络]]（DNN）在语音合成<ref>Zhizheng Wu, Oliver Watts, and Simon King.  Merlin:  An open source neural network speech synthesis system.  In SSW, pages 202–207, 2016.</ref>、图像识别<ref name=":2">Alex  Krizhevsky,  Ilya  Sutskever,  and  Geoffrey  E  Hinton.  Imagenet  classification  with  deep  convolutional  neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.</ref><ref>Edgar  Xi,  Selina  Bing,  and  Yang  Jin. Capsule  network  performance  on  complex  data. arXiv  preprint arXiv:1712.03480, 2017.</ref>和翻译<ref>I Sutskever, O Vinyals, and QV Le. Sequence to sequence learning with neural networks. Advances in NIPS, 2014.</ref>等各个领域都表现出了目前最先进的性能。这些巨大的进步是由于将深度学习技术<ref>Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436–444, 2015.</ref>引入[[人工神经网络]]，以及将GPU用于高速计算<ref>Rajat Raina, Anand Madhavan, and Andrew Y Ng. Large-scale deep unsupervised learning using graphics processors. In Proceedings of the 26th annual international conference on machine learning, pages 873–880. ACM, 2009.</ref>。然而，DNN的性能在多个方面仍然是神秘的。例如，基本的[[机器学习]]理论表明，具有足够参数来完全记忆大型图像数据集的模型应该大过拟合训练数据，导致泛化能力差，特别是在未正则化的模型中<ref>Chiyuan  Zhang,  Samy  Bengio,  Moritz  Hardt,  Benjamin  Recht,  and  Oriol  Vinyals.  Understanding  deep  learning requires rethinking generalization. arXiv preprint arXiv:1611.03530, 2016.</ref>。然而，在实践中，即使没有显式正则化，深度神经网络也具有良好的泛化性能<ref>Behnam  Neyshabur,  Srinadh  Bhojanapalli,  David  McAllester,  and  Nati  Srebro.  Exploring  generalization  in  deep learning. In Advances in Neural Information Processing Systems, pages 5947–5956, 2017.</ref>。虽然众所周知，人工神经网络可以近似任何给定的函数<ref>Kurt Hornik, Maxwell Stinchcombe, and Halbert White.  Multilayer feedforward networks are universal approximators. Neural networks, 2(5):359–366, 1989.</ref>，但他们得出的函数是如何超越训练数据进行泛化的，这一点还不太清楚。

[[深度神经网络]]（DNN）在语音合成<ref>Zhizheng Wu, Oliver Watts, and Simon King.  Merlin:  An open source neural network speech synthesis system.  In SSW, pages 202–207, 2016.</ref>、图像识别<ref name=":2">Alex  Krizhevsky,  Ilya  Sutskever,  and  Geoffrey  E  Hinton.  Imagenet  classification  with  deep  convolutional  neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012.</ref><ref>Edgar  Xi,  Selina  Bing,  and  Yang  Jin. Capsule  network  performance  on  complex  data. arXiv  preprint arXiv:1712.03480, 2017.</ref>和翻译<ref>I Sutskever, O Vinyals, and QV Le. Sequence to sequence learning with neural networks. Advances in NIPS, 2014.</ref>等各个领域都表现出了目前最先进的性能。这些巨大的进步是由于将深度学习技术<ref>Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436–444, 2015.</ref>引入[[人工神经网络]]，以及将GPU用于高速计算<ref>Rajat Raina, Anand Madhavan, and Andrew Y Ng. Large-scale deep unsupervised learning using graphics processors. In Proceedings of the 26th annual international conference on machine learning, pages 873–880. ACM, 2009.</ref>。然而，DNN的性能在多个方面仍然是神秘的。例如，基本的[[机器学习]]理论表明，具有足够参数来完全记忆大型图像数据集的模型应该大过拟合训练数据，导致泛化能力差，特别是在未正则化的模型中<ref>Chiyuan  Zhang,  Samy  Bengio,  Moritz  Hardt,  Benjamin  Recht,  and  Oriol  Vinyals.  Understanding  deep  learning requires rethinking generalization. arXiv preprint arXiv:1611.03530, 2016.</ref>。然而，在实践中，即使没有显式正则化，深度神经网络也具有良好的泛化性能<ref>Behnam  Neyshabur,  Srinadh  Bhojanapalli,  David  McAllester,  and  Nati  Srebro.  Exploring  generalization  in  deep learning. In Advances in Neural Information Processing Systems, pages 5947–5956, 2017.</ref>。虽然众所周知，人工神经网络可以近似任何给定的函数<ref>Kurt Hornik, Maxwell Stinchcombe, and Halbert White.  Multilayer feedforward networks are universal approximators. Neural networks, 2(5):359–366, 1989.</ref>，但他们得出的函数是如何超越训练数据进行泛化的，这一点还不太清楚。