更改

[https://en.wikipedia.org/wiki/Vanishing_gradient_problem 梯度消失问题]影响使用反向传播的多层[前馈神经网络https://en.wikipedia.org/wiki/Feedforward_neural_network 前馈神经网络] 和[https://en.wikipedia.org/wiki/Recurrent_neural_network 循环神经网络]（RNN）。<ref name="HOCH19912">S. Hochreiter., "[http://people.idsia.ch/~juergen/SeppHochreiter1991ThesisAdvisorSchmidhuber.pdf Untersuchungen zu dynamischen neuronalen Netzen]," ''Diploma thesis. Institut f. Informatik, Technische Univ. Munich. Advisor: J. Schmidhuber'', 1991.</ref><ref name="HOCH2001">{{cite book|url=https://books.google.com/books?id=NWOcMVA64aAC|title=A Field Guide to Dynamical Recurrent Networks|last=Hochreiter|first=S.|last2=et al.|date=15 January 2001|publisher=John Wiley & Sons|year=|location=|pages=|chapter=Gradient flow in recurrent nets: the difficulty of learning long-term dependencies|editor-last2=Kremer|editor-first2=Stefan C.|editor-first1=John F.|editor-last1=Kolen}}</ref> 由于梯度从一层到另一层传播，它们随着层数指数级减小，这样阻碍了依赖这些误差的的神经元权重的调整，尤其影响深度网络。

[https://en.wikipedia.org/wiki/Vanishing_gradient_problem 梯度消失问题]影响使用反向传播的多层[[前馈神经网络]] [[前馈神经网络|和]][https://en.wikipedia.org/wiki/Recurrent_neural_network 循环神经网络]（RNN）。<ref name="HOCH19912">S. Hochreiter., "[http://people.idsia.ch/~juergen/SeppHochreiter1991ThesisAdvisorSchmidhuber.pdf Untersuchungen zu dynamischen neuronalen Netzen]," ''Diploma thesis. Institut f. Informatik, Technische Univ. Munich. Advisor: J. Schmidhuber'', 1991.</ref><ref name="HOCH2001">{{cite book|url=https://books.google.com/books?id=NWOcMVA64aAC|title=A Field Guide to Dynamical Recurrent Networks|last=Hochreiter|first=S.|last2=et al.|date=15 January 2001|publisher=John Wiley & Sons|year=|location=|pages=|chapter=Gradient flow in recurrent nets: the difficulty of learning long-term dependencies|editor-last2=Kremer|editor-first2=Stefan C.|editor-first1=John F.|editor-last1=Kolen}}</ref> 由于梯度从一层到另一层传播，它们随着层数指数级减小，这样阻碍了依赖这些误差的的神经元权重的调整，尤其影响深度网络。

为了解决这个问题，[https://en.wikipedia.org/wiki/J%C3%BCrgen_Schmidhuber Schmidhuber]采用了一种多层网络结构，通过[https://en.wikipedia.org/wiki/Unsupervised_learning 无监督学习]每次预训练一级然后使用反向传播很好地调整<ref name="SCHMID1992">J. Schmidhuber., "Learning complex, extended sequences using the principle of history compression," ''Neural Computation'', 4, pp. 234–242, 1992.</ref>。例如，Behnke 在图像重建和人脸定位中只依赖梯度符号。<ref>{{cite book|url=http://www.ais.uni-bonn.de/books/LNCS2766.pdf|title=Hierarchical Neural Networks for Image Interpretation.|publisher=Springer|year=2003|series=Lecture Notes in Computer Science|volume=2766|author=Sven Behnke}}</ref>

为了解决这个问题，[https://en.wikipedia.org/wiki/J%C3%BCrgen_Schmidhuber Schmidhuber]采用了一种多层网络结构，通过[https://en.wikipedia.org/wiki/Unsupervised_learning 无监督学习]每次预训练一级然后使用反向传播很好地调整<ref name="SCHMID1992">J. Schmidhuber., "Learning complex, extended sequences using the principle of history compression," ''Neural Computation'', 4, pp. 234–242, 1992.</ref>。例如，Behnke 在图像重建和人脸定位中只依赖梯度符号。<ref>{{cite book|url=http://www.ais.uni-bonn.de/books/LNCS2766.pdf|title=Hierarchical Neural Networks for Image Interpretation.|publisher=Springer|year=2003|series=Lecture Notes in Computer Science|volume=2766|author=Sven Behnke}}</ref>

==模型==

==模型==

一个“人工神经网络”是一个称为[https://en.wikipedia.org/wiki/Artificial_neurons 人工神经元]的简单元素的网络，它们接收输入，根据输入改变内部状态（“激活”），然后依靠输入和激活产生输出，通过连接某些神经元的输出到其他神经元的输入的“网络”形式构成了一个[https://en.wikipedia.org/wiki/Directed_graph 有向的][https://en.wikipedia.org/wiki/Weighted_graph 有权图]。权重和计算[https://en.wikipedia.org/wiki/Activation_function 激活的函数]可以被称为“学习”的过程改变，这被[https://en.wikipedia.org/wiki/Learning_rule 学习规则]控制。<ref name=Zell1994ch5.2>{{cite book |last=Zell |first=Andreas |year=1994 |title=Simulation Neuronaler Netze |trans-title=Simulation of Neural Networks |language=German |edition=1st |publisher=Addison-Wesley  |chapter=chapter 5.2}}</ref>

一个“人工神经网络”是一个称为[https://en.wikipedia.org/wiki/Artificial_neurons 人工神经元]的简单元素的网络，它们接收输入，根据输入改变内部状态（“激活”），然后依靠输入和激活产生输出，通过连接某些神经元的输出到其他神经元的输入的“网络”形式构成了一个[https://en.wikipedia.org/wiki/Directed_graph 有向的][https://en.wikipedia.org/wiki/Weighted_graph 有权图]。权重和计算[https://en.wikipedia.org/wiki/Activation_function 激活的函数]可以被称为“学习”的过程改变，这被[https://en.wikipedia.org/wiki/Learning_rule 学习规则]控制。<ref name="Zell1994ch5.2">{{cite book |last=Zell |first=Andreas |year=1994 |title=Simulation Neuronaler Netze |trans-title=Simulation of Neural Networks |language=German |edition=1st |publisher=Addison-Wesley  |chapter=chapter 5.2}}</ref>

===人工神经网络的组成部分（Components of an artificial neural network）===

===人工神经网络的组成部分（Components of an artificial neural network）===