人工神经网络 - 版本历史

2025年6月14日 (六) 07:12 Word2vec

2025-06-14T07:12:01Z

薄荷：/* 面向复杂系统的人工智能读书会 */

2022-01-09T04:11:19Z

面向复杂系统的人工智能读书会

薄荷：/* 混合层级深度模型（Compound hierarchical-deep models） */

2022-01-09T04:10:53Z

混合层级深度模型（Compound hierarchical-deep models）

薄荷：/* 混合层级深度模型（Compound hierarchical-deep models） */

2022-01-09T04:10:23Z

混合层级深度模型（Compound hierarchical-deep models）

2022年1月9日 (日) 04:09 薄荷

2022-01-09T04:09:30Z

显示更改

2022年1月9日 (日) 04:06 薄荷

2022-01-09T04:06:15Z

薄荷：/* 图片 */

2022-01-09T04:01:12Z

图片

2022年1月9日 (日) 03:59 薄荷

2022-01-09T03:59:10Z

显示更改

2022年1月9日 (日) 03:10 薄荷

2022-01-09T03:10:57Z

是趣木木呀：/* 编者推荐 */

2022-01-07T07:32:17Z

编者推荐

@@ 第49行： / 第49行： @@
-[https://en.wikipedia.org/wiki/Vanishing_gradient_problem 梯度消失问题]影响使用反向传播的多层[前馈神经网络https://en.wikipedia.org/wiki/Feedforward_neural_network 前馈神经网络] 和[https://en.wikipedia.org/wiki/Recurrent_neural_network 循环神经网络]（RNN）。<ref name="HOCH19912">S. Hochreiter., "[http://people.idsia.ch/~juergen/SeppHochreiter1991ThesisAdvisorSchmidhuber.pdf Untersuchungen zu dynamischen neuronalen Netzen]," ''Diploma thesis. Institut f. Informatik, Technische Univ. Munich. Advisor: J. Schmidhuber'', 1991.</ref><ref name="HOCH2001">{{cite book|url=https://books.google.com/books?id=NWOcMVA64aAC|title=A Field Guide to Dynamical Recurrent Networks|last=Hochreiter|first=S.|last2=et al.|date=15 January 2001|publisher=John Wiley & Sons|year=|location=|pages=|chapter=Gradient flow in recurrent nets: the difficulty of learning long-term dependencies|editor-last2=Kremer|editor-first2=Stefan C.|editor-first1=John F.|editor-last1=Kolen}}</ref> 由于梯度从一层到另一层传播，它们随着层数指数级减小，这样阻碍了依赖这些误差的的神经元权重的调整，尤其影响深度网络。
+[https://en.wikipedia.org/wiki/Vanishing_gradient_problem 梯度消失问题]影响使用反向传播的多层[[前馈神经网络]] [[前馈神经网络|和]][https://en.wikipedia.org/wiki/Recurrent_neural_network 循环神经网络]（RNN）。<ref name="HOCH19912">S. Hochreiter., "[http://people.idsia.ch/~juergen/SeppHochreiter1991ThesisAdvisorSchmidhuber.pdf Untersuchungen zu dynamischen neuronalen Netzen]," ''Diploma thesis. Institut f. Informatik, Technische Univ. Munich. Advisor: J. Schmidhuber'', 1991.</ref><ref name="HOCH2001">{{cite book|url=https://books.google.com/books?id=NWOcMVA64aAC|title=A Field Guide to Dynamical Recurrent Networks|last=Hochreiter|first=S.|last2=et al.|date=15 January 2001|publisher=John Wiley & Sons|year=|location=|pages=|chapter=Gradient flow in recurrent nets: the difficulty of learning long-term dependencies|editor-last2=Kremer|editor-first2=Stefan C.|editor-first1=John F.|editor-last1=Kolen}}</ref> 由于梯度从一层到另一层传播，它们随着层数指数级减小，这样阻碍了依赖这些误差的的神经元权重的调整，尤其影响深度网络。
 为了解决这个问题，[https://en.wikipedia.org/wiki/J%C3%BCrgen_Schmidhuber Schmidhuber]采用了一种多层网络结构，通过[https://en.wikipedia.org/wiki/Unsupervised_learning 无监督学习]每次预训练一级然后使用反向传播很好地调整<ref name="SCHMID1992">J. Schmidhuber., "Learning complex, extended sequences using the principle of history compression," ''Neural Computation'', 4, pp. 234–242, 1992.</ref>。例如，Behnke 在图像重建和人脸定位中只依赖梯度符号。<ref>{{cite book|url=http://www.ais.uni-bonn.de/books/LNCS2766.pdf|title=Hierarchical Neural Networks for Image Interpretation.|publisher=Springer|year=2003|series=Lecture Notes in Computer Science|volume=2766|author=Sven Behnke}}</ref>
@@ 第81行： / 第81行： @@
 ==模型==
-一个“人工神经网络”是一个称为[https://en.wikipedia.org/wiki/Artificial_neurons 人工神经元]的简单元素的网络，它们接收输入，根据输入改变内部状态（“激活”），然后依靠输入和激活产生输出，通过连接某些神经元的输出到其他神经元的输入的“网络”形式构成了一个[https://en.wikipedia.org/wiki/Directed_graph 有向的][https://en.wikipedia.org/wiki/Weighted_graph 有权图]。权重和计算[https://en.wikipedia.org/wiki/Activation_function 激活的函数]可以被称为“学习”的过程改变，这被[https://en.wikipedia.org/wiki/Learning_rule 学习规则]控制。<ref name=Zell1994ch5.2>{{cite book |last=Zell |first=Andreas |year=1994 |title=Simulation Neuronaler Netze |trans-title=Simulation of Neural Networks |language=German |edition=1st |publisher=Addison-Wesley  |chapter=chapter 5.2}}</ref>
+一个“人工神经网络”是一个称为[https://en.wikipedia.org/wiki/Artificial_neurons 人工神经元]的简单元素的网络，它们接收输入，根据输入改变内部状态（“激活”），然后依靠输入和激活产生输出，通过连接某些神经元的输出到其他神经元的输入的“网络”形式构成了一个[https://en.wikipedia.org/wiki/Directed_graph 有向的][https://en.wikipedia.org/wiki/Weighted_graph 有权图]。权重和计算[https://en.wikipedia.org/wiki/Activation_function 激活的函数]可以被称为“学习”的过程改变，这被[https://en.wikipedia.org/wiki/Learning_rule 学习规则]控制。<ref name="Zell1994ch5.2">{{cite book |last=Zell |first=Andreas |year=1994 |title=Simulation Neuronaler Netze |trans-title=Simulation of Neural Networks |language=German |edition=1st |publisher=Addison-Wesley  |chapter=chapter 5.2}}</ref>
 ===人工神经网络的组成部分（Components of an artificial neural network）===

←上一版本		2022年1月9日 (日) 04:11的版本
第702行：		第702行：


−	~~本词条内容翻译自 en.wikipedia.org，遵守~~ CC3.~~0协议~~	+	'''本词条内容源自wikipedia及公开资料，遵守 CC3.0协议。'''
	[[Category:人工智能]]		[[Category:人工智能]]

@@ 第428行： / 第428行： @@
 === 混合层级深度模型（Compound hierarchical-deep models） ===
-混合层级深度模型构成了带非参数[https://en.wikipedia.org/wiki/Bayesian_network 贝叶斯模型]的深度网络。[https://en.wikipedia.org/wiki/Feature_(machine_learning) 特征]可以使用像DBN<ref name="hinton2006" />，DBM<ref name="ref3">{{cite journal|last1=Hinton|first1=Geoffrey|last2=Salakhutdinov|first2=Ruslan|date=2009|title=Efficient Learning of Deep Boltzmann Machines|url=http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS09_SalakhutdinovH.pdf|volume=3|pages=448–455}}</ref>,深度自动编码器<ref name="ref15">{{cite journal|last2=Bengio|first2=Yoshua|last3=Louradour|first3=Jerdme|last4=Lamblin|first4=Pascal|date=2009|title=Exploring Strategies for Training Deep Neural Networks|url=http://dl.acm.org/citation.cfm?id=1577070|journal=The Journal of Machine Learning Research|volume=10|pages=1–40|last1=Larochelle|first1=Hugo}}</ref>，卷积变体<ref name="ref39">{{cite journal|last2=Carpenter|first2=Blake|date=2011|title=Text Detection and Character Recognition in Scene Images with Unsupervised Feature Learning|url=http://www.iapr-tc11.org/archive/icdar2011/fileup/PDF/4520a440.pdf|journal=|volume=|pages=440–445|via=|last1=Coates|first1=Adam}}</ref><ref name="ref40">{{cite journal|last2=Grosse|first2=Roger|date=2009|title=Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations|url=http://portal.acm.org/citation.cfm?|journal=Proceedings of the 26th Annual International Conference on Machine Learning|pages=1–8|last1=Lee|first1=Honglak}}</ref>，ssRAM，<ref name="ref32" />深度编码网络，<ref name="ref41">{{cite journal|last2=Zhang|first2=Tong|date=2010|title=Deep Coding Network|url=http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2010_1077.pdf|journal=Advances in Neural . . .|pages=1–9|last1=Lin|first1=Yuanqing}}</ref>带稀疏特征学习的DBN，<ref name="ref42">{{cite journal|last2=Boureau|first2=Y-Lan|date=2007|title=Sparse Feature Learning for Deep Belief Networks|url=http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2007_1118.pdf|journal=Advances in Neural Information Processing Systems|volume=23|pages=1–8|last1=Ranzato|first1=Marc Aurelio}}</ref>RNN,<ref name="ref43">{{cite journal|last2=Lin|first2=Clif|date=2011|title=Parsing Natural Scenes and Natural Language with Recursive Neural Networks|url=http://machinelearning.wustl.edu/mlpapers/paper_files/ICML2011Socher_125.pdf|journal=Proceedings of the 26th International Conference on Machine Learning|last1=Socher|first1=Richard}}</ref>条件DBN，<ref name="ref44">{{cite journal|last2=Hinton|first2=Geoffrey|date=2006|title=Modeling Human Motion Using Binary Latent Variables|url=http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2006_693.pdf|journal=Advances in Neural Information Processing Systems|last1=Taylor|first1=Graham}}</ref>去噪自动编码器的深度结构学习<ref name="ref45">{{cite journal|last2=Larochelle|first2=Hugo|date=2008|title=Extracting and composing robust features with denoising autoencoders|url=http://portal.acm.org/citation.cfm?journal=Proceedings of the 25th international conference on Machine learning – ICML '08|pages=1096–1103|last1=Vincent|first1=Pascal}}</ref>。这提供了更好的表示，允许更快的学习和高维数据下更精确的分类。然而，这些结果在学习带少示例的异常类时表现很差，因为所有的网络单元都参与表示输入（分布式表征）并且必须一起被调整（高[https://en.wikipedia.org/wiki/Degree_of_freedom 自由度]）。限制自由度减少了要学习的参数数量，使从新的例子中的新的类学习更容易。[https://en.wikipedia.org/wiki/Hierarchical_Bayesian_model 层次贝叶斯模型]允许从少量示例中学习，例如<ref name="ref34">{{cite journal|last2=Perfors|first2=Amy|last3=Tenenbaum|first3=Joshua|date=2007|title=Learning overhypotheses with hierarchical Bayesian models|journal=Developmental Science|volume=10|issue=3|pages=307–21|last1=Kemp|first1=Charles}}</ref><ref name="ref37">{{cite journal|last2=Tenenbaum|first2=Joshua|date=2007|title=Word learning as Bayesian inference|journal=Psychol. Rev.|volume=114|issue=2|pages=245–72|last1=Xu|first1=Fei}}</ref><ref name="ref46">{{cite journal|last2=Polatkan|first2=Gungor|date=2011|title=The Hierarchical Beta Process for Convolutional Factor Analysis and Deep Learning|url=http://machinelearning.wustl.edu/mlpapers/paper_files/ICML2011Chen_251.pdf|journal=Machine Learning . . .|last1=Chen|first1=Bo}}</ref><ref name="ref47">{{cite journal|last2=Fergus|first2=Rob|date=2006|title=One-shot learning of object categories|journal=IEEE Transactions on Pattern Analysis and Machine Intelligence|volume=28|issue=4|pages=594–611|last1=Fei-Fei|first1=Li}}</ref><ref name="ref48">{{cite journal|last2=Dunson|first2=David|date=2008|title=The Nested Dirichlet Process|url=http://amstat.tandfonline.com/doi/full/10.1198/016214508000000553|journal=Journal of the American Statistical Association|volume=103|issue=483|pages=1131–1154|last1=Rodriguez|first1=Abel}}</ref>计算机视觉，[https://en.wikipedia.org/wiki/Statistics 统计学] 和认知科学。
+混合层级深度模型构成了带非参数[https://en.wikipedia.org/wiki/Bayesian_network 贝叶斯模型]的深度网络。[https://en.wikipedia.org/wiki/Feature_(machine_learning) 特征]可以使用像DBN<ref name="hinton2006" />，DBM<ref name="ref3">{{cite journal|last1=Hinton|first1=Geoffrey|last2=Salakhutdinov|first2=Ruslan|date=2009|title=Efficient Learning of Deep Boltzmann Machines|url=http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS09_SalakhutdinovH.pdf|volume=3|pages=448–455}}</ref>,深度自动编码器<ref name="ref15">{{cite journal|last2=Bengio|first2=Yoshua|last3=Louradour|first3=Jerdme|last4=Lamblin|first4=Pascal|date=2009|title=Exploring Strategies for Training Deep Neural Networks|url=http://dl.acm.org/citation.cfm?id=1577070|journal=The Journal of Machine Learning Research|volume=10|pages=1–40|last1=Larochelle|first1=Hugo}}</ref>，卷积变体<ref name="ref39">{{cite journal|last2=Carpenter|first2=Blake|date=2011|title=Text Detection and Character Recognition in Scene Images with Unsupervised Feature Learning|url=http://www.iapr-tc11.org/archive/icdar2011/fileup/PDF/4520a440.pdf|journal=|volume=|pages=440–445|via=|last1=Coates|first1=Adam}}</ref><ref name="ref40">{{cite journal|last2=Grosse|first2=Roger|date=2009|title=Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations|url=http://portal.acm.org/citation.cfm?|journal=Proceedings of the 26th Annual International Conference on Machine Learning|pages=1–8|last1=Lee|first1=Honglak}}</ref>，ssRAM，<ref name="ref32" />深度编码网络，<ref name="ref41">{{cite journal|last2=Zhang|first2=Tong|date=2010|title=Deep Coding Network|url=http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2010_1077.pdf|journal=Advances in Neural . . .|pages=1–9|last1=Lin|first1=Yuanqing}}</ref>带稀疏特征学习的DBN，<ref name="ref42">{{cite journal|last2=Boureau|first2=Y-Lan|date=2007|title=Sparse Feature Learning for Deep Belief Networks|url=http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2007_1118.pdf|journal=Advances in Neural Information Processing Systems|volume=23|pages=1–8|last1=Ranzato|first1=Marc Aurelio}}</ref>RNN,<ref name="ref43">{{cite journal|last2=Lin|first2=Clif|date=2011|title=Parsing Natural Scenes and Natural Language with Recursive Neural Networks|url=http://machinelearning.wustl.edu/mlpapers/paper_files/ICML2011Socher_125.pdf|journal=Proceedings of the 26th International Conference on Machine Learning|last1=Socher|first1=Richard}}</ref>条件DBN，<ref name="ref44">{{cite journal|last2=Hinton|first2=Geoffrey|date=2006|title=Modeling Human Motion Using Binary Latent Variables|url=http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2006_693.pdf|journal=Advances in Neural Information Processing Systems|last1=Taylor|first1=Graham}}</ref>去噪自动编码器的深度结构学习<ref name="ref45">{{cite journal|last2=Larochelle|first2=Hugo|date=2008|title=Extracting and composing robust features with denoising autoencoders|url=http://portal.acm.org/citation.cfm?|journal=Proceedings of the 25th international conference on Machine learning – ICML '08|pages=1096–1103|last1=Vincent|first1=Pascal}}</ref>。这提供了更好的表示，允许更快的学习和高维数据下更精确的分类。然而，这些结果在学习带少示例的异常类时表现很差，因为所有的网络单元都参与表示输入（分布式表征）并且必须一起被调整（高[https://en.wikipedia.org/wiki/Degree_of_freedom 自由度]）。限制自由度减少了要学习的参数数量，使从新的例子中的新的类学习更容易。[https://en.wikipedia.org/wiki/Hierarchical_Bayesian_model 层次贝叶斯模型]允许从少量示例中学习，例如<ref name="ref34">{{cite journal|last2=Perfors|first2=Amy|last3=Tenenbaum|first3=Joshua|date=2007|title=Learning overhypotheses with hierarchical Bayesian models|journal=Developmental Science|volume=10|issue=3|pages=307–21|last1=Kemp|first1=Charles}}</ref><ref name="ref37">{{cite journal|last2=Tenenbaum|first2=Joshua|date=2007|title=Word learning as Bayesian inference|journal=Psychol. Rev.|volume=114|issue=2|pages=245–72|last1=Xu|first1=Fei}}</ref><ref name="ref46">{{cite journal|last2=Polatkan|first2=Gungor|date=2011|title=The Hierarchical Beta Process for Convolutional Factor Analysis and Deep Learning|url=http://machinelearning.wustl.edu/mlpapers/paper_files/ICML2011Chen_251.pdf|journal=Machine Learning . . .|last1=Chen|first1=Bo}}</ref><ref name="ref47">{{cite journal|last2=Fergus|first2=Rob|date=2006|title=One-shot learning of object categories|journal=IEEE Transactions on Pattern Analysis and Machine Intelligence|volume=28|issue=4|pages=594–611|last1=Fei-Fei|first1=Li}}</ref><ref name="ref48">{{cite journal|last2=Dunson|first2=David|date=2008|title=The Nested Dirichlet Process|url=http://amstat.tandfonline.com/doi/full/10.1198/016214508000000553|journal=Journal of the American Statistical Association|volume=103|issue=483|pages=1131–1154|last1=Rodriguez|first1=Abel}}</ref>计算机视觉，[https://en.wikipedia.org/wiki/Statistics 统计学] 和认知科学。
 混合HD结构目的是整合HB和深度网络的特征。混合HDP-DBM结构是一种作为层级模型的[https://en.wikipedia.org/wiki/Hierarchical_Dirichlet_process 层级狄利克雷过程]与DBM结构合并。这是全[https://en.wikipedia.org/wiki/Generative_model 生成模型]，从流经模型层的抽象概念中生成，它可以分析在异常类中看起来“合理的”自然的新例子。所以的层级通过最大化一个共同[https://en.wikipedia.org/wiki/Log_probability 对数概率][https://en.wikipedia.org/wiki/Score_(statistics) 分数]被共同学习。<ref name="ref38">{{cite journal|last2=Joshua|first2=Tenenbaum|date=2012|title=Learning with Hierarchical-Deep Models|journal=IEEE Transactions on Pattern Analysis and Machine Intelligence|volume=35|issue=8|pages=1958–71|last1=Ruslan|first1=Salakhutdinov}}</ref>

@@ 第428行： / 第428行： @@
 === 混合层级深度模型（Compound hierarchical-deep models） ===
-混合层级深度模型构成了带非参数[https://en.wikipedia.org/wiki/Bayesian_network 贝叶斯模型]的深度网络。[https://en.wikipedia.org/wiki/Feature_(machine_learning) 特征]可以使用像DBN<ref name="hinton2006" />，DBM<ref name="ref3">{{cite journal|last1=Hinton|first1=Geoffrey|last2=Salakhutdinov|first2=Ruslan|date=2009|title=Efficient Learning of Deep Boltzmann Machines|url=http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS09_SalakhutdinovH.pdf|volume=3|pages=448–455}}</ref>,深度自动编码器<ref name="ref15">{{cite journal|last2=Bengio|first2=Yoshua|last3=Louradour|first3=Jerdme|last4=Lamblin|first4=Pascal|date=2009|title=Exploring Strategies for Training Deep Neural Networks|url=http://dl.acm.org/citation.cfm?id=1577070|journal=The Journal of Machine Learning Research|volume=10|pages=1–40|last1=Larochelle|first1=Hugo}}</ref>，卷积变体<ref name="ref39">{{cite journal|last2=Carpenter|first2=Blake|date=2011|title=Text Detection and Character Recognition in Scene Images with Unsupervised Feature Learning|url=http://www.iapr-tc11.org/archive/icdar2011/fileup/PDF/4520a440.pdf|journal=|volume=|pages=440–445|via=|last1=Coates|first1=Adam}}</ref><ref name="ref40">{{cite journal|last2=Grosse|first2=Roger|date=2009|title=Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations|url=http://portal.acm.org/citation.cfm?journal=Proceedings of the 26th Annual International Conference on Machine Learning|pages=1–8|last1=Lee|first1=Honglak}}</ref>，ssRAM，<ref name="ref32" />深度编码网络，<ref name="ref41">{{cite journal|last2=Zhang|first2=Tong|date=2010|title=Deep Coding Network|url=http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2010_1077.pdf|journal=Advances in Neural . . .|pages=1–9|last1=Lin|first1=Yuanqing}}</ref>带稀疏特征学习的DBN，<ref name="ref42">{{cite journal|last2=Boureau|first2=Y-Lan|date=2007|title=Sparse Feature Learning for Deep Belief Networks|url=http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2007_1118.pdf|journal=Advances in Neural Information Processing Systems|volume=23|pages=1–8|last1=Ranzato|first1=Marc Aurelio}}</ref>RNN,<ref name="ref43">{{cite journal|last2=Lin|first2=Clif|date=2011|title=Parsing Natural Scenes and Natural Language with Recursive Neural Networks|url=http://machinelearning.wustl.edu/mlpapers/paper_files/ICML2011Socher_125.pdf|journal=Proceedings of the 26th International Conference on Machine Learning|last1=Socher|first1=Richard}}</ref>条件DBN，<ref name="ref44">{{cite journal|last2=Hinton|first2=Geoffrey|date=2006|title=Modeling Human Motion Using Binary Latent Variables|url=http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2006_693.pdf|journal=Advances in Neural Information Processing Systems|last1=Taylor|first1=Graham}}</ref>去噪自动编码器的深度结构学习<ref name="ref45">{{cite journal|last2=Larochelle|first2=Hugo|date=2008|title=Extracting and composing robust features with denoising autoencoders|url=http://portal.acm.org/citation.cfm?journal=Proceedings of the 25th international conference on Machine learning – ICML '08|pages=1096–1103|last1=Vincent|first1=Pascal}}</ref>。这提供了更好的表示，允许更快的学习和高维数据下更精确的分类。然而，这些结果在学习带少示例的异常类时表现很差，因为所有的网络单元都参与表示输入（分布式表征）并且必须一起被调整（高[https://en.wikipedia.org/wiki/Degree_of_freedom 自由度]）。限制自由度减少了要学习的参数数量，使从新的例子中的新的类学习更容易。[https://en.wikipedia.org/wiki/Hierarchical_Bayesian_model 层次贝叶斯模型]允许从少量示例中学习，例如<ref name="ref34">{{cite journal|last2=Perfors|first2=Amy|last3=Tenenbaum|first3=Joshua|date=2007|title=Learning overhypotheses with hierarchical Bayesian models|journal=Developmental Science|volume=10|issue=3|pages=307–21|last1=Kemp|first1=Charles}}</ref><ref name="ref37">{{cite journal|last2=Tenenbaum|first2=Joshua|date=2007|title=Word learning as Bayesian inference|journal=Psychol. Rev.|volume=114|issue=2|pages=245–72|last1=Xu|first1=Fei}}</ref><ref name="ref46">{{cite journal|last2=Polatkan|first2=Gungor|date=2011|title=The Hierarchical Beta Process for Convolutional Factor Analysis and Deep Learning|url=http://machinelearning.wustl.edu/mlpapers/paper_files/ICML2011Chen_251.pdf|journal=Machine Learning . . .|last1=Chen|first1=Bo}}</ref><ref name="ref47">{{cite journal|last2=Fergus|first2=Rob|date=2006|title=One-shot learning of object categories|journal=IEEE Transactions on Pattern Analysis and Machine Intelligence|volume=28|issue=4|pages=594–611|last1=Fei-Fei|first1=Li}}</ref><ref name="ref48">{{cite journal|last2=Dunson|first2=David|date=2008|title=The Nested Dirichlet Process|url=http://amstat.tandfonline.com/doi/full/10.1198/016214508000000553|journal=Journal of the American Statistical Association|volume=103|issue=483|pages=1131–1154|last1=Rodriguez|first1=Abel}}</ref>计算机视觉，[https://en.wikipedia.org/wiki/Statistics 统计学] 和认知科学。
+混合层级深度模型构成了带非参数[https://en.wikipedia.org/wiki/Bayesian_network 贝叶斯模型]的深度网络。[https://en.wikipedia.org/wiki/Feature_(machine_learning) 特征]可以使用像DBN<ref name="hinton2006" />，DBM<ref name="ref3">{{cite journal|last1=Hinton|first1=Geoffrey|last2=Salakhutdinov|first2=Ruslan|date=2009|title=Efficient Learning of Deep Boltzmann Machines|url=http://machinelearning.wustl.edu/mlpapers/paper_files/AISTATS09_SalakhutdinovH.pdf|volume=3|pages=448–455}}</ref>,深度自动编码器<ref name="ref15">{{cite journal|last2=Bengio|first2=Yoshua|last3=Louradour|first3=Jerdme|last4=Lamblin|first4=Pascal|date=2009|title=Exploring Strategies for Training Deep Neural Networks|url=http://dl.acm.org/citation.cfm?id=1577070|journal=The Journal of Machine Learning Research|volume=10|pages=1–40|last1=Larochelle|first1=Hugo}}</ref>，卷积变体<ref name="ref39">{{cite journal|last2=Carpenter|first2=Blake|date=2011|title=Text Detection and Character Recognition in Scene Images with Unsupervised Feature Learning|url=http://www.iapr-tc11.org/archive/icdar2011/fileup/PDF/4520a440.pdf|journal=|volume=|pages=440–445|via=|last1=Coates|first1=Adam}}</ref><ref name="ref40">{{cite journal|last2=Grosse|first2=Roger|date=2009|title=Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations|url=http://portal.acm.org/citation.cfm?|journal=Proceedings of the 26th Annual International Conference on Machine Learning|pages=1–8|last1=Lee|first1=Honglak}}</ref>，ssRAM，<ref name="ref32" />深度编码网络，<ref name="ref41">{{cite journal|last2=Zhang|first2=Tong|date=2010|title=Deep Coding Network|url=http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2010_1077.pdf|journal=Advances in Neural . . .|pages=1–9|last1=Lin|first1=Yuanqing}}</ref>带稀疏特征学习的DBN，<ref name="ref42">{{cite journal|last2=Boureau|first2=Y-Lan|date=2007|title=Sparse Feature Learning for Deep Belief Networks|url=http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2007_1118.pdf|journal=Advances in Neural Information Processing Systems|volume=23|pages=1–8|last1=Ranzato|first1=Marc Aurelio}}</ref>RNN,<ref name="ref43">{{cite journal|last2=Lin|first2=Clif|date=2011|title=Parsing Natural Scenes and Natural Language with Recursive Neural Networks|url=http://machinelearning.wustl.edu/mlpapers/paper_files/ICML2011Socher_125.pdf|journal=Proceedings of the 26th International Conference on Machine Learning|last1=Socher|first1=Richard}}</ref>条件DBN，<ref name="ref44">{{cite journal|last2=Hinton|first2=Geoffrey|date=2006|title=Modeling Human Motion Using Binary Latent Variables|url=http://machinelearning.wustl.edu/mlpapers/paper_files/NIPS2006_693.pdf|journal=Advances in Neural Information Processing Systems|last1=Taylor|first1=Graham}}</ref>去噪自动编码器的深度结构学习<ref name="ref45">{{cite journal|last2=Larochelle|first2=Hugo|date=2008|title=Extracting and composing robust features with denoising autoencoders|url=http://portal.acm.org/citation.cfm?journal=Proceedings of the 25th international conference on Machine learning – ICML '08|pages=1096–1103|last1=Vincent|first1=Pascal}}</ref>。这提供了更好的表示，允许更快的学习和高维数据下更精确的分类。然而，这些结果在学习带少示例的异常类时表现很差，因为所有的网络单元都参与表示输入（分布式表征）并且必须一起被调整（高[https://en.wikipedia.org/wiki/Degree_of_freedom 自由度]）。限制自由度减少了要学习的参数数量，使从新的例子中的新的类学习更容易。[https://en.wikipedia.org/wiki/Hierarchical_Bayesian_model 层次贝叶斯模型]允许从少量示例中学习，例如<ref name="ref34">{{cite journal|last2=Perfors|first2=Amy|last3=Tenenbaum|first3=Joshua|date=2007|title=Learning overhypotheses with hierarchical Bayesian models|journal=Developmental Science|volume=10|issue=3|pages=307–21|last1=Kemp|first1=Charles}}</ref><ref name="ref37">{{cite journal|last2=Tenenbaum|first2=Joshua|date=2007|title=Word learning as Bayesian inference|journal=Psychol. Rev.|volume=114|issue=2|pages=245–72|last1=Xu|first1=Fei}}</ref><ref name="ref46">{{cite journal|last2=Polatkan|first2=Gungor|date=2011|title=The Hierarchical Beta Process for Convolutional Factor Analysis and Deep Learning|url=http://machinelearning.wustl.edu/mlpapers/paper_files/ICML2011Chen_251.pdf|journal=Machine Learning . . .|last1=Chen|first1=Bo}}</ref><ref name="ref47">{{cite journal|last2=Fergus|first2=Rob|date=2006|title=One-shot learning of object categories|journal=IEEE Transactions on Pattern Analysis and Machine Intelligence|volume=28|issue=4|pages=594–611|last1=Fei-Fei|first1=Li}}</ref><ref name="ref48">{{cite journal|last2=Dunson|first2=David|date=2008|title=The Nested Dirichlet Process|url=http://amstat.tandfonline.com/doi/full/10.1198/016214508000000553|journal=Journal of the American Statistical Association|volume=103|issue=483|pages=1131–1154|last1=Rodriguez|first1=Abel}}</ref>计算机视觉，[https://en.wikipedia.org/wiki/Statistics 统计学] 和认知科学。
 混合HD结构目的是整合HB和深度网络的特征。混合HDP-DBM结构是一种作为层级模型的[https://en.wikipedia.org/wiki/Hierarchical_Dirichlet_process 层级狄利克雷过程]与DBM结构合并。这是全[https://en.wikipedia.org/wiki/Generative_model 生成模型]，从流经模型层的抽象概念中生成，它可以分析在异常类中看起来“合理的”自然的新例子。所以的层级通过最大化一个共同[https://en.wikipedia.org/wiki/Log_probability 对数概率][https://en.wikipedia.org/wiki/Score_(statistics) 分数]被共同学习。<ref name="ref38">{{cite journal|last2=Joshua|first2=Tenenbaum|date=2012|title=Learning with Hierarchical-Deep Models|journal=IEEE Transactions on Pattern Analysis and Machine Intelligence|volume=35|issue=8|pages=1958–71|last1=Ruslan|first1=Salakhutdinov}}</ref>

@@ 第3行： / 第3行： @@
 |description=人工神经网络,机器学习,深度学习
 }}
 [[File:Colored_neural_network.svg.png|thumb|300px|人工神经网络是一组互相连接的节点，与[https://en.wikipedia.org/wiki/Brain 大脑]中的大量[https://en.wikipedia.org/wiki/Neuron 神经元]类似。这里，每个圆形节点代表一个[https://en.wikipedia.org/wiki/Artificial_neuron 人工神经元] ，一个箭头表示从一个人工神经元的输出连接到另一个的输入。]]
@@ 第636行： / 第634行： @@
 * [https://en.wikipedia.org/wiki/Tensor_product_network 张量积网络]
 * [https://en.wikipedia.org/wiki/Time_delay_neural_network 时延神经网络] (TDNN)
 ==引用==
 <references/>
 ==参考书目==
@@ 第677行： / 第677行： @@
 ==编者推荐==
-突触修饰的自反向传播提高了脉冲和人工神经网络的效率 | 复杂性科学顶刊精选6篇https://swarma.org/?p=29861
+===集智文章===
-重磅Science论文：用人工神经网络解决量子多体问题https://swarma.org/?p=3792
+===集智课程===
-[[Category:人工智能]]
+====[https://campus.swarma.org/course/1775 面向复杂系统的人工智能读书会]====
 本词条内容翻译自 en.wikipedia.org，遵守 CC3.0协议

@@ 第577行： / 第577行： @@
 <gallery widths="260">
-[[File:Single_layer_ann.svg.png|一个单层前馈人工神经网络。从<math>\scriptstyle x_2</math>开始的箭头为了清晰省略了。这个网络有p个输入和q个输出。在这个系统中，第q个输出的值<math>\scriptstyle {y_q}</math>被以<math>\scriptstyle {y_q} = K*({\sum({x_i}*{w_{iq}})}-{b_q}) </math>计算]]
+File:Single_layer_ann.svg.png|一个单层前馈人工神经网络。从<math>\scriptstyle x_2</math>开始的箭头为了清晰省略了。这个网络有p个输入和q个输出。在这个系统中，第q个输出的值<math>\scriptstyle {y_q}</math>被以<math>\scriptstyle {y_q} = K*({\sum({x_i}*{w_{iq}})}-{b_q}) </math>计算]]
-[[File:Two_layer_ann.svg.png|一个两层前馈人工神经网络]]
+File:Two_layer_ann.svg.png|一个两层前馈人工神经网络
-[[File:Artificial_neural_network.svg.png|一个人工神经网络]]
+File:Artificial_neural_network.svg.png|一个人工神经网络
-[[File:Ann_dependency_(graph).svg.png|一个ANN依赖图]]
+File:Ann_dependency_(graph).svg.png|一个ANN依赖图
-[[File:Single-layer_feedforward_artificial_neural_network.png|有4输入，6隐藏单元和2输出的单层前馈神经网络。给定位置状态和方向，输出转动基于控制值。 ]]
+File:Single-layer_feedforward_artificial_neural_network.png|有4输入，6隐藏单元和2输出的单层前馈神经网络。给定位置状态和方向，输出转动基于控制值。
-[[File:Two-layer_feedforward_artificial_neural_network.png|有8输入，2x8隐藏单元和2输出的两层前馈人工神经网络。给定位置状态，方向和其他环境值，输出推进基于控制值。]]
+File:Two-layer_feedforward_artificial_neural_network.png|有8输入，2x8隐藏单元和2输出的两层前馈人工神经网络。给定位置状态，方向和其他环境值，输出推进基于控制值。
-[[File:Cmac.jpg|CMAC神经网络的并行流水线结构。这种学习算法可以一步收敛。]]
+File:Cmac.jpg|CMAC神经网络的并行流水线结构。这种学习算法可以一步收敛。
 </gallery>

@@ 第184行： / 第184行： @@
 * 拟牛顿 （[https://en.wikipedia.org/wiki/Broyden%E2%80%93Fletcher%E2%80%93Goldfarb%E2%80%93Shanno_algorithm Broyden-Fletcher-Goldfarb-Shanno]），单[https://en.wikipedia.org/wiki/Secant_method 步割线]；
 * [https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm Levenberg-Marquardt]和[https://en.wikipedia.org/wiki/Conjugate_gradient_method 共轭梯度](Fletcher-Reeves 更新, Polak-Ribiére 更新, Powell-Beale 重启,标度共轭梯度)。<ref>{{cite journal|author1=M. Forouzanfar |author2=H. R. Dajani |author3=V. Z. Groza |author4=M. Bolic |author5=S. Rajan  |last-author-amp=yes |date=July 2010 | title = Comparison of Feed-Forward Neural Network Training Algorithms for Oscillometric Blood Pressure Estimation | journal = 4th Int. Workshop Soft Computing Applications | publisher = IEEE| location = Arad, Romania |url=https://www.researchgate.net/profile/Mohamad_Forouzanfar/publication/224173336_Comparison_of_Feed-Forward_Neural_Network_training_algorithms_for_oscillometric_blood_pressure_estimation/links/00b7d533829c3a7484000000.pdf?ev=pub_int_doc_dl&origin=publication_detail&inViewer=true&msrp=TyT96%2BjWOHJo%2BVhkMF4IzwHPAImSd442n%2BAkEuXj9qBmQSZ495CpxqlaOYon%2BSlEzWQElBGyJmbBCiiUOV8ImeEqPFXiIRivcrWsWmlPBYU%3D }}</ref>
-[https://en.wikipedia.org/wiki/Evolutionary_methods 进化法]<ref>{{cite journal| author1 = de Rigo, D. | author2 = Castelletti, A. | author3 = Rizzoli, A. E. | author4 = Soncini-Sessa, R. | author5 = Weber, E. |date=January 2005 | title = A selective improvement technique for fastening Neuro-Dynamic Programming in Water Resources Network Management | journal = 16th IFAC World Congress | conferenceurl = http://www.nt.ntnu.no/users/skoge/prost/proceedings/ifac2005/Index.html | editor = Pavel Zítek | volume = 16 | publisher = IFAC | location = Prague, Czech Republic | url = http://www.nt.ntnu.no/users/skoge/prost/proceedings/ifac2005/Papers/Paper4269.html
+[https://en.wikipedia.org/wiki/Evolutionary_methods 进化法]<ref>{{cite journal| author1 = de Rigo, D. | author2 = Castelletti, A. | author3 = Rizzoli, A. E. | author4 = Soncini-Sessa, R. | author5 = Weber, E. |date=January 2005 | title = A selective improvement technique for fastening Neuro-Dynamic Programming in Water Resources Network Management | journal = 16th IFAC World Congress | url = http://www.nt.ntnu.no/users/skoge/prost/proceedings/ifac2005/Index.html | editor = Pavel Zítek | volume = 16 | publisher = IFAC | location = Prague, Czech Republic | url = http://www.nt.ntnu.no/users/skoge/prost/proceedings/ifac2005/Papers/Paper4269.html
-  | accessdate = 30 December 2011}}</ref>，[https://en.wikipedia.org/wiki/Gene_expression_programming 基因表达式编程]<ref>{{cite journal|last=Ferreira|first=C.|year=2006|title=Designing Neural Networks Using Gene Expression Programming|url= http://www.gene-expression-programming.com/webpapers/Ferreira-ASCT2006.pdf|publisher= In A. Abraham, B. de Baets, M. Köppen, and B. Nickolay, eds., Applied Soft Computing Technologies: The Challenge of Complexity, pages 517–536, Springer-Verlag}}</ref>，[https://en.wikipedia.org/wiki/Simulated_annealing 模拟退火]<ref>{{cite journal| author = Da, Y. |author2=Xiurun, G. |date=July 2005 | title = An improved PSO-based ANN with simulated annealing technique | journal = New Aspects in Neurocomputing: 11th European Symposium on Artificial Neural Networks | conferenceurl = http://www.dice.ucl.ac.be/esann/proceedings/electronicproceedings.htm | editor = T. Villmann | publisher = Elsevier }}<!--| accessdate = 30 December 2011 --></ref>，[https://en.wikipedia.org/wiki/Expectation-maximization 期望最大化]，[https://en.wikipedia.org/wiki/Non-parametric_methods 非参数方法]和[https://en.wikipedia.org/wiki/Particle_swarm_optimization 粒子群算法]<ref>{{cite journal| author = Wu, J. |author2=Chen, E. |date=May 2009 | title = A Novel Nonparametric Regression Ensemble for Rainfall Forecasting Using Particle Swarm Optimization Technique Coupled with Artificial Neural Network | journal = 6th International Symposium on Neural Networks, ISNN 2009 | conferenceurl = http://www2.mae.cuhk.edu.hk/~isnn2009/ | editors = Wang, H., Shen, Y., Huang, T., Zeng, Z. | publisher = Springer }}<!--| accessdate = 1 January 2012 --></ref>是训练神经网络的其他方法。
+  | accessdate = 30 December 2011}}</ref>，[https://en.wikipedia.org/wiki/Gene_expression_programming 基因表达式编程]<ref>{{cite journal|last=Ferreira|first=C.|year=2006|title=Designing Neural Networks Using Gene Expression Programming|url= http://www.gene-expression-programming.com/webpapers/Ferreira-ASCT2006.pdf|publisher= In A. Abraham, B. de Baets, M. Köppen, and B. Nickolay, eds., Applied Soft Computing Technologies: The Challenge of Complexity, pages 517–536, Springer-Verlag}}</ref>，[https://en.wikipedia.org/wiki/Simulated_annealing 模拟退火]<ref>{{cite journal| author = Da, Y. |author2=Xiurun, G. |date=July 2005 | title = An improved PSO-based ANN with simulated annealing technique | journal = New Aspects in Neurocomputing: 11th European Symposium on Artificial Neural Networks | url = http://www.dice.ucl.ac.be/esann/proceedings/electronicproceedings.htm | editor = T. Villmann | publisher = Elsevier }}<!--| accessdate = 30 December 2011 --></ref>，[https://en.wikipedia.org/wiki/Expectation-maximization 期望最大化]，[https://en.wikipedia.org/wiki/Non-parametric_methods 非参数方法]和[https://en.wikipedia.org/wiki/Particle_swarm_optimization 粒子群算法]<ref>{{cite journal| author = Wu, J. |author2=Chen, E. |date=May 2009 | title = A Novel Nonparametric Regression Ensemble for Rainfall Forecasting Using Particle Swarm Optimization Technique Coupled with Artificial Neural Network | journal = 6th International Symposium on Neural Networks, ISNN 2009 | url = http://www2.mae.cuhk.edu.hk/~isnn2009/ | editors = Wang, H., Shen, Y., Huang, T., Zeng, Z. | publisher = Springer }}<!--| accessdate = 1 January 2012 --></ref>是训练神经网络的其他方法。
 ==== 递推收敛学习算法（Convergent recursive learning algorithm） ====