更改

==合理性==

==合理性==

许多著名的技术专家和学者都对技术奇点的合理性提出质疑，包括Paul Allen、Jeff Hawkins、John Holland、Jaron Lanier和Gordon Moore，他的摩尔定律经常被引用来支持这一概念。<ref name="spectrum.ieee.org"/><ref name="ieee"/><ref name="Allen"/>

许多著名的技术专家和学者都对技术奇点的合理性提出质疑，包括Paul Allen、Jeff Hawkins、John Holland、Jaron Lanier和Gordon Moore，他的摩尔定律经常被引用来支持这一概念。<ref name="spectrum.ieee.org"/><ref name="ieee"/><ref name="Allen"/>

大多数创造超人或超越人类智能的方法分为两类：人脑的智能增强和人工智能。据推测，智能增强的方法很多，包括生物工程、基因工程、益智药物、AI 助手、直接脑机接口和思维上传。因为人们正在探索通向智能爆炸的多种途径，这使得奇点出现的可能性变得更大；如果奇点不发生，所有这些方法都必将失败。<ref name="singinst.org">{{cite web|url=http://singinst.org/overview/whatisthesingularity |title=What is the Singularity? &#124; Singularity Institute for Artificial Intelligence |publisher=Singinst.org |accessdate=2011-09-09 |url-status=dead |archiveurl=https://web.archive.org/web/20110908014050/http://singinst.org/overview/whatisthesingularity/ |archivedate=2011-09-08 }}</ref>

大多数创造超人或超越人类智能的方法分为两类：人脑的智能增强和人工智能。据推测，智能增强的方法很多，包括生物工程、基因工程、益智药物、AI 助手、直接脑机接口和思维上传。因为人们正在探索通向智能爆炸的多种途径，这使得奇点出现的可能性变得更大；如果奇点不发生，所有这些方法都必将失败。<ref name="singinst.org">{{cite web|url=http://singinst.org/overview/whatisthesingularity |title=What is the Singularity? &#124; Singularity Institute for Artificial Intelligence |publisher=Singinst.org |accessdate=2011-09-09 |url-status=dead |archiveurl=https://web.archive.org/web/20110908014050/http://singinst.org/overview/whatisthesingularity/ |archivedate=2011-09-08 }}</ref>

[[Robin Hanson]] expressed skepticism of human intelligence augmentation, writing that once the "low-hanging fruit" of easy methods for increasing human intelligence have been exhausted, further improvements will become increasingly difficult to find.<ref name="hanson">{{cite web |url=https://mason.gmu.edu/~rhanson/vc.html#hanson |title=Some Skepticism |date=1998 |first=Robin |last=Hanson |author-link=Robin Hanson |accessdate=April 8, 2020}}</ref> Despite all of the speculated ways for amplifying human intelligence, non-human artificial intelligence (specifically seed AI) is the most popular option among the hypotheses that would advance the singularity.{{citation needed|date=July 2012}}

Robin Hanson对人类智能增强表示怀疑，他写道，一旦提高人类智力的“唾手可得的”简单方法用尽，进一步的改进将变得越来越难。<ref name="hanson">{{cite web |url=https://mason.gmu.edu/~rhanson/vc.html#hanson |title=Some Skepticism |date=1998 |first=Robin |last=Hanson |author-link=Robin Hanson |accessdate=April 8, 2020}}</ref>尽管有各种提高人类智能的方法，但非人类人工智能（特别是种子人工智能）仍是所有能推进奇点的假说中最受欢迎的一个。

 

Robin Hanson 对人类智能增强表示怀疑，他写道，一旦提高人类智力的“唾手可得的”简单方法用尽，进一步的改进将变得越来越难。<ref name="hanson">{{cite web |url=https://mason.gmu.edu/~rhanson/vc.html#hanson |title=Some Skepticism |date=1998 |first=Robin |last=Hanson |author-link=Robin Hanson |accessdate=April 8, 2020}}</ref>尽管有各种提高人类智能的方法，但非人类人工智能(特别是种子人工智能)仍是所有能推进奇点的假说中最受欢迎的一个。{{citation needed|date=July 2012}}

 

 

There are two logically independent, but mutually reinforcing, causes of intelligence improvements: increases in the speed of computation, and improvements to the [[algorithm]]s used.<ref name="consc.net">[http://consc.net/papers/singularity.pdf The Singularity: A Philosophical Analysis, David J. Chalmers]</ref> The former is predicted by [[Moore's law|Moore's Law]] and the forecasted improvements in hardware,<ref name="itrs">{{cite web |url=http://www.itrs.net/Links/2007ITRS/ExecSum2007.pdf |title=ITRS |accessdate=2011-09-09 |url-status=dead |archiveurl=https://web.archive.org/web/20110929173755/http://www.itrs.net/Links/2007ITRS/ExecSum2007.pdf |archivedate=2011-09-29 }}</ref> and is comparatively similar to previous technological advances. But there are some AI researchers{{who|date=March 2017}} who believe software is more important than hardware.<ref>{{Cite web|url=https://blog.chronicled.com/why-software-is-more-important-than-hardware-right-now-6a4b58feaa7c|title=Why Software Is More Important Than Hardware Right Now|last=Kulkarni|first=Ajit|date=2017-12-12|website=Chronicled|access-date=2019-02-23}}</ref>{{citation needed|date=July 2012}}

智能爆炸是否发生取决于三个因素。<ref name="david_chalmers_singularity_lecture_resources_available">David Chalmers John Locke Lecture, 10 May, Exam Schools, Oxford, [http://www.fhi.ox.ac.uk/news/2010/david_chalmers_singularity_lecture_resources_available Presenting a philosophical analysis of the possibility of a technological singularity or "intelligence explosion" resulting from recursively self-improving AI] {{webarchive|url=https://web.archive.org/web/20130115205558/http://www.fhi.ox.ac.uk/news/2010/david_chalmers_singularity_lecture_resources_available |date=2013-01-15 }}.</ref>第一个加速因素是过去的每一次改进都使新的智能增强成为可能。相反，随着智能的进步，进一步的发展将变得越来越复杂，可能会抵消智力增长的优势。平均而言，每一次改进都应该至少带来一次改进，以便继续朝着奇点的方向发展。最后，物理定律最终会阻止任何进一步的改进。

智能改进有两个逻辑上独立但又相互加强的原因：计算速度的提高和使用的算法的改进。<ref name="consc.net">[http://consc.net/papers/singularity.pdf The Singularity: A Philosophical Analysis, David J. Chalmers]</ref>前者由摩尔定律和硬件方面的改进进行预测，<ref name="itrs">{{cite web |url=http://www.itrs.net/Links/2007ITRS/ExecSum2007.pdf |title=ITRS |accessdate=2011-09-09 |url-status=dead |archiveurl=https://web.archive.org/web/20110929173755/http://www.itrs.net/Links/2007ITRS/ExecSum2007.pdf |archivedate=2011-09-29 }}</ref>与以前的技术进步比较相似。但也有一些人工智能研究人员认为软件比硬件更重要<ref>{{Cite web|url=https://blog.chronicled.com/why-software-is-more-important-than-hardware-right-now-6a4b58feaa7c|title=Why Software Is More Important Than Hardware Right Now|last=Kulkarni|first=Ajit|date=2017-12-12|website=Chronicled|access-date=2019-02-23}}</ref>{{citation needed|date=July 2012}}

智能改进有两个逻辑上独立但又相互加强的原因：计算速度的提高和使用的算法的改进。<ref name="consc.net">[http://consc.net/papers/singularity.pdf The Singularity: A Philosophical Analysis, David J. Chalmers]</ref>前者由摩尔定律和硬件方面的改进进行预测，<ref name="itrs">{{cite web |url=http://www.itrs.net/Links/2007ITRS/ExecSum2007.pdf |title=ITRS |accessdate=2011-09-09 |url-status=dead |archiveurl=https://web.archive.org/web/20110929173755/http://www.itrs.net/Links/2007ITRS/ExecSum2007.pdf |archivedate=2011-09-29 }}</ref>与以前的技术进步比较相似。但也有一些人工智能研究人员认为软件比硬件更重要<ref>{{Cite web|url=https://blog.chronicled.com/why-software-is-more-important-than-hardware-right-now-6a4b58feaa7c|title=Why Software Is More Important Than Hardware Right Now|last=Kulkarni|first=Ajit|date=2017-12-12|website=Chronicled|access-date=2019-02-23}}</ref>

 

 

在2017年，一项对2015年 NeurIPS 和 ICML 机器学习会议上发表论文的作者的电子邮件调查询问了智能爆炸的可能性。在受访者中，12% 的人认为“很有可能” ，17% 的人认为“有可能” ，21% 的人认为“可能性中等” ，24% 的人认为“不太可能” ，26% 的人认为“非常不可能”。<ref>{{cite arxiv|last1=Grace|first1=Katja|last2=Salvatier|first2=John|last3=Dafoe|first3=Allan|last4=Zhang|first4=Baobao|last5=Evans|first5=Owain|title=When Will AI Exceed Human Performance? Evidence from AI Experts|eprint=1705.08807|date=24 May 2017|class=cs.AI}}</ref>

===速度改进 ===

===速度改进 ===

Both for human and artificial intelligence, hardware improvements increase the rate of future hardware improvements. Simply put,<ref name="arstechnica">{{cite web|last=Siracusa |first=John |url=https://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/8 |title=Mac OS X 10.6 Snow Leopard: the Ars Technica review |publisher=Arstechnica.com |date=2009-08-31 |accessdate=2011-09-09}}</ref> [[Moore's Law]] suggests that if the first doubling of speed took 18 months, the second would take 18 subjective months; or 9 external months, whereafter, four months, two months, and so on towards a speed singularity.<ref name="singularity6">Eliezer Yudkowsky, 1996 [http://www.yudkowsky.net/obsolete/singularity.html "Staring into the Singularity"]</ref> An upper limit on speed may eventually be reached, although it is unclear how high this would be.  Jeff Hawkins has stated that a self-improving computer system would inevitably run into upper limits on computing power: "in the end there are limits to how big and fast computers can run. We would end up in the same place; we'd just get there a bit faster. There would be no singularity."<ref name="Hawkins">{{cite magazine |url=https://spectrum.ieee.org/computing/hardware/tech-luminaries-address-singularity |title=Tech Luminaries Address Singularity |date=1 June 2008 |magazine=[[IEEE Spectrum]]}}</ref>

很难直接将基于硅的硬件与神经元相比较。但是Berglas（2008）<ref name=:"Berglas">Berglas, Anthony (2008), Artificial Intelligence will Kill our Grandchildren, archived from the original on 2014-07-23, retrieved 2008-06-13</ref>指出计算机语音识别正在接近人类的能力，而且这种能力似乎只需要0.01%的脑容量。这个类比表明，现代计算机硬件与人脑一样强大，只差几个数量级。

 

无论对于人类智能还是人工智能，硬件改进都会提高未来硬件改进的速度。简单地说，<ref name="arstechnica">{{cite web|last=Siracusa |first=John |url=https://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/8 |title=Mac OS X 10.6 Snow Leopard: the Ars Technica review |publisher=Arstechnica.com |date=2009-08-31 |accessdate=2011-09-09}}</ref>摩尔定律认为，如果第一次速度翻倍需要18个月，第二次则需要18个月；或者额外的9个月，之后，4个月、2个月，以此类推，走向速度奇点<ref name="singularity6">Eliezer Yudkowsky, 1996 [http://www.yudkowsky.net/obsolete/singularity.html "Staring into the Singularity"]</ref> 速度的上限最终可能会达到，尽管还不清楚这会有多高。杰夫·霍金斯（Jeff Hawkins）曾表示，一个自我完善的计算机系统不可避免地会遇到计算能力的上限：“最终，计算机的规模和速度都是有限的。我们最终会在停留在同一个地方；我们只会更快到达那里。不会有奇点。”<ref name="Hawkins">{{cite magazine |url=https://spectrum.ieee.org/computing/hardware/tech-luminaries-address-singularity |title=Tech Luminaries Address Singularity |date=1 June 2008 |magazine=[[IEEE Spectrum]]}}</ref>

 

 

很难直接将基于硅的硬件与神经元相比较。但是{Harvtxt| Berglas|2008}指出计算机语音识别正在接近人类的能力，而且这种能力似乎只需要0.01%的脑容量。这个类比表明，现代计算机硬件与人脑一样强大，只差几个数量级。

 

====指数增长====

====指数增长====

 

马丁·福特 Martin Ford的《隧道中的灯光：自动化，加速技术和未来经济 The Lights in the Tunnel: Automation, Accelerating Technology and the Economy of the Future》

 

马丁·福特 Martin Ford 的《隧道中的灯光: 自动化，加速技术和未来经济 The Lights in the Tunnel: Automation, Accelerating Technology and the Economy of the Future》

[[Image:PPTMooresLawai.jpg|thumb|Ray Kurzweil writes that, due to [[paradigm shift]]s, a trend of exponential growth extends [[Moore's law]] from [[integrated circuits]] to earlier [[transistor]]s, [[vacuum tube]]s, [[relay]]s, and [[electromechanics|electromechanical]] computers. He predicts that the exponential growth will continue, and that in a few decades the computing power of all computers will exceed that of ("unenhanced") human brains, with superhuman [[artificial intelligence]] appearing around the same time.|链接=Special:FilePath/PPTMooresLawai.jpg]]

[[Image:PPTMooresLawai.jpg|thumb|Ray Kurzweil writes that, due to [[paradigm shift]]s, a trend of exponential growth extends [[Moore's law]] from [[integrated circuits]] to earlier [[transistor]]s, [[vacuum tube]]s, [[relay]]s, and [[electromechanics|electromechanical]] computers. He predicts that the exponential growth will continue, and that in a few decades the computing power of all computers will exceed that of ("unenhanced") human brains, with superhuman [[artificial intelligence]] appearing around the same time.|链接=Special:FilePath/PPTMooresLawai.jpg]]

[[资料图：摩尔超过120的定律年.png|拇指|左|摩尔定律120年的更新版本（基于[[Ray Kurzweil | Kurzweil's]][[c:文件：PPTMooresLawai.jpg|图形]]）。最近的7个数据点都是[[Nvidia GPU | Nvidia GPU]].]]

[[资料图：摩尔超过120的定律年.png|拇指|左|摩尔定律120年的更新版本（基于[[Ray Kurzweil | Kurzweil's]][[c:文件：PPTMooresLawai.jpg|图形]]）。最近的7个数据点都是[[Nvidia GPU | Nvidia GPU]].]]

摩尔定律所建议的计算技术的指数增长通常被认为是在相对不远的将来出现奇点的一个理由，许多作者已经提出了摩尔定律的推广。计算机科学家和未来主义者Hans Moravec在1998年的一本书中提到<ref>{{cite book|author=Moravec, Hans|title=Robot: Mere Machine to Transcendent Mind|year=1999|publisher=Oxford U. Press|page=61|url=https://books.google.com/books?id=fduW6KHhWtQC&pg=PA61|isbn=978-0-19-513630-2}}</ref>指数型增长曲线可以沿着集成电路之前的早期计算技术进行延伸。

摩尔定律所建议的计算技术的指数增长通常被认为是在相对不远的将来出现奇点的一个理由，许多作者已经提出了摩尔定律的推广。计算机科学家和未来主义者Hans Moravec在1998年的一本书中提到<ref>{{cite book|author=Moravec, Hans|title=Robot: Mere Machine to Transcendent Mind|year=1999|publisher=Oxford U. Press|page=61|url=https://books.google.com/books?id=fduW6KHhWtQC&pg=PA61|isbn=978-0-19-513630-2}}</ref>指数型增长曲线可以沿着集成电路之前的早期计算技术进行延伸。

[[Ray Kurzweil]] postulates a [[law of accelerating returns]] in which the speed of technological change (and more generally, all evolutionary processes<ref name="google"/>) increases exponentially, generalizing Moore's law in the same manner as Moravec's proposal, and also including material technology (especially as applied to [[nanotechnology]]), [[Medical Technology|medical technology]] and others.<ref name="singularity2"/> Between 1986 and 2007, machines' application-specific capacity to compute information per capita roughly doubled every 14 months; the per capita capacity of the world's general-purpose computers has doubled every 18 months; the global telecommunication capacity per capita doubled every 34 months; and the world's storage capacity per capita doubled every 40 months.<ref name="HilbertLopez2011">[http://www.sciencemag.org/content/332/6025/60 "The World's Technological Capacity to Store, Communicate, and Compute Information"], Martin Hilbert and Priscila López (2011), [[Science (journal)|Science]], 332(6025), 60–65; free access to the article through here: martinhilbert.net/WorldInfoCapacity.html</ref> On the other hand, it has been argued that the global acceleration pattern having the 21st century singularity as its parameter should be characterized as [[Hyperbolic growth|hyperbolic]] rather than exponential.<ref>[https://link.springer.com/book/10.1007/978-3-030-33730-8 ''The 21st Century Singularity and Global Futures. A Big History Perspective''] (Springer, 2020)</ref>

Ray Kurzweil假设了一个加速回报定律，其中技术变革的速度（更广泛地说，所有进化过程<ref name="google"/>）急剧增长。从1986年到2007年，摩尔定律以与莫拉维克提案相同的方式呈指数增长，并包括材料技术（尤其是应用于纳米技术）、医疗技术和其他技术，<ref name="singularity2"/>计算机计算人均信息的特定应用能力大约每14个月翻一番；世界通用计算机的人均容量每18个月翻一番；全球人均电信容量每34个月翻一番；世界人均存储容量每40个月翻一番。<ref name="HilbertLopez2011">[http://www.sciencemag.org/content/332/6025/60 "The World's Technological Capacity to Store, Communicate, and Compute Information"], Martin Hilbert and Priscila López (2011), [[Science (journal)|Science]], 332(6025), 60–65; free access to the article through here: martinhilbert.net/WorldInfoCapacity.html</ref> 另一方面，有人认为，以21世纪奇点为参数的全球加速度模式应该被描述为双曲线增长而不是指数型。<ref>[https://link.springer.com/book/10.1007/978-3-030-33730-8 ''The 21st Century Singularity and Global Futures. A Big History Perspective''] (Springer, 2020)</ref>

 

[[Ray Kurzweil]]假设了一个[[加速回报定律]]，其中技术变革的速度（更广泛地说，所有进化过程<ref name="google"/>）急剧增长。从1986年到2007年，摩尔定律以与莫拉维克提案相同的方式呈指数增长，并包括材料技术（尤其是应用于[[纳米技术]]）、[[医疗技术|医疗技术]]和其他技术，计算机计算人均信息的特定应用能力大约每14个月翻一番；世界通用计算机的人均容量每18个月翻一番；全球人均电信容量每34个月翻一番；世界人均存储容量每40个月翻一番。<ref name="HilbertLopez2011">[http://www.sciencemag.org/content/332/6025/60 "The World's Technological Capacity to Store, Communicate, and Compute Information"], Martin Hilbert and Priscila López (2011), [[Science (journal)|Science]], 332(6025), 60–65; free access to the article through here: martinhilbert.net/WorldInfoCapacity.html</ref> 另一方面，有人认为，以21世纪奇点为参数的全球加速度模式应该被描述为[[双曲线增长|双曲]]而不是指数型<ref>[https://link.springer.com/book/10.1007/978-3-030-33730-8 ''The 21st Century Singularity and Global Futures. A Big History Perspective''] (Springer, 2020)</ref>

 

Kurzweil reserves the term "singularity" for a rapid increase in artificial intelligence (as opposed to other technologies), writing for example that "The Singularity will allow us to transcend these limitations of our biological bodies and brains ... There will be no distinction, post-Singularity, between human and machine".<ref name="singularity3"/> He also defines his predicted date of the singularity (2045) in terms of when he expects computer-based intelligences to significantly exceed the sum total of human brainpower, writing that advances in computing before that date "will not represent the Singularity" because they do "not yet correspond to a profound expansion of our intelligence."<ref name="transformation"/>

====加速变革====

====加速变革====

[[Image:ParadigmShiftsFrr15Events.svg|thumb|According to Kurzweil, his [[logarithmic scale|logarithmic graph]] of 15 lists of [[paradigm shift]]s for key [[human history|historic]] events shows an [[exponential growth|exponential]] trend|链接=Special:FilePath/ParadigmShiftsFrr15Events.svg]]

[[Image:ParadigmShiftsFrr15Events.svg|thumb|According to Kurzweil, his [[logarithmic scale|logarithmic graph]] of 15 lists of [[paradigm shift]]s for key [[human history|historic]] events shows an [[exponential growth|exponential]] trend|链接=Special:FilePath/ParadigmShiftsFrr15Events.svg]]

[[图片：ParadigmShiftsFrr15Events.svg|thumb |根据Kurzweil的说法，他对关键的[[人类历史|历史]]事件的15个[[范式转移]]列表的[[对数标度|对数图]]显示了[[指数增长|指数]]趋势]]

[[图片：ParadigmShiftsFrr15Events.svg|thumb |根据Kurzweil的说法，他对关键的[[人类历史|历史]]事件的15个[[范式转移]]列表的[[对数标度|对数图]]显示了[[指数增长|指数]]趋势]]

一些奇点论的支持者认为，通过对过去趋势的推断，特别是那些缩减技术进步的差距有关的趋势，奇点是不可避免的。技术进步的背景下，较早使用“奇点”一词时，Stanislaw Ulam讲述了与冯·诺伊曼关于加速变革的一次谈话：

一些奇点论的支持者认为，通过对过去趋势的推断，特别是那些缩减技术进步的差距有关的趋势，奇点是不可避免的。技术进步的背景下，较早使用“奇点”一词时，Stanislaw Ulam讲述了与冯·诺伊曼关于加速变革的一次谈话：

一次围绕不断加速的技术进步和生活方式变化的对话，它使得人类历史上出现了一些基本的奇点，超过了这些奇点，如我们所知的那些人类的事务，将无法继续下去。

::一次围绕不断加速的技术进步和生活方式变化的对话，它使得人类历史上出现了一些基本的奇点，超过了这些奇点，如我们所知的那些人类的事务，将无法继续下去。<ref name=mathematical/>

经常被引用的危险包括那些与分子纳米技术和基因工程有关的技术。这些威胁是奇点论的倡导者和批评者面临的主要议题，也是比尔·乔伊《连线 Wired》杂志上所发表文章《为什么未来不需要我们 Why the future doesn't need us》的主题。

 

 

 

经常被引用的危险包括那些与分子纳米技术和基因工程有关的技术。这些威胁是奇点论的倡导者和批评者面临的主要议题，也是比尔 · 乔伊《连线 Wired》杂志上所发表文章《为什么未来不需要我们 Why the future doesn't need us》的主题。

===算法改进 ===

===算法改进 ===

Some intelligence technologies, like "seed AI",<ref name="Yampolskiy, Roman V 2015"/><ref name="ReferenceA"/> may also have the potential to not just make themselves faster, but also more efficient, by modifying their [[source code]]. These improvements would make further improvements possible, which would make further improvements possible, and so on.

递归自我改进算法的机制在两个方面不同于原始计算速度的提高。首先，它不需要外部影响：设计更快的硬件的机器仍然需要人类来创造改进的硬件，或者对工厂进行适当的编程。AI可以既身处一个AI盒 AI box里面，又同时改进自己的源代码。

 

 

 

Second, as with [[Vernor Vinge]]’s conception of the singularity, it is much harder to predict the outcome. While speed increases seem to be only a quantitative difference from human intelligence, actual algorithm improvements would be qualitatively different. [[Eliezer Yudkowsky]] compares it to the changes that human intelligence brought: humans changed the world thousands of times more rapidly than evolution had done, and in totally different ways. Similarly, the evolution of life was a massive departure and acceleration from the previous geological rates of change, and improved intelligence could cause change to be as different again.<ref name="yudkowsky">{{cite web|author=Eliezer S. Yudkowsky |url=http://yudkowsky.net/singularity/power |title=Power of Intelligence |publisher=Yudkowsky |accessdate=2011-09-09}}</ref>

第二，和Vernor Vinge关于奇点的概念一样，对结果的预测要困难得多。虽然速度的提高似乎与人类的智能只是数量上的区别，但实际的算法改进在质量上是不同的。Eliezer Yudkowsky将其与人类智能带来的变化相比较：人类改变世界的速度比进化速度快数千倍，而且方式完全不同。同样地，生命的进化与以前的地质变化又有着巨大的不同和加速，而智能的提高可能会使变化再次变得不同。<ref name="yudkowsky">{{cite web|author=Eliezer S. Yudkowsky |url=http://yudkowsky.net/singularity/power |title=Power of Intelligence |publisher=Yudkowsky |accessdate=2011-09-09}}</ref>

第二，和 Vernor Vinge 关于奇点的概念一样，对结果的预测要困难得多。虽然速度的提高似乎与人类的智能只是数量上的区别，但实际的算法改进在质量上是不同的。Eliezer Yudkowsky将其与人类智能带来的变化相比较：人类改变世界的速度比进化速度快数千倍，而且方式完全不同。同样地，生命的进化与以前的地质变化又有着巨大的不同和加速，而智能的提高可能会使变化再次变得不同。<ref name="yudkowsky">{{cite web|author=Eliezer S. Yudkowsky |url=http://yudkowsky.net/singularity/power |title=Power of Intelligence |publisher=Yudkowsky |accessdate=2011-09-09}}</ref>

由递归式自我改进的算法集合引起的智能爆炸存在着巨大的危险。首先，人工智能的目标结构在自我完善的情况下可能不是一成不变的，这可能会导致人工智能对原本计划之外的东西进行优化。<ref name="selfawaresystems">[http://selfawaresystems.com/2007/11/30/paper-on-the-basic-ai-drives/ Omohundro, Stephen M., "The Basic AI Drives." Artificial General Intelligence, 2008 proceedings of the First AGI Conference, eds. Pei Wang, Ben Goertzel, and Stan Franklin. Vol. 171. Amsterdam: IOS, 2008 ]</ref><ref name="kurzweilai">{{cite web|url=http://www.kurzweilai.net/artificial-general-intelligence-now-is-the-time |title=Artificial General Intelligence: Now Is the Time |publisher=KurzweilAI |accessdate=2011-09-09}}</ref>第二，人工智能可以与人类竞争赖以生存的稀缺资源。<ref name="selfawaresystems.com">[http://selfawaresystems.com/2007/10/05/paper-on-the-nature-of-self-improving-artificial-intelligence/ Omohundro, Stephen M., "The Nature of Self-Improving Artificial Intelligence." Self-Aware Systems. 21 Jan. 2008. Web. 07 Jan. 2010.]</ref><ref>{{cite book|last1=Barrat|first1=James|title=Our Final Invention|year=2013|publisher=St. Martin's Press|location=New York|isbn=978-0312622374|pages=78–98|edition=First|chapter=6, "Four Basic Drives"|title-link=Our Final Invention}}</ref>

There are substantial dangers associated with an intelligence explosion singularity originating from a recursively self-improving set of algorithms. First, the goal structure of the AI might not be invariant under self-improvement, potentially causing the AI to optimise for something other than what was originally intended.<ref name="selfawaresystems">[http://selfawaresystems.com/2007/11/30/paper-on-the-basic-ai-drives/ Omohundro, Stephen M., "The Basic AI Drives." Artificial General Intelligence, 2008 proceedings of the First AGI Conference, eds. Pei Wang, Ben Goertzel, and Stan Franklin. Vol. 171. Amsterdam: IOS, 2008 ]</ref><ref name="kurzweilai">{{cite web|url=http://www.kurzweilai.net/artificial-general-intelligence-now-is-the-time |title=Artificial General Intelligence: Now Is the Time |publisher=KurzweilAI |accessdate=2011-09-09}}</ref> Secondly, AIs could compete for the same scarce resources mankind uses to survive.<ref name="selfawaresystems.com">[http://selfawaresystems.com/2007/10/05/paper-on-the-nature-of-self-improving-artificial-intelligence/ Omohundro, Stephen M., "The Nature of Self-Improving Artificial Intelligence." Self-Aware Systems. 21 Jan. 2008. Web. 07 Jan. 2010.]</ref><ref>{{cite book|last1=Barrat|first1=James|title=Our Final Invention|year=2013|publisher=St. Martin's Press|location=New York|isbn=978-0312622374|pages=78–98|edition=First|chapter=6, "Four Basic Drives"|title-link=Our Final Invention}}</ref>

虽然不是恶意的，但没有理由认为人工智能会积极促进人类目标的实现，除非这些目标可以被编程，否则，它们就可能利用目前用于支持人类的资源来促进自己的目标，从而导致人类灭绝。<ref name="kurzweilai.net">{{cite web|url=http://www.kurzweilai.net/max-more-and-ray-kurzweil-on-the-singularity-2 |title=Max More and Ray Kurzweil on the Singularity |publisher=KurzweilAI |accessdate=2011-09-09}}</ref><ref name="ReferenceB">{{cite web|url=http://singinst.org/riskintro/index.html |title=Concise Summary &#124; Singularity Institute for Artificial Intelligence |publisher=Singinst.org |accessdate=2011-09-09}}</ref><ref name="nickbostrom7">[http://www.nickbostrom.com/fut/evolution.html Bostrom, Nick, The Future of Human Evolution, Death and Anti-Death: Two Hundred Years After Kant, Fifty Years After Turing, ed. Charles Tandy, pp. 339–371, 2004, Ria University Press.]</ref>

 

Carl Shulman和Anders Sandberg认为，算法改进可能是奇点的限制因素；虽然硬件效率趋于稳步提高，但软件创新更不具可预测性，可能会受到连续、累积的研究的限制。他们认为，智能爆炸在受软件限制的奇点情况中发生的可能性实际上比在受硬件限制的奇点更高，因为在软件受限的情况下，一旦开发出人类水平的人工智能，它可以在非常快的硬件上连续运行，廉价硬件的丰富将使人工智能研究不那么受限制。<ref name=ShulmanSandberg2010>{{cite journal|last=Shulman|first=Carl|author2=Anders Sandberg |title=Implications of a Software-Limited Singularity|journal=ECAP10: VIII European Conference on Computing and Philosophy|year=2010|url=http://intelligence.org/files/SoftwareLimited.pdf|accessdate=17 May 2014|editor1-first=Klaus|editor1-last=Mainzer}}</ref>一旦软件知道如何使用硬件，大量的硬件就可以被释放出来，这被称为“计算过剩”。<ref name="MuehlhauserSalamon2012">{{cite book|last=Muehlhauser|first=Luke|title=Singularity Hypotheses: A Scientific and Philosophical Assessment|year=2012|publisher=Springer|chapter-url=http://intelligence.org/files/IE-EI.pdf|author2=Anna Salamon |editor=Amnon Eden |editor2=Johnny Søraker |editor3=James H. Moor |editor4=Eric Steinhart|chapter=Intelligence Explosion: Evidence and Import}}</ref>

 

 

 

[[Carl Shulman]] and [[Anders Sandberg]] suggest that algorithm improvements may be the limiting factor for a singularity; while hardware efficiency tends to improve at a steady pace, software innovations are more unpredictable and may be bottlenecked by serial, cumulative research. They suggest that in the case of a software-limited singularity, intelligence explosion would actually become more likely than with a hardware-limited singularity, because in the software-limited case, once human-level AI is developed, it could run serially on very fast hardware, and the abundance of cheap hardware would make AI research less constrained.<ref name=ShulmanSandberg2010>{{cite journal|last=Shulman|first=Carl|author2=Anders Sandberg |title=Implications of a Software-Limited Singularity|journal=ECAP10: VIII European Conference on Computing and Philosophy|year=2010|url=http://intelligence.org/files/SoftwareLimited.pdf|accessdate=17 May 2014|editor1-first=Klaus|editor1-last=Mainzer}}</ref> An abundance of accumulated hardware that can be unleashed once the software figures out how to use it has been called "computing overhang."<ref name="MuehlhauserSalamon2012">{{cite book|last=Muehlhauser|first=Luke|title=Singularity Hypotheses: A Scientific and Philosophical Assessment|year=2012|publisher=Springer|chapter-url=http://intelligence.org/files/IE-EI.pdf|author2=Anna Salamon |editor=Amnon Eden |editor2=Johnny Søraker |editor3=James H. Moor |editor4=Eric Steinhart|chapter=Intelligence Explosion: Evidence and Import}}</ref>

 

Carl Shulman 和 Anders Sandberg 认为，算法改进可能是奇点的限制因素；虽然硬件效率趋于稳步提高，但软件创新更不具可预测性，可能会受到连续、累积的研究的限制。他们认为，智能爆炸在受软件限制的奇点情况中发生的可能性实际上比在受硬件限制的奇点更高，因为在软件受限的情况下，一旦开发出人类水平的人工智能，它可以在非常快的硬件上连续运行，廉价硬件的丰富将使人工智能研究不那么受限制。<ref name=ShulmanSandberg2010>{{cite journal|last=Shulman|first=Carl|author2=Anders Sandberg |title=Implications of a Software-Limited Singularity|journal=ECAP10: VIII European Conference on Computing and Philosophy|year=2010|url=http://intelligence.org/files/SoftwareLimited.pdf|accessdate=17 May 2014|editor1-first=Klaus|editor1-last=Mainzer}}</ref> 一旦软件知道如何使用硬件，大量的硬件就可以被释放出来，这被称为“计算过剩”。<ref name="MuehlhauserSalamon2012">{{cite book|last=Muehlhauser|first=Luke|title=Singularity Hypotheses: A Scientific and Philosophical Assessment|year=2012|publisher=Springer|chapter-url=http://intelligence.org/files/IE-EI.pdf|author2=Anna Salamon |editor=Amnon Eden |editor2=Johnny Søraker |editor3=James H. Moor |editor4=Eric Steinhart|chapter=Intelligence Explosion: Evidence and Import}}</ref>

===危机===

===危机===

 

一些批评者，如哲学家Hubert Dreyfus断言计算机或机器无法实现人类智能，而其他人，如物理学家史蒂芬·霍金 Stephen Hawking，则认为如果最终结果是相同的，那么智力的定义其实无关紧要。<ref name="dreyfus"/>

Some critics, like philosopher [[Hubert Dreyfus]], assert that computers or machines cannot achieve [[human intelligence]], while others, like physicist [[Stephen Hawking]], hold that the definition of intelligence is irrelevant if the net result is the same.<ref name="dreyfus"/>

 

 

早在1932年，约翰·W·坎贝尔 John Wood Campbell Jr. 的短篇小说《最后的进化 The last evolution》中就对这个想法进行了描述。

早在1932年，约翰·W·坎贝尔 John Wood Campbell Jr. 的短篇小说《最后的进化 The last evolution》中就对这个想法进行了描述。

心理学家史蒂芬·平克 Steven Pinker 在2008年指出：

心理学家史蒂芬·平克 Steven Pinker 在2008年指出：

加州大学伯克利分校，哲学教授约翰·塞尔 John Searle写道：

 

 

[[University of California, Berkeley]], [[philosophy]] professor [[John Searle]] writes:

 

{{blockquote|[Computers] have, literally ..., no [[intelligence]], no [[motivation]], no [[autonomy]], and no agency.  We design them to behave as if they had certain sorts of [[psychology]], but there is no psychological reality to the corresponding processes or behavior. ...  [T]he machinery has no beliefs, desires, [or] motivations.<ref>[[John R. Searle]], “What Your Computer Can’t Know”, ''[[The New York Review of Books]]'', 9 October 2014, p. 54.</ref>}}

{{blockquote|[Computers] have, literally ..., no [[intelligence]], no [[motivation]], no [[autonomy]], and no agency.  We design them to behave as if they had certain sorts of [[psychology]], but there is no psychological reality to the corresponding processes or behavior. ...  [T]he machinery has no beliefs, desires, [or] motivations.<ref>[[John R. Searle]], “What Your Computer Can’t Know”, ''[[The New York Review of Books]]'', 9 October 2014, p. 54.</ref>}}

::毫不夸张地说……，计算机没有智能，没有动机，没有自主，也没有智能体。我们设计他们，使他们的行为好像表示他们有某种心理，但其实没有对应这些过程或行为的心理现实……机器没有信仰、愿望或动机。<ref>John R. Searle, “What Your Computer Can’t Know”, ''The New York Review of Books'', 9 October 2014, p. 54.</ref>

 

 

[[Martin Ford (author)|Martin Ford]] in ''The Lights in the Tunnel: Automation, Accelerating Technology and the Economy of the Future''<ref name="thelightsinthetunnel"/> postulates a "technology paradox" in that before the singularity could occur most routine jobs in the economy would be automated, since this would require a level of technology inferior to that of the singularity. This would cause massive unemployment and plummeting consumer demand, which in turn would destroy the incentive to invest in the technologies that would be required to bring about the Singularity. Job displacement is increasingly no longer limited to work traditionally considered to be "routine."<ref name="nytimes"/>

 

Martin Ford 在“隧道中的灯光：自动化、加速技术和未来经济 The Lights in the Tunnel: Automation, Accelerating Technology and the Economy of the Future”<ref name="thelightsinthetunnel"/>中提出了一个“技术悖论”：在奇点出现之前，经济体中的大多数日常工作都将自动化，因为这所需的技术水平低于奇点。这将导致大规模的失业和消费者需求的骤降，这反过来又会破坏投资于实现奇点所需技术的动机。工作的替代越来越不再局限于那些传统上被认为是“例行公事”的工作。<ref name="nytimes"/>

 

[[Theodore Modis]]<ref name="google13"/><ref name="Singularity Myth"/> and [[Jonathan Huebner]]<ref name="technological14"/> argue that the rate of technological innovation has not only ceased to rise, but is actually now declining. Evidence for this decline is that the rise in computer [[clock rate]]s is slowing, even while Moore's prediction of exponentially increasing circuit density continues to hold. This is due to excessive heat build-up from the chip, which cannot be dissipated quickly enough to prevent the chip from melting when operating at higher speeds. Advances in speed may be possible in the future by virtue of more power-efficient CPU designs and multi-cell processors.<ref name="cnet"/> While Kurzweil used Modis' resources, and Modis' work was around accelerating change, Modis distanced himself from Kurzweil's thesis of a "technological singularity", claiming that it lacks scientific rigor.<ref name="Singularity Myth"/>

 

 

 

In a detailed empirical accounting, ''The Progress of Computing'', [[William Nordhaus]] argued that, prior to 1940, computers followed the much slower growth of a traditional industrial economy, thus rejecting extrapolations of Moore's law to 19th-century computers.<ref>{{Cite journal | doi=10.1017/S0022050707000058|title = Two Centuries of Productivity Growth in Computing| journal=The Journal of Economic History| volume=67|pages = 128–159|year = 2007|last1 = Nordhaus|first1 = William D.| citeseerx=10.1.1.330.1871}}</ref>

 

 

In a 2007 paper, Schmidhuber stated that the frequency of subjectively "notable events" appears to be approaching a 21st-century singularity, but cautioned readers to take such plots of subjective events with a grain of salt: perhaps differences in memory of recent and distant events could create an illusion of accelerating change where none exists.<ref>Schmidhuber, Jürgen. "New millennium AI and the convergence of history." Challenges for computational intelligence. Springer Berlin Heidelberg, 2007. 15–35.</ref>

在2007年的一篇论文中，Schmidhuber 指出主观上“值得注意的事件”出现的频率似乎正在接近21世纪的奇点，但他提醒读者，对这些主观事件的情节要持保留态度：也许对近期和远期事件的记忆差异，可能会造成一种在根本不存在的情况下变化加速的错觉。<ref>Schmidhuber, Jürgen. "New millennium AI and the convergence of history." Challenges for computational intelligence. Springer Berlin Heidelberg, 2007. 15–35.</ref>

Martin Ford 在《隧道中的灯光：自动化、加速技术和未来经济 The Lights in the Tunnel: Automation, Accelerating Technology and the Economy of the Future》<ref name="thelightsinthetunnel"/>中提出了一个“技术悖论”：在奇点出现之前，经济体中的大多数日常工作都将自动化，因为这所需的技术水平低于奇点。这将导致大规模的失业和消费者需求的骤降，这反过来又会破坏投资于实现奇点所需技术的动机。工作的替代越来越不再局限于那些传统上被认为是“例行公事”的工作。<ref name="nytimes"/>

Paul Allen argued the opposite of accelerating returns, the complexity brake; the more progress science makes towards understanding intelligence, the more difficult it becomes to make additional progress. A study of the number of patents shows that human creativity does not show accelerating returns, but in fact, as suggested by Joseph Tainter in his The Collapse of Complex Societies, a law of diminishing returns. The number of patents per thousand peaked in the period from 1850 to 1900, and has been declining since.[60] The growth of complexity eventually becomes self-limiting, and leads to a widespread "general systems collapse".

Theodore Modis<ref name="google13"/><ref name="Singularity Myth"/>和Jonathan Huebner<ref name="technological14"/>认为技术创新的速度不仅停止上升，而且现在实际上正在下降。这种下降的证据是计算机时钟速率的增长正在放缓，尽管摩尔关于电路密度指数增长的预测仍然成立。这是由于芯片产生过多的热量，当它们以较高的速度运行时，这些热量不能足够快地散去，可能导致芯片熔化。在未来，随着更节能的CPU设计和多单元处理器的发明，速度的提高可能实现。<ref name="cnet"/>尽管 Kurzweil利用了Modis的（工作成果）作为资源，同时 Modis 的工作围绕着加速变革，但 Modis对Kurzweil的“技术奇点”理论保持距离，称其缺乏科学严谨性。<ref name="Singularity Myth"/>

保罗·艾伦 Paul Allen 认为，与加速回报相反的是<font color="#ff8000">复杂性制动 complexity brake</font>；科学在理解智力方面取得的进展越多，就越难取得更多的进展。一项对专利数量的研究表明，人类的创造力并没有表现出加速的回报。事实上，正如Joseph Tainter 在他的《复杂社会的崩溃 The Collapse of Complex Societies 》中所指出的那样，存在一个收益递减定律 a law of diminishing returns 的限制。每千人的专利的数量在1850年至1900年期间达到顶峰，此后一直在下降。复杂性的增长最终会自我限制，并导致广泛的“一般系统崩溃 general systems collapse”。

在2007年的一篇论文中，Schmidhuber指出主观上“值得注意的事件”出现的频率似乎正在接近21世纪的奇点，但他提醒读者，对这些主观事件的情节要持保留态度：也许对近期和远期事件的记忆差异，可能会造成一种在根本不存在的情况下变化加速的错觉。<ref>Schmidhuber, Jürgen. "New millennium AI and the convergence of history." Challenges for computational intelligence. Springer Berlin Heidelberg, 2007. 15–35.</ref>

[[Jaron Lanier]] refutes the idea that the Singularity is inevitable. He states: "I do not think the technology is creating itself. It's not an autonomous process."<ref name="lanier">{{cite web |author=Jaron Lanier |title=Who Owns the Future? |work=New York: Simon & Schuster |date=2013 |url=http://www.epubbud.com/read.php?g=JCB8D9LA&tocp=59}}</ref> He goes on to assert: "The reason to believe in human agency over technological determinism is that you can then have an economy where people earn their own way and invent their own lives. If you structure a society on ''not'' emphasizing individual human agency, it's the same thing operationally as denying people clout, dignity, and self-determination ... to embrace [the idea of the Singularity] would be a celebration of bad data and bad politics."<ref name="lanier" />

保罗·艾伦 Paul Allen认为，与加速回报相反的是'''复杂性制动 complexity brake'''；科学在理解智力方面取得的进展越多，就越难取得更多的进展。一项对专利数量的研究表明，人类的创造力并没有表现出加速的回报。事实上，正如Joseph Tainter在他的《复杂社会的崩溃 The Collapse of Complex Societies 》中所指出的那样，存在一个收益递减定律 a law of diminishing returns的限制。每千人的专利的数量在1850年至1900年期间达到顶峰，此后一直在下降。复杂性的增长最终会自我限制，并导致广泛的“一般系统崩溃 general systems collapse”。

Jaron Lanier 驳斥了奇点不可避免的观点。他说：“我不认为这项技术是在创造自我。这不是一个自主的过程。”<ref name="lanier">{{cite web |author=Jaron Lanier |title=Who Owns the Future? |work=New York: Simon & Schuster |date=2013 |url=http://www.epubbud.com/read.php?g=JCB8D9LA&tocp=59}}</ref>他接着断言：“相信人的能动性而不是技术决定论的原因是，这样你就可以有一个经济体，人们在其中可以自己挣钱，创造自己的生活。如果你在不强调个人主观能动性的基础上构建社会，就等于在操作上否认人们的影响力、尊严和自决...接受 [奇点的想法] 将是对糟糕数据和糟糕政治的颂扬。<ref name="lanier" />

Jaron Lanier驳斥了奇点不可避免的观点。他说：“我不认为这项技术是在创造自我。这不是一个自主的过程。”<ref name="lanier">{{cite web |author=Jaron Lanier |title=Who Owns the Future? |work=New York: Simon & Schuster |date=2013 |url=http://www.epubbud.com/read.php?g=JCB8D9LA&tocp=59}}</ref>他接着断言：“相信人的能动性而不是技术决定论的原因是，这样你就可以有一个经济体，人们在其中可以自己挣钱，创造自己的生活。如果你在不强调个人主观能动性的基础上构建社会，就等于在操作上否认人们的影响力、尊严和自决……接受奇点的想法将是对糟糕数据和糟糕政治的颂扬。<ref name="lanier" />

[[Economics|Economist]] [[Robert J. Gordon]], in ''The Rise and Fall of American Growth: The U.S. Standard of Living Since the Civil War'' (2016), points out that measured economic growth has slowed around 1970 and slowed even further since the [[financial crisis of 2007–2008]], and argues that the economic data show no trace of a coming Singularity as imagined by mathematician [[I.J. Good]].<ref>[[William D. Nordhaus]], "Why Growth Will Fall" (a review of [[Robert J. Gordon]], ''The Rise and Fall of American Growth:  The U.S. Standard of Living Since the Civil War'', Princeton University Press, 2016, {{ISBN|978-0691147727}}, 762 pp., $39.95), ''[[The New York Review of Books]]'', vol. LXIII, no. 13 (August 18, 2016), p. 68.</ref>

经济学家 Robert J.Gordon在《美国经济增长的兴衰：内战以来的美国生活水平 The Rise and Fall of American Growth: The U.S. Standard of Living Since the Civil War》（2016）中指出，据测量，经济增长在1970年左右放缓，自2007-2008年金融危机以来甚至进一步放缓，并认为经济数据显示没有迹象表明数学家I.J.Good所设想的奇点将会到来。<ref>[[William D. Nordhaus]], "Why Growth Will Fall" (a review of [[Robert J. Gordon]], ''The Rise and Fall of American Growth:  The U.S. Standard of Living Since the Civil War'', Princeton University Press, 2016, {{ISBN|978-0691147727}}, 762 pp., $39.95), ''[[The New York Review of Books]]'', vol. LXIII, no. 13 (August 18, 2016), p. 68.</ref>

除了对奇点概念的一般性批评外，一些批评者还对 Kurzweil的标志性图表提出了质疑。一种批评是，这种性质的对数图像本质上就会存在倾向于直线的有偏差结果。其他人批评Kurzweil在数据点的使用上存在选择偏差。<ref name="PZMyers"/>例如，生物学家 P. Z. Myers指出，许多早期的进化“事件”都是随意挑选的。Kurzweil反驳了这一点，他绘制了15个中立来源的进化事件图，并表明它们都符合一条直线。《经济学人》用一张图表来嘲讽这个概念：一把剃须刀上的刀片数在过去几年里从一个增加到多达五个，并且它将以更快的速度增长到无穷大。<ref name="moreblades"/>

 

 

除了对奇点概念的一般性批评外，一些批评者还对 Kurzweil 的标志性图表提出了质疑。一种批评是，这种性质的对数图像本质上就会存在倾向于直线的有偏差结果。其他人批评 Kurzweil 在数据点的使用上存在选择偏差。<ref name="PZMyers"/>例如，生物学家 P. Z. Myers 指出，许多早期的进化“事件”都是随意挑选的。Kurzweil 反驳了这一点，他绘制了15个中立来源的进化事件图，并表明它们都符合一条直线.《经济学人》用一张图表来嘲讽这个概念：一把剃须刀上的刀片数在过去几年里从一个增加到多达五个，并且它将以更快的速度增长到无穷大。<ref name="moreblades"/>

==潜在影响==

==潜在影响==

 

过去由于一些技术进步，经济增长率发生了巨大变化。以人口增长为基础，从旧石器时代到新石器时代，经济每25万年翻一番。新农业经济每900年翻一番，增长显著。在当今时代，从工业革命开始，世界经济产出每15年翻一番，比农业时代快60倍。罗宾·汉森 Robin Hanson认为，如果超人智能的兴起引发了类似的革命，人们会预期经济至少每季度翻一番，甚至可能每周翻一番。<ref name="Hanson">{{Citation |url=http://www.spectrum.ieee.org/robotics/robotics-software/economics-of-the-singularity |title=Economics Of The Singularity |author=Robin Hanson |work=IEEE Spectrum Special Report: The Singularity }} & [http://hanson.gmu.edu/longgrow.pdf Long-Term Growth As A Sequence of Exponential Modes]</ref>

Dramatic changes in the rate of economic growth have occurred in the past because of some technological advancement. Based on population growth, the economy doubled every 250,000 years from the [[Paleolithic]] era until the [[Neolithic Revolution]]. The new agricultural economy doubled every 900 years, a remarkable increase. In the current era, beginning with the Industrial Revolution, the world's economic output doubles every fifteen years, sixty times faster than during the agricultural era. If the rise of superhuman intelligence causes a similar revolution, argues Robin Hanson, one would expect the economy to double at least quarterly and possibly on a weekly basis.<ref name="Hanson">{{Citation |url=http://www.spectrum.ieee.org/robotics/robotics-software/economics-of-the-singularity |title=Economics Of The Singularity |author=Robin Hanson |work=IEEE Spectrum Special Report: The Singularity }} & [http://hanson.gmu.edu/longgrow.pdf Long-Term Growth As A Sequence of Exponential Modes]</ref>

 

 

===不确定性和风险===

===不确定性和风险===

物理学家史蒂芬·霍金在2014年表示，“成功创造人工智能将是人类历史上最大的事件。不幸的是，这也可能是最后一次，除非我们学会如何规避风险。”<ref name=hawking_2014/>霍金认为，在未来几十年里，人工智能可能会带来“无法估量的利益和风险”，例如“技术超越金融市场的聪明程度，超越人类研究人员的创造力，超越人类领袖的操控力，开发我们甚至无法理解的武器”。霍金建议，<ref name=hawking_2014/>人们应该更认真地对待人工智能，并做更多的工作来为奇点做准备：<ref name=hawking_2014>{{cite web |url=https://www.independent.co.uk/news/science/stephen-hawking-transcendence-looks-at-the-implications-of-artificial-intelligence--but-are-we-taking-ai-seriously-enough-9313474.html |title=Stephen Hawking: 'Transcendence looks at the implications of artificial intelligence - but are we taking AI seriously enough?'  |work=[[The Independent]] |author=Stephen Hawking |date=1 May 2014 |accessdate=May 5, 2014|author-link=Stephen Hawking }}</ref>

::所以，面对可能的收益和风险难以估量的未来，专家们肯定会尽一切可能确保最好的结果，对吗？错了。如果一个高级的外星文明给我们发了一条信息说，“我们几十年后就会到达”，我们会不会只回答，“好吧，你到了这里就打电话给我们——我们会开着灯的”？可能不会——但这或多或少就是人工智能正在发生的事情。

 

 

 

 

Physicist [[Stephen Hawking]] said in 2014 that "Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks."<ref name=hawking_2014/> Hawking believed that in the coming decades, AI could offer "incalculable benefits and risks" such as "technology outsmarting financial markets, out-inventing human researchers, out-manipulating human leaders, and developing weapons we cannot even understand."<ref name=hawking_2014/> Hawking suggested that artificial intelligence should be taken more seriously and that more should be done to prepare for the singularity:<ref name=hawking_2014>{{cite web |url=https://www.independent.co.uk/news/science/stephen-hawking-transcendence-looks-at-the-implications-of-artificial-intelligence--but-are-we-taking-ai-seriously-enough-9313474.html |title=Stephen Hawking: 'Transcendence looks at the implications of artificial intelligence - but are we taking AI seriously enough?'  |work=[[The Independent]] |author=Stephen Hawking |date=1 May 2014 |accessdate=May 5, 2014|author-link=Stephen Hawking }}</ref>

 

物理学家史蒂芬·霍金在2014年表示，“成功创造人工智能将是人类历史上最大的事件。不幸的是，这也可能是最后一次，除非我们学会如何规避风险。”<ref name=hawking_2014/>  霍金认为，在未来几十年里，人工智能可能会带来“无法估量的利益和风险”，例如“技术超越金融市场的聪明程度，超越人类研究人员的创造力，超越人类领袖的操控力，开发我们甚至无法理解的武器”。霍金建议<ref name=hawking_2014/> ，人们应该更认真地对待人工智能，并做更多的工作来为奇点做准备：<ref name=hawking_2014>{{cite web |url=https://www.independent.co.uk/news/science/stephen-hawking-transcendence-looks-at-the-implications-of-artificial-intelligence--but-are-we-taking-ai-seriously-enough-9313474.html |title=Stephen Hawking: 'Transcendence looks at the implications of artificial intelligence - but are we taking AI seriously enough?'  |work=[[The Independent]] |author=Stephen Hawking |date=1 May 2014 |accessdate=May 5, 2014|author-link=Stephen Hawking }}</ref>

 

{{Harvtxt|Berglas|2008}} claims that there is no direct evolutionary motivation for an AI to be friendly to humans. Evolution has no inherent tendency to produce outcomes valued by humans, and there is little reason to expect an arbitrary optimisation process to promote an outcome desired by mankind, rather than inadvertently leading to an AI behaving in a way not intended by its creators.<ref name="nickbostrom8">Nick Bostrom, [http://www.nickbostrom.com/ethics/ai.html "Ethical Issues in Advanced Artificial Intelligence"], in ''Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence'', Vol. 2, ed. I. Smit et al., Int. Institute of Advanced Studies in Systems Research and Cybernetics, 2003, pp. 12–17</ref><ref name="singinst">[[Eliezer Yudkowsky]]: [http://singinst.org/upload/artificial-intelligence-risk.pdf Artificial Intelligence as a Positive and Negative Factor in Global Risk] {{webarchive|url=https://web.archive.org/web/20120611190606/http://singinst.org/upload/artificial-intelligence-risk.pdf |date=2012-06-11 }}. Draft for a publication in ''Global Catastrophic Risk'' from August 31, 2006, retrieved July 18, 2011 (PDF file)</ref><ref name="singinst9">[http://www.singinst.org/blog/2007/06/11/the-stamp-collecting-device/ The Stamp Collecting Device, Nick Hay]</ref> [[Anders Sandberg]] has also elaborated on this scenario, addressing various common counter-arguments.<ref name="aleph">[http://www.aleph.se/andart/archives/2011/02/why_we_should_fear_the_paperclipper.html 'Why we should fear the Paperclipper'], 2011-02-14 entry of Sandberg's blog 'Andart'</ref> AI researcher [[Hugo de Garis]] suggests that artificial intelligences may simply eliminate the human race [[instrumental convergence|for access to scarce resources]],<ref name="selfawaresystems.com" /><ref name="selfawaresystems10">[http://selfawaresystems.com/2007/11/30/paper-on-the-basic-ai-drives/ Omohundro, Stephen M., "The Basic AI Drives." Artificial General Intelligence, 2008 proceedings of the First AGI Conference, eds. Pei Wang, Ben Goertzel, and Stan Franklin. Vol. 171. Amsterdam: IOS, 2008.]</ref> and humans would be powerless to stop them.<ref name="forbes">de Garis, Hugo. [https://www.forbes.com/2009/06/18/cosmist-terran-cyborgist-opinions-contributors-artificial-intelligence-09-hugo-de-garis.html "The Coming Artilect War"], Forbes.com, 22 June 2009.</ref> Alternatively, AIs developed under evolutionary pressure to promote their own survival could outcompete humanity.<ref name="nickbostrom7" />

::当我们创造出第一个超级智能实体时，我们可能会犯错并给它一个导致人类毁灭的目标（假设它巨大的智力优势赋予它这样做的力量）。例如，我们可能会错误地将子目标提升为超级目标。我们告诉它去解决一个数学问题，然后它将太阳系中的所有物质变成一个巨大的计算装置，在这个过程中杀死了提出这个问题的人。

按照Eliezer Yudkowsky的观点，人工智能安全的一个重要问题是，不友好的人工智能可能比友好的人工智能更容易创建。虽然两者都需要递归优化过程的进步，但友好的人工智能还需要目标结构在自我改进过程中保持不变（否则人工智能可以将自己转变成不友好的东西），以及一个与人类价值观相一致且不会自动毁灭人类的目标结构。另一方面，一个不友好的人工智能可以针对任意的目标结构进行优化，<ref name="singinst12">[http://singinst.org/upload/CEV.html Coherent Extrapolated Volition, Eliezer S. Yudkowsky, May 2004 ] {{webarchive|url=https://web.archive.org/web/20100815055725/http://singinst.org/upload/CEV.html |date=2010-08-15 }}</ref>而目标结构不需要在自我改进过程中保持不变。Bill Hibbard（2014）提出了一种人工智能设计，<ref name="JAGI2012">{{Citation| journal=Journal of Artificial General Intelligence| year=2012| volume=3| issue=1| title=Model-Based Utility Functions| first=Bill| last=Hibbard| postscript=.| doi=10.2478/v10229-011-0013-5| page=1|arxiv = 1111.3934 |bibcode = 2012JAGI....3....1H | s2cid=8434596}}</ref>可以避免包括自欺欺人、<ref name="selfawaresystems"/><ref name="AGI-12a">[http://agi-conference.org/2012/wp-content/uploads/2012/12/paper_56.pdf  Avoiding Unintended AI Behaviors.] Bill Hibbard. 2012 proceedings of the Fifth Conference on Artificial General Intelligence, eds. Joscha Bach, Ben Goertzel and Matthew Ikle. [http://intelligence.org/2012/12/19/december-2012-newsletter/ This paper won the Machine Intelligence Research Institute's 2012 Turing Prize for the Best AGI Safety Paper].</ref>无意的工具性行为和奖励机制<ref name="AGI-12a"/>的腐败等一些危险。他还讨论了人工智能<ref name="JET2008">{{Citation| url=http://jetpress.org/v17/hibbard.htm| journal=Journal of Evolution and Technology| year=2008| volume=17| title=The Technology of Mind and a New Social Contract| first=Bill| last=Hibbard| postscript=.}}</ref>和人工智能测试的社会影响。<ref name="AGI-12b">[http://agi-conference.org/2012/wp-content/uploads/2012/12/paper_57.pdf  Decision Support for Safe AI Design|.] Bill Hibbard. 2012 proceedings of the Fifth Conference on Artificial General Intelligence, eds. Joscha Bach, Ben Goertzel and Matthew Ikle.</ref>他在2001年出版的《超级智能机器 Super-Intelligent Machines》一书中提倡对人工智能的公共教育和公众控制。该书还提出了一个简单的易受奖励机制的腐败影响的设计。

 

 

 

 

 

 

 

 

 

 

 

 

按照[[Eliezer Yudkowsky]]的观点，人工智能安全的一个重要问题是，不友好的人工智能可能比友好的人工智能更容易创建。虽然两者都需要递归优化过程的进步，但友好的人工智能还需要目标结构在自我改进过程中保持不变（否则人工智能可以将自己转变成不友好的东西），以及一个与人类价值观相一致且不会自动毁灭人类的目标结构。另一方面，一个不友好的人工智能可以针对任意的目标结构进行优化，<ref name="singinst12">[http://singinst.org/upload/CEV.html Coherent Extrapolated Volition, Eliezer S. Yudkowsky, May 2004 ] {{webarchive|url=https://web.archive.org/web/20100815055725/http://singinst.org/upload/CEV.html |date=2010-08-15 }}</ref> 而目标结构不需要在自我改进过程中保持不变。Bill Hibbard (2014)提出了一种人工智能设计<ref name="JAGI2012">{{Citation| journal=Journal of Artificial General Intelligence| year=2012| volume=3| issue=1| title=Model-Based Utility Functions| first=Bill| last=Hibbard| postscript=.| doi=10.2478/v10229-011-0013-5| page=1|arxiv = 1111.3934 |bibcode = 2012JAGI....3....1H | s2cid=8434596}}</ref> ，可以避免包括自欺欺人<ref name="selfawaresystems"/><ref name="AGI-12a">[http://agi-conference.org/2012/wp-content/uploads/2012/12/paper_56.pdf  Avoiding Unintended AI Behaviors.] Bill Hibbard. 2012 proceedings of the Fifth Conference on Artificial General Intelligence, eds. Joscha Bach, Ben Goertzel and Matthew Ikle. [http://intelligence.org/2012/12/19/december-2012-newsletter/ This paper won the Machine Intelligence Research Institute's 2012 Turing Prize for the Best AGI Safety Paper].</ref> 、无意的工具性行为和奖励机制<ref name="AGI-12a"/>的腐败等一些危险。他还讨论了人工智能<ref name="JET2008">{{Citation| url=http://jetpress.org/v17/hibbard.htm| journal=Journal of Evolution and Technology| year=2008| volume=17| title=The Technology of Mind and a New Social Contract| first=Bill| last=Hibbard| postscript=.}}</ref>和人工智能测试的社会影响<ref name="AGI-12b">[http://agi-conference.org/2012/wp-content/uploads/2012/12/paper_57.pdf  Decision Support for Safe AI Design|.] Bill Hibbard. 2012 proceedings of the Fifth Conference on Artificial General Intelligence, eds. Joscha Bach, Ben Goertzel and Matthew Ikle.</ref> 。他在2001年出版的“超级智能机器Super-Intelligent Machines”一书中提倡对人工智能的公共教育和公众控制。该书还提出了一个简单的易受奖励机制的腐败影响的设计。It also proposed a simple design that was vulnerable to corruption of the reward generator.

===社会生物进化的下一步===

===社会生物进化的下一步===

[[File:Major Evolutionary Transitions digital.jpg|thumb|upright=1.6|Schematic Timeline of Information and Replicators in the Biosphere: Gillings et al.'s "[[The Major Transitions in Evolution|major evolutionary transitions]]" in information processing.<ref name="InfoBiosphere2016" />|链接=Special:FilePath/Major_Evolutionary_Transitions_digital.jpg]]

[[File:Major Evolutionary Transitions digital.jpg|thumb|upright=1.6|Schematic Timeline of Information and Replicators in the Biosphere: Gillings et al.'s "[[The Major Transitions in Evolution|major evolutionary transitions]]" in information processing.<ref name="InfoBiosphere2016" />|链接=Special:FilePath/Major_Evolutionary_Transitions_digital.jpg]]

[[档案：生物vs。数字信息.jpg|2014年，全球数字信息总量（5{e | 21}字节）与全球人类基因组信息（10¹⁹字节）的对比。<ref name="InfoBiosphere2016" />]][[Index.php?title=技术奇点#cite%20note-InfoBiosphere2016-99|<span class="mw-reflink-text">[99]</span>]][[Index.php?title=技术奇点#cite%20note-InfoBiosphere2016-89|<span class="mw-reflink-text">[89]</span>]][[Index.php?title=技术奇点#cite%20note-InfoBiosphere2016-89|<span class="mw-reflink-text">[89]</span>]][[Index.php?title=技术奇点#cite%20note-InfoBiosphere2016-89|<span class="mw-reflink-text">[89]</span>]][[Index.php?title=技术奇点#cite%20note-InfoBiosphere2016-89|<span class="mw-reflink-text">[89]</span>]][[Index.php?title=技术奇点#cite%20note-InfoBiosphere2016-89|<span class="mw-reflink-text">[89]</span>]]

[[档案：生物vs。数字信息.jpg|2014年，全球数字信息总量（5{e | 21}字节）与全球人类基因组信息（10¹⁹字节）的对比。<ref name="InfoBiosphere2016" />]][[Index.php?title=技术奇点#cite%20note-InfoBiosphere2016-99|<span class="mw-reflink-text">[99]</span>]][[Index.php?title=技术奇点#cite%20note-InfoBiosphere2016-89|<span class="mw-reflink-text">[89]</span>]][[Index.php?title=技术奇点#cite%20note-InfoBiosphere2016-89|<span class="mw-reflink-text">[89]</span>]][[Index.php?title=技术奇点#cite%20note-InfoBiosphere2016-89|<span class="mw-reflink-text">[89]</span>]][[Index.php?title=技术奇点#cite%20note-InfoBiosphere2016-89|<span class="mw-reflink-text">[89]</span>]][[Index.php?title=技术奇点#cite%20note-InfoBiosphere2016-89|<span class="mw-reflink-text">[89]</span>]]

虽然技术奇点通常被视为一个突发事件，但一些学者认为目前的变化速度已经符合这种描述。

虽然技术奇点通常被视为一个突发事件，但一些学者认为目前的变化速度已经符合这种描述。

In addition, some argue that we are already in the midst of a [[The Major Transitions in Evolution|major evolutionary transition]] that merges technology, biology, and society. Digital technology has infiltrated the fabric of human society to a degree of indisputable and often life-sustaining dependence.

此外，有人认为，我们已经处在一个融合了技术、生物学和社会学的'''进化巨变 major evolutionary transition'''之中。数字技术已经无可争辩地渗透到人类社会的结构中，而且生命的维持常常依赖数字技术。

人类创造的数字信息已经达到了与生物圈中生物信息相似的规模。自20世纪80年代以来，存储的数字信息量大约每2.5年翻一番，2014年达到约5泽字节（5e21字节）。

 

 

 

 

 

 

人类创造的数字信息已经达到了与生物圈中生物信息相似的规模。自20世纪80年代以来，存储的数字信息量大约每2.5年翻一番，2014年达到约5泽字节（5e21字节）。{{Citation needed|date=April 2019}}

In biological terms, there are 7.2&nbsp;billion humans on the planet, each having a genome of 6.2&nbsp;billion nucleotides. Since one byte can encode four nucleotide pairs, the individual genomes of every human on the planet could be encoded by approximately 1{{e|19}} bytes. The digital realm stored 500 times more information than this in 2014 (see figure). The total amount of DNA contained in all of the cells on Earth is estimated to be about 5.3{{e|37}} base pairs, equivalent to 1.325{{e|37}} bytes of information.

In biological terms, there are 7.2&nbsp;billion humans on the planet, each having a genome of 6.2&nbsp;billion nucleotides. Since one byte can encode four nucleotide pairs, the individual genomes of every human on the planet could be encoded by approximately 1{{e|19}} bytes. The digital realm stored 500 times more information than this in 2014 (see figure). The total amount of DNA contained in all of the cells on Earth is estimated to be about 5.3{{e|37}} base pairs, equivalent to 1.325{{e|37}} bytes of information.

在生物学方面，地球上有72亿人，每个人的基因组有62亿个核苷酸。由于一个字节可以编码四个核苷酸对，地球上每个人类的个体基因组可以编码大约1{e | 19}字节。2014年，数字领域存储的信息是这个数字的500倍（见图）。据估计，地球上所有细胞所含的DNA总量约为5.3{e | 37}碱基对，相当于1.325{e | 37}字节的信息。

在生物学方面，地球上有72亿人，每个人的基因组有62亿个核苷酸。由于一个字节可以编码四个核苷酸对，地球上每个人类的个体基因组可以编码大约1×1019字节。2014年，数字领域存储的信息是这个数字的500倍（见图）。据估计，地球上所有细胞所含的DNA总量约为5.3{e | 37}碱基对，相当于1.325{e | 37}字节的信息。

 

 

如果数字存储以目前每年30-38%的复合年增长率继续增长，它将在大约110年内与地球上所有细胞中的所有DNA所包含的信息总量相抗衡。这将意味着在仅仅150年的时间里，生物圈中储存的信息量翻了一番”。

如果数字存储以目前每年30-38%的复合年增长率继续增长，<ref name="HilbertLopez2011" />它将在大约110年内与地球上所有细胞中的所有DNA所包含的信息总量相抗衡。这将意味着在仅仅150年的时间里，生物圈中储存的信息量翻了一番。<ref name="InfoBiosphere2016">{{Cite journal |url=http://escholarship.org/uc/item/38f4b791 |doi=10.1016/j.tree.2015.12.013|pmid=26777788|title=Information in the Biosphere: Biological and Digital Worlds|journal=Trends in Ecology & Evolution|volume=31|issue=3|pages=180–189|year=2016|last1=Kemp|first1=D. J.|last2=Hilbert|first2=M.|last3=Gillings|first3=M. R.}}</ref>

===对人类社会的影响===

===对人类社会的影响===

 

2009年2月，在'''人工智能促进协会 Association for the Advancement of Artificial Intelligence(AAAI)'''的主持下，Eric Horvitz在加利福尼亚州Pacific Grove的Asilomar主持了一次由主要计算机科学家、人工智能研究人员和机器人学家参加的会议。其目的是讨论机器人能够自给自足并能够自己做决定的假设可能性的潜在影响。他们讨论了计算机和机器人能够在多大程度上获得自主性 autonomy，以及在多大程度上可以利用这些能力对人类构成威胁或危险。<ref name="nytimes july09" />

 

 

In February 2009, under the auspices of the [[Association for the Advancement of Artificial Intelligence]] (AAAI), [[Eric Horvitz]] chaired a meeting of leading computer scientists, artificial intelligence researchers and roboticists at Asilomar in Pacific Grove, California. The goal was to discuss the potential impact of the hypothetical possibility that robots could become self-sufficient and able to make their own decisions. They discussed the extent to which computers and robots might be able to acquire [[autonomy]], and to what degree they could use such abilities to pose threats or hazards.<ref name="nytimes july09" />

 

2009年2月，在<font color="#ff8000">人工智能促进协会Association for the Advancement of Artificial Intelligence(AAAI) </font>的主持下，Eric Horvitz 在加利福尼亚州 Pacific Grove 的 Asilomar 主持了一次由主要计算机科学家、人工智能研究人员和机器人学家参加的会议。其目的是讨论机器人能够自给自足并能够自己做决定的假设可能性的潜在影响。他们讨论了计算机和机器人能够在多大程度上获得自主性 autonomy，以及在多大程度上可以利用这些能力对人类构成威胁或危险。<ref name="nytimes july09" />