更改

跳到导航 跳到搜索
删除26,220字节 、 2020年9月26日 (六) 14:08
无编辑摘要
第9行: 第9行:  
尽管相关工作可以追溯到更早,但自然语言处理(NLP)还是通常被认为始于20世纪50年代。
 
尽管相关工作可以追溯到更早,但自然语言处理(NLP)还是通常被认为始于20世纪50年代。
   −
=== Symbolic NLP (1950s - early 1990s) ===
+
=== 符号型 NLP (1950s - 1990s) ===
    
* '''1950s''':1950年,艾伦 · 图灵发表《计算机器与智能》一文,提出'''[[图灵测试 Turing Test]]'''作为判断机器智能程度的标准。
 
* '''1950s''':1950年,艾伦 · 图灵发表《计算机器与智能》一文,提出'''[[图灵测试 Turing Test]]'''作为判断机器智能程度的标准。
第23行: 第23行:  
  --[[用户:打豆豆|打豆豆]]([[用户讨论:打豆豆|讨论]])"一个重要的发展(最终导致1990年代的统计转变)是在此期间定量​​评估的重要性日益提高"一句为意译
 
  --[[用户:打豆豆|打豆豆]]([[用户讨论:打豆豆|讨论]])"一个重要的发展(最终导致1990年代的统计转变)是在此期间定量​​评估的重要性日益提高"一句为意译
   −
===统计自然语言处理(1990s-2010s) ===
+
=== 统计自然语言处理(1990s-2010s) ===
    
直到20世纪80年代,大多数自然语言处理系统仍依赖于复杂的、人工制定的规则。然而从20世纪80年代末开始,随着语言处理'''[[机器学习 Machine Learning]]'''算法的引入,自然语言处理领域掀起了一场革命。这是由于计算能力的稳步增长(参见'''[[摩尔定律 Moore's Law]]''')和'''[[乔姆斯基语言学理论 Chomskyan Theories of Linguistics]]的'''主导地位的削弱(如'''[[转换语法 Transformational Grammar]]''')。乔姆斯基语言学理论并不认同语料库语言学,而'''[[语料库语言学 Corpus Linguistic]]'''却是语言处理机器学习方法的基础。<ref>Chomskyan linguistics encourages the investigation of "[[corner case]]s" that stress the limits of its theoretical models (comparable to [[pathological (mathematics)|pathological]] phenomena in mathematics), typically created using [[thought experiment]]s, rather than the systematic investigation of typical phenomena that occur in real-world data, as is the case in [[corpus linguistics]].  The creation and use of such [[text corpus|corpora]] of real-world data is a fundamental part of machine-learning algorithms for natural language processing.  In addition, theoretical underpinnings of Chomskyan linguistics such as the so-called "[[poverty of the stimulus]]" argument entail that general learning algorithms, as are typically used in machine learning, cannot be successful in language processing.  As a result, the Chomskyan paradigm discouraged the application of such models to language processing.</ref>一些最早被使用的机器学习算法,比如'''[[决策树Decision Tree]]''',使用“如果...那么..."(if-then)硬判决系统,类似于之前既有的人工制定的规则。然而,'''[[词性标注 Part-of-speech Tagging]]'''将'''[[隐马尔可夫模型 Hidden Markov Models ]]'''引入到自然语言处理中,并且研究重点被放在了统计模型上。统计模型将输入数据的各个特征都赋上实值权重,从而做出'''[[软判决 Soft Decision]]'''和'''[[概率决策 Probabilistic Decision]]'''。许多语音识别系统现所依赖的缓存语言模型就是这种统计模型的例子。这种模型在给定非预期输入,尤其是包含错误的输入(在实际数据中这是非常常见的),并且将多个子任务整合到较大系统中时,结果通常更加可靠。
 
直到20世纪80年代,大多数自然语言处理系统仍依赖于复杂的、人工制定的规则。然而从20世纪80年代末开始,随着语言处理'''[[机器学习 Machine Learning]]'''算法的引入,自然语言处理领域掀起了一场革命。这是由于计算能力的稳步增长(参见'''[[摩尔定律 Moore's Law]]''')和'''[[乔姆斯基语言学理论 Chomskyan Theories of Linguistics]]的'''主导地位的削弱(如'''[[转换语法 Transformational Grammar]]''')。乔姆斯基语言学理论并不认同语料库语言学,而'''[[语料库语言学 Corpus Linguistic]]'''却是语言处理机器学习方法的基础。<ref>Chomskyan linguistics encourages the investigation of "[[corner case]]s" that stress the limits of its theoretical models (comparable to [[pathological (mathematics)|pathological]] phenomena in mathematics), typically created using [[thought experiment]]s, rather than the systematic investigation of typical phenomena that occur in real-world data, as is the case in [[corpus linguistics]].  The creation and use of such [[text corpus|corpora]] of real-world data is a fundamental part of machine-learning algorithms for natural language processing.  In addition, theoretical underpinnings of Chomskyan linguistics such as the so-called "[[poverty of the stimulus]]" argument entail that general learning algorithms, as are typically used in machine learning, cannot be successful in language processing.  As a result, the Chomskyan paradigm discouraged the application of such models to language processing.</ref>一些最早被使用的机器学习算法,比如'''[[决策树Decision Tree]]''',使用“如果...那么..."(if-then)硬判决系统,类似于之前既有的人工制定的规则。然而,'''[[词性标注 Part-of-speech Tagging]]'''将'''[[隐马尔可夫模型 Hidden Markov Models ]]'''引入到自然语言处理中,并且研究重点被放在了统计模型上。统计模型将输入数据的各个特征都赋上实值权重,从而做出'''[[软判决 Soft Decision]]'''和'''[[概率决策 Probabilistic Decision]]'''。许多语音识别系统现所依赖的缓存语言模型就是这种统计模型的例子。这种模型在给定非预期输入,尤其是包含错误的输入(在实际数据中这是非常常见的),并且将多个子任务整合到较大系统中时,结果通常更加可靠。
第33行: 第33行:  
*'''2000年代''':近期研究更多地集中在'''[[无监督学习 Unsupervised Learning]]'''和'''[[半监督学习 Semi-supervised Learning]]'''算法上。这些算法可以从无标注但有预期答案的数据或标注和未标注兼有的数据中学习。一般而言,这种任务比'''[[监督学习 Supervised Learning]]'''困难,并且在同量数据下,产生的结果通常不精确。然而如果算法具有较低的'''[[时间复杂度 Time Complexity]]''',且无标注的数据量巨大(包括万维网),可以有效弥补结果不精确的问题。
 
*'''2000年代''':近期研究更多地集中在'''[[无监督学习 Unsupervised Learning]]'''和'''[[半监督学习 Semi-supervised Learning]]'''算法上。这些算法可以从无标注但有预期答案的数据或标注和未标注兼有的数据中学习。一般而言,这种任务比'''[[监督学习 Supervised Learning]]'''困难,并且在同量数据下,产生的结果通常不精确。然而如果算法具有较低的'''[[时间复杂度 Time Complexity]]''',且无标注的数据量巨大(包括万维网),可以有效弥补结果不精确的问题。
   −
===神经NLP(2010s-至今)===
+
=== 神经NLP(2010s-至今) ===
    
二十一世纪一零年代,'''[[表示学习 Representation Learning]]'''和'''[[深度神经网络Deep Neural Network]]'''式的机器学习方法在自然语言处理中得到了广泛的应用,部分原因是一系列的结果表明这些技术可以在许多自然语言任务中获得最先进的结果<ref name=goldberg:nnlp17>{{cite journal |last=Goldberg |first=Yoav |year=2016 |arxiv=1807.10854 |title=A Primer on Neural Network Models for Natural Language Processing |journal=Journal of Artificial Intelligence Research |volume=57 |pages=345–420 |doi=10.1613/jair.4992 }}</ref><ref name=goodfellow:book16>{{cite book |first=Ian |last=Goodfellow |first2=Yoshua |last2=Bengio |first3=Aaron |last3=Courville |url=http://www.deeplearningbook.org/ |title=Deep Learning |location= |publisher=MIT Press |year=2016 |isbn= }}</ref>,比如语言建模、语法分析等<ref name=jozefowicz:lm16>{{cite book |first=Rafal |last=Jozefowicz |first2=Oriol |last2=Vinyals |first3=Mike |last3=Schuster |first4=Noam |last4=Shazeer |first5=Yonghui |last5=Wu |year=2016 |arxiv=1602.02410 |title=Exploring the Limits of Language Modeling |bibcode=2016arXiv160202410J }}</ref><ref name=choe:emnlp16>{{cite journal |first=Do Kook |last=Choe |first2=Eugene |last2=Charniak |journal=Emnlp 2016 |url=https://aclanthology.coli.uni-saarland.de/papers/D16-1257/d16-1257 |title=Parsing as Language Modeling }}</ref><ref name="vinyals:nips15">{{cite journal |last=Vinyals |first=Oriol |last2=Kaiser |first2=Lukasz |displayauthors=1 |journal=Nips2015 |title=Grammar as a Foreign Language |year=2014 |arxiv=1412.7449 |bibcode=2014arXiv1412.7449V |url=https://papers.nips.cc/paper/5635-grammar-as-a-foreign-language.pdf }}</ref>。流行的技术包括使用'''[[词嵌入Word Embedding]]'''来获取单词的语义属性,以及增加高级任务的端到端学习(如问答) ,而不是依赖于分立的中间任务流程(如词性标记和依赖性分析)。在某些领域,这种转变使得NLP系统的设计发生了重大变化,因此,基于深层神经网络的方法可以被视为一种有别于统计自然语言处理的新范式。例如,神经机器翻译(neural machine translation,NMT)一词强调了这样一个事实:基于深度学习的机器翻译方法直接学习序列到序列变换,从而避免了统计机器翻译(statistical machine translation,SMT)中使用的词对齐和语言建模等中间步骤。
 
二十一世纪一零年代,'''[[表示学习 Representation Learning]]'''和'''[[深度神经网络Deep Neural Network]]'''式的机器学习方法在自然语言处理中得到了广泛的应用,部分原因是一系列的结果表明这些技术可以在许多自然语言任务中获得最先进的结果<ref name=goldberg:nnlp17>{{cite journal |last=Goldberg |first=Yoav |year=2016 |arxiv=1807.10854 |title=A Primer on Neural Network Models for Natural Language Processing |journal=Journal of Artificial Intelligence Research |volume=57 |pages=345–420 |doi=10.1613/jair.4992 }}</ref><ref name=goodfellow:book16>{{cite book |first=Ian |last=Goodfellow |first2=Yoshua |last2=Bengio |first3=Aaron |last3=Courville |url=http://www.deeplearningbook.org/ |title=Deep Learning |location= |publisher=MIT Press |year=2016 |isbn= }}</ref>,比如语言建模、语法分析等<ref name=jozefowicz:lm16>{{cite book |first=Rafal |last=Jozefowicz |first2=Oriol |last2=Vinyals |first3=Mike |last3=Schuster |first4=Noam |last4=Shazeer |first5=Yonghui |last5=Wu |year=2016 |arxiv=1602.02410 |title=Exploring the Limits of Language Modeling |bibcode=2016arXiv160202410J }}</ref><ref name=choe:emnlp16>{{cite journal |first=Do Kook |last=Choe |first2=Eugene |last2=Charniak |journal=Emnlp 2016 |url=https://aclanthology.coli.uni-saarland.de/papers/D16-1257/d16-1257 |title=Parsing as Language Modeling }}</ref><ref name="vinyals:nips15">{{cite journal |last=Vinyals |first=Oriol |last2=Kaiser |first2=Lukasz |displayauthors=1 |journal=Nips2015 |title=Grammar as a Foreign Language |year=2014 |arxiv=1412.7449 |bibcode=2014arXiv1412.7449V |url=https://papers.nips.cc/paper/5635-grammar-as-a-foreign-language.pdf }}</ref>。流行的技术包括使用'''[[词嵌入Word Embedding]]'''来获取单词的语义属性,以及增加高级任务的端到端学习(如问答) ,而不是依赖于分立的中间任务流程(如词性标记和依赖性分析)。在某些领域,这种转变使得NLP系统的设计发生了重大变化,因此,基于深层神经网络的方法可以被视为一种有别于统计自然语言处理的新范式。例如,神经机器翻译(neural machine translation,NMT)一词强调了这样一个事实:基于深度学习的机器翻译方法直接学习序列到序列变换,从而避免了统计机器翻译(statistical machine translation,SMT)中使用的词对齐和语言建模等中间步骤。
第39行: 第39行:  
==基于规则的NLP vs. 统计NLP (Rule-based vs. statistical NLP{{anchor|Statistical natural language processing (SNLP)}})==
 
==基于规则的NLP vs. 统计NLP (Rule-based vs. statistical NLP{{anchor|Statistical natural language processing (SNLP)}})==
   −
In the early days, many language-processing systems were designed by hand-coding a set of rules:<ref name=winograd:shrdlu71>{{cite thesis |last=Winograd |first=Terry |year=1971 |title=Procedures as a Representation for Data in a Computer Program for Understanding Natural Language |url=http://hci.stanford.edu/winograd/shrdlu/ }}</ref><ref name=schank77>{{cite book |first=Roger C. |last=Schank |first2=Robert P. |last2=Abelson |year=1977 |title=Scripts, Plans, Goals, and Understanding: An Inquiry Into Human Knowledge Structures |location=Hillsdale |publisher=Erlbaum |isbn=0-470-99033-3 }}</ref> such as by writing grammars or devising heuristic rules for [[stemming]].  
+
在早期,许多语言处理系统是通过人工编码一组规则来设计的<ref name=winograd:shrdlu71>{{cite thesis |last=Winograd |first=Terry |year=1971 |title=Procedures as a Representation for Data in a Computer Program for Understanding Natural Language |url=http://hci.stanford.edu/winograd/shrdlu/ }}</ref><ref name=schank77>{{cite book |first=Roger C. |last=Schank |first2=Robert P. |last2=Abelson |year=1977 |title=Scripts, Plans, Goals, and Understanding: An Inquiry Into Human Knowledge Structures |location=Hillsdale |publisher=Erlbaum |isbn=0-470-99033-3 }}</ref>: 例如通过编写语法或设计用于词干提取的'''[[启发式 Heuristic]]'''规则。
 +
.
   −
In the early days, many language-processing systems were designed by hand-coding a set of rules: such as by writing grammars or devising heuristic rules for stemming.
+
自20世纪80年代末和90年代中期的“统计革命”ref name=johnson:eacl:ilcl09>[http://www.aclweb.org/anthology/W09-0103 Mark Johnson. How the statistical revolution changes (computational) linguistics.] Proceedings of the EACL 2009 Workshop on the Interaction between Linguistics and Computational Linguistics.</ref><ref name=resnik:langlog11>[http://languagelog.ldc.upenn.edu/nll/?p=2946 Philip Resnik. Four revolutions.] Language Log, February 5, 2011.</ref>以来,许多自然语言处理研究都深度依赖机器学习。机器学习的范式要求通过分析大型语料库(corpora,语料库corpus的复数形式,是一组可能带有人或计算机标注的文档)使用统计学推论自动学习这些规则。
 
  −
在早期,许多语言处理系统是通过人工编码一组规则来设计的: 例如通过编写语法或设计用于词干提取的'''[[启发式 Heuristic]]'''规则。
  −
 
  −
 
  −
 
  −
Since the so-called "statistical revolution"<ref name=johnson:eacl:ilcl09>[http://www.aclweb.org/anthology/W09-0103 Mark Johnson. How the statistical revolution changes (computational) linguistics.] Proceedings of the EACL 2009 Workshop on the Interaction between Linguistics and Computational Linguistics.</ref><ref name=resnik:langlog11>[http://languagelog.ldc.upenn.edu/nll/?p=2946 Philip Resnik. Four revolutions.] Language Log, February 5, 2011.</ref> in the late 1980s and mid-1990s, much natural language processing research has relied heavily on [[machine learning]]. The machine-learning paradigm calls instead for using [[statistical inference]] to automatically learn such rules through the analysis of large ''[[text corpus|corpora]]'' (the plural form of ''corpus'', is a set of documents, possibly with human or computer annotations) of typical real-world examples.
  −
 
  −
Since the so-called "statistical revolution" in the late 1980s and mid-1990s, much natural language processing research has relied heavily on machine learning. The machine-learning paradigm calls instead for using statistical inference to automatically learn such rules through the analysis of large corpora (the plural form of corpus, is a set of documents, possibly with human or computer annotations) of typical real-world examples.
  −
 
  −
自20世纪80年代末和90年代中期的“统计革命”以来,许多自然语言处理研究都深度依赖机器学习。机器学习的范式要求通过分析大型语料库(corpora,语料库corpus的复数形式,是一组可能带有人或计算机标注的文档)使用统计学推论自动学习这些规则。
  −
 
  −
 
  −
 
  −
Many different classes of machine-learning algorithms have been applied to natural-language-processing tasks. These algorithms take as input a large set of "features" that are generated from the input data. Some of the earliest-used algorithms, such as [[decision tree]]s, produced systems of hard if-then rules similar to the systems of handwritten rules that were then common. Increasingly, however, research has focused on [[statistical models]], which make soft, [[probabilistic]] decisions based on attaching [[real-valued]] weights to each input feature. Such models have the advantage that they can express the relative certainty of many different possible answers rather than only one, producing more reliable results when such a model is included as a component of a larger system.
  −
 
  −
Many different classes of machine-learning algorithms have been applied to natural-language-processing tasks. These algorithms take as input a large set of "features" that are generated from the input data. Some of the earliest-used algorithms, such as decision trees, produced systems of hard if-then rules similar to the systems of handwritten rules that were then common. Increasingly, however, research has focused on statistical models, which make soft, probabilistic decisions based on attaching real-valued weights to each input feature. Such models have the advantage that they can express the relative certainty of many different possible answers rather than only one, producing more reliable results when such a model is included as a component of a larger system.
      
许多不同类型的机器学习算法已被应用在自然语言处理任务中。这些算法将输入数据的大量“特性”作为输入。一些最早被使用的算法,比如'''[[决策树Decision Tree]]''',使用“如果...那么..."(if-then)硬判决系统,类似于之前既有的人工制定的规则。然而后来人们将研究重点聚焦在统计模型上。统计模型将输入数据的各个特征都赋上实值权重,从而做出'''[[软判决 Soft Decision]]'''和'''[[概率决策 Probabilistic Decision]]'''。这种模型的优点是,它们可以表示出许多不同的可能答案的相对确定性,而不仅仅是一个答案。当这种模型作为一个更大系统的模块时,产生的结果更加可靠。
 
许多不同类型的机器学习算法已被应用在自然语言处理任务中。这些算法将输入数据的大量“特性”作为输入。一些最早被使用的算法,比如'''[[决策树Decision Tree]]''',使用“如果...那么..."(if-then)硬判决系统,类似于之前既有的人工制定的规则。然而后来人们将研究重点聚焦在统计模型上。统计模型将输入数据的各个特征都赋上实值权重,从而做出'''[[软判决 Soft Decision]]'''和'''[[概率决策 Probabilistic Decision]]'''。这种模型的优点是,它们可以表示出许多不同的可能答案的相对确定性,而不仅仅是一个答案。当这种模型作为一个更大系统的模块时,产生的结果更加可靠。
  −
Systems based on machine-learning algorithms have many advantages over hand-produced rules:
  −
  −
Systems based on machine-learning algorithms have many advantages over hand-produced rules:
      
基于机器学习算法的系统比人工制定的规则有许多优点:
 
基于机器学习算法的系统比人工制定的规则有许多优点:
  −
*The learning procedures used during machine learning automatically focus on the most common cases, whereas when writing rules by hand it is often not at all obvious where the effort should be directed.
      
*机器学习的学习过程自动聚焦于最常见的例子,然而人工制定的规则常常不知道从何下手
 
*机器学习的学习过程自动聚焦于最常见的例子,然而人工制定的规则常常不知道从何下手
  −
*Automatic learning procedures can make use of statistical inference algorithms to produce models that are robust to unfamiliar input (e.g. containing words or structures that have not been seen before) and to erroneous input (e.g. with misspelled words or words accidentally omitted). Generally, handling such input gracefully with handwritten rules, or, more generally, creating systems of handwritten rules that make soft decisions, is extremely difficult, error-prone and time-consuming.
      
*自动学习过程中可以利用统计推断算法生成对不常见输入(包含未见过的字词或结构)、错误输入(如拼错或无意遗漏词语)有较好鲁棒性的模型。通常用人工制定的规则或建立一个人工制定规则的软决策系统处理这样的输入是极其困难、易于出错且耗费时间的。
 
*自动学习过程中可以利用统计推断算法生成对不常见输入(包含未见过的字词或结构)、错误输入(如拼错或无意遗漏词语)有较好鲁棒性的模型。通常用人工制定的规则或建立一个人工制定规则的软决策系统处理这样的输入是极其困难、易于出错且耗费时间的。
  −
*Systems based on automatically learning the rules can be made more accurate simply by supplying more input data. However, systems based on handwritten rules can only be made more accurate by increasing the complexity of the rules, which is a much more difficult task. In particular, there is a limit to the complexity of systems based on handcrafted rules, beyond which the systems become more and more unmanageable. However, creating more data to input to machine-learning systems simply requires a corresponding increase in the number of man-hours worked, generally without significant increases in the complexity of the annotation process.
      
基于自动学习规则的系统可以仅用更多的输出就能得到更精确的结果。然而基于人工制定规则的系统只能通过把规则变得复杂来实现提高结果精确度,而制定更复杂的规则这件事本身就很困难。而且基于人工制定规则的系统有一定的限制,超过限制后系统就会变得不可控。然而制造更多数据供给机器学习系统只需要增加相应的人工标注的时间,而且这个过程的复杂度不会有显著改变。
 
基于自动学习规则的系统可以仅用更多的输出就能得到更精确的结果。然而基于人工制定规则的系统只能通过把规则变得复杂来实现提高结果精确度,而制定更复杂的规则这件事本身就很困难。而且基于人工制定规则的系统有一定的限制,超过限制后系统就会变得不可控。然而制造更多数据供给机器学习系统只需要增加相应的人工标注的时间,而且这个过程的复杂度不会有显著改变。
第81行: 第56行:  
==主要评估及任务(Major evaluations and tasks)==
 
==主要评估及任务(Major evaluations and tasks)==
   −
The following is a list of some of the most commonly researched tasks in natural language processing. Some of these tasks have direct real-world applications, while others more commonly serve as subtasks that are used to aid in solving larger tasks.
  −
  −
The following is a list of some of the most commonly researched tasks in natural language processing. Some of these tasks have direct real-world applications, while others more commonly serve as subtasks that are used to aid in solving larger tasks.
      
以下列出了自然语言处理中最常被研究的任务。其中一些任务在具有直接的实际应用,而其他任务则通常作为子任务,用于帮助解决更大的任务。
 
以下列出了自然语言处理中最常被研究的任务。其中一些任务在具有直接的实际应用,而其他任务则通常作为子任务,用于帮助解决更大的任务。
  −
  −
  −
Though natural language processing tasks are closely intertwined, they are frequently subdivided into categories for convenience. A coarse division is given below.
  −
  −
Though natural language processing tasks are closely intertwined, they are frequently subdivided into categories for convenience. A coarse division is given below.
      
尽管自然语言处理的各种任务紧密交错,但为了方便,它们常被细分为不同的类别。下面给出一个粗略的分类。
 
尽管自然语言处理的各种任务紧密交错,但为了方便,它们常被细分为不同的类别。下面给出一个粗略的分类。
第99行: 第65行:  
===句法(Syntax)===
 
===句法(Syntax)===
   −
; [[Grammar induction]]<ref>{{cite journal |last=Klein |first=Dan |first2=Christopher D. |last2=Manning |url=http://papers.nips.cc/paper/1945-natural-language-grammar-induction-using-a-constituent-context-model.pdf |title=Natural language grammar induction using a constituent-context model |journal=Advances in Neural Information Processing Systems |year=2002 }}</ref>: Generate a [[formal grammar]] that describes a language's syntax.
+
'''[[语法归纳 Grammar Induction]]'''<ref>{{cite journal |last=Klein |first=Dan |first2=Christopher D. |last2=Manning |url=http://papers.nips.cc/paper/1945-natural-language-grammar-induction-using-a-constituent-context-model.pdf |title=Natural language grammar induction using a constituent-context model |journal=Advances in Neural Information Processing Systems |year=2002 }}</ref>: 生成描述语言句法结构的规范语法。
 
  −
Grammar induction: Generate a formal grammar that describes a language's syntax.
  −
 
  −
'''[[语法归纳 Grammar Induction]]''': 生成描述语言句法结构的规范语法。
  −
 
  −
; [[Lemmatisation|Lemmatization]]: The task of removing inflectional endings only and to return the base dictionary form of a word which is also known as a lemma.
  −
 
  −
Lemmatization: The task of removing inflectional endings only and to return the base dictionary form of a word which is also known as a lemma.
      
'''[[词形还原 Lemmatization]]''': 只去掉词形变化的词尾,并返回词的基本形式,也称'''[[词目 Lemma]]'''。
 
'''[[词形还原 Lemmatization]]''': 只去掉词形变化的词尾,并返回词的基本形式,也称'''[[词目 Lemma]]'''。
   −
; [[Morphology (linguistics)|Morphological segmentation]]: Separate words into individual [[morpheme]]s and identify the class of the morphemes. The difficulty of this task depends greatly on the complexity of the [[Morphology (linguistics)|morphology]] (''i.e.'', the structure of words) of the language being considered. [[English language|English]] has fairly simple morphology, especially [[inflectional morphology]], and thus it is often possible to ignore this task entirely and simply model all possible forms of a word (''e.g.'', "open, opens, opened, opening") as separate words. In languages such as [[Turkish language|Turkish]] or [[Meitei language|Meitei]],<ref>{{cite journal |last=Kishorjit |first=N. |last2=Vidya |first2=Raj RK. |last3=Nirmal |first3=Y. |last4=Sivaji |first4=B. |year=2012 |url=http://aclweb.org/anthology//W/W12/W12-5008.pdf |title=Manipuri Morpheme Identification |journal=Proceedings of the 3rd Workshop on South and Southeast Asian Natural Language Processing (SANLP) |pages=95–108 |location=COLING 2012, Mumbai, December 2012 }}</ref> a highly [[Agglutination|agglutinated]] Indian language, however, such an approach is not possible, as each dictionary entry has thousands of possible word forms.
+
'''[[语素切分  Morphological Segmentation]]''': 将单词分成独立的'''[[语素 Morpheme]]''',并确定语素的类别。这项任务的难度很大程度上取决于所考虑的语言的形态(即句子的结构)的复杂性。英语有相当简单的语素,特别是'''[[屈折语素 Inflectional Morphology]]''',因此通常可以完全忽略这个任务,而简单地将一个单词的所有可能形式(例如,"open,opens,opened,opening")作为单独的单词。然而,在诸如土耳其语或曼尼普尔语这样的语言中<ref>{{cite journal |last=Kishorjit |first=N. |last2=Vidya |first2=Raj RK. |last3=Nirmal |first3=Y. |last4=Sivaji |first4=B. |year=2012 |url=http://aclweb.org/anthology//W/W12/W12-5008.pdf |title=Manipuri Morpheme Identification |journal=Proceedings of the 3rd Workshop on South and Southeast Asian Natural Language Processing (SANLP) |pages=95–108 |location=COLING 2012, Mumbai, December 2012 }}</ref> ,这种方法是不可取的,因为每个词都有成千上万种可能的词形。
 
  −
Morphological segmentation: Separate words into individual morphemes and identify the class of the morphemes. The difficulty of this task depends greatly on the complexity of the morphology (i.e., the structure of words) of the language being considered. English has fairly simple morphology, especially inflectional morphology, and thus it is often possible to ignore this task entirely and simply model all possible forms of a word (e.g., "open, opens, opened, opening") as separate words. In languages such as Turkish or Meitei, a highly agglutinated Indian language, however, such an approach is not possible, as each dictionary entry has thousands of possible word forms.
  −
 
  −
'''[[语素切分  Morphological Segmentation]]''': 将单词分成独立的'''[[语素 Morpheme]]''',并确定语素的类别。这项任务的难度很大程度上取决于所考虑的语言的形态(即句子的结构)的复杂性。英语有相当简单的语素,特别是'''[[屈折语素 Inflectional Morphology]]''',因此通常可以完全忽略这个任务,而简单地将一个单词的所有可能形式(例如,"open,opens,opened,opening")作为单独的单词。然而,在诸如土耳其语或曼尼普尔语这样的语言中,这种方法是不可取的,因为每个词都有成千上万种可能的词形。
  −
 
  −
; [[Part-of-speech tagging]]: Given a sentence, determine the [[part of speech]] (POS) for each word. Many words, especially common ones, can serve as multiple [[parts of speech]]. For example, "book" can be a [[noun]] ("the book on the table") or [[verb]] ("to book a flight"); "set" can be a [[noun]], [[verb]] or [[adjective]]; and "out" can be any of at least five different parts of speech. Some languages have more such ambiguity than others.{{dubious|date=June 2018}} Languages with little [[inflectional morphology]], such as [[English language|English]], are particularly prone to such ambiguity. [[Chinese language|Chinese]] is prone to such ambiguity because it is a [[tonal language]] during verbalization. Such inflection is not readily conveyed via the entities employed within the orthography to convey the intended meaning.
  −
 
  −
Part-of-speech tagging: Given a sentence, determine the part of speech (POS) for each word. Many words, especially common ones, can serve as multiple parts of speech. For example, "book" can be a noun ("the book on the table") or verb ("to book a flight"); "set" can be a noun, verb or adjective; and "out" can be any of at least five different parts of speech. Some languages have more such ambiguity than others. Languages with little inflectional morphology, such as English, are particularly prone to such ambiguity. Chinese is prone to such ambiguity because it is a tonal language during verbalization. Such inflection is not readily conveyed via the entities employed within the orthography to convey the intended meaning.
      
'''[[词性标注 Part-of-speech Tagging]]''': 给定一个句子,确定每个词的词性(part of speech, POS)。许多单词,尤其是常见的单词,可以拥有多种词性。例如,“book”可以是名词(书本)(“ the book on the table”)或动词(预订)(“to book a flight”); “set”可以是名词、动词或形容词; “out”至少有五种不同的词性。有些语言比其他语言有更多的这种模糊性。像英语这样几乎没有屈折形态的语言尤其容易出现这种歧义。汉语是一种在动词化过程中会变音调的语言,所以容易出现歧义现象。这样的词形变化不容易通过正字法中使用的实体来传达预期的意思。
 
'''[[词性标注 Part-of-speech Tagging]]''': 给定一个句子,确定每个词的词性(part of speech, POS)。许多单词,尤其是常见的单词,可以拥有多种词性。例如,“book”可以是名词(书本)(“ the book on the table”)或动词(预订)(“to book a flight”); “set”可以是名词、动词或形容词; “out”至少有五种不同的词性。有些语言比其他语言有更多的这种模糊性。像英语这样几乎没有屈折形态的语言尤其容易出现这种歧义。汉语是一种在动词化过程中会变音调的语言,所以容易出现歧义现象。这样的词形变化不容易通过正字法中使用的实体来传达预期的意思。
    
  --[[用户:Thingamabob|Thingamabob]]([[用户讨论:Thingamabob|讨论]])“‘out’至少有五种不同的词性”一句为意译
 
  --[[用户:Thingamabob|Thingamabob]]([[用户讨论:Thingamabob|讨论]])“‘out’至少有五种不同的词性”一句为意译
  −
; [[Parsing]]: Determine the [[parse tree]] (grammatical analysis) of a given sentence. The [[grammar]] for [[natural language|natural languages]] is [[ambiguous]] and typical sentences have multiple possible analyses: perhaps surprisingly, for a typical sentence there may be thousands of potential parses (most of which will seem completely nonsensical to a human). There are two primary types of parsing: ''dependency parsing'' and ''constituency parsing''. Dependency parsing focuses on the relationships between words in a sentence (marking things like primary objects and predicates), whereas constituency parsing focuses on building out the parse tree using a [[probabilistic context-free grammar]] (PCFG) (see also ''[[stochastic grammar]]'').
  −
  −
Parsing: Determine the parse tree (grammatical analysis) of a given sentence. The grammar for natural languages is ambiguous and typical sentences have multiple possible analyses: perhaps surprisingly, for a typical sentence there may be thousands of potential parses (most of which will seem completely nonsensical to a human). There are two primary types of parsing: dependency parsing and constituency parsing. Dependency parsing focuses on the relationships between words in a sentence (marking things like primary objects and predicates), whereas constituency parsing focuses on building out the parse tree using a probabilistic context-free grammar (PCFG) (see also stochastic grammar).
      
'''[[语法分析 Parsing]]''': 确定给定句子的'''[[语法树 Parse tree]]'''(语法分析)。自然语言的语法是模糊的,典型的句子有多种可能的分析: 也许会让人有些吃惊,一个典型的句子可能有成千上万个潜在的语法分析(其中大多数对于人类来说是毫无意义的)。分析类型主要有两种: '''[[依存分析 Dependency Parsing]]'''和'''[[成分分析 Constituency Parsing]]'''。依存句法分析侧重于句子中单词之间的关系(标记主要对象和谓语等) ,而成分分析侧重于使用'''[[概率上下文无关文法 Probabilistic Context-free Grammar, PCFG]]'''(probabilistic context-free grammar,PCFG)构建语法树(参见'''[[随机语法 Stochastic Grammar]]''')。
 
'''[[语法分析 Parsing]]''': 确定给定句子的'''[[语法树 Parse tree]]'''(语法分析)。自然语言的语法是模糊的,典型的句子有多种可能的分析: 也许会让人有些吃惊,一个典型的句子可能有成千上万个潜在的语法分析(其中大多数对于人类来说是毫无意义的)。分析类型主要有两种: '''[[依存分析 Dependency Parsing]]'''和'''[[成分分析 Constituency Parsing]]'''。依存句法分析侧重于句子中单词之间的关系(标记主要对象和谓语等) ,而成分分析侧重于使用'''[[概率上下文无关文法 Probabilistic Context-free Grammar, PCFG]]'''(probabilistic context-free grammar,PCFG)构建语法树(参见'''[[随机语法 Stochastic Grammar]]''')。
  −
; [[Sentence breaking]] (also known as "[[sentence boundary disambiguation]]"): Given a chunk of text, find the sentence boundaries. Sentence boundaries are often marked by [[Full stop|periods]] or other [[punctuation mark|punctuation marks]], but these same characters can serve other purposes (''e.g.'', marking [[abbreviation|abbreviations]]).
  −
  −
Sentence breaking (also known as "sentence boundary disambiguation"): Given a chunk of text, find the sentence boundaries. Sentence boundaries are often marked by periods or other punctuation marks, but these same characters can serve other purposes (e.g., marking abbreviations).
      
'''[[断句 Sentence breaking]]'''(也被称为'''[[句子边界消歧 Sentence Boundary Disambiguation]]''') : 给定一段文本,找到句子边界。句子的边界通常用句号或其他标点符号来标记,但是这些标点符号也会被用于其他目的(例如,标记缩写)。
 
'''[[断句 Sentence breaking]]'''(也被称为'''[[句子边界消歧 Sentence Boundary Disambiguation]]''') : 给定一段文本,找到句子边界。句子的边界通常用句号或其他标点符号来标记,但是这些标点符号也会被用于其他目的(例如,标记缩写)。
  −
; [[Stemming]]: The process of reducing inflected (or sometimes derived) words to their root form. (''e.g.'', "close" will be the root for "closed", "closing", "close", "closer" etc.).
  −
  −
Stemming: The process of reducing inflected (or sometimes derived) words to their root form. (e.g., "close" will be the root for "closed", "closing", "close", "closer" etc.).
      
'''[[词根化 Stemming]]''': 把词形变化(或者派生出来的)的词缩减到其词根形式的过程。(例如,close 是“ closed”、“ closing”、“ close”、“ closer”等的词根。).
 
'''[[词根化 Stemming]]''': 把词形变化(或者派生出来的)的词缩减到其词根形式的过程。(例如,close 是“ closed”、“ closing”、“ close”、“ closer”等的词根。).
  −
; [[Word segmentation]]: Separate a chunk of continuous text into separate words. For a language like [[English language|English]], this is fairly trivial, since words are usually separated by spaces. However, some written languages like [[Chinese language|Chinese]], [[Japanese language|Japanese]] and [[Thai language|Thai]] do not mark word boundaries in such a fashion, and in those languages text segmentation is a significant task requiring knowledge of the [[vocabulary]] and [[Morphology (linguistics)|morphology]] of words in the language. Sometimes this process is also used in cases like bag of words (BOW) creation in data mining.
  −
  −
Word segmentation: Separate a chunk of continuous text into separate words. For a language like English, this is fairly trivial, since words are usually separated by spaces. However, some written languages like Chinese, Japanese and Thai do not mark word boundaries in such a fashion, and in those languages text segmentation is a significant task requiring knowledge of the vocabulary and morphology of words in the language. Sometimes this process is also used in cases like bag of words (BOW) creation in data mining.
      
'''[[分词 Word Segmentation]]''': 把一段连续的文本分割成单独的词语。对于像英语之类的语言是相对简单的,因为单词通常由空格分隔。然而,对于汉语、日语和泰语的文字,并没有类似这种方式的词语边界标记,在这些语言中,文本分词是一项重要的任务,要求掌握语言中词汇和词形的知识。有时这个过程也被用于数据挖掘中创建[[词包]](bag of words,BOW)。
 
'''[[分词 Word Segmentation]]''': 把一段连续的文本分割成单独的词语。对于像英语之类的语言是相对简单的,因为单词通常由空格分隔。然而,对于汉语、日语和泰语的文字,并没有类似这种方式的词语边界标记,在这些语言中,文本分词是一项重要的任务,要求掌握语言中词汇和词形的知识。有时这个过程也被用于数据挖掘中创建[[词包]](bag of words,BOW)。
  −
; [[Terminology extraction]]: The goal of terminology extraction is to automatically extract relevant terms from a given corpus.
  −
  −
Terminology extraction: The goal of terminology extraction is to automatically extract relevant terms from a given corpus.
      
'''[[术语抽取 Terminology Extraction]]''': 术语抽取的目标是从给定的语料库中自动提取相关术语。
 
'''[[术语抽取 Terminology Extraction]]''': 术语抽取的目标是从给定的语料库中自动提取相关术语。
    
===语义(Semantics)===
 
===语义(Semantics)===
  −
; [[Lexical semantics]]: What is the computational meaning of individual words in context?
  −
  −
Lexical semantics: What is the computational meaning of individual words in context?
      
'''[[词汇语义学  Lexical Semantics]]''': 每个词在上下文中的计算意义是什么?
 
'''[[词汇语义学  Lexical Semantics]]''': 每个词在上下文中的计算意义是什么?
  −
; [[Distributional semantics]]: How can we learn semantic representations from data?
  −
  −
Distributional semantics: How can we learn semantic representations from data?
      
'''[[分布语义  Distributional semantics]]''': 我们如何从数据中学习语义表示?
 
'''[[分布语义  Distributional semantics]]''': 我们如何从数据中学习语义表示?
  −
; [[Machine translation]]: Automatically translate text from one human language to another.  This is one of the most difficult problems, and is a member of a class of problems colloquially termed "[[AI-complete]]", i.e. requiring all of the different types of knowledge that humans possess (grammar, semantics, facts about the real world, etc.) to solve properly.
  −
  −
Machine translation: Automatically translate text from one human language to another.  This is one of the most difficult problems, and is a member of a class of problems colloquially termed "AI-complete", i.e. requiring all of the different types of knowledge that humans possess (grammar, semantics, facts about the real world, etc.) to solve properly.
      
'''[[机器翻译 Machine Translation]]''': 将文本从一种语言自动翻译成另一种语言。这是最困难的问题之一,也是“人工智能完备”问题的一部分,即需要人类拥有的所有不同类型的知识(语法、语义、对现实世界的事实的认知等)才能妥善解决。
 
'''[[机器翻译 Machine Translation]]''': 将文本从一种语言自动翻译成另一种语言。这是最困难的问题之一,也是“人工智能完备”问题的一部分,即需要人类拥有的所有不同类型的知识(语法、语义、对现实世界的事实的认知等)才能妥善解决。
  −
; [[Named entity recognition]] (NER): Given a stream of text, determine which items in the text map to proper names, such as people or places, and what the type of each such name is (e.g. person, location, organization). Although [[capitalization]] can aid in recognizing named entities in languages such as English, this information cannot aid in determining the type of named entity, and in any case, is often inaccurate or insufficient.  For example, the first letter of a sentence is also capitalized, and named entities often span several words, only some of which are capitalized.  Furthermore, many other languages in non-Western scripts (e.g. [[Chinese language|Chinese]] or [[Arabic language|Arabic]]) do not have any capitalization at all, and even languages with capitalization may not consistently use it to distinguish names. For example, [[German language|German]] capitalizes all [[noun]]s, regardless of whether they are names, and [[French language|French]] and [[Spanish language|Spanish]] do not capitalize names that serve as [[adjective]]s.
  −
  −
Named entity recognition (NER): Given a stream of text, determine which items in the text map to proper names, such as people or places, and what the type of each such name is (e.g. person, location, organization). Although capitalization can aid in recognizing named entities in languages such as English, this information cannot aid in determining the type of named entity, and in any case, is often inaccurate or insufficient.  For example, the first letter of a sentence is also capitalized, and named entities often span several words, only some of which are capitalized.  Furthermore, many other languages in non-Western scripts (e.g. Chinese or Arabic) do not have any capitalization at all, and even languages with capitalization may not consistently use it to distinguish names. For example, German capitalizes all nouns, regardless of whether they are names, and French and Spanish do not capitalize names that serve as adjectives.
      
'''[[命名实体识别 Named entity Recognition, NER]]''': 给定一个文本流,确定文本中的哪些词能映射到专有名称,如人或地点,以及这些名称的类型(例如:人名、地点名、组织名)。虽然大写有助于识别英语等语言中的命名实体,但这种信息对于确定命名实体的类型无用,而且,在多数情况下,这种信息是不准确、不充分的。比如,一个句子的第一个字母也是大写的,以及命名实体通常跨越几个单词,只有某些是大写的。此外,许多其他非西方文字的语言(如汉语或阿拉伯语)没有大写,甚至有大写的语言也不一定能用它来区分名字。例如,德语中多有名词都大写,法语和西班牙语中作为形容词的名称不大写。
 
'''[[命名实体识别 Named entity Recognition, NER]]''': 给定一个文本流,确定文本中的哪些词能映射到专有名称,如人或地点,以及这些名称的类型(例如:人名、地点名、组织名)。虽然大写有助于识别英语等语言中的命名实体,但这种信息对于确定命名实体的类型无用,而且,在多数情况下,这种信息是不准确、不充分的。比如,一个句子的第一个字母也是大写的,以及命名实体通常跨越几个单词,只有某些是大写的。此外,许多其他非西方文字的语言(如汉语或阿拉伯语)没有大写,甚至有大写的语言也不一定能用它来区分名字。例如,德语中多有名词都大写,法语和西班牙语中作为形容词的名称不大写。
  −
; [[Natural language generation]]: Convert information from computer databases or semantic intents into readable human language.
  −
  −
Natural language generation: Convert information from computer databases or semantic intents into readable human language.
      
'''[[自然语言生成]]''': 将计算机数据库或语义意图中的信息转换为人类可读的语言。
 
'''[[自然语言生成]]''': 将计算机数据库或语义意图中的信息转换为人类可读的语言。
   −
; [[Natural language understanding]]: Convert chunks of text into more formal representations such as [[first-order logic]] structures that are easier for [[computer]] programs to manipulate. Natural language understanding involves the identification of the intended semantic from the multiple possible semantics which can be derived from a natural language expression which usually takes the form of organized notations of natural language concepts. Introduction and creation of language metamodel and ontology are efficient however empirical solutions. An explicit formalization of natural language semantics without confusions with implicit assumptions such as [[closed-world assumption]] (CWA) vs. [[open-world assumption]], or subjective Yes/No vs. objective True/False is expected for the construction of a basis of semantics formalization.<ref>{{cite journal |first=Yucong |last=Duan |first2=Christophe |last2=Cruz |year=2011 |url=http://www.ijimt.org/abstract/100-E00187.htm |title=Formalizing Semantic of Natural Language through Conceptualization from Existence |archiveurl=https://web.archive.org/web/20111009135952/http://www.ijimt.org/abstract/100-E00187.htm |archivedate=2011-10-09 |journal=International Journal of Innovation, Management and Technology |volume=2 |issue=1 |pages=37–42 }}</ref>
+
'''[[自然语言理解 Natural Language Understanding]]''': 将文本块转换成更加正式的表示形式,比如更易于计算机程序处理的'''[[一阶逻辑结构 First-order Logic Structure]]'''。自然语言理解包括从多种可能的语义中识别预期的语义,这些语义可以由有序符号表现的自然语言表达中派生出来。引入和创建语言元模型和本体是有效但经验化的做法。自然语言语义<font color=#32cd32>形式化]]要求清楚明了,而不能是混有隐含的猜测,如封闭世界假设与开放世界假设、主观的是 / 否与客观的真 / <ref>{{cite journal |first=Yucong |last=Duan |first2=Christophe |last2=Cruz |year=2011 |url=http://www.ijimt.org/abstract/100-E00187.htm |title=Formalizing Semantic of Natural Language through Conceptualization from Existence |archiveurl=https://web.archive.org/web/20111009135952/http://www.ijimt.org/abstract/100-E00187.htm |archivedate=2011-10-09 |journal=International Journal of Innovation, Management and Technology |volume=2 |issue=1 |pages=37–42 }}</ref>
 
  −
Natural language understanding: Convert chunks of text into more formal representations such as first-order logic structures that are easier for computer programs to manipulate. Natural language understanding involves the identification of the intended semantic from the multiple possible semantics which can be derived from a natural language expression which usually takes the form of organized notations of natural language concepts. Introduction and creation of language metamodel and ontology are efficient however empirical solutions. An explicit formalization of natural language semantics without confusions with implicit assumptions such as closed-world assumption (CWA) vs. open-world assumption, or subjective Yes/No vs. objective True/False is expected for the construction of a basis of semantics formalization.
  −
 
  −
'''[[自然语言理解 Natural Language Understanding]]''': 将文本块转换成更加正式的表示形式,比如更易于计算机程序处理的'''[[一阶逻辑结构 First-order Logic Structure]]'''。自然语言理解包括从多种可能的语义中识别预期的语义,这些语义可以由有序符号表现的自然语言表达中派生出来。引入和创建语言元模型和本体是有效但经验化的做法。自然语言语义<font color=#32cd32>形式化]]要求清楚明了,而不能是混有隐含的猜测,如封闭世界假设与开放世界假设、主观的是 / 否与客观的真 / 假。
  −
 
  −
; [[Optical character recognition]] (OCR): Given an image representing printed text, determine the corresponding text.
  −
 
  −
Optical character recognition (OCR): Given an image representing printed text, determine the corresponding text.
      
'''[[光学字符识别 Optical Character Recognition,OCR)]]''' : 给定一幅印有文字的图像,识别相应的文本。
 
'''[[光学字符识别 Optical Character Recognition,OCR)]]''' : 给定一幅印有文字的图像,识别相应的文本。
   −
; [[Question answering]]: Given a human-language question, determine its answer.  Typical questions have a specific right answer (such as "What is the capital of Canada?"), but sometimes open-ended questions are also considered (such as "What is the meaning of life?"). Recent works have looked at even more complex questions.<ref>{{cite journal |title=Versatile question answering systems: seeing in synthesis |last=Mittal |journal= International Journal of Intelligent Information and Database Systems|volume=5 |issue=2 |pages=119–142 |year=2011 |doi=10.1504/IJIIDS.2011.038968 |url=https://hal.archives-ouvertes.fr/hal-01104648/file/Mittal_VersatileQA_IJIIDS.pdf }}</ref>
+
问答: 给出一个用人类语言表述的问题,确定它的答案。典型的问题都有一个明确的正确答案(例如“加拿大的首都是哪里? ”),但有时候也需要考虑开放式的问题(比如“生命的意义是什么? ”)。最近一些工作在研究更复杂的问题.<ref>{{cite journal |title=Versatile question answering systems: seeing in synthesis |last=Mittal |journal= International Journal of Intelligent Information and Database Systems|volume=5 |issue=2 |pages=119–142 |year=2011 |doi=10.1504/IJIIDS.2011.038968 |url=https://hal.archives-ouvertes.fr/hal-01104648/file/Mittal_VersatileQA_IJIIDS.pdf }}</ref>
 
  −
Question answering: Given a human-language question, determine its answer.  Typical questions have a specific right answer (such as "What is the capital of Canada?"), but sometimes open-ended questions are also considered (such as "What is the meaning of life?"). Recent works have looked at even more complex questions.
  −
 
  −
问答: 给出一个用人类语言表述的问题,确定它的答案。典型的问题都有一个明确的正确答案(例如“加拿大的首都是哪里? ”),但有时候也需要考虑开放式的问题(比如“生命的意义是什么? ”)。最近一些工作在研究更复杂的问题。
  −
 
  −
; [[Textual entailment|Recognizing Textual entailment]]: Given two text fragments, determine if one being true entails the other, entails the other's negation, or allows the other to be either true or false.<ref name=rte:11>PASCAL Recognizing Textual Entailment Challenge (RTE-7) https://tac.nist.gov//2011/RTE/</ref>
  −
 
  −
Recognizing Textual entailment: Given two text fragments, determine if one being true entails the other, entails the other's negation, or allows the other to be either true or false.
  −
 
  −
'''[[文本蕴涵识别 Recognizing Textual Entailment]]''': 给定两个文本片段,确定其中一个是否蕴含了另一个,或者是否蕴含了另一个的否定,或者是否允许另一个文本中立。
     −
; [[Relationship extraction]]: Given a chunk of text, identify the relationships among named entities (e.g. who is married to whom).
+
'''[[文本蕴涵识别 Recognizing Textual Entailment]]''': 给定两个文本片段,确定其中一个是否蕴含了另一个,或者是否蕴含了另一个的否定,或者是否允许另一个文本中立<ref name=rte:11>PASCAL Recognizing Textual Entailment Challenge (RTE-7) https://tac.nist.gov//2011/RTE/</ref>。
   −
Relationship extraction: Given a chunk of text, identify the relationships among named entities (e.g. who is married to whom).
      
'''[[关系抽取 Relation Extraction]]''': 给定一个文本块,识别命名实体之间的关系(例如:谁嫁给了谁)。
 
'''[[关系抽取 Relation Extraction]]''': 给定一个文本块,识别命名实体之间的关系(例如:谁嫁给了谁)。
  −
; [[Sentiment analysis]] (see also [[multimodal sentiment analysis]]): Extract subjective information usually from a set of documents, often using online reviews to determine "polarity" about specific objects. It is especially useful for identifying trends of public opinion in social media, for marketing.
  −
  −
Sentiment analysis (see also multimodal sentiment analysis): Extract subjective information usually from a set of documents, often using online reviews to determine "polarity" about specific objects. It is especially useful for identifying trends of public opinion in social media, for marketing.
      
'''[[情感分析 Sentiment Analysis]]'''(参见'''[[多模态情感分析 Multimodal Sentiment Analysis]]'''): 从一组文档中提取主观信息,通常使用在线评论来确定特定对象的“极性”。情感分析在识别社会媒体中的舆论趋势和市场营销中尤其有效。
 
'''[[情感分析 Sentiment Analysis]]'''(参见'''[[多模态情感分析 Multimodal Sentiment Analysis]]'''): 从一组文档中提取主观信息,通常使用在线评论来确定特定对象的“极性”。情感分析在识别社会媒体中的舆论趋势和市场营销中尤其有效。
   −
; [[Topic segmentation]] and recognition: Given a chunk of text, separate it into segments each of which is devoted to a topic, and identify the topic of the segment.
+
'''[[话题分割和识别]]''': 给定一个文本块,将其分成几个部分,每个部分都有一个主题,并确定各个部分的主题。
 
  −
Topic segmentation and recognition: Given a chunk of text, separate it into segments each of which is devoted to a topic, and identify the topic of the segment.
  −
 
  −
话题分割和识别: 给定一个文本块,将其分成几个部分,每个部分都有一个主题,并确定各个部分的主题。
  −
 
  −
; [[Word sense disambiguation]]: Many words have more than one [[Meaning (linguistics)|meaning]]; we have to select the meaning which makes the most sense in context.  For this problem, we are typically given a list of words and associated word senses, e.g. from a dictionary or an online resource such as [[WordNet]].
  −
 
  −
Word sense disambiguation: Many words have more than one meaning; we have to select the meaning which makes the most sense in context.  For this problem, we are typically given a list of words and associated word senses, e.g. from a dictionary or an online resource such as WordNet.
      
'''[[词义消歧 Word Sense Disambiguation]]''': 从词语的多个意思中选出最符合上下文的一个意思。为了解决这个问题,我们通常会从字典或如WordNet的在线资源中取一系列的单词和相关的词义。
 
'''[[词义消歧 Word Sense Disambiguation]]''': 从词语的多个意思中选出最符合上下文的一个意思。为了解决这个问题,我们通常会从字典或如WordNet的在线资源中取一系列的单词和相关的词义。
         
===话语(Discourse)===
 
===话语(Discourse)===
  −
; [[Automatic summarization]]:Produce a readable summary of a chunk of text.  Often used to provide summaries of the text of a known type, such as research papers, articles in the financial section of a newspaper.
  −
  −
Automatic summarization:Produce a readable summary of a chunk of text.  Often used to provide summaries of the text of a known type, such as research papers, articles in the financial section of a newspaper.
      
'''[[自动摘要 Automatic Summarization]]''':自动生成一个可读的文本摘要。常用于提供已知类型如研究论文、报纸财经版的文章等文本的摘要。
 
'''[[自动摘要 Automatic Summarization]]''':自动生成一个可读的文本摘要。常用于提供已知类型如研究论文、报纸财经版的文章等文本的摘要。
  −
; [[Coreference|Coreference resolution]]: Given a sentence or larger chunk of text, determine which words ("mentions") refer to the same objects ("entities"). [[Anaphora resolution]] is a specific example of this task, and is specifically concerned with matching up [[pronoun]]s with the nouns or names to which they refer. The more general task of coreference resolution also includes identifying so-called "bridging relationships" involving [[referring expression]]s. For example, in a sentence such as "He entered John's house through the front door", "the front door" is a referring expression and the bridging relationship to be identified is the fact that the door being referred to is the front door of John's house (rather than of some other structure that might also be referred to).
  −
  −
Coreference resolution: Given a sentence or larger chunk of text, determine which words ("mentions") refer to the same objects ("entities"). Anaphora resolution is a specific example of this task, and is specifically concerned with matching up pronouns with the nouns or names to which they refer. The more general task of coreference resolution also includes identifying so-called "bridging relationships" involving referring expressions. For example, in a sentence such as "He entered John's house through the front door", "the front door" is a referring expression and the bridging relationship to be identified is the fact that the door being referred to is the front door of John's house (rather than of some other structure that might also be referred to).
      
'''[[共指消解 Coreference Resolution]]''': 给定一个句子或更大的文本块,确定哪些单词(“指称”)指的是相同的对象(“实体”)。指代消解就是这项任务的一个具体实例,它专门研究代词与所指名词或名称的匹配问题。共指消解的一般任务还包括识别指称之间的“桥接关系”。例如,在“他从前门进入了约翰的房子”这句话中,“前门”是一种指称,需要确定的桥接关系是:所指的门是约翰的房子的前门(而不是其他一些也可以指称的结构)。
 
'''[[共指消解 Coreference Resolution]]''': 给定一个句子或更大的文本块,确定哪些单词(“指称”)指的是相同的对象(“实体”)。指代消解就是这项任务的一个具体实例,它专门研究代词与所指名词或名称的匹配问题。共指消解的一般任务还包括识别指称之间的“桥接关系”。例如,在“他从前门进入了约翰的房子”这句话中,“前门”是一种指称,需要确定的桥接关系是:所指的门是约翰的房子的前门(而不是其他一些也可以指称的结构)。
  −
; [[Discourse analysis]]: This rubric includes several related tasks.  One task is identifying the [[discourse]] structure of a connected text, i.e. the nature of the discourse relationships between sentences (e.g. elaboration, explanation, contrast).  Another possible task is recognizing and classifying the [[speech act]]s in a chunk of text (e.g. yes-no question, content question, statement, assertion, etc.).
  −
  −
Discourse analysis: This rubric includes several related tasks.  One task is identifying the discourse structure of a connected text, i.e. the nature of the discourse relationships between sentences (e.g. elaboration, explanation, contrast).  Another possible task is recognizing and classifying the speech acts in a chunk of text (e.g. yes-no question, content question, statement, assertion, etc.).
      
'''[[话语分析 Discourse Analysis]]''':这个部分包括几个相关任务。一个是识别相连文本的语篇结构,即句子之间的话语关系(例如:详述、解释、对比)。还有识别和分类文本块中的言语行为(例如:是-否问题,内容问题,陈述,断言等)
 
'''[[话语分析 Discourse Analysis]]''':这个部分包括几个相关任务。一个是识别相连文本的语篇结构,即句子之间的话语关系(例如:详述、解释、对比)。还有识别和分类文本块中的言语行为(例如:是-否问题,内容问题,陈述,断言等)
      
===语音(Speech)===
 
===语音(Speech)===
  −
; [[Speech recognition]]: Given a sound clip of a person or people speaking, determine the textual representation of the speech.  This is the opposite of [[text to speech]] and is one of the extremely difficult problems colloquially termed "[[AI-complete]]" (see above).  In [[natural speech]] there are hardly any pauses between successive words, and thus [[speech segmentation]] is a necessary subtask of speech recognition (see below). In most spoken languages, the sounds representing successive letters blend into each other in a process termed [[coarticulation]], so the conversion of the [[analog signal]] to discrete characters can be a very difficult process. Also, given that words in the same language are spoken by people with different accents, the speech recognition software must be able to recognize the wide variety of input as being identical to each other in terms of its textual equivalent.
  −
  −
Speech recognition: Given a sound clip of a person or people speaking, determine the textual representation of the speech.  This is the opposite of text to speech and is one of the extremely difficult problems colloquially termed "AI-complete" (see above).  In natural speech there are hardly any pauses between successive words, and thus speech segmentation is a necessary subtask of speech recognition (see below). In most spoken languages, the sounds representing successive letters blend into each other in a process termed coarticulation, so the conversion of the analog signal to discrete characters can be a very difficult process. Also, given that words in the same language are spoken by people with different accents, the speech recognition software must be able to recognize the wide variety of input as being identical to each other in terms of its textual equivalent.
      
'''[[语音识别 Speech Recognition]]''': 给定一个或多个人说话的声音片段,确定语音的文本内容。这是文本转语音的反过程,是一个极其困难被称为“人工智能完备”(见上文)的问题。自然语音中连续的单词之间几乎没有停顿,因此语音分割是语音识别的一个必要的子任务(见下文)。在大多数口语中,连续字母的声音在“协同发音”中相互融合,因此将模拟信号转换为离散字符会是一个非常困难的过程。此外,由于说同一个词时不同人的口音不同,所以语音识别软件必须能够识别文本相同的不同输入。
 
'''[[语音识别 Speech Recognition]]''': 给定一个或多个人说话的声音片段,确定语音的文本内容。这是文本转语音的反过程,是一个极其困难被称为“人工智能完备”(见上文)的问题。自然语音中连续的单词之间几乎没有停顿,因此语音分割是语音识别的一个必要的子任务(见下文)。在大多数口语中,连续字母的声音在“协同发音”中相互融合,因此将模拟信号转换为离散字符会是一个非常困难的过程。此外,由于说同一个词时不同人的口音不同,所以语音识别软件必须能够识别文本相同的不同输入。
  −
; [[Speech segmentation]]: Given a sound clip of a person or people speaking, separate it into words.  A subtask of [[speech recognition]] and typically grouped with it.
  −
  −
Speech segmentation: Given a sound clip of a person or people speaking, separate it into words.  A subtask of speech recognition and typically grouped with it.
      
'''[[语音分割 Speech Segmentation]]''': 给一个人或人说话的声音片段,将其分成单词。这是语音识别的一个子任务,通常两者一起出现。
 
'''[[语音分割 Speech Segmentation]]''': 给一个人或人说话的声音片段,将其分成单词。这是语音识别的一个子任务,通常两者一起出现。
   −
; [[Text-to-speech]]:Given a text, transform those units and produce a spoken representation. Text-to-speech can be used to aid the visually impaired.<ref>{{Citation|last=Yi|first=Chucai|title=Assistive Text Reading from Complex Background for Blind Persons|date=2012|work=Camera-Based Document Analysis and Recognition|pages=15–28|publisher=Springer Berlin Heidelberg|language=en|doi=10.1007/978-3-642-29364-1_2|isbn=9783642293634|last2=Tian|first2=Yingli|citeseerx=10.1.1.668.869}}</ref>
+
'''[[语音合成 Text-to-speech ]]''': 给定一个文本,把这些文字转换为口语表达。语音合成可以用来帮助视力受损的人<ref>{{Citation|last=Yi|first=Chucai|title=Assistive Text Reading from Complex Background for Blind Persons|date=2012|work=Camera-Based Document Analysis and Recognition|pages=15–28|publisher=Springer Berlin Heidelberg|language=en|doi=10.1007/978-3-642-29364-1_2|isbn=9783642293634|last2=Tian|first2=Yingli|citeseerx=10.1.1.668.869}}</ref>
 
  −
Text-to-speech:Given a text, transform those units and produce a spoken representation. Text-to-speech can be used to aid the visually impaired.
  −
 
  −
'''[[语音合成 Text-to-speech ]]''': 给定一个文本,把这些文字转换为口语表达。语音合成可以用来帮助视力受损的人。
  −
 
  −
 
      
===对话(Dialogue)===
 
===对话(Dialogue)===
  −
The first published work by an artificial intelligence was published in 2018, ''[[1 the Road]]'', marketed as a novel, contains sixty million words.
  −
  −
The first published work by an artificial intelligence was published in 2018, 1 the Road, marketed as a novel, contains sixty million words.
      
第一部由人工智能创作的作品于2018年出版,名为《路》(1 the Road) ,以小说的形式发售,包含6000万字。
 
第一部由人工智能创作的作品于2018年出版,名为《路》(1 the Road) ,以小说的形式发售,包含6000万字。
421

个编辑

导航菜单