机器学习 Machine Learning

机器学习 Machine Learning，ML是计算机科学的分支——人工智能的一个子集，它通常使用统计学方法，借助数据，赋予计算机“学习”的能力（例如，逐渐提高在特定任务上的表现）而不需要明确编写学习过程^[1]。

机器学习的名字是Arthur Samuel ^[2] 在1959年创建的。它由人工智能 ^[3] 中的模式识别与计算学习理论研究发展而来，探究从数据 ^[4] 中学习、做出预测的算法的研究与结构——这些算法通过从样本输入中建立模型，做出数据驱动的预测或决定 ^[5]，避免了一系列严格的静态程序指令。机器学习被应用于那些设计运转良好、清晰明确的算法十分困难或根本不可行的计算任务中；其应用范围包括电子邮件过滤、网络入侵者与针对数据泄露的恶意内部人员的探查、光学字符识别、学习排序、计算机视觉。

机器学习与计算统计学密切相关(而且常常与之重叠)，计算统计学也专注于通过使用计算机进行预测。它与数学优化有着很强的联系，后者为之提供了方法、理论和应用领域。机器学习有时与数据挖掘 ^[6] 混为一谈，后者更侧重于探索性数据分析，被称为无监督学习 ^[5] ^[7]。机器学习也可以是无监督的，并可用于为各种实体，学习和建立基线行为轮廓，然后用于寻找有意义的异常。

在数据分析领域，机器学习是设计复杂模型和算法的一种方法。这有助于预测；在商业用途中，这被称为预测分析。这些分析模型使研究人员、数据科学家、工程师和分析人员能够“产生可靠的、可重复的决策和结果”，并通过从数据中的历史关系和趋势中学习，来发现“隐藏的洞见”。 ^[8]

概述

Tom M.Mitchell对机器学习领域中研究的算法提供了一个被广泛引用的、更正式的定义：“计算机程序被描述为从经验E中学习某些类型的任务T和性能度量P，如果它在T中的性能(用P来衡量)随着经验E而提高。” ^[9] 这一将机器学习与任务进行关联的定义提供了一个基本的可执行定义，而不是认知术语形式的定义。这与艾伦·图灵在他的论文“计算机器和智能”中的提议一脉相承，其中“机器能思考吗？”被替换为“机器能做我们(作为思维实体)能做的事情吗？” ^[10] 。图灵的建议揭露了思维机器可能具有的各种特性以及构建机器时的诸多暗含之意。

机器学习的任务

机器学习任务通常分为两大类，取决于学习系统是否存在学习“信号”或“反馈”:

监督学习：向计算机展示由“教师”提供的示例输入及其期望的输出，目标是学习将输入映射到输出的一般规则。作为特例，输入信号只能被部分提供，或仅限于特定反馈：
- 半监督学习：只提供给计算机一个不完整的训练信号：一个训练集，其中有一些（通常很多）的目标输出丢失。
- 主动学习：计算机只能获得有限的实例集(基于预算)的训练标签，还必须优化对象的选择以获取标签。当交互使用时，可以向用户展示这些对象以供标签。
- 强化学习：训练数据(以奖励和惩罚的形式)只作为对程序在动态环境中的行为的反馈，例如自动驾驶或与对手玩游戏^[5]。
无监督学习：没有标签给学习算法，留下它自己在其输入中找到结构。无监督学习本身可以是一个目标（发现数据中隐藏的模式），或者是一种达到目的的手段（特征学习）。

机器学习的应用

当考虑机器学习系统的期望输出时，机器学习任务的另一种分类出现了。 ^[5]

在分类中，输入被分成两个或多个类，学习者必须生成一个模型，将未被分类的输入分配给这些类的一个或多个(多标签分类)。这通常是以监督的方式处理的。垃圾邮件过滤是分类的一个例子，其中输入是电子邮件(或其他)消息，类是“垃圾邮件”和“非垃圾邮件”。
在回归中，也是一个有监督学习问题，输出是连续的而不是离散的。
在聚类中，一组输入被分为不同组。与分类不同的是，这些组事先是未知的，因此这通常是无监督学习的任务。
密度估计挖掘输入在某一空间的分布。
维数约简通过将输入映射到低维空间来简化输入。主题建模是一个相关的问题，在这个问题中，程序会得到一个人类语言文档的列表，并负责找出哪些文档涵盖了类似的主题。

在其他类型的机器学习问题中，元学习在以往经验的基础上学会了自己的归纳偏好。发育学习是为机器人学习而发展起来的，它通过自主的自我探索和与人类教师的社会互动，利用主动学习、成熟、运动协同和模仿等引导机制，产生自己的学习情境序列(也称为课程)，累积获得一系列新技能。

历史、和其它领域的关系

参见：机器学习时间线

美国计算机游戏和人工智能领域的先驱Arthur Samuel于1959年在IBM时发明了“机器学习”一词。作为一项科学探索，机器学习源于对人工智能的探索 ^[11]. 。早在人工智能作为一门学术学科的时期，一些研究人员就对让机器从数据中学习产生了兴趣。他们试图用各种符号方法以及后来被称为“神经网络”的方法来解决这个问题；这些模型大多是感知器和其他模型，后来被发现是广义线性统计模型的再发明 ^[12]。统计学推理也被使用，特别是在自动医疗诊断中 ^[13]。

然而，对逻辑的、基于知识的方法的日益重视，导致人工智能和机器学习之间产生了裂痕，概率系统受到数据采集和数据表示的理论和实践问题的困扰 ^[13]。到1980年，专家系统已经主导了人工智能，而统计方法不再受欢迎 ^[14]。基于符号与知识的学习的工作仍然属于AI领域，这促成了归纳逻辑编程，但更多的在模式识别和信息检索 ^[13] 方面的统计方法的研究已经超出了人工智能本身的范围。神经网络的研究几乎同时被人工智能和计算机科学所抛弃。而在AI/CS领域之外，这条路线也被其他学科的研究人员奉为“连接主义”而继续存在，包括Hopfield、Rumelhart和Hinton。他们的主要成功是在上世纪80年代中期重新发明了反向传播算法 ^[13]。

机器学习,作为一个独立的领域，在20世纪90年代开始蓬勃发展。机器学习的目标从实现人工智能转变为解决可解决的实践性问题。它将重点从AI中继承的符号方法转向了来自于统计学和概率论的方法和模型^[14] ，同时也受益于数字化信息日益增长的普及性，以及互联网传播信息的能力。

机器学习和数据挖掘通常采用相同的方法，重叠程度很大。但是机器学习侧重于预测——基于从训练数据中学习到的已知属性，而数据挖掘则侧重于在数据中发现(以前的)未知属性(这是数据库中知识发现的分析步骤)。数据挖掘采用多种机器学习方法而目标不同；另一方面，机器学习也采用数据挖掘的方法作为“无监督学习”或作为预处理步骤来提高学习精度。这两个研究对象之间的许多混淆(它们通常有单独的会议和单独的期刊，ECML PKDD是一个主要的例外)来自于它们所使用的基本假设：在机器学习中，性能通常是根据重复生产已知知识的能力来评估的，而在知识发现和数据挖掘(KDD)中，关键任务是发现以前未知的知识。在对学习已知知识进行评估时，其他监督学习方法很容易优于无监督学习方法，而在典型的KDD任务中，由于训练数据的不可得性，则无法使用监督学习的方法。

机器学习与优化也有着密切的联系：许多学习问题被描述为训练集上的一些损失函数的最小化。损失函数表示被训练模型的预测与实际问题之间的差异(例如，在分类中，要为实例分配一个标签，且模型要被训练以正确预测一组预先分配过标签的示例)。这两个领域的区别来自于一般化的目的：优化算法可以最小化训练集上的损失，而机器学习则涉及到最小化未知样本的损失 ^[15]。

与统计学的关系

机器学习和统计是密切相关的领域。据Michael I. Jordan说，机器学习的思想，从方法论原理到理论工具，都在统计学上有着悠久的历史 ^[16]。他还建议用数据科学这个术语来称呼整个领域 ^[16]。

Leo Breiman区分了两种统计建模范式：数据模型和算法模型 ^[17] ，其中“算法模型”或多或少是指像随机森林那样的机器学习算法。

一些统计学家采用机器学习的方法，形成了一个他们称之为统计学习的交叉领域^[18]。

与人工智能 Artificial intelligence的关系Relation to artificial intelligence

As a scientific endeavor, machine learning grew out of the quest for artificial intelligence. In the early days of AI as an academic discipline, some researchers were interested in having machines learn from data. They attempted to approach the problem with various symbolic methods, as well as what were then termed "neural networks"; these were mostly perceptrons and other models that were later found to be reinventions of the generalized linear models of statistics.^[19] Probabilistic reasoning was also employed, especially in automated medical diagnosis.^[13]^:488

As a scientific endeavor, machine learning grew out of the quest for artificial intelligence. In the early days of AI as an academic discipline, some researchers were interested in having machines learn from data. They attempted to approach the problem with various symbolic methods, as well as what were then termed "neural networks"; these were mostly perceptrons and other models that were later found to be reinventions of the generalized linear models of statistics. Probabilistic reasoning was also employed, especially in automated medical diagnosis.

作为一项科研成果，机器学习源于对人工智能的探索。在人工智能这一学科研究的早期，一些研究人员对于让机器从数据中进行学习这一问题很感兴趣。他们试图用各种符号方法甚至是当时被称为”神经网络 Neural Networks”的方法来处理这个问题;但这些方法大部分是感知器或其他模型。后来这些模型随着统计学中广义线性模型的发展而重新出现在大众视野中，与此同时概率推理的方法也被广泛使用，特别是在自动医疗诊断问题上。

However, an increasing emphasis on the logical, knowledge-based approach caused a rift between AI and machine learning. Probabilistic systems were plagued by theoretical and practical problems of data acquisition and representation.^[13]^:488 By 1980, expert systems had come to dominate AI, and statistics was out of favor.^[14] Work on symbolic/knowledge-based learning did continue within AI, leading to inductive logic programming, but the more statistical line of research was now outside the field of AI proper, in pattern recognition and information retrieval.^[13]^{:708–710; 755} Neural networks research had been abandoned by AI and computer science around the same time. This line, too, was continued outside the AI/CS field, as "connectionism", by researchers from other disciplines including Hopfield, Rumelhart and Hinton. Their main success came in the mid-1980s with the reinvention of backpropagation.^[13]^:25

However, an increasing emphasis on the logical, knowledge-based approach caused a rift between AI and machine learning. Probabilistic systems were plagued by theoretical and practical problems of data acquisition and representation. Work on symbolic/knowledge-based learning did continue within AI, leading to inductive logic programming, but the more statistical line of research was now outside the field of AI proper, in pattern recognition and information retrieval. Neural networks research had been abandoned by AI and computer science around the same time. This line, too, was continued outside the AI/CS field, as "connectionism", by researchers from other disciplines including Hopfield, Rumelhart and Hinton. Their main success came in the mid-1980s with the reinvention of backpropagation.

然而，日益强调的基于知识的逻辑方法 Knowledge-based Approach导致了人工智能和机器学习之间的裂痕。概率系统一直被数据获取和表示的理论和实际问题所困扰。在人工智能内部，符号/知识学习的工作确实在继续，导致了归纳逻辑编程，但更多的统计研究现在已经超出了人工智能本身的领域，即模式识别和信息检索。神经网络的研究几乎在同一时间被人工智能和计算机科学领域所抛弃，但这种思路却在人工智能/计算机之外的领域被延续了下来，被其他学科的研究人员称为“连接主义”。（编者注：连接主义又称为仿生学派或生理学派，其主要原理为神经网络及神经网络间的连接机制与学习算法。）包括霍普菲尔德 Hopfield、鲁梅尔哈特 Rumelhart和辛顿 Hinton。他们的主要成就集中在20世纪80年代中期，在这一阶段神经网络的方法随着反向传播算法的出现而重新被世人所重视。

Machine learning, reorganized as a separate field, started to flourish in the 1990s. The field changed its goal from achieving artificial intelligence to tackling solvable problems of a practical nature. It shifted focus away from the symbolic approaches it had inherited from AI, and toward methods and models borrowed from statistics and probability theory. As of 2019, many sources continue to assert that machine learning remains a sub field of AI. Yet some practitioners, for example Dr Daniel Hulme, who both teaches AI and runs a company operating in the field, argues that machine learning and AI are separate.

机器学习，在重组为一个独立的领域之后，与20世纪90年代开始蓬勃发展。该领域的目标从实现人工智能转变为解决实际中的可解决问题。它将焦点从它从人工智能继承的符号方法转移到从统计学和概率论中借鉴的方法和模型。截止至2019年，许多资料都继续断言机器学习仍然是人工智能的一个子领域。然而，一些该领域的从业者（例如丹尼尔 · 休姆 Daniel Hulme博士，他既教授人工智能，又经营着一家在该领域运营的公司），则认为机器学习和人工智能是分开的。

Machine learning, reorganized as a separate field, started to flourish in the 1990s. The field changed its goal from achieving artificial intelligence to tackling solvable problems of a practical nature. It shifted focus away from the symbolic approaches it had inherited from AI, and toward methods and models borrowed from statistics and probability theory.^[14] As of 2019, many sources continue to assert that machine learning remains a sub field of AI. Yet some practitioners, for example Dr Daniel Hulme, who both teaches AI and runs a company operating in the field, argues that machine learning and AI are separate. ^[20]引用错误：无效<ref>标签；未填name属性的引用必须填写内容^[21]^[22]

与数据挖掘 Data mining的关系Relation to data mining

Machine learning and data mining often employ the same methods and overlap significantly, but while machine learning focuses on prediction, based on known properties learned from the training data, data mining focuses on the discovery of (previously) unknown properties in the data (this is the analysis step of knowledge discovery in databases). Data mining uses many machine learning methods, but with different goals; on the other hand, machine learning also employs data mining methods as "unsupervised learning" or as a preprocessing step to improve learner accuracy. Much of the confusion between these two research communities (which do often have separate conferences and separate journals, ECML PKDD being a major exception) comes from the basic assumptions they work with: in machine learning, performance is usually evaluated with respect to the ability to reproduce known knowledge, while in knowledge discovery and data mining (KDD) the key task is the discovery of previously unknown knowledge. Evaluated with respect to known knowledge, an uninformed (unsupervised) method will easily be outperformed by other supervised methods, while in a typical KDD task, supervised methods cannot be used due to the unavailability of training data.

Machine learning and data mining often employ the same methods and overlap significantly, but while machine learning focuses on prediction, based on known properties learned from the training data, data mining focuses on the discovery of (previously) unknown properties in the data (this is the analysis step of knowledge discovery in databases). Data mining uses many machine learning methods, but with different goals; on the other hand, machine learning also employs data mining methods as "unsupervised learning" or as a preprocessing step to improve learner accuracy. Much of the confusion between these two research communities (which do often have separate conferences and separate journals, ECML PKDD being a major exception) comes from the basic assumptions they work with: in machine learning, performance is usually evaluated with respect to the ability to reproduce known knowledge, while in knowledge discovery and data mining (KDD) the key task is the discovery of previously unknown knowledge. Evaluated with respect to known knowledge, an uninformed (unsupervised) method will easily be outperformed by other supervised methods, while in a typical KDD task, supervised methods cannot be used due to the unavailability of training data.

机器学习和数据挖掘虽然在使用方法上有些相似并且有很大的重叠，但是机器学习的重点是预测，基于从训练数据中学到的已知属性，而数据挖掘的重点则是发现数据中（以前）未知的属性（这是数据库中知识发现 Knowledge Discovery in Database, KDD的基本分析步骤），也就是说数据挖掘虽然使用了许多机器学习方法，但二者的目标不同; 另一方面，机器学习也使用数据挖掘方法作为“无监督学习”或作为提高学习者准确性的预处理步骤。这两个研究领域之间的混淆（这两个领域通常有各自单独的会议和单独的期刊，ECML PKDD是一个例外）来自他们工作的基本假设: 在机器学习中，算法性能通常是根据再现已知知识的能力来评估，而在知识发现和数据挖掘中，其关键任务是发现以前未知的知识，因此在对已知知识进行评价时，其他监督方法很容易超过未知（无监督）方法，而在典型的知识发现任务中，由于缺乏训练数据，无法使用有监督的学习算法。

与优化的关系 Relation to optimization

Machine learning also has intimate ties to optimization: many learning problems are formulated as minimization of some loss function on a training set of examples. Loss functions express the discrepancy between the predictions of the model being trained and the actual problem instances (for example, in classification, one wants to assign a label to instances, and models are trained to correctly predict the pre-assigned labels of a set of examples). The difference between the two fields arises from the goal of generalization: while optimization algorithms can minimize the loss on a training set, machine learning is concerned with minimizing the loss on unseen samples.^[23]

Machine learning also has intimate ties to optimization: many learning problems are formulated as minimization of some loss function on a training set of examples. Loss functions express the discrepancy between the predictions of the model being trained and the actual problem instances (for example, in classification, one wants to assign a label to instances, and models are trained to correctly predict the pre-assigned labels of a set of examples). The difference between the two fields arises from the goal of generalization: while optimization algorithms can minimize the loss on a training set, machine learning is concerned with minimizing the loss on unseen samples.

机器学习与优化也有着密切的联系: 许多学习问题被表述为最小化训练样本集上的某些损失函数 Loss Function。损失函数表示正在训练的模型预测结果与实际数据之间的差异（例如，在分类问题中，人们的目标是给一个未知的实例分配其对应标签，而模型经过训练学习到的是如何正确地为一组实例标记事先已知的标签）。这两个领域之间的差异源于泛化的目标: 优化算法可以最小化训练集上的损失，而机器学习关注于最小化未知样本上的损失。

理论

主文章：计算学习理论

学习者的一个核心目标是从经验中举一反三 ^[5] ^[24]。这里的举一反三是指学习机器在经历了学习数据集之后，能够准确地对新的、未见过的示例/任务进行精确的操作。训练实例通常来自一些未知的概率分布（常被认为是样本空间的代表），并且学习者必须建立一个关于这个空间的通用模型，使得它能够在新的情况下产生足够精确的预测。

机器学习算法及其性能的计算分析是理论计算机科学的一个分支，称为计算学习理论。由于训练集是有限的但是未来是不确定的，所以学习理论通常不能保证算法的性能。然而，算法性能的概率的界限是相当普适的。偏差-方差分解是量化泛化误差的一种方法。

为了在泛化的情境中获得最佳性能，假设的复杂性应该与数据基础上的函数的复杂性相匹配。如果假设没有函数复杂，那么模型就不适合数据。如果模型的复杂度因此增加，则训练误差减小。但如果假设过于复杂，则模型会被过度拟合，泛化也将变得更差^[25]。

除了性能界限外，计算学习理论家还研究了学习的时间复杂性和可行性。在计算学习理论中，如果可以在多项式时间内进行计算，则认为计算是可行的。有两种时间复杂度的结果。正结果表明，一类函数可以在多项式时间内学习。负结果表明，某些类函数不能在多项式时间内学习。

方法

主文章：机器学习算法列表

强化学习

主文章：强化学习

强化学习是指一个智能体（agent）应该如何在环境中采取行动，从而最大限度地获得长期报酬的概念。强化学习算法试图找到一种策略，将世界状态映射到智能体在这些状态中应该采取的行动。强化学习不同于监督学习问题，因为不会提供正确的输入/输出对，也没有明确地修正次优行为。

相似性与度量学习

主文章：相似性学习

在这个问题中，学习机器给出了一对被认为相似的对象和一对不太相似的对象。然后，它需要学习一个相似函数(或距离度量函数)，该函数可以预测新对象是否相似。该算法有时用于推荐系统。

稀疏字典学习

主文章：稀疏字典学习

该方法将数据表示为基函数的线性组合，并假定系数是稀疏的。设x是d维数据，D是d乘n矩阵，其中D的每一列代表一个基函数，r是用D表示x的系数。数学上，稀疏字典学习意味着求解 [math]\displaystyle{ x\approx Dr ,r }[/math]是稀疏的。一般说来，假设n大于d，以便稀疏表示。

学习字典和稀疏表示是强NP难解的，也很难近似求解^[26] 。稀疏字典学习的一种流行的启发式方法是K-SVD。

稀疏字典学习已经在几种环境中得到了应用。在分类中，问题是确定以前看不见的数据属于哪些类。假设已经为每个类构建了一个字典。然后，将一个新的数据与类相关联，使得它被相应的字典最优表示。稀疏字典学习也被应用于图像去噪。关键的思想是一个干净的图像补丁可以用图像字典来稀疏地表示，但是噪声却不能^[27]。

遗传算法

主文章：遗传算法

遗传算法(GeneticAlgorithm，GA)是一种模拟自然选择过程的搜索启发式算法，它利用变异和重组等方法生成新的基因型，以期为给定问题找到好的解决方案。在机器学习中，遗传算法在20世纪80年代和90年代得到了一些应用 ^[28]^[29]。相反，机器学习技术被用来改进遗传算法和进化算法的性能 ^[30]。

学习算法的分类 Types of learning algorithms

The types of machine learning algorithms differ in their approach, the type of data they input and output, and the type of task or problem that they are intended to solve.

不同类型的机器学习算法的方法、输入和输出的数据类型以及它们要解决的任务或问题的类型都有所不同。

监督学习 Supervised learning

A support vector machine is a supervised learning model that divides the data into regions separated by a linear boundary. Here, the linear boundary divides the black circles from the white.

支持向量机是一个有监督学习模型，它将数据划分为由线性边界分隔的区域。在这里，有一个线性边界可以将黑色圆圈和白色圆圈分开。]

Supervised learning algorithms build a mathematical model of a set of data that contains both the inputs and the desired outputs.^[31] The data is known as training data, and consists of a set of training examples. Each training example has one or more inputs and the desired output, also known as a supervisory signal. In the mathematical model, each training example is represented by an array or vector, sometimes called a feature vector, and the training data is represented by a matrix. Through iterative optimization of an objective function, supervised learning algorithms learn a function that can be used to predict the output associated with new inputs.^[32] An optimal function will allow the algorithm to correctly determine the output for inputs that were not a part of the training data. An algorithm that improves the accuracy of its outputs or predictions over time is said to have learned to perform that task.^[33]

Supervised learning algorithms build a mathematical model of a set of data that contains both the inputs and the desired outputs. The data is known as training data, and consists of a set of training examples. Each training example has one or more inputs and the desired output, also known as a supervisory signal. In the mathematical model, each training example is represented by an array or vector, sometimes called a feature vector, and the training data is represented by a matrix. Through iterative optimization of an objective function, supervised learning algorithms learn a function that can be used to predict the output associated with new inputs. An optimal function will allow the algorithm to correctly determine the output for inputs that were not a part of the training data. An algorithm that improves the accuracy of its outputs or predictions over time is said to have learned to perform that task.

有监督学习算法会建立一个包含输入和期望输出的数据集上的的数学模型。这些数据被称为训练数据，由一组组训练样本组成。每个训练样本都有一个或多个输入和期望的输出，也称为监督信号。在数学模型中，每个训练样本由一个数组或向量表示，有时也称为特征向量 Feature Vector，训练数据由一个矩阵表示。通过对目标函数的迭代优化，监督式学习算法可以学习到一个用来预测与新输入相关的输出的函数。一个达到最优的目标函数可以实现算法对未知输入的输出结果有正确的预判，这种正确的预判并不仅限于训练数据上（即模型具有良好的泛化能力）。随着时间的推移，提高输出或预测精度的算法被称为已学会执行该任务。

Types of supervised learning algorithms include Active learning , classification and regression.^[34] Classification algorithms are used when the outputs are restricted to a limited set of values, and regression algorithms are used when the outputs may have any numerical value within a range. As an example, for a classification algorithm that filters emails, the input would be an incoming email, and the output would be the name of the folder in which to file the email.

Types of supervised learning algorithms include Active learning , classification and regression. Classification algorithms are used when the outputs are restricted to a limited set of values, and regression algorithms are used when the outputs may have any numerical value within a range. As an example, for a classification algorithm that filters emails, the input would be an incoming email, and the output would be the name of the folder in which to file the email.

监督式学习算法的类型包括主动学习 Active Learning、分类 Classification和回归 Regression。当输出被限制在一个有限的值集内时使用分类算法，当输出在一个范围内可能有任何数值时使用回归算法。例如，对于过滤电子邮件的分类算法，输入将是一封收到的电子邮件，输出将是用于将电子邮件归档的文件夹的名称。

Similarity learning is an area of supervised machine learning closely related to regression and classification, but the goal is to learn from examples using a similarity function that measures how similar or related two objects are. It has applications in ranking, recommendation systems, visual identity tracking, face verification, and speaker verification.

相似性学习 Similarity Learning是监督学习领域中与回归和分类密切相关的一个领域，但其目标是从实例中学习如何通过使用相似性函数来衡量两个对象之间的相似程度。它在排名、推荐系统、视觉身份跟踪、人脸验证和语者验证 Speaker Verification等方面都有应用。

无监督学习 Unsupervised learning

Unsupervised learning algorithms take a set of data that contains only inputs, and find structure in the data, like grouping or clustering of data points. The algorithms, therefore, learn from test data that has not been labeled, classified or categorized. Instead of responding to feedback, unsupervised learning algorithms identify commonalities in the data and react based on the presence or absence of such commonalities in each new piece of data. A central application of unsupervised learning is in the field of density estimation in statistics, such as finding the probability density function.^[35] Though unsupervised learning encompasses other domains involving summarizing and explaining data features.

Unsupervised learning algorithms take a set of data that contains only inputs, and find structure in the data, like grouping or clustering of data points. The algorithms, therefore, learn from test data that has not been labeled, classified or categorized. Instead of responding to feedback, unsupervised learning algorithms identify commonalities in the data and react based on the presence or absence of such commonalities in each new piece of data. A central application of unsupervised learning is in the field of density estimation in statistics, such as finding the probability density function. Though unsupervised learning encompasses other domains involving summarizing and explaining data features.

无监督学习 Unsupervised Learning算法只需要一组只包含输入的数据，通过寻找数据中潜在结构、规律，对数据点进行分组或聚类。因此，算法是从未被标记、分类或分类的测试数据中学习，而不是通过响应反馈来改进策略。无监督式学习算法可以识别数据中的共性，并根据每个新数据中是否存在这些共性而做出反应。无监督学习的一个核心应用是统计学中的密度估计领域，比如寻找概率密度函数。尽管非监督式学习也包含了其他领域，如总结和解释数据特性。

聚类

主文章：聚类分析

聚类分析是将一组观测数据分配到子集(称为聚类)中，使同一簇内的观测按照某些预先指定的准则相似，而从不同的簇中提取的观测值则不同。不同的聚类技术对数据的结构提出了不同的假设，通常用某种相似性度量来定义，并通过内部紧密性(同一聚类成员之间的相似性)和不同聚类之间的分离性来评估。其他方法基于估计的密度和图的连通性。聚类是一种无监督学习方法，是一种常用的统计数据分析技术。

半监督学习 Semi-supervised learning

Semi-supervised learning falls between unsupervised learning (without any labeled training data) and supervised learning (with completely labeled training data). Some of the training examples are missing training labels, yet many machine-learning researchers have found that unlabeled data, when used in conjunction with a small amount of labeled data, can produce a considerable improvement in learning accuracy.

半监督学习 Semi-supervised Learning介于无监督式学习（没有任何标记的训练数据）和有监督学习（完全标记的训练数据）之间。有些训练样本缺少训练标签，但许多机器学习研究人员发现，如果将未标记的数据与少量标记的数据结合使用，可以大大提高学习的准确性。

In weakly supervised learning, the training labels are noisy, limited, or imprecise; however, these labels are often cheaper to obtain, resulting in larger effective training sets.^[36]

In weakly supervised learning, the training labels are noisy, limited, or imprecise; however, these labels are often cheaper to obtain, resulting in larger effective training sets.

在弱监督学习 Weak Supervision中，训练标签是有噪声的、有限的或不精确的; 然而，这些标签使用起来往往更加“实惠”——这种数据更容易得到、更容易拥有更大的有效训练集。

强化学习 Reinforcement learning

Reinforcement learning is an area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward. Due to its generality, the field is studied in many other disciplines, such as game theory, control theory, operations research, information theory, simulation-based optimization, multi-agent systems, swarm intelligence, statistics and genetic algorithms. In machine learning, the environment is typically represented as a Markov Decision Process (MDP). Many reinforcement learning algorithms use dynamic programming techniques.^[37] Reinforcement learning algorithms do not assume knowledge of an exact mathematical model of the MDP, and are used when exact models are infeasible. Reinforcement learning algorithms are used in autonomous vehicles or in learning to play a game against a human opponent.

Reinforcement learning is an area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward. Due to its generality, the field is studied in many other disciplines, such as game theory, control theory, operations research, information theory, simulation-based optimization, multi-agent systems, swarm intelligence, statistics and genetic algorithms. In machine learning, the environment is typically represented as a Markov Decision Process (MDP). Many reinforcement learning algorithms use dynamic programming techniques. Reinforcement learning algorithms do not assume knowledge of an exact mathematical model of the MDP, and are used when exact models are infeasible. Reinforcement learning algorithms are used in autonomous vehicles or in learning to play a game against a human opponent.

强化学习是机器学习的一个领域，它研究软件组件应该如何在某个环境中进行行动决策，以便最大化某种累积收益的概念。由于其存在的普遍性，该领域的研究在许多其他学科，如博弈论 Game Theory，控制理论 Control Theory，运筹学 Operations Research，信息论 Information Theory，基于仿真的优化 Simulation-based Optimization，多主体系统 Multi-agent System，群体智能 Swarm Intelligence，统计学 Statistics和遗传算法 Genetic Algorithm。在机器学习中，环境通常被表示为马可夫决策过程 Markov Decision Process ，MDP。许多强化学习算法使用动态编程技术。强化学习算法不需要知道 MDP 的精确数学模型，而是在精确模型不可行的情况下使用。强化学习算法常用于车辆自动驾驶问题或人机游戏场景。

自学习 Self learning

Self-learning as machine learning paradigm was introduced in 1982 along with a neural network capable of self-learning named Crossbar Adaptive Array (CAA). ^[38] It is a learning with no external rewards and no external teacher advices. The CAA self-learning algorithm computes, in a crossbar fashion, both decisions about actions and emotions (feelings) about consequence situations. The system is driven by the interaction between cognition and emotion. ^[39]

Self-learning as machine learning paradigm was introduced in 1982 along with a neural network capable of self-learning named Crossbar Adaptive Array (CAA). It is a learning with no external rewards and no external teacher advices. The CAA self-learning algorithm computes, in a crossbar fashion, both decisions about actions and emotions (feelings) about consequence situations. The system is driven by the interaction between cognition and emotion.

自学习作为一种机器学习范式，于1982年提出，并提出了一种具有自学习能力的神经网络叫做交叉自适应矩阵 Crossbar Adaptive Array，CAA。这是一种没有外部激励和学习器建议的学习方法。CAA自学习算法以交叉方式计算关于行为的决策和关于后果情况的情绪（感觉）。这个系统是由认知和情感的相互作用所驱动的。

The self-learning algorithm updates a memory matrix W =||w(a,s)|| such that in each iteration executes the following machine learning routine:

 In situation s perform action a;

 In situation s perform action a;

在情境中执行动作 a;

 Receive consequence situation s’;

 Receive consequence situation s’;

接受结果状态 s’ ;

 Compute emotion of being in consequence situation v(s’);

 Compute emotion of being in consequence situation v(s’);

计算处于结果情境 v (s’)中的情绪;

 Update crossbar memory  w’(a,s) = w(a,s) + v(s’).

 Update crossbar memory  w’(a,s) = w(a,s) + v(s’).

更新交叉条记忆存储 w’(a，s) w (a，s) + v (s’)。

It is a system with only one input, situation s, and only one output, action (or behavior) a. There is neither a separate reinforcement input nor an advice input from the environment. The backpropagated value (secondary reinforcement) is the emotion toward the consequence situation. The CAA exists in two environments, one is behavioral environment where it behaves, and the other is genetic environment, wherefrom it initially and only once receives initial emotions about situations to be encountered in the behavioral environment. After receiving the genome (species) vector from the genetic environment, the CAA learns a goal seeking behavior, in an environment that contains both desirable and undesirable situations. ^[40]

It is a system with only one input, situation s, and only one output, action (or behavior) a. There is neither a separate reinforcement input nor an advice input from the environment. The backpropagated value (secondary reinforcement) is the emotion toward the consequence situation. The CAA exists in two environments, one is behavioral environment where it behaves, and the other is genetic environment, wherefrom it initially and only once receives initial emotions about situations to be encountered in the behavioral environment. After receiving the genome (species) vector from the genetic environment, the CAA learns a goal seeking behavior, in an environment that contains both desirable and undesirable situations.

它是一个只有一个输入、情景和一个输出、动作(或行为)的系统。既没有单独的强化输入，也没有来自环境的通知输入。反向传播价值（二次强化）是对结果情境的情感信息。CAA 存在于两种环境中，一种是行为环境，另一种是遗传环境，CAA将从这样的环境中获取且仅获取到一次关于它自身的初始情绪（这种情绪信息描述了算法应该对这样环境下对应的结果持有何种态度）。在从遗传环境中获得基因组(物种)载体后，CAA 会在一个既包含理想情况又包含不理想情况的环境中学习一种寻求目标的行为。

特征学习 Feature learning

主文章：表示学习

一些学习算法，大多是无监督学习算法，旨在发现更好的输入的训练数据的表示。经典的例子包括主成分分析和聚类分析。表示学习算法通常试图在输入中保留信息，并将其转换成有用的方式，通常是在执行分类或预测之前的预处理步骤，允许重构来自未知数据生成分布的输入，而不一定对不太可能服从该分布的结构可靠。

流形学习算法尝试处理被学习的数据表示是低维的情况。稀疏编码算法尝试处理被学习的数据表示是稀疏(有多个零)的情况。多线性子空间学习算法的目的是直接从多维数据的张量表示中学习低维表示，而不将它们重构成(高维)向量^[41]。深度学习算法能发现数据表示的多个层次，或者由低级特征定义(或生成)的更高、更抽象的特征层次。有人认为，智能机器是一种学习表示法的机器，它能找出那些解释观测数据的潜在变异因素^[42]。

Feature learning can be either supervised or unsupervised. In supervised feature learning, features are learned using labeled input data. Examples include artificial neural networks, multilayer perceptrons, and supervised dictionary learning. In unsupervised feature learning, features are learned with unlabeled input data. Examples include dictionary learning, independent component analysis, autoencoders, matrix factorization^[43] and various forms of clustering.引用错误：没有找到与</ref>对应的<ref>标签^[44]^[45]

}}</ref>

{} / ref

特征学习可以是有监督的，也可以是无监督的。在有监督的特征学习中，可以利用标记输入数据学习特征。例如人工神经网络 Artificial Neural Networks，ANN、多层感知机 Multilayer Perceptrons，MLP和受控字典式学习模型 Supervised Dictionary Learning Model，SDLM。在无监督的特征学习中，特征是通过未标记的输入数据进行学习的。例如，字典学习 Dictionary learning、独立元素分析 Independent Component Analysis、自动编码器 Autoencoders、矩阵分解 Matrix Factorization和各种形式的聚类。

Manifold learning algorithms attempt to do so under the constraint that the learned representation is low-dimensional. Sparse coding algorithms attempt to do so under the constraint that the learned representation is sparse, meaning that the mathematical model has many zeros. Multilinear subspace learning algorithms aim to learn low-dimensional representations directly from tensor representations for multidimensional data, without reshaping them into higher-dimensional vectors.^[46] Deep learning algorithms discover multiple levels of representation, or a hierarchy of features, with higher-level, more abstract features defined in terms of (or generating) lower-level features. It has been argued that an intelligent machine is one that learns a representation that disentangles the underlying factors of variation that explain the observed data.^[47]

Manifold learning algorithms attempt to do so under the constraint that the learned representation is low-dimensional. Sparse coding algorithms attempt to do so under the constraint that the learned representation is sparse, meaning that the mathematical model has many zeros. Multilinear subspace learning algorithms aim to learn low-dimensional representations directly from tensor representations for multidimensional data, without reshaping them into higher-dimensional vectors. Deep learning algorithms discover multiple levels of representation, or a hierarchy of features, with higher-level, more abstract features defined in terms of (or generating) lower-level features. It has been argued that an intelligent machine is one that learns a representation that disentangles the underlying factors of variation that explain the observed data.

流形学习 Manifold Learning算法试图在学习表示为低维的约束条件下进行流形学习。稀疏编码算法 Sparse Coding Algorithms试图在学习表示为稀疏的约束条件下进行编码，这意味着数学模型有许多零点 Zeros。多线性子空间学习算法 Multilinear Subspace Learning Algorithms旨在直接从多维数据的张量表示中学习低维的表示，而不是将它们重塑为高维向量。深度学习算法 Deep Learning Algorithms发现了多层次的表示，或者是一个特征层次结构，具有更高层次、更抽象的特征，这些特征定义为（或可以生成）低层次的特征。有人认为，一个智能机器的表现是可以学习到一种表示的方法，并能够解释数据观测值变化背后的机理或潜在影响。

Feature learning is motivated by the fact that machine learning tasks such as classification often require input that is mathematically and computationally convenient to process. However, real-world data such as images, video, and sensory data has not yielded to attempts to algorithmically define specific features. An alternative is to discover such features or representations through examination, without relying on explicit algorithms.

特征学习的动力来自于机器学习任务，如分类中，通常需要数学上和计算上方便处理的输入。然而，真实世界的数据，如图像、视频和感官数据，并没有那么简单就可以用通过算法定义特定特征。另一种方法是通过检查发现这些特征或表示，而不依赖于显式算法。

稀疏字典学习 Sparse dictionary learning

Sparse dictionary learning is a feature learning method where a training example is represented as a linear combination of basis functions, and is assumed to be a sparse matrix. The method is strongly NP-hard and difficult to solve approximately.^[48] A popular heuristic method for sparse dictionary learning is the K-SVD algorithm. Sparse dictionary learning has been applied in several contexts. In classification, the problem is to determine the class to which a previously unseen training example belongs. For a dictionary where each class has already been built, a new training example is associated with the class that is best sparsely represented by the corresponding dictionary. Sparse dictionary learning has also been applied in image de-noising. The key idea is that a clean image patch can be sparsely represented by an image dictionary, but the noise cannot.^[49]

Sparse dictionary learning is a feature learning method where a training example is represented as a linear combination of basis functions, and is assumed to be a sparse matrix. The method is strongly NP-hard and difficult to solve approximately. A popular heuristic method for sparse dictionary learning is the K-SVD algorithm. Sparse dictionary learning has been applied in several contexts. In classification, the problem is to determine the class to which a previously unseen training example belongs. For a dictionary where each class has already been built, a new training example is associated with the class that is best sparsely represented by the corresponding dictionary. Sparse dictionary learning has also been applied in image de-noising. The key idea is that a clean image patch can be sparsely represented by an image dictionary, but the noise cannot.

稀疏词典学习是一种特征学习方法，在这种方法中，一个训练样本被表示为基函数的线性组合，并假设为稀疏矩阵。该方法具有强 NP- Hard性并且近似求解困难。一种流行的启发式 Heuristic稀疏字典学习方法是 K-SVD 算法。稀疏词典学习已经应用于以下几种情况下：在分类中，问题在于如何确定先前未见的训练样本所属的类；对于已经构建了每个类的字典，一个新的训练示例将与相应的字典最好地稀疏表示的类相关联。稀疏字典学习也被广泛应用到图像去噪的问题中。其关键思想是，一个干净的图像补丁 patch可以由图像字典稀疏地表示，但噪声不能。

异常检测 Anomaly detection

In data mining, anomaly detection, also known as outlier detection, is the identification of rare items, events or observations which raise suspicions by differing significantly from the majority of the data.^[50] Typically, the anomalous items represent an issue such as bank fraud, a structural defect, medical problems or errors in a text. Anomalies are referred to as outliers, novelties, noise, deviations and exceptions.^[51]

In data mining, anomaly detection, also known as outlier detection, is the identification of rare items, events or observations which raise suspicions by differing significantly from the majority of the data. Typically, the anomalous items represent an issue such as bank fraud, a structural defect, medical problems or errors in a text. Anomalies are referred to as outliers, novelties, noise, deviations and exceptions.

在数据挖掘中，异常检测 Anomaly / Outlier detection是指识别那些引起怀疑的稀有项目、事件或者观测结果，它们与其他大多数数据有很大的不同。一般来说，这些不正常的项目都可以反映出数据背后的一个问题，如银行欺诈、结构缺陷、医疗问题或文本中的错误。异常也被称为异常值 Outliers、奇异值 Novelties、噪音 Noise、偏差 Deviations和异常 Exceptions。

In particular, in the context of abuse and network intrusion detection, the interesting objects are often not rare objects, but unexpected bursts in activity. This pattern does not adhere to the common statistical definition of an outlier as a rare object, and many outlier detection methods (in particular, unsupervised algorithms) will fail on such data, unless it has been aggregated appropriately. Instead, a cluster analysis algorithm may be able to detect the micro-clusters formed by these patterns.^[52]

In particular, in the context of abuse and network intrusion detection, the interesting objects are often not rare objects, but unexpected bursts in activity. This pattern does not adhere to the common statistical definition of an outlier as a rare object, and many outlier detection methods (in particular, unsupervised algorithms) will fail on such data, unless it has been aggregated appropriately. Instead, a cluster analysis algorithm may be able to detect the micro-clusters formed by these patterns.

特别是在滥用和网络入侵检测的背景下，人们感兴趣的往往不是罕见的对象，而是突发性的活动。这种模式并不符合异常值作为稀有对象的通用统计学定义，而且许多异常检测方法（特别是无监督的算法）将无法处理这类数据，除非它已经被适当地聚合处理。相反地，数据聚类算法可以检测到这些模式形成的微团簇。

Three broad categories of anomaly detection techniques exist.^[53] Unsupervised anomaly detection techniques detect anomalies in an unlabeled test data set under the assumption that the majority of the instances in the data set are normal, by looking for instances that seem to fit least to the remainder of the data set. Supervised anomaly detection techniques require a data set that has been labeled as "normal" and "abnormal" and involves training a classifier (the key difference to many other statistical classification problems is the inherently unbalanced nature of outlier detection). Semi-supervised anomaly detection techniques construct a model representing normal behavior from a given normal training data set and then test the likelihood of a test instance to be generated by the model.

Three broad categories of anomaly detection techniques exist. Unsupervised anomaly detection techniques detect anomalies in an unlabeled test data set under the assumption that the majority of the instances in the data set are normal, by looking for instances that seem to fit least to the remainder of the data set. Supervised anomaly detection techniques require a data set that has been labeled as "normal" and "abnormal" and involves training a classifier (the key difference to many other statistical classification problems is the inherently unbalanced nature of outlier detection). Semi-supervised anomaly detection techniques construct a model representing normal behavior from a given normal training data set and then test the likelihood of a test instance to be generated by the model.

异常检测技术有3大类。无监督的异常检测 / 测试技术在假设数据集中大多数实例都是正常的情况下，通过是来寻找数据集中最违和的实例，从而实现检测未被标记的测试数据集中的异常。监督式的异常检测分析技术需要一个被标记为“正常”和“异常”的数据集，还需要训练一个分类器（和许多其他分类分析问题的关键区别在于异常检测本身的不平衡性）。半监督的异常检测技术从给定的正常训练数据集构建一个表示正常行为的模型，然后测试由该模型生成的测试实例的可能性。

机器人学习 Robot learning

In developmental robotics, robot learning algorithms generate their own sequences of learning experiences, also known as a curriculum, to cumulatively acquire new skills through self-guided exploration and social interaction with humans. These robots use guidance mechanisms such as active learning, maturation, motor synergies and imitation.

在发展型机器人 Developmental robotics学习中，机器人学习算法能够产生自己的学习经验序列，也称为课程，通过自我引导的探索来与人类社会进行互动，累积获得新技能。这些机器人在学习的过程中会使用诸如主动学习、成熟、协同运动和模仿等引导机制。

基于规则的机器学习算法

基于规则的机器学习是任何机器学习方法的通用术语，它通过识别、学习或演化“规则”来存储、操作或应用知识。基于规则的机器学习者的定义特征是识别和使用一组关系规则，这些规则共同表示系统获取的知识。这与其他机器学习者不同，这些机器学习者通常会识别出一个可以普遍应用于任何实例的奇异模型，以便进行预测^[54] 。基于规则的机器学习方法包括学习分类器系统、关联规则学习和人工免疫系统。

关联规则 Association rules

主文章：关联规则学习

Association rule learning is a rule-based machine learning method for discovering relationships between variables in large databases. It is intended to identify strong rules discovered in databases using some measure of "interestingness".^[55]

Association rule learning is a rule-based machine learning method for discovering relationships between variables in large databases. It is intended to identify strong rules discovered in databases using some measure of "interestingness".

关联规则学习 Association Rule Learning是一种基于规则的机器学习 Rule-based machine learning方法，用于发现大型数据库中变量之间的关系。它旨在利用某种“有趣度”的度量，识别在数据库中新发现的强大规则。

Based on the concept of strong rules, Rakesh Agrawal, Tomasz Imieliński and Arun Swami introduced association rules for discovering regularities between products in large-scale transaction data recorded by point-of-sale (POS) systems in supermarkets.^[56] For example, the rule [math]\displaystyle{ \{\mathrm{onions, potatoes}\} \Rightarrow \{\mathrm{burger}\} }[/math] found in the sales data of a supermarket would indicate that if a customer buys onions and potatoes together, they are likely to also buy hamburger meat. Such information can be used as the basis for decisions about marketing activities such as promotional pricing or product placements. In addition to market basket analysis, association rules are employed today in application areas including Web usage mining, intrusion detection, continuous production, and bioinformatics. In contrast with sequence mining, association rule learning typically does not consider the order of items either within a transaction or across transactions.

Based on the concept of strong rules, Rakesh Agrawal, Tomasz Imieliński and Arun Swami introduced association rules for discovering regularities between products in large-scale transaction data recorded by point-of-sale (POS) systems in supermarkets. For example, the rule [math]\displaystyle{ \{\mathrm{onions, potatoes}\} \Rightarrow \{\mathrm{burger}\} }[/math] found in the sales data of a supermarket would indicate that if a customer buys onions and potatoes together, they are likely to also buy hamburger meat. Such information can be used as the basis for decisions about marketing activities such as promotional pricing or product placements. In addition to market basket analysis, association rules are employed today in application areas including Web usage mining, intrusion detection, continuous production, and bioinformatics. In contrast with sequence mining, association rule learning typically does not consider the order of items either within a transaction or across transactions.

在基于强规则的原理中，Rakesh Agrawal、 Tomasz imieli ski 和 Arun Swami 引入了关联规则这一概念，用于在超市销售点（POS）系统记录的大规模交易数据中发现产品之间的规则。例如，在超市的销售数据中发现的规则表明，如果某位顾客同时购买洋葱和土豆，那么他也很可能会购买汉堡肉。这些信息可以作为市场决策的依据，如促销价格或产品植入。除了市场篮子分析之外，关联规则还应用于 Web 使用挖掘 Web Usage Mining、入侵检测 Intrusion Detection、连续生产和生物信息学 Bioinformatics等应用领域。与序列挖掘相比，关联规则学习通常不考虑事务内或事务之间的先后顺序。

学习分类器

主文章：学习分类器系统

学习分类器系统 Learning Classifier Systems，LCS是一组基于规则的机器学习算法，它将发现组件(通常是遗传算法)与学习组件(执行有监督学习、强化学习或无监督学习)结合起来。他们试图找出一套与情境相关的规则，这些规则以一种分段的方式，集体存储和应用知识，以便进行预测^[57]。

归纳逻辑规划

Inductive logic programming (ILP) is an approach to rule-learning using logic programming as a uniform representation for input examples, background knowledge, and hypotheses. Given an encoding of the known background knowledge and a set of examples represented as a logical database of facts, an ILP system will derive a hypothesized logic program that entails all positive and no negative examples. Inductive programming is a related field that considers any kind of programming languages for representing hypotheses (and not only logic programming), such as functional programs.

归纳逻辑规划 Inductive Logic Programming，ILP是一种用逻辑规划作为输入示例、背景知识和假设的统一表示的规则学习方法。如果将已知的背景知识进行编码，并将一组示例表示为事实的逻辑数据库，ILP 系统将推导出一个假设的逻辑程序，其中包含所有正面和负面的样例。归纳编程是一个与其相关的领域，它考虑用任何一种编程语言来表示假设（不仅仅是逻辑编程），比如函数编程 Functional programs。

主文章：归纳逻辑编程

而归纳逻辑程序设计 Inductive Logic Programming是一种将逻辑编程作为输入实例、背景知识和假设的统一表示的规则学习方法。给定已知背景知识的编码和作为事实逻辑数据库表示的一组示例，ILP系统将导出一个假设逻辑程序，其中包含所有正负示例。归纳编程是一个相关的领域，它考虑任何类型的编程语言来表示假设(而不仅仅是逻辑编程)，例如函数程序。

Inductive logic programming is particularly useful in bioinformatics and natural language processing. Gordon Plotkin and Ehud Shapiro laid the initial theoretical foundation for inductive machine learning in a logical setting.^[58]^[59]^[60] Shapiro built their first implementation (Model Inference System) in 1981: a Prolog program that inductively inferred logic programs from positive and negative examples.^[61] The term inductive here refers to philosophical induction, suggesting a theory to explain observed facts, rather than mathematical induction, proving a property for all members of a well-ordered set.

Inductive logic programming is particularly useful in bioinformatics and natural language processing. Gordon Plotkin and Ehud Shapiro laid the initial theoretical foundation for inductive machine learning in a logical setting. Shapiro built their first implementation (Model Inference System) in 1981: a Prolog program that inductively inferred logic programs from positive and negative examples. The term inductive here refers to philosophical induction, suggesting a theory to explain observed facts, rather than mathematical induction, proving a property for all members of a well-ordered set.

它在生物信息学和自然语言处理 Natural Language Processing中特别有用。戈登 · 普洛特金 Gordon Plotkin和埃胡德 · 夏皮罗 Ehud Shapiro为归纳机器学习在逻辑上奠定了最初的理论基础。夏皮罗 Shapiro在1981年实现了他们的第一个模型推理系统: 一个从正反例中归纳推断逻辑程序的 Prolog 程序。这里的”归纳“指的是哲学上的归纳，通过提出一个理论来解释观察到的事实，而不是数学归纳法证明了一个有序集合的所有成员的性质。

模型 Models

Performing machine learning involves creating a model, which is trained on some training data and then can process additional data to make predictions. Various types of models have been used and researched for machine learning systems.

执行机器学习需要建立一个算法模型，该模型根据一些训练数据进行训练，然后可以处理额外的数据进行预测。机器学习系统已经使用和研究了各种类型的模型。

人工神经网络 Artificial neural networks

文件:Colored neural network.svg

An artificial neural network is an interconnected group of nodes, akin to the vast network of neurons in a brain. Here, each circular node represents an artificial neuron and an arrow represents a connection from the output of one artificial neuron to the input of another.人工神经网络 Artificial Neural Network，ANN是一组相互连接的节点，类似于大脑中庞大的神经元网络。在这里，每个圆形节点代表一个人工神经元 Neuron，一个箭头代表从一个人工神经元的输出到另一个输入的连接

主文章：人工神经网络

人工神经网络(ANN)学习算法，通常称为神经网络(NN)，是一种受生物神经网络启发的学习算法。计算是根据一组相互关联的人工神经元来构造的，它们使用连接主义的计算方法来处理信息。现代神经网络是非线性统计数据建模工具。它们通常用于模拟输入和输出之间的复杂关系，在数据中找到模式，或在观测变量之间的未知联合概率分布中寻找统计结构。

An ANN is a model based on a collection of connected units or nodes called "artificial neurons", which loosely model the neurons in a biological brain. Each connection, like the synapses in a biological brain, can transmit information, a "signal", from one artificial neuron to another. An artificial neuron that receives a signal can process it and then signal additional artificial neurons connected to it. In common ANN implementations, the signal at a connection between artificial neurons is a real number, and the output of each artificial neuron is computed by some non-linear function of the sum of its inputs. The connections between artificial neurons are called "edges". Artificial neurons and edges typically have a weight that adjusts as learning proceeds. The weight increases or decreases the strength of the signal at a connection. Artificial neurons may have a threshold such that the signal is only sent if the aggregate signal crosses that threshold. Typically, artificial neurons are aggregated into layers. Different layers may perform different kinds of transformations on their inputs. Signals travel from the first layer (the input layer) to the last layer (the output layer), possibly after traversing the layers multiple times.

人工神经网络是一种基于一组被称为“人工神经元”的连接单元或节点的模型，人工神经元可以对生物大脑中的神经元进行松散的建模。每一个连接，就像生物大脑中的突触一样，可以将信息，一个“信号” ，从一个人工神经元传递到另一个。接收到信号的人工神经元可以处理它，然后发送信号给连接到它的其他人工神经元。在通常的人工神经网络实现中，人工神经元之间连接处的信号是一个实数，每个人工神经元的输出是由一些输入和的非线性函数计算出来的。人造神经元之间的连接称为“边缘”。人工神经元和边缘通常有一个权重，可以随着学习的进行而调整。重量增加或减少连接处信号的强度。人工神经元可能有一个阈值，这样只有当聚合信号超过这个阈值时才发送信号。通常，人造神经元聚集成层。不同的层可以对其输入执行不同类型的转换。信号从第一层(输入层)传输到最后一层(输出层) ，可能是在多次遍历这些层之后。

The original goal of the ANN approach was to solve problems in the same way that a human brain would. However, over time, attention moved to performing specific tasks, leading to deviations from biology. Artificial neural networks have been used on a variety of tasks, including computer vision, speech recognition, machine translation, social network filtering, playing board and video games and medical diagnosis.

人工神经网络方法的最初目标是用人类大脑解决问题的同样方式。然而，随着时间的推移，其注意力转移到执行特定的任务上，导致了与生物学的偏差。现在人工神经网络已被用于各种任务中，包括计算机视觉 Computer Visio、语音识别 Speech Recognition、机器翻译 Machine Translation、社会网络过滤 Social Network Filtering、玩棋盘和视频游戏以及医疗诊断。

深度学习

主文章：深度学习

近几年来，硬件价格的下降和个人用GPU的发展促进了深度学习 Deep Learning概念的发展，该概念由人工神经网络中的多个隐层组成。这种方法试图模拟人脑将光和声音处理成视觉和听觉的方式。深入学习的一些成功应用是计算机视觉和语音识别 ^[62]。

决策树 Decision trees

主文章：决策树学习

Decision tree learning uses a decision tree as a predictive model to go from observations about an item (represented in the branches) to conclusions about the item's target value (represented in the leaves). It is one of the predictive modeling approaches used in statistics, data mining and machine learning. Tree models where the target variable can take a discrete set of values are called classification trees; in these tree structures, leaves represent class labels and branches represent conjunctions of features that lead to those class labels. Decision trees where the target variable can take continuous values (typically real numbers) are called regression trees. In decision analysis, a decision tree can be used to visually and explicitly represent decisions and decision making. In data mining, a decision tree describes data, but the resulting classification tree can be an input for decision making.

决策树学习使用决策树作为预测模型，它将问题相关的观察结果映射为问题目标值的结论，是统计学、数据挖掘和机器学习中常用的预测建模方法之一。目标变量接受到的一组离散值的树模型称为分类树; 在这些树结构中，叶子代表类标签，分支代表连接到这些类标签的特征。其中目标变量可以取连续值（通常是实数）的决策树称为回归树。在决策分析中，可以使用决策树直观地表示决策和决策。在数据挖掘中，决策树是用来描述数据的，但得到的分类树可以作为决策的输入。

支持向量机 Support vector machines

主文章：SVM支持向量机

SVM支持向量机(SupportVectorMachine，SVMs)是一种用于分类和回归的监督学习算法。给出一组训练实例，每个样本会被标记为属于两类中的一个，SVM算法建立了一个模型来预测一个新的例子是否属于一个类别或另一个类别。

Support vector machines (SVMs), also known as support vector networks, are a set of related supervised learning methods used for classification and regression. Given a set of training examples, each marked as belonging to one of two categories, an SVM training algorithm builds a model that predicts whether a new example falls into one category or the other.^[63] An SVM training algorithm is a non-probabilistic, binary, linear classifier, although methods such as Platt scaling exist to use SVM in a probabilistic classification setting. In addition to performing linear classification, SVMs can efficiently perform a non-linear classification using what is called the kernel trick, implicitly mapping their inputs into high-dimensional feature spaces.

Support vector machines (SVMs), also known as support vector networks, are a set of related supervised learning methods used for classification and regression. Given a set of training examples, each marked as belonging to one of two categories, an SVM training algorithm builds a model that predicts whether a new example falls into one category or the other. An SVM training algorithm is a non-probabilistic, binary, linear classifier, although methods such as Platt scaling exist to use SVM in a probabilistic classification setting. In addition to performing linear classification, SVMs can efficiently perform a non-linear classification using what is called the kernel trick, implicitly mapping their inputs into high-dimensional feature spaces.

支持向量机的训练算法用到的是一种非概率的二进制线性分类器，尽管在概率分类环境中也存在使用支持向量机的方法，如 Platt 缩放法。除了执行线性分类，支持向量机可以有效地执行非线性分类使用所谓的核技巧 Kernel trick，隐式地将模型输入映射到高维特征空间。

文件:Linear regression.svg

Illustration of linear regression on a data set.数据集上的线性回归

回归分析 Regression analysis

Regression analysis encompasses a large variety of statistical methods to estimate the relationship between input variables and their associated features. Its most common form is linear regression, where a single line is drawn to best fit the given data according to a mathematical criterion such as ordinary least squares. The latter is often extended by regularization (mathematics) methods to mitigate overfitting and bias, as in ridge regression. When dealing with non-linear problems, go-to models include polynomial regression (for example, used for trendline fitting in Microsoft Excel ^[64]), Logistic regression (often used in statistical classification) or even kernel regression, which introduces non-linearity by taking advantage of the kernel trick to implicitly map input variables to higher dimensional space.

Regression analysis encompasses a large variety of statistical methods to estimate the relationship between input variables and their associated features. Its most common form is linear regression, where a single line is drawn to best fit the given data according to a mathematical criterion such as ordinary least squares. The latter is often extended by regularization (mathematics) methods to mitigate overfitting and bias, as in ridge regression. When dealing with non-linear problems, go-to models include polynomial regression (for example, used for trendline fitting in Microsoft Excel ), Logistic regression (often used in statistical classification) or even kernel regression, which introduces non-linearity by taking advantage of the kernel trick to implicitly map input variables to higher dimensional space.

回归分析 Regression Analysis包含了大量的统计方法来估计输入变量和它们的相关特征之间的关系。它最常见的形式是线性回归 Linear Regression，根据一个数学标准，比如一般最小平方法，画一条线来最好地拟合给定的数据。后者通常通过正则化(数学)方法来扩展，以减少过拟合和偏差，如岭回归。在处理非线性问题时，常用的模型包括多项式回归(例如，在 Microsoft Excel 中用于趋势线拟合)、 Logit模型回归（通常用于分类）甚至核回归，它利用核技巧将输入变量隐式地映射到更高维度空间，从而引入了非线性。

贝叶斯网络 Bayesian networks

文件:SimpleBayesNetNodes.svg

A simple Bayesian network. Rain influences whether the sprinkler is activated, and both rain and the sprinkler influence whether the grass is wet. 一个简单的贝叶斯网路。雨水会影响喷头是否被激活，而雨水和喷头都会影响草地是否湿润。

主文章：贝叶斯网络

贝叶斯网路 Bayesian Network，或称信任网络或者有向无环图形模型是通过有向无环图 DAG表示一组随机变量及其条件独立性的概率图形模型。例如，贝叶斯网络可以表示疾病和症状之间的概率关系。给定症状，网络可以用来计算各种疾病出现的概率。有效的算法可以进行推理和学习。现有的高效算法可以执行推理和学习。贝叶斯网络模型的变量序列，如语音信号或蛋白质序列，被称为动态贝叶斯网络。而贝叶斯网络能够表示和解决不确定性决策问题的推广称为影响图。

A Bayesian network, belief network or directed acyclic graphical model is a probabilistic graphical model that represents a set of random variables and their conditional independence with a directed acyclic graph (DAG). For example, a Bayesian network could represent the probabilistic relationships between diseases and symptoms. Given symptoms, the network can be used to compute the probabilities of the presence of various diseases. Efficient algorithms exist that perform inference and learning. Bayesian networks that model sequences of variables, like speech signals or protein sequences, are called dynamic Bayesian networks. Generalizations of Bayesian networks that can represent and solve decision problems under uncertainty are called influence diagrams.

遗传算法 Genetic algorithms

A genetic algorithm (GA) is a search algorithm and heuristic technique that mimics the process of natural selection, using methods such as mutation and crossover to generate new genotypes in the hope of finding good solutions to a given problem. In machine learning, genetic algorithms were used in the 1980s and 1990s.^[65]^[66] Conversely, machine learning techniques have been used to improve the performance of genetic and evolutionary algorithms.^[67]

A genetic algorithm (GA) is a search algorithm and heuristic technique that mimics the process of natural selection, using methods such as mutation and crossover to generate new genotypes in the hope of finding good solutions to a given problem. In machine learning, genetic algorithms were used in the 1980s and 1990s. Conversely, machine learning techniques have been used to improve the performance of genetic and evolutionary algorithms.

遗传算法 Genetic Algorithm，GA是一种模仿自然选择过程的搜索算法和启发式技术，利用变异和交叉等方法产生新的基因型，以期为给定的问题找到最优解。在机器学习中，遗传算法在20世纪80年代和90年代被广泛使用，而现在的机器学习技术已经可以被用来改善遗传和进化算法的性能。

训练模型 Training models

Usually, machine learning models require a lot of data in order for them to perform well. Usually, when training a machine learning model, one needs to collect a large, representative sample of data from a training set. Data from the training set can be as varied as a corpus of text, a collection of images, and data collected from individual users of a service. Overfitting is something to watch out for when training a machine learning model.

通常情况下，机器学习模型需要大量的数据才能有良好的性能，因此当训练一个机器学习模型时，需要从一个训练集中收集大量有代表性的数据样本。来自训练集的数据可以像文本语料库、图像集合和从服务的单个用户收集的数据一样多种多样。当训练一个机器学习模型时，需要特别注意过拟合问题。

联合学习 Federated learning

Federated learning is a new approach to training machine learning models that decentralizes the training process, allowing for users' privacy to be maintained by not needing to send their data to a centralized server. This also increases efficiency by decentralizing the training process to many devices. For example, Gboard uses federated machine learning to train search query prediction models on users' mobile phones without having to send individual searches back to Google.^[68]

Federated learning is a new approach to training machine learning models that decentralizes the training process, allowing for users' privacy to be maintained by not needing to send their data to a centralized server. This also increases efficiency by decentralizing the training process to many devices. For example, Gboard uses federated machine learning to train search query prediction models on users' mobile phones without having to send individual searches back to Google.

联合学习 Federated Learning是一种新的训练机器学习模型的方法，它分散了训练的过程，允许用户不需要将他们的数据发送到一个集中的服务器这样的做法来维护他们的隐私。通过将模型的训练过程分散到许多设备上，提升了算法效率。例如，谷歌董事会使用联合机器学习刚发来训练用户手机上的搜索查询预测模型，而不必将每个人地搜索信息发送回谷歌。

应用

机器学习的应用包括：
农业
 机器证明 ^[69]^[70]
适应性网站
 情感计算
 生物信息学
 脑-机接口
 化学信息学
分类DNA序列
 计算解剖学
 计算机网络
 电信
 计算机视觉，包括目标识别
检测信用卡诈骗
 通用游戏^[71]
信息检索
 互联网诈骗检测^[25]
语言学
 营销
 机器学习控制
 机器感知
 医学诊断
 经济学
 保险
 自然语言处理
 自然语言理解^[72]
优化和元启发式
 在线广告
 推荐系统
 机器人移动
 搜索引擎
 情感分析（意见挖掘)
序列挖掘
 软件工程
 语音识别和手写识别
 金融市场分析
结构健康监测
 句法模式识别
 时间序列预测
 用户行为分析
 翻译 ^[73]

2006年，在线电影公司Netflix举办了第一届“Netflix奖”竞赛，目的是寻找一个更好地预测用户喜好的程序，并将现有Cinematch电影推荐算法的准确性提高至少10%。由AT&T实验室的研究人员组成的一个联合团队与“大混乱与务实理论”合作，以100万美元的价格建立了一个集成模型^[74] ，赢得了2009年的大奖。在颁奖后不久，Netflix意识到，观众的收视率并不是他们观看模式的最佳指标(“一切都是推荐的”)，于是他们相应地改变了推荐引擎 ^[75]。

2010年，《华尔街日报》发表了Rebellion研究公司及它们利用机器学习预测金融危机的相关报道。^[76]。

2012年，Sun Microsystems的联合创始人VinodKhosla预测，在接下来的20年里，80%的医生将因自动机器学习医疗诊断软件而失业 ^[77]。

2014年，有报道称机器学习算法在艺术史领域被用于研究美术绘画，并可能揭示了艺术家之间先前未被认识到的影响^[78]。

局限

虽然机器学习在某些领域是革命性的，但有效的机器学习仍是困难的，因为找出模式很难，而且往往没有足够的训练数据；因此，许多机器学习程序往往无法达到预期值 ^[79] ^[80] ^[81] 。造成这种情况的原因很多：缺乏(适当的)数据、无法访问数据、数据偏见、隐私问题、错误的任务选择和算法、错误的工具和人员、缺乏资源和评估问题^[82]。

机器学习方法尤其会受到不同数据偏见的影响。只针对当前客户进行训练的机器学习系统可能无法预测训练数据中未表示的新客户组的需求。当接受人工数据训练时，机器学习很可能会产生与社会上已经存在的相同的成体制偏见和无意识偏见 ^[83] 。从数据中学到的语言模型已被证明含有类似人类的偏见 ^[84] ^[85] 。用于犯罪风险评估的机器学习系统被发现对黑人有偏见^[86] ^[87] 。在2015年，谷歌上黑人的照片常常被贴上大猩猩的标签 ^[88] ，而到2018年，这仍然没有得到很好的解决，但据报道，谷歌仍在使用变通方法将所有大猩猩从训练数据中删除，因此根本无法识别真正的大猩猩 ^[89]。在许多其他系统中 ^[90] ，也发现了识别非白人的类似问题。2016年，微软测试了一个从Twitter上学习的聊天机器人，而后者却很快学会了种族主义和性别歧视的语言 ^[91]。由于这些挑战，机器在其他领域的有效使用仍有很长的路要走 ^[92]。2018年，Uber的一辆自动驾驶汽车未能检测到行人并导致其在事故中丧生。 ^[93]。 IBM Watson系统在医疗保健领域使用机器学习的尝试，即便经过多年的时间和数十亿美元的投资，也未能实现 ^[94] ^[95]。

模型评估

分类机学习模型可以通过例如 Holdout 方法等精度估计技术来验证，该方法将训练和测试集(通常指定2/3为训练集，1/3为测试集)中的数据分开，并对训练模型在测试集上的性能进行评估。比较而言，N重交叉验证方法将数据随机分成k个子集，其中使用k-1个实例来训练模型，而使用第k个实例来测试训练模型的预测能力。除了Holdout方法和交叉验证方法之外，Bootstrap还可以使用从数据集中替换的n个实例来评估模型的准确性。

除了总体准确性外，调查人员还经常报告敏感性和特异性，即真阳性率(TPR)和真阴性率(TNR)。同样，调查人员有时报告假阳性率(FPR)和假阴性率(FNR)。然而，这些比率不能显示它们的分子和分母。总运行特性(TOC)是表示模型诊断能力的一种有效方法。TOC显示上述比率的分子和分母，因此TOC提供的信息比常用的受试者工作特征(ROC)和曲线下ROC的关联区域(AUC)提供的信息更多^[96]。

规范准则

机器学习导致了许多伦理问题。利用偏见收集的数据集进行训练的系统可能会在使用时表现出这些偏见(算法偏见)，从而将文化偏见数字化 ^[97] 。例如，使用具有种族主义雇用政策的公司的招聘数据可能导致机器学习系统复制这种偏见，方法是将求职者与先前成功申请者的相似性打分 ^[98] ^[99] 。负责任的数据收集和系统使用的算法规则的记录是机器学习的一个关键部分。

因为语言包含偏见，受过语料库训练的机器也必然会学习偏见 ^[100]。

其他与个人偏见无关的道德挑战则更多地体现在医疗保健方面。医疗保健专业人士担心，这些系统可能不是为了公众利益而设计的，而是作为创收机器设计的。这在改善医疗保健和增加利润方面存在长期的道德困境的美国尤其如此。例如，这些算法可以设计为患者提供不必要的测试或药物治疗，而该算法的专属所有者在这些测试或药物中持有股份。在医疗保健领域，机器学习在成为为专业人员提供诊断、药物治疗甚至计划病人康复路径的伟大工具上有着巨大的潜力，但这在前面提到的个人偏见和这些“贪婪”偏见得到解决之前是不可能发生的^[101]。

软件

包含各种机器学习算法的软件包包括：

免费开源软件

·CNTK                        ·MLPACK                         ·Spark MLlib
·Deeplearning4j              ·MXNet                          ·TensorFlow
·ELKI                        ·OpenNN                         ·Torch/PyTorch
·H2O                         ·Orange                         ·Weka / MOA
·Mahout                      ·scikit-learn                   ·Yooreeka
·Mallet                      ·Shogun

有免费开源版本的专利软件

   • KNIME                            • RapidMiner

专利软件

期刊

会议

Conference on Neural Information Processing Systems
International Conference on Machine Learning
International Conference on Learning Representations
Open Data Science Conference (ODSC)

参见

参考文献

↑ The "without being explicitly programmed" definition is often attributed to Arthur Samuel, who coined the term "machine learning" in 1959. But the phrase is not found literally in this publication, and may be a paraphrase that appeared later. Confer "Paraphrasing Arthur Samuel (1959), the question is: How can computers learn to solve problems without being explicitly programmed?" in Koza, John R.; Bennett, Forrest H.; Andre, David; Keane, Martin A. (1996). Automated Design of Both the Topology and Sizing of Analog Electrical Circuits Using Genetic Programming. Artificial Intelligence in Design '96. Springer, Dordrecht. pp. 151–170. doi:10.1007/978-94-009-0279-4_9
↑ Samuel, Arthur (1959). "Some Studies in Machine Learning Using the Game of Checkers". IBM Journal of Research and Development. 3 (3): 210–229. CiteSeerX 10.1.1.368.2254. {{cite journal}}: External link in |title= (help)
↑ http://www.britannica.com/EBchecked/topic/1116194/machine-learning
↑ Ron Kohavi; Foster Provost (1998). "Glossary of terms". Machine Learning. 30: 271–274.
↑ ^5.0 ^5.1 ^5.2 ^5.3 ^5.4 Bishop, C. M. (2006), Pattern Recognition and Machine Learning, Springer, ISBN 978-0-387-31073-2
↑ Mannila Heikki(1996). Data mining: machine learning, statistics, and databases. Int'l Conf. Scientific and Statistical Database Management.IEEE Computer Society.
↑ Friedman, Jerome H. (1998). "Data Mining and Statistics: What's the connection?". Computing Science and Statistics. 29 (1): 3–9.
↑ "Machine Learning: What it is and why it matters". www.sas.com. Retrieved 2016-03-29.
↑ Mitchell, T. (1997). Machine Learning. McGraw Hill. pp. 2.
↑ Harnad, Stevan(2008), "The Annotation Game: On Turing (1950) on Computing, Machinery, and Intelligence", in Epstein, Robert; Peters, Grace, The Turing Test Sourcebook: Philosophical and Methodological Issues in the Quest for the Thinking Computer , Kluwer
↑ R. Kohavi and F. Provost, "Glossary of terms," Machine Learning, vol. 30, no. 2–3, pp. 271–274, 1998.
↑ Sarle Warren."Neural Networks and statistical models". 10.1.1.27.699
↑ ^13.0 ^13.1 ^13.2 ^13.3 ^13.4 ^13.5 ^13.6 ^13.7 Russell, Stuart; Norvig, Peter (2003) [1995]. Artificial Intelligence: A Modern Approach (2nd ed.). Prentice Hall. ISBN 978-0137903955. 引用错误：无效<ref>标签；name属性“aima”使用不同内容定义了多次
↑ ^14.0 ^14.1 ^14.2 ^14.3 Langley, Pat (2011). "The changing science of machine learning". Machine Learning. 82 (3): 275–279. {{cite journal}}: Text "doi : 10.1007/s10994-011-5242-y" ignored (help) 引用错误：无效<ref>标签；name属性“changing”使用不同内容定义了多次
↑ Le Roux, Nicolas; Bengio, Yoshua;Fitzgibbon, Andrew(2012). "Improving First and Second-Order Methods by Modeling Uncertainty Optimization for Machine Learning" In Sra, Suvrit; Nowozin, Sebastian; Wright, Stephen J. Optimization for Machine Learning. MIT Press. p. 404.
↑ ^16.0 ^16.1 Michael I. Jordan (2014-09-10). "statistics and machine learning." reddit. Retrieved 2014-10-01.
↑ Cornell University Library. "Breiman: Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)". Retrieved 8 August 2015.
↑ Gareth James; Daniela Witten; Trevor Hastie; Robert Tibshirani (2013). An Introduction to Statistical Learning. Springer. p. vii. http://www-bcf.usc.edu/~gareth/ISL/.
↑ 模板:Cite citeseerx
↑ Ref name"elements" {{cite web {{cite web { cite web |url= https://course.elementsofai.com/ |url= https://course.elementsofai.com/ Https://course.elementsofai.com/ |title= The Elements of AI |title= The Elements of AI 人工智能的元素 |publisher= University of Helsinki |publisher= University of Helsinki 出版商赫尔辛基大学 |date = Dec 2019 |date = Dec 2019 2019年12月 |accessdate=7 April 2020}} |accessdate=7 April 2020}} 2020年4月7日}
↑ / ref {{cite web {{cite web { cite web |url= https://www.techworld.com/tech-innovation/satalia-ceo-no-one-is-doing-ai-optimisation-can-change-that-3775689/ |url= https://www.techworld.com/tech-innovation/satalia-ceo-no-one-is-doing-ai-optimisation-can-change-that-3775689/ Https://www.techworld.com/tech-innovation/satalia-ceo-no-one-is-doing-ai-optimisation-can-change-that-3775689/ |title= Satalia CEO Daniel Hulme has a plan to overcome the limitations of machine learning |title= Satalia CEO Daniel Hulme has a plan to overcome the limitations of machine learning 萨塔利亚公司首席执行官丹尼尔 · 休姆计划克服机器学习的局限性 |publisher= Techworld |publisher= Techworld 出版商 Techworld |date = October 2019 |date = October 2019 2019年10月 |accessdate=7 April 2020}} |accessdate=7 April 2020}} 2020年4月7日}
↑ 引用错误：无效<ref>标签；未给name属性为Alpaydin2020的引用提供文字
↑ Le Roux, Nicolas; Bengio, Yoshua; Fitzgibbon, Andrew (2012). "Improving+First+and+Second-Order+Methods+by+Modeling+Uncertainty "Improving First and Second-Order Methods by Modeling Uncertainty". In Sra, Suvrit; Nowozin, Sebastian; Wright, Stephen J. (eds.). Optimization for Machine Learning. MIT Press. p. 404. ISBN 9780262016469.
↑ Mohri, Mehryar; Rostamizadeh, Afshin; Talwalkar, Ameet (2012). Foundations of Machine Learning. USA, Massachusetts: MIT Press. ISBN 9780262018258.
↑ ^25.0 ^25.1 Alpaydin, Ethem (2010). Introduction to Machine Learning. London: The MIT Press. https://mitpress.mit.edu/books/introduction-machine-learning.
↑ Tillmann, A. M. (2015). "On the Computational Intractability of Exact and Approximate Dictionary Learning". IEEE Signal Processing Letters. 22 (1): 45–49. {{cite journal}}: Text "arxiv:1405.6664" ignored (help); Text "bibcode;2015ISPL...22...45T" ignored (help)
↑ Aharon, M, M Elad, and A Bruckstein. 2006. "K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation." Signal Processing, IEEE Transactions on 54 (11): 4311–4322
↑ Goldberg, David E.; Holland, John H. (1988). "Genetic algorithms and machine learning". Machine Learning. 3 (2): 95–99.
↑ Michie, D.; Spiegelhalter, D. J.; Taylor, C. C. (1994). Machine Learning, Neural and Statistical Classification. Ellis Horwood.
↑ Zhang, Jun; Zhan, Zhi-hui; Lin, Ying; Chen, Ni; Gong, Yue-jiao; Zhong, Jing-hui; Chung, Henry S.H.; Li, Yun; Shi, Yu-hui (2011). "Evolutionary Computation Meets Machine Learning: A Survey" (PDF). Computational Intelligence Magazine. 6 (4): 68–75.
↑ Russell, Stuart J.; Norvig, Peter (2010). Artificial Intelligence: A Modern Approach (Third ed.). Prentice Hall. ISBN 9780136042594.
↑ Mohri, Mehryar; Rostamizadeh, Afshin; Talwalkar, Ameet (2012). Foundations of Machine Learning. The MIT Press. ISBN 9780262018258.
↑ 引用错误：无效<ref>标签；未给name属性为Mitchell-1997的引用提供文字
↑ Alpaydin, Ethem (2010). Introduction to Machine Learning. MIT Press. p. 9. ISBN 978-0-262-01243-0. https://books.google.com/books?id=7f5bBAAAQBAJ&printsec=frontcover#v=onepage&q=classification&f=false.
↑ Jordan, Michael I.; Bishop, Christopher M. (2004). "Neural Networks". In Allen B. Tucker. Computer Science Handbook, Second Edition (Section VII: Intelligent Systems). Boca Raton, Florida: Chapman & Hall/CRC Press LLC. ISBN 978-1-58488-360-9.
↑ Alex Ratner; Stephen Bach; Paroma Varma; Chris. "Weak Supervision: The New Programming Paradigm for Machine Learning". hazyresearch.github.io. referencing work by many other members of Hazy Research. Retrieved 2019-06-06.
↑ van Otterlo, M.; Wiering, M. (2012). Reinforcement learning and markov decision processes. Adaptation, Learning, and Optimization. 12. pp. 3–42. doi:10.1007/978-3-642-27645-3_1. ISBN 978-3-642-27644-6.

↑ Bozinovski, S. (1982). "A self-learning system using secondary reinforcement" . In Trappl, Robert (ed.). Cybernetics and Systems Research: Proceedings of the Sixth European Meeting on Cybernetics and Systems Research. North Holland. pp. 397–402.

.

↑ Bozinovski, Stevo (2014) "Modeling mechanisms of cognition-emotion interaction in artificial neural networks, since 1981." Procedia Computer Science p. 255-263
↑ Bozinovski, S. (2001) "Self-learning agents: A connectionist theory of emotion based on crossbar value judgment." Cybernetics and Systems 32(6) 637-667.
↑ Lu, Haiping; Plataniotis, K.N.; Venetsanopoulos, A.N. (2011). "A Survey of Multilinear Subspace Learning for Tensor Data" (PDF). Pattern Recognition. 44 (7): 1540–1551.
↑ Yoshua Bengio (2009). Learning Deep Architectures for AI. Now Publishers Inc.. pp. 1–3. https://books.google.com/books?id=cq5ewg7FniMC&pg=PA3.
↑ Nathan Srebro; Jason D. M. Rennie; Tommi S. Jaakkola (2004). Maximum-Margin Matrix Factorization. NIPS.
↑ Csurka, Gabriella; Dance, Christopher C.; Fan, Lixin; Willamowski, Jutta; Bray, Cédric (2004). Visual categorization with bags of keypoints (PDF). ECCV Workshop on Statistical Learning in Computer Vision.
↑ Daniel Jurafsky; James H. Martin (2009). Speech and Language Processing. Pearson Education International. pp. 145–146.
↑ Lu, Haiping; Plataniotis, K.N.; Venetsanopoulos, A.N. (2011). "A Survey of Multilinear Subspace Learning for Tensor Data" (PDF). Pattern Recognition. 44 (7): 1540–1551. doi:10.1016/j.patcog.2011.01.004.
↑ Yoshua Bengio (2009). Learning Deep Architectures for AI. Now Publishers Inc.. pp. 1–3. ISBN 978-1-60198-294-0. https://books.google.com/books?id=cq5ewg7FniMC&pg=PA3.
↑ Tillmann, A. M. (2015). "On the Computational Intractability of Exact and Approximate Dictionary Learning". IEEE Signal Processing Letters. 22 (1): 45–49. arXiv:1405.6664. Bibcode:2015ISPL...22...45T. doi:10.1109/LSP.2014.2345761.
↑ Aharon, M, M Elad, and A Bruckstein. 2006. "K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation." Signal Processing, IEEE Transactions on 54 (11): 4311–4322
↑ Zimek, Arthur; Schubert, Erich (2017), "Outlier Detection", Encyclopedia of Database Systems (in English), Springer New York, pp. 1–5, doi:10.1007/978-1-4899-7993-3_80719-1, ISBN 9781489979933
↑ Hodge, V. J.; Austin, J. (2004). "A Survey of Outlier Detection Methodologies" (PDF). Artificial Intelligence Review. 22 (2): 85–126. CiteSeerX 10.1.1.318.4023. doi:10.1007/s10462-004-4304-y.
↑ Dokas, Paul; Ertoz, Levent; Kumar, Vipin; Lazarevic, Aleksandar; Srivastava, Jaideep; Tan, Pang-Ning (2002). "Data mining for network intrusion detection" (PDF). Proceedings NSF Workshop on Next Generation Data Mining.
↑ Chandola, V.; Banerjee, A.; Kumar, V. (2009). "Anomaly detection: A survey". ACM Computing Surveys. 41 (3): 1–58. doi:10.1145/1541880.1541882.
↑ Bassel, George W.; Glaab, Enrico; Marquez, Julietta; Holdsworth, Michael J.; Bacardit, Jaume (2011-09-01). "Functional Network Construction in Arabidopsis Using Rule-Based Machine Learning on Large-Scale Data Sets". The Plant Cell (in English). 23 (9): 3101–3116. {{cite journal}}: Text "doi:10.1105/tpc.111.088153" ignored (help); Text "issn:1532-298X" ignored (help); Text "pmc:3203449" ignored (help); Text "pmid:21896882" ignored (help)
↑ Piatetsky-Shapiro, Gregory (1991), Discovery, analysis, and presentation of strong rules, in Piatetsky-Shapiro, Gregory; and Frawley, William J.; eds., Knowledge Discovery in Databases, AAAI/MIT Press, Cambridge, MA.
↑ Agrawal, R.; Imieliński, T.; Swami, A. (1993). "Mining association rules between sets of items in large databases". Proceedings of the 1993 ACM SIGMOD international conference on Management of data - SIGMOD '93. pp. 207. doi:10.1145/170035.170072. ISBN 978-0897915922.
↑ Urbanowicz, Ryan J.; Moore, Jason H. (2009-09-22). "Learning Classifier Systems: A Complete Introduction, Review, and Roadmap". Journal of Artificial Evolution and Applications (in English). 2009: 1–25. {{cite journal}}: Text "issn:1687-6229" ignored (help)
↑ Plotkin G.D. Automatic Methods of Inductive Inference, PhD thesis, University of Edinburgh, 1970.
↑ Shapiro, Ehud Y. Inductive inference of theories from facts, Research Report 192, Yale University, Department of Computer Science, 1981. Reprinted in J.-L. Lassez, G. Plotkin (Eds.), Computational Logic, The MIT Press, Cambridge, MA, 1991, pp. 199–254.

↑ Shapiro, Ehud Y. (1983). Algorithmic program debugging. Cambridge, Mass: MIT Press.

↑ Shapiro, Ehud Y. "The model inference system." Proceedings of the 7th international joint conference on Artificial intelligence-Volume 2. Morgan Kaufmann Publishers Inc., 1981.
↑ Honglak Lee, Roger Grosse, Rajesh Ranganath, Andrew Y. Ng. "Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations" Proceedings of the 26th Annual International Conference on Machine Learning, 2009.
↑ Cortes, Corinna; Vapnik, Vladimir N. (1995). "Support-vector networks". Machine Learning. 20 (3): 273–297. doi:10.1007/BF00994018.
↑ Stevenson, Christopher. "Tutorial: Polynomial Regression in Excel". facultystaff.richmond.edu. Retrieved 22 January 2017.
↑ Goldberg, David E.; Holland, John H. (1988). "Genetic algorithms and machine learning" (PDF). Machine Learning. 3 (2): 95–99. doi:10.1007/bf00113892.
↑ Michie, D.; Spiegelhalter, D. J.; Taylor, C. C. (1994). "Machine Learning, Neural and Statistical Classification". Ellis Horwood Series in Artificial Intelligence. Bibcode:1994mlns.book.....M.
↑ Zhang, Jun; Zhan, Zhi-hui; Lin, Ying; Chen, Ni; Gong, Yue-jiao; Zhong, Jing-hui; Chung, Henry S.H.; Li, Yun; Shi, Yu-hui (2011). "Evolutionary Computation Meets Machine Learning: A Survey". Computational Intelligence Magazine. 6 (4): 68–75. doi:10.1109/mci.2011.942584.
↑ "Federated Learning: Collaborative Machine Learning without Centralized Training Data". Google AI Blog (in English). Retrieved 2019-06-08.
↑ Bridge, James P., Sean B. Holden, and Lawrence C. Paulson. "Machine learning for first-order theorem proving." Journal of automated reasoning 53.2 (2014): 141–172.
↑ Loos, Sarah, et al. "Deep Network Guided Proof Search." arXiv preprint arXiv:1701.06972 (2017).
↑ Finnsson, Hilmar, and Yngvi Björnsson. "Simulation-Based Approach to General Game Playing." AAAI. Vol. 8. 2008.
↑ Sarikaya, Ruhi, Geoffrey E. Hinton, and Anoop Deoras. "Application of deep belief networks for natural language understanding." IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP) 22.4 (2014): 778–784.
↑ "AI-based translation to soon reach human levels: industry officials". Yonhap news agency. Retrieved 4 Mar 2017.
↑ "BelKor Home Page" research.att.com
↑ "The Netflix Tech Blog: Netflix Recommendations: Beyond the 5 stars (Part 1)". Retrieved 8 August 2015.
↑ Scott Patterson (13 July 2010). "Letting the Machines Decide". The Wall Street Journal. Retrieved 24 June 2018.
↑ Vonod Khosla (January 10, 2012). "Do We Need Doctors or Algorithms?". Tech Crunch.
↑ When A Machine Learning Algorithm Studied Fine Art Paintings, It Saw Things Art Historians Had Never Noticed, The Physics at ArXiv blog
↑ "Why Machine Learning Models Often Fail to Learn: QuickTake Q&A". Bloomberg.com. 2016-11-10. Retrieved 2017-04-10.
↑ "The First Wave of Corporate AI Is Doomed to Fail". Harvard Business Review. 2017-04-18. Retrieved 2018-08-20.
↑ "Why the A.I. euphoria is doomed to fail". VentureBeat. 2016-09-18. Retrieved 2018-08-20.
↑ Reasons why your machine learning project will fail "9 Reasons why your machine learning project will fail". www.kdnuggets.com. Retrieved 2018-08-20.
↑ Garcia, Megan (2016). "Racist in the Machine". World Policy Journal (in English). 33 (4): 111–117. {{cite journal}}: Text "issn:0740-2775" ignored (help)
↑ Caliskan, Aylin; Bryson, Joanna J.; Narayanan, Arvind (2017-04-14). "Semantics derived automatically from language corpora contain human-like biases". Science (in English). 356 (6334): 183–186. {{cite journal}}: Text "doi:10.1126/science.aal4230" ignored (help); Text "issn:0036-8075" ignored (help); Text "pmid:28408601" ignored (help)
↑ Wang, Xinan; Dasgupta, Sanjoy (2016), Lee, D. D.; Sugiyama, M.; Luxburg, U. V.; Guyon, I., eds., "An algorithm for L1 nearest neighbor search via monotonic embedding" (PDF), Advances in Neural Information Processing Systems 29, Curran Associates, Inc., pp. 983–991, Retrieved 2018-08-20
↑ "Machine Bias". ProPublica. Julia Angwin, Jeff Larson, Lauren Kirchner, Surya Mattu. 2016-05-23. Retrieved 2018-08-20.
↑ "Opinion | When an Algorithm Helps Send You to Prison". New York Times. Retrieved 2018-08-20.
↑ "Google apologises for racist blunder". BBC News. 2015-07-01. Retrieved 2018-08-20.
↑ "Google 'fixed' its racist algorithm by removing gorillas from its image-labeling tech". The Verge. Retrieved 2018-08-20.
↑ "Opinion | Artificial Intelligence's White Guy Problem". New York Times. Retrieved 2018-08-20.
↑ Metz, Rachel. "Why Microsoft's teen chatbot, Tay, said lots of awful things online". MIT Technology Review. Retrieved 2018-08-20.
↑ Simonite, Tom. "Microsoft says its racist chatbot illustrates how AI isn't adaptable enough to help most businesses". MIT Technology Review. Retrieved 2018-08-20.
↑ "Why Uber's self-driving car killed a pedestrian". The Economist. Retrieved 2018-08-20.
↑ "IBM's Watson recommended 'unsafe and incorrect' cancer treatments - STAT". STAT. 2018-07-25. Retrieved 2018-08-21.
↑ Hernandez, Daniela; Greenwald, Ted (2018-08-11). "IBM Has a Watson Dilemma". Wall Street Journal. ISSN 0099-9660. Retrieved 2018-08-21.
↑ Pontius, Robert Gilmore; Si, Kangping (2014). "The total operating characteristic to measure diagnostic ability for multiple thresholds". International Journal of Geographical Information Science. 28 (3): 570–583. {{cite journal}}: Text "doi:10.1080/13658816.2013.862623" ignored (help)
↑ Bostrom, Nick (2011). "The Ethics of Artificial Intelligence" (PDF). Retrieved 11 April 2016.
↑ Edionwe, Tolulope. "The fight against racist algorithms". The Outline. Retrieved 17 November 2017.
↑ Jeffries, Adrianne. "Machine learning is racist because the internet is racist". The Outline. Retrieved 17 November 2017.
↑ Narayanan, Arvind (August 24, 2016). "Language necessarily contains human biases, and so will machines trained on language corpora". Freedom to Tinker.
↑ Char, D. S., Shah, N. H., & Magnus, D. (2018). Implementing Machine Learning in Health Care—Addressing Ethical Challenges.New England Journal of Medicine,378(11), 981-983. doi:10.1056/nejmp1714229

拓展阅读

Nils J. Nilsson, Introduction to Machine Learning.
Trevor Hastie, Robert Tibshirani and Jerome H. Friedman (2001). The Elements of Statistical Learning, Springer. ISBN 0-387-95284-5.
Pedro Domingos (September 2015), The Master Algorithm, Basic Books, ISBN 978-0-465-06570-7
Ian H. Witten and Eibe Frank (2011). Data Mining: Practical machine learning tools and techniques Morgan Kaufmann, 664pp., ISBN 978-0-12-374856-0.
Ethem Alpaydin (2004). Introduction to Machine Learning, MIT Press, ISBN 978-0-262-01243-0.
David J. C. MacKay. Information Theory, Inference, and Learning Algorithms Cambridge: Cambridge University Press, 2003. ISBN 0-521-64298-1
Richard O. Duda, Peter E. Hart, David G. Stork (2001) Pattern classification (2nd edition), Wiley, New York, ISBN 0-471-05669-3.
Christopher Bishop (1995). Neural Networks for Pattern Recognition, Oxford University Press. ISBN 0-19-853864-2.
Stuart Russell & Peter Norvig, (2002). Artificial Intelligence – A Modern Approach. Prentice Hall, ISBN 0-136-04259-7.
Ray Solomonoff, An Inductive Inference Machine, IRE Convention Record, Section on Information Theory, Part 2, pp., 56–62, 1957.
Ray Solomonoff, An Inductive Inference Machine A privately circulated report from the 1956 Dartmouth Summer Research Conference on AI.

外部链接

International Machine Learning Society
Popular online course by Andrew Ng, at Coursera. It uses GNU Octave. The course is a free version of Stanford University's actual course taught by Ng, whose lectures are also available for free.
mloss is an academic database of open-source machine learning software.
Machine Learning Crash Course by Google. This is a free course on machine learning through the use of TensorFlow.

编者推荐

集智相关课程

课程：当复杂网络遇上机器学习

本词条内容源自wikipedia及公开资料，遵守 CC3.0协议。

[2w-1] The "without being explicitly programmed" definition is often attributed to Arthur Samuel, who coined the term "machine learning" in 1959. But the phrase is not found literally in this publication, and may be a paraphrase that appeared later. Confer "Paraphrasing Arthur Samuel (1959), the question is: How can computers learn to solve problems without being explicitly programmed?" in Koza, John R.; Bennett, Forrest H.; Andre, David; Keane, Martin A. (1996). Automated Design of Both the Topology and Sizing of Analog Electrical Circuits Using Genetic Programming. Artificial Intelligence in Design '96. Springer, Dordrecht. pp. 151–170. doi:10.1007/978-94-009-0279-4_9

[Samuel-2] Samuel, Arthur (1959). "Some Studies in Machine Learning Using the Game of Checkers". IBM Journal of Research and Development. 3 (3): 210–229. CiteSeerX 10.1.1.368.2254. {{cite journal}}: External link in |title= (help)

[Britannica-3] ttp://www.britannica.com/EBchecked/topic/1116194/machine-learning

[4] Ron Kohavi; Foster Provost (1998). "Glossary of terms". Machine Learning. 30: 271–274.

[bishop2006-5] 5.0 ^5.1 ^5.2 ^5.3 ^5.4 Bishop, C. M. (2006), Pattern Recognition and Machine Learning, Springer, ISBN 978-0-387-31073-2

[6] Mannila Heikki(1996). Data mining: machine learning, statistics, and databases. Int'l Conf. Scientific and Statistical Database Management.IEEE Computer Society.

[7] Friedman, Jerome H. (1998). "Data Mining and Statistics: What's the connection?". Computing Science and Statistics. 29 (1): 3–9.

[8] "Machine Learning: What it is and why it matters". www.sas.com. Retrieved 2016-03-29.

[9] Mitchell, T. (1997). Machine Learning. McGraw Hill. pp. 2.

[10] Harnad, Stevan(2008), "The Annotation Game: On Turing (1950) on Computing, Machinery, and Intelligence", in Epstein, Robert; Peters, Grace, The Turing Test Sourcebook: Philosophical and Methodological Issues in the Quest for the Thinking Computer , Kluwer

[11] R. Kohavi and F. Provost, "Glossary of terms," Machine Learning, vol. 30, no. 2–3, pp. 271–274, 1998.

[12] Sarle Warren."Neural Networks and statistical models". 10.1.1.27.699

[aima-13] 13.0 ^13.1 ^13.2 ^13.3 ^13.4 ^13.5 ^13.6 ^13.7 Russell, Stuart; Norvig, Peter (2003) [1995]. Artificial Intelligence: A Modern Approach (2nd ed.). Prentice Hall. ISBN 978-0137903955. 引用错误：无效<ref>标签；name属性“aima”使用不同内容定义了多次

[changing-14] 14.0 ^14.1 ^14.2 ^14.3 Langley, Pat (2011). "The changing science of machine learning". Machine Learning. 82 (3): 275–279. {{cite journal}}: Text "doi : 10.1007/s10994-011-5242-y" ignored (help) 引用错误：无效<ref>标签；name属性“changing”使用不同内容定义了多次

[15] Le Roux, Nicolas; Bengio, Yoshua;Fitzgibbon, Andrew(2012). "Improving First and Second-Order Methods by Modeling Uncertainty Optimization for Machine Learning" In Sra, Suvrit; Nowozin, Sebastian; Wright, Stephen J. Optimization for Machine Learning. MIT Press. p. 404.

[mi_jordan_ama-16] 16.0 ^16.1 Michael I. Jordan (2014-09-10). "statistics and machine learning." reddit. Retrieved 2014-10-01.

[17] Cornell University Library. "Breiman: Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)". Retrieved 8 August 2015.

[islr-18] Gareth James; Daniela Witten; Trevor Hastie; Robert Tibshirani (2013). An Introduction to Statistical Learning. Springer. p. vii. http://www-bcf.usc.edu/~gareth/ISL/.

[19] 模板:Cite citeseerx

[elements-20] Ref name"elements" {{cite web {{cite web { cite web |url= https://course.elementsofai.com/ |url= https://course.elementsofai.com/ Https://course.elementsofai.com/ |title= The Elements of AI |title= The Elements of AI 人工智能的元素 |publisher= University of Helsinki |publisher= University of Helsinki 出版商赫尔辛基大学 |date = Dec 2019 |date = Dec 2019 2019年12月 |accessdate=7 April 2020}} |accessdate=7 April 2020}} 2020年4月7日}

[21] / ref {{cite web {{cite web { cite web |url= https://www.techworld.com/tech-innovation/satalia-ceo-no-one-is-doing-ai-optimisation-can-change-that-3775689/ |url= https://www.techworld.com/tech-innovation/satalia-ceo-no-one-is-doing-ai-optimisation-can-change-that-3775689/ Https://www.techworld.com/tech-innovation/satalia-ceo-no-one-is-doing-ai-optimisation-can-change-that-3775689/ |title= Satalia CEO Daniel Hulme has a plan to overcome the limitations of machine learning |title= Satalia CEO Daniel Hulme has a plan to overcome the limitations of machine learning 萨塔利亚公司首席执行官丹尼尔 · 休姆计划克服机器学习的局限性 |publisher= Techworld |publisher= Techworld 出版商 Techworld |date = October 2019 |date = October 2019 2019年10月 |accessdate=7 April 2020}} |accessdate=7 April 2020}} 2020年4月7日}

[Alpaydin2020-22] 引用错误：无效<ref>标签；未给name属性为Alpaydin2020的引用提供文字

[23] Le Roux, Nicolas; Bengio, Yoshua; Fitzgibbon, Andrew (2012). "Improving+First+and+Second-Order+Methods+by+Modeling+Uncertainty "Improving First and Second-Order Methods by Modeling Uncertainty". In Sra, Suvrit; Nowozin, Sebastian; Wright, Stephen J. (eds.). Optimization for Machine Learning. MIT Press. p. 404. ISBN 9780262016469.

[24] Mohri, Mehryar; Rostamizadeh, Afshin; Talwalkar, Ameet (2012). Foundations of Machine Learning. USA, Massachusetts: MIT Press. ISBN 9780262018258.

[alpaydin-25] 25.0 ^25.1 Alpaydin, Ethem (2010). Introduction to Machine Learning. London: The MIT Press. https://mitpress.mit.edu/books/introduction-machine-learning.

[26] Tillmann, A. M. (2015). "On the Computational Intractability of Exact and Approximate Dictionary Learning". IEEE Signal Processing Letters. 22 (1): 45–49. {{cite journal}}: Text "arxiv:1405.6664" ignored (help); Text "bibcode;2015ISPL...22...45T" ignored (help)

[27] Aharon, M, M Elad, and A Bruckstein. 2006. "K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation." Signal Processing, IEEE Transactions on 54 (11): 4311–4322

[28] Goldberg, David E.; Holland, John H. (1988). "Genetic algorithms and machine learning". Machine Learning. 3 (2): 95–99.

[29] Michie, D.; Spiegelhalter, D. J.; Taylor, C. C. (1994). Machine Learning, Neural and Statistical Classification. Ellis Horwood.

[30] Zhang, Jun; Zhan, Zhi-hui; Lin, Ying; Chen, Ni; Gong, Yue-jiao; Zhong, Jing-hui; Chung, Henry S.H.; Li, Yun; Shi, Yu-hui (2011). "Evolutionary Computation Meets Machine Learning: A Survey" (PDF). Computational Intelligence Magazine. 6 (4): 68–75.

[31] Russell, Stuart J.; Norvig, Peter (2010). Artificial Intelligence: A Modern Approach (Third ed.). Prentice Hall. ISBN 9780136042594.

[32] Mohri, Mehryar; Rostamizadeh, Afshin; Talwalkar, Ameet (2012). Foundations of Machine Learning. The MIT Press. ISBN 9780262018258.

[Mitchell-1997-33] 引用错误：无效<ref>标签；未给name属性为Mitchell-1997的引用提供文字

[34] Alpaydin, Ethem (2010). Introduction to Machine Learning. MIT Press. p. 9. ISBN 978-0-262-01243-0. https://books.google.com/books?id=7f5bBAAAQBAJ&printsec=frontcover#v=onepage&q=classification&f=false.

[JordanBishop2004-35] Jordan, Michael I.; Bishop, Christopher M. (2004). "Neural Networks". In Allen B. Tucker. Computer Science Handbook, Second Edition (Section VII: Intelligent Systems). Boca Raton, Florida: Chapman & Hall/CRC Press LLC. ISBN 978-1-58488-360-9.

[36] Alex Ratner; Stephen Bach; Paroma Varma; Chris. "Weak Supervision: The New Programming Paradigm for Machine Learning". hazyresearch.github.io. referencing work by many other members of Hazy Research. Retrieved 2019-06-06.

[37] van Otterlo, M.; Wiering, M. (2012). Reinforcement learning and markov decision processes. Adaptation, Learning, and Optimization. 12. pp. 3–42. doi:10.1007/978-3-642-27645-3_1. ISBN 978-3-642-27644-6.

[38] Bozinovski, S. (1982). "A self-learning system using secondary reinforcement" . In Trappl, Robert (ed.). Cybernetics and Systems Research: Proceedings of the Sixth European Meeting on Cybernetics and Systems Research. North Holland. pp. 397–402.
v
t
e
Inline tags
Attribution

{{Attribution needed}}

{{By whom}}

{{Weasel-inline}}

{{Which?}}

{{Who}}

Citation

{{Better source}}

{{Citation needed}}
{{Citation needed span}}

{{Reference necessary}}

{{Citation needed (lead)}}

{{Failed verification}}

{{Full}}

{{Primary source-inline}}

{{Request quotation}}

{{Retracted}}

{{Third-party-inline}}

Incomplete

{{Author missing}}

{{Author incomplete}}

{{Date missing}}

{{ISBN missing}}

{{Page needed}}

{{Publisher missing}}

{{Title incomplete}}

{{Year missing}}

{{Dead link}}

Content

{{Contradict-inline}}

{{Contradiction-inline}}

{{Dubious|talk page section name}}

{{Examples}}

{{Inconsistent}}

{{List fact}}

{{Lopsided}}

{{POV-statement}}

Date and place

{{Clarify timeframe}}

{{Quantify}}

{{Update after}}

{{Update-small}}

{{When}}

{{Where}}

{{Year needed}}

Wikification

{{Disambiguation needed}}

{{Pronunciation needed}}

Wording

{{Ambiguous}}

{{Awkward}}

{{Buzz}}

{{Clarify}}

{{Clarify span}}

{{Definition}}

{{Elucidate}}

{{Expand acronym}}

{{Technical-statement}}

{{Vague|optional message to be displayed on mouseover}}

{{Why?}}

Category
.

[39] v

[40] t

[41] e

[42] {{Attribution needed}}

[43] {{By whom}}

[44] {{Weasel-inline}}

[45] {{Which?}}

[46] {{Who}}

[47] {{Better source}}

[48] {{Citation needed}}
{{Citation needed span}}

{{Reference necessary}}

[49] {{Citation needed span}}

[50] {{Reference necessary}}

[51] {{Citation needed (lead)}}

[52] {{Failed verification}}

[53] {{Full}}

[54] {{Primary source-inline}}

[55] {{Request quotation}}

[56] {{Retracted}}

[57] {{Third-party-inline}}

[58] {{Author missing}}

[59] {{Author incomplete}}

[60] {{Date missing}}

[61] {{ISBN missing}}

[62] {{Page needed}}

[63] {{Publisher missing}}

[64] {{Title incomplete}}

[65] {{Year missing}}

[66] {{Dead link}}

[67] {{Contradict-inline}}

[68] {{Contradiction-inline}}

[69] {{Dubious|talk page section name}}

[70] {{Examples}}

[71] {{Inconsistent}}

[72] {{List fact}}

[73] {{Lopsided}}

[74] {{POV-statement}}

[75] {{Clarify timeframe}}

[76] {{Quantify}}

[77] {{Update after}}

[78] {{Update-small}}

[79] {{When}}

[80] {{Where}}

[81] {{Year needed}}

[82] {{Disambiguation needed}}

[83] {{Pronunciation needed}}

[84] {{Ambiguous}}

[85] {{Awkward}}

[86] {{Buzz}}

[87] {{Clarify}}

[88] {{Clarify span}}

[89] {{Definition}}

[90] {{Elucidate}}

[91] {{Expand acronym}}

[92] {{Technical-statement}}

[93] {{Vague|optional message to be displayed on mouseover}}

[94] {{Why?}}

[39] Bozinovski, Stevo (2014) "Modeling mechanisms of cognition-emotion interaction in artificial neural networks, since 1981." Procedia Computer Science p. 255-263

[40] Bozinovski, S. (2001) "Self-learning agents: A connectionist theory of emotion based on crossbar value judgment." Cybernetics and Systems 32(6) 637-667.

[41] Lu, Haiping; Plataniotis, K.N.; Venetsanopoulos, A.N. (2011). "A Survey of Multilinear Subspace Learning for Tensor Data" (PDF). Pattern Recognition. 44 (7): 1540–1551.

[42] Yoshua Bengio (2009). Learning Deep Architectures for AI. Now Publishers Inc.. pp. 1–3. https://books.google.com/books?id=cq5ewg7FniMC&pg=PA3.

[43] Nathan Srebro; Jason D. M. Rennie; Tommi S. Jaakkola (2004). Maximum-Margin Matrix Factorization. NIPS.

[44] Csurka, Gabriella; Dance, Christopher C.; Fan, Lixin; Willamowski, Jutta; Bray, Cédric (2004). Visual categorization with bags of keypoints (PDF). ECCV Workshop on Statistical Learning in Computer Vision.

[jurafsky-45] Daniel Jurafsky; James H. Martin (2009). Speech and Language Processing. Pearson Education International. pp. 145–146.

[46] Lu, Haiping; Plataniotis, K.N.; Venetsanopoulos, A.N. (2011). "A Survey of Multilinear Subspace Learning for Tensor Data" (PDF). Pattern Recognition. 44 (7): 1540–1551. doi:10.1016/j.patcog.2011.01.004.

[47] Yoshua Bengio (2009). Learning Deep Architectures for AI. Now Publishers Inc.. pp. 1–3. ISBN 978-1-60198-294-0. https://books.google.com/books?id=cq5ewg7FniMC&pg=PA3.

[48] Tillmann, A. M. (2015). "On the Computational Intractability of Exact and Approximate Dictionary Learning". IEEE Signal Processing Letters. 22 (1): 45–49. arXiv:1405.6664. Bibcode:2015ISPL...22...45T. doi:10.1109/LSP.2014.2345761.

[49] Aharon, M, M Elad, and A Bruckstein. 2006. "K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation." Signal Processing, IEEE Transactions on 54 (11): 4311–4322

[:0-50] Zimek, Arthur; Schubert, Erich (2017), "Outlier Detection", Encyclopedia of Database Systems (in English), Springer New York, pp. 1–5, doi:10.1007/978-1-4899-7993-3_80719-1, ISBN 9781489979933

[51] Hodge, V. J.; Austin, J. (2004). "A Survey of Outlier Detection Methodologies" (PDF). Artificial Intelligence Review. 22 (2): 85–126. CiteSeerX 10.1.1.318.4023. doi:10.1007/s10462-004-4304-y.

[52] Dokas, Paul; Ertoz, Levent; Kumar, Vipin; Lazarevic, Aleksandar; Srivastava, Jaideep; Tan, Pang-Ning (2002). "Data mining for network intrusion detection" (PDF). Proceedings NSF Workshop on Next Generation Data Mining.

[ChandolaSurvey-53] Chandola, V.; Banerjee, A.; Kumar, V. (2009). "Anomaly detection: A survey". ACM Computing Surveys. 41 (3): 1–58. doi:10.1145/1541880.1541882.

[54] Bassel, George W.; Glaab, Enrico; Marquez, Julietta; Holdsworth, Michael J.; Bacardit, Jaume (2011-09-01). "Functional Network Construction in Arabidopsis Using Rule-Based Machine Learning on Large-Scale Data Sets". The Plant Cell (in English). 23 (9): 3101–3116. {{cite journal}}: Text "doi:10.1105/tpc.111.088153" ignored (help); Text "issn:1532-298X" ignored (help); Text "pmc:3203449" ignored (help); Text "pmid:21896882" ignored (help)

[piatetsky-55] Piatetsky-Shapiro, Gregory (1991), Discovery, analysis, and presentation of strong rules, in Piatetsky-Shapiro, Gregory; and Frawley, William J.; eds., Knowledge Discovery in Databases, AAAI/MIT Press, Cambridge, MA.

[mining-56] Agrawal, R.; Imieliński, T.; Swami, A. (1993). "Mining association rules between sets of items in large databases". Proceedings of the 1993 ACM SIGMOD international conference on Management of data - SIGMOD '93. pp. 207. doi:10.1145/170035.170072. ISBN 978-0897915922.

[57] Urbanowicz, Ryan J.; Moore, Jason H. (2009-09-22). "Learning Classifier Systems: A Complete Introduction, Review, and Roadmap". Journal of Artificial Evolution and Applications (in English). 2009: 1–25. {{cite journal}}: Text "issn:1687-6229" ignored (help)

[58] Plotkin G.D. Automatic Methods of Inductive Inference, PhD thesis, University of Edinburgh, 1970.

[59] Shapiro, Ehud Y. Inductive inference of theories from facts, Research Report 192, Yale University, Department of Computer Science, 1981. Reprinted in J.-L. Lassez, G. Plotkin (Eds.), Computational Logic, The MIT Press, Cambridge, MA, 1991, pp. 199–254.

[60] Shapiro, Ehud Y. (1983). Algorithmic program debugging. Cambridge, Mass: MIT Press.
v
t
e
Inline tags
Attribution

{{Attribution needed}}

{{By whom}}

{{Weasel-inline}}

{{Which?}}

{{Who}}

Citation

{{Better source}}

{{Citation needed}}
{{Citation needed span}}

{{Reference necessary}}

{{Citation needed (lead)}}

{{Failed verification}}

{{Full}}

{{Primary source-inline}}

{{Request quotation}}

{{Retracted}}

{{Third-party-inline}}

Incomplete

{{Author missing}}

{{Author incomplete}}

{{Date missing}}

{{ISBN missing}}

{{Page needed}}

{{Publisher missing}}

{{Title incomplete}}

{{Year missing}}

{{Dead link}}

Content

{{Contradict-inline}}

{{Contradiction-inline}}

{{Dubious|talk page section name}}

{{Examples}}

{{Inconsistent}}

{{List fact}}

{{Lopsided}}

{{POV-statement}}

Date and place

{{Clarify timeframe}}

{{Quantify}}

{{Update after}}

{{Update-small}}

{{When}}

{{Where}}

{{Year needed}}

Wikification

{{Disambiguation needed}}

{{Pronunciation needed}}

Wording

{{Ambiguous}}

{{Awkward}}

{{Buzz}}

{{Clarify}}

{{Clarify span}}

{{Definition}}

{{Elucidate}}

{{Expand acronym}}

{{Technical-statement}}

{{Vague|optional message to be displayed on mouseover}}

{{Why?}}

Category

[117] v

[118] t

[119] e

[120] {{Attribution needed}}

[121] {{By whom}}

[122] {{Weasel-inline}}

[123] {{Which?}}

[124] {{Who}}

[125] {{Better source}}

[126] {{Citation needed}}
{{Citation needed span}}

{{Reference necessary}}

[127] {{Citation needed span}}

[128] {{Reference necessary}}

[129] {{Citation needed (lead)}}

[130] {{Failed verification}}

[131] {{Full}}

[132] {{Primary source-inline}}

[133] {{Request quotation}}

[134] {{Retracted}}

[135] {{Third-party-inline}}

[136] {{Author missing}}

[137] {{Author incomplete}}

[138] {{Date missing}}

[139] {{ISBN missing}}

[140] {{Page needed}}

[141] {{Publisher missing}}

[142] {{Title incomplete}}

[143] {{Year missing}}

[144] {{Dead link}}

[145] {{Contradict-inline}}

[146] {{Contradiction-inline}}

[147] {{Dubious|talk page section name}}

[148] {{Examples}}

[149] {{Inconsistent}}

[150] {{List fact}}

[151] {{Lopsided}}

[152] {{POV-statement}}

[153] {{Clarify timeframe}}

[154] {{Quantify}}

[155] {{Update after}}

[156] {{Update-small}}

[157] {{When}}

[158] {{Where}}

[159] {{Year needed}}

[160] {{Disambiguation needed}}

[161] {{Pronunciation needed}}

[162] {{Ambiguous}}

[163] {{Awkward}}

[164] {{Buzz}}

[165] {{Clarify}}

[166] {{Clarify span}}

[167] {{Definition}}

[168] {{Elucidate}}

[169] {{Expand acronym}}

[170] {{Technical-statement}}

[171] {{Vague|optional message to be displayed on mouseover}}

[172] {{Why?}}

[61] Shapiro, Ehud Y. "The model inference system." Proceedings of the 7th international joint conference on Artificial intelligence-Volume 2. Morgan Kaufmann Publishers Inc., 1981.

[62] Honglak Lee, Roger Grosse, Rajesh Ranganath, Andrew Y. Ng. "Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations" Proceedings of the 26th Annual International Conference on Machine Learning, 2009.

[CorinnaCortes-63] Cortes, Corinna; Vapnik, Vladimir N. (1995). "Support-vector networks". Machine Learning. 20 (3): 273–297. doi:10.1007/BF00994018.

[64] Stevenson, Christopher. "Tutorial: Polynomial Regression in Excel". facultystaff.richmond.edu. Retrieved 22 January 2017.

[65] Goldberg, David E.; Holland, John H. (1988). "Genetic algorithms and machine learning" (PDF). Machine Learning. 3 (2): 95–99. doi:10.1007/bf00113892.

[66] Michie, D.; Spiegelhalter, D. J.; Taylor, C. C. (1994). "Machine Learning, Neural and Statistical Classification". Ellis Horwood Series in Artificial Intelligence. Bibcode:1994mlns.book.....M.

[67] Zhang, Jun; Zhan, Zhi-hui; Lin, Ying; Chen, Ni; Gong, Yue-jiao; Zhong, Jing-hui; Chung, Henry S.H.; Li, Yun; Shi, Yu-hui (2011). "Evolutionary Computation Meets Machine Learning: A Survey". Computational Intelligence Magazine. 6 (4): 68–75. doi:10.1109/mci.2011.942584.

[68] "Federated Learning: Collaborative Machine Learning without Centralized Training Data". Google AI Blog (in English). Retrieved 2019-06-08.

[69] Bridge, James P., Sean B. Holden, and Lawrence C. Paulson. "Machine learning for first-order theorem proving." Journal of automated reasoning 53.2 (2014): 141–172.

[70] Loos, Sarah, et al. "Deep Network Guided Proof Search." arXiv preprint arXiv:1701.06972 (2017).

[71] Finnsson, Hilmar, and Yngvi Björnsson. "Simulation-Based Approach to General Game Playing." AAAI. Vol. 8. 2008.

[72] Sarikaya, Ruhi, Geoffrey E. Hinton, and Anoop Deoras. "Application of deep belief networks for natural language understanding." IEEE/ACM Transactions on Audio, Speech and Language Processing (TASLP) 22.4 (2014): 778–784.

[73] "AI-based translation to soon reach human levels: industry officials". Yonhap news agency. Retrieved 4 Mar 2017.

[74] "BelKor Home Page" research.att.com

[75] "The Netflix Tech Blog: Netflix Recommendations: Beyond the 5 stars (Part 1)". Retrieved 8 August 2015.

[76] Scott Patterson (13 July 2010). "Letting the Machines Decide". The Wall Street Journal. Retrieved 24 June 2018.

[77] Vonod Khosla (January 10, 2012). "Do We Need Doctors or Algorithms?". Tech Crunch.

[78] When A Machine Learning Algorithm Studied Fine Art Paintings, It Saw Things Art Historians Had Never Noticed, The Physics at ArXiv blog

[79] "Why Machine Learning Models Often Fail to Learn: QuickTake Q&A". Bloomberg.com. 2016-11-10. Retrieved 2017-04-10.

[80] "The First Wave of Corporate AI Is Doomed to Fail". Harvard Business Review. 2017-04-18. Retrieved 2018-08-20.

[81] "Why the A.I. euphoria is doomed to fail". VentureBeat. 2016-09-18. Retrieved 2018-08-20.

[82] Reasons why your machine learning project will fail "9 Reasons why your machine learning project will fail". www.kdnuggets.com. Retrieved 2018-08-20.

[83] Garcia, Megan (2016). "Racist in the Machine". World Policy Journal (in English). 33 (4): 111–117. {{cite journal}}: Text "issn:0740-2775" ignored (help)

[84] Caliskan, Aylin; Bryson, Joanna J.; Narayanan, Arvind (2017-04-14). "Semantics derived automatically from language corpora contain human-like biases". Science (in English). 356 (6334): 183–186. {{cite journal}}: Text "doi:10.1126/science.aal4230" ignored (help); Text "issn:0036-8075" ignored (help); Text "pmid:28408601" ignored (help)

[85] Wang, Xinan; Dasgupta, Sanjoy (2016), Lee, D. D.; Sugiyama, M.; Luxburg, U. V.; Guyon, I., eds., "An algorithm for L1 nearest neighbor search via monotonic embedding" (PDF), Advances in Neural Information Processing Systems 29, Curran Associates, Inc., pp. 983–991, Retrieved 2018-08-20

[86] "Machine Bias". ProPublica. Julia Angwin, Jeff Larson, Lauren Kirchner, Surya Mattu. 2016-05-23. Retrieved 2018-08-20.

[87] "Opinion | When an Algorithm Helps Send You to Prison". New York Times. Retrieved 2018-08-20.

[88] "Google apologises for racist blunder". BBC News. 2015-07-01. Retrieved 2018-08-20.

[89] "Google 'fixed' its racist algorithm by removing gorillas from its image-labeling tech". The Verge. Retrieved 2018-08-20.

[90] "Opinion | Artificial Intelligence's White Guy Problem". New York Times. Retrieved 2018-08-20.

[91] Metz, Rachel. "Why Microsoft's teen chatbot, Tay, said lots of awful things online". MIT Technology Review. Retrieved 2018-08-20.

[92] Simonite, Tom. "Microsoft says its racist chatbot illustrates how AI isn't adaptable enough to help most businesses". MIT Technology Review. Retrieved 2018-08-20.

[93] "Why Uber's self-driving car killed a pedestrian". The Economist. Retrieved 2018-08-20.

[94] "IBM's Watson recommended 'unsafe and incorrect' cancer treatments - STAT". STAT. 2018-07-25. Retrieved 2018-08-21.

[95] Hernandez, Daniela; Greenwald, Ted (2018-08-11). "IBM Has a Watson Dilemma". Wall Street Journal. ISSN 0099-9660. Retrieved 2018-08-21.

[96] Pontius, Robert Gilmore; Si, Kangping (2014). "The total operating characteristic to measure diagnostic ability for multiple thresholds". International Journal of Geographical Information Science. 28 (3): 570–583. {{cite journal}}: Text "doi:10.1080/13658816.2013.862623" ignored (help)

[97] Bostrom, Nick (2011). "The Ethics of Artificial Intelligence" (PDF). Retrieved 11 April 2016.

[Edionwe_Outline-98] Edionwe, Tolulope. "The fight against racist algorithms". The Outline. Retrieved 17 November 2017.

[Jeffries_Outline-99] Jeffries, Adrianne. "Machine learning is racist because the internet is racist". The Outline. Retrieved 17 November 2017.

[100] Narayanan, Arvind (August 24, 2016). "Language necessarily contains human biases, and so will machines trained on language corpora". Freedom to Tinker.

[101] Char, D. S., Shah, N. H., & Magnus, D. (2018). Implementing Machine Learning in Health Care—Addressing Ethical Challenges.New England Journal of Medicine,378(11), 981-983. doi:10.1056/nejmp1714229

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

[50]

[51]

[52]

[53]

[54]

[55]

[56]

[57]

[58]

[59]

[60]

[61]

[62]

[63]

[64]

[65]

[66]

[67]

[68]

[69]

[70]

[71]

[72]

[73]

[74]

[75]

[76]

[77]

[78]

[79]

[80]

[81]

[82]

[83]

[84]

[85]

[86]

[87]

[88]

[89]

[90]

[91]

[92]

[93]

[94]

[95]

[96]

[97]

[98]

[99]

[100]