463
个编辑
更改
跳到导航
跳到搜索
第987行：
第987行： − 在许多应用程序中，需要最大化互信息（从而增加依赖关系） ，这通常相当于最小化条件熵。例如:+ − − 第1,015行：
第1,013行： − + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
→应用 Applications
In many applications, one wants to maximize mutual information (thus increasing dependencies), which is often equivalent to minimizing conditional entropy. Examples include:
In many applications, one wants to maximize mutual information (thus increasing dependencies), which is often equivalent to minimizing conditional entropy. Examples include:
在许多应用场景中，需要最大化互信息（从而增加依赖关系），这通常相当于最小化条件熵。例如:
* In [[search engine technology]], mutual information between phrases and contexts is used as a feature for [[k-means clustering]] to discover semantic clusters (concepts).<ref name=magerman>[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.78.4178&rep=rep1&type=pdf Parsing a Natural Language Using Mutual Information Statistics] by David M. Magerman and Mitchell P. Marcus</ref> For example, the mutual information of a bigram might be calculated as:
* In [[search engine technology]], mutual information between phrases and contexts is used as a feature for [[k-means clustering]] to discover semantic clusters (concepts).<ref name=magerman>[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.78.4178&rep=rep1&type=pdf Parsing a Natural Language Using Mutual Information Statistics] by David M. Magerman and Mitchell P. Marcus</ref> For example, the mutual information of a bigram might be calculated as:
{{Equation box 1
{{Equation box 1
where <math>f_{XY}</math> is the number of times the bigram xy appears in the corpus, <math>f_{X}</math> is the number of times the unigram x appears in the corpus, B is the total number of bigrams, and U is the total number of unigrams.
where <math>f_{XY}</math> is the number of times the bigram xy appears in the corpus, <math>f_{X}</math> is the number of times the unigram x appears in the corpus, B is the total number of bigrams, and U is the total number of unigrams.
其中 math f { XY } / math 是 bigram XY 在语料库中出现的次数，math f { x } / math 是 unigram x 在语料库中出现的次数，b 是 bigrams 的总数，u 是 unigrams 的总数。
其中<math>f_{XY}</math>是 bigram XY 在语料库中出现的次数，<math>f_{X}</math>是 unigram x 在语料库中出现的次数，b 是 bigrams 的总数，u 是 unigrams 的总数。
* In [[telecommunications]], the [[channel capacity]] is equal to the mutual information, maximized over all input distributions.
* In [[telecommunications]], the [[channel capacity]] is equal to the mutual information, maximized over all input distributions.
In telecommunications, the channel capacity is equal to the mutual information, maximized over all input distributions.
在所有的通信信道中，信息的最大化分配是在所有的通信信道上进行的。
* [[Discriminative model|Discriminative training]] procedures for [[hidden Markov model]]s have been proposed based on the [[maximum mutual information]] (MMI) criterion.
* [[Discriminative model|Discriminative training]] procedures for [[hidden Markov model]]s have been proposed based on the [[maximum mutual information]] (MMI) criterion.
Discriminative training procedures for hidden Markov models have been proposed based on the maximum mutual information (MMI) criterion.
现在已经提出了基于最大互信息（MMI）准则的隐马尔可夫模型判别训练方法。
* [[Nucleic acid secondary structure|RNA secondary structure]] prediction from a [[multiple sequence alignment]].
* [[Nucleic acid secondary structure|RNA secondary structure]] prediction from a [[multiple sequence alignment]].
RNA secondary structure prediction from a multiple sequence alignment.
从多序列比对预测RNA二级结构。
* [[Phylogenetic profiling]] prediction from pairwise present and disappearance of functionally link [[gene]]s.
* [[Phylogenetic profiling]] prediction from pairwise present and disappearance of functionally link [[gene]]s.
Phylogenetic profiling prediction from pairwise present and disappearance of functionally link genes.
功能连锁基因成对存在与消失的系统发育模式预测。
* Mutual information has been used as a criterion for [[feature selection]] and feature transformations in [[machine learning]]. It can be used to characterize both the relevance and redundancy of variables, such as the [[minimum redundancy feature selection]].
* Mutual information has been used as a criterion for [[feature selection]] and feature transformations in [[machine learning]]. It can be used to characterize both the relevance and redundancy of variables, such as the [[minimum redundancy feature selection]].
Mutual information has been used as a criterion for feature selection and feature transformations in machine learning. It can be used to characterize both the relevance and redundancy of variables, such as the minimum redundancy feature selection.
在机器学习中，互信息作为特征选择和特征转换的准则。它可以用来表征变量的相关性和冗余性，例如最小冗余特征选择。
* Mutual information is used in determining the similarity of two different [[cluster analysis|clusterings]] of a dataset. As such, it provides some advantages over the traditional [[Rand index]].
* Mutual information is used in determining the similarity of two different [[cluster analysis|clusterings]] of a dataset. As such, it provides some advantages over the traditional [[Rand index]].
Mutual information is used in determining the similarity of two different clusterings of a dataset. As such, it provides some advantages over the traditional Rand index.
互信息用于确定数据集中两个不同聚类的相似性。因此，它与传统的Rand指数相比具有一定的优势。
* Mutual information of words is often used as a significance function for the computation of [[collocation]]s in [[corpus linguistics]]. This has the added complexity that no word-instance is an instance to two different words; rather, one counts instances where 2 words occur adjacent or in close proximity; this slightly complicates the calculation, since the expected probability of one word occurring within <math>N</math> words of another, goes up with <math>N</math>.
* Mutual information of words is often used as a significance function for the computation of [[collocation]]s in [[corpus linguistics]]. This has the added complexity that no word-instance is an instance to two different words; rather, one counts instances where 2 words occur adjacent or in close proximity; this slightly complicates the calculation, since the expected probability of one word occurring within <math>N</math> words of another, goes up with <math>N</math>.
Mutual information of words is often used as a significance function for the computation of collocations in corpus linguistics. This has the added complexity that no word-instance is an instance to two different words; rather, one counts instances where 2 words occur adjacent or in close proximity; this slightly complicates the calculation, since the expected probability of one word occurring within 𝑁 words of another, goes up with 𝑁.
在语料库语言学中，词的互信息常常被用作计算搭配的意义函数。这增加了复杂性，即没有一个单词实例是两个不同单词的实例；相反，我们统计两个单词相邻或非常接近的实例；这稍微使计算复杂化，因为一个单词出现在另一个单词的𝑁单词中的预期概率会增加。
* Mutual information is used in [[medical imaging]] for [[image registration]]. Given a reference image (for example, a brain scan), and a second image which needs to be put into the same [[coordinate system]] as the reference image, this image is deformed until the mutual information between it and the reference image is maximized.
* Mutual information is used in [[medical imaging]] for [[image registration]]. Given a reference image (for example, a brain scan), and a second image which needs to be put into the same [[coordinate system]] as the reference image, this image is deformed until the mutual information between it and the reference image is maximized.
Mutual information is used in medical imaging for image registration. Given a reference image (for example, a brain scan), and a second image which needs to be put into the same coordinate system as the reference image, this image is deformed until the mutual information between it and the reference image is maximized.
在医学成像中，利用互信息进行图像配准。给定一个参考图像（例如，脑部扫描），以及需要将第二个图像放入与参考图像相同的坐标系中，该图像会发生变形，直到其与参考图像之间的互信息最大化。
* Detection of [[phase synchronization]] in [[time series]] analysis
* Detection of [[phase synchronization]] in [[time series]] analysis
Detection of phase synchronization in time series analysis
时间序列分析中的相位同步检测。
* In the [[infomax]] method for neural-net and other machine learning, including the infomax-based [[Independent component analysis]] algorithm
* In the [[infomax]] method for neural-net and other machine learning, including the infomax-based [[Independent component analysis]] algorithm
In the infomax method for neural-net and other machine learning, including the infomax-based Independent component analysis algorithm.
在infomax方法中用于神经网络等机器学习，包括基于infomax的独立分量分析算法
* Average mutual information in [[delay embedding theorem]] is used for determining the ''embedding delay'' parameter.
* Average mutual information in [[delay embedding theorem]] is used for determining the ''embedding delay'' parameter.
Average mutual information in delay embedding theorem is used for determining the embedding delay parameter.
利用延迟嵌入定理中的平均互信息确定嵌入延迟参数。
* Mutual information between [[genes]] in [[microarray|expression microarray]] data is used by the ARACNE algorithm for reconstruction of [[gene regulatory network|gene networks]].
* Mutual information between [[genes]] in [[microarray|expression microarray]] data is used by the ARACNE algorithm for reconstruction of [[gene regulatory network|gene networks]].
Mutual information between genes in expression microarray data is used by the ARACNE algorithm for reconstruction of gene networks.
ARACNE算法利用表达微阵列数据中基因间的互信息来重构基因网络。
* In [[statistical mechanics]], [[Loschmidt's paradox]] may be expressed in terms of mutual information.<ref name=everett56>[[Hugh Everett]] [https://www.pbs.org/wgbh/nova/manyworlds/pdf/dissertation.pdf Theory of the Universal Wavefunction], Thesis, Princeton University, (1956, 1973), pp 1–140 (page 30)</ref><ref name=everett57>{{cite journal | last1 = Everett | first1 = Hugh | authorlink = Hugh Everett | year = 1957 | title = Relative State Formulation of Quantum Mechanics | url = http://www.univer.omsk.su/omsk/Sci/Everett/paper1957.html | journal = Reviews of Modern Physics | volume = 29 | issue = 3 | pages = 454–462 | doi = 10.1103/revmodphys.29.454 | bibcode = 1957RvMP...29..454E | access-date = 2012-07-16 | archive-url = https://web.archive.org/web/20111027191052/http://www.univer.omsk.su/omsk/Sci/Everett/paper1957.html | archive-date = 2011-10-27 | url-status = dead }}</ref> Loschmidt noted that it must be impossible to determine a physical law which lacks [[time reversal symmetry]] (e.g. the [[second law of thermodynamics]]) only from physical laws which have this symmetry. He pointed out that the [[H-theorem]] of [[Boltzmann]] made the assumption that the velocities of particles in a gas were permanently uncorrelated, which removed the time symmetry inherent in the H-theorem. It can be shown that if a system is described by a probability density in [[phase space]], then [[Liouville's theorem (Hamiltonian)|Liouville's theorem]] implies that the joint information (negative of the joint entropy) of the distribution remains constant in time. The joint information is equal to the mutual information plus the sum of all the marginal information (negative of the marginal entropies) for each particle coordinate. Boltzmann's assumption amounts to ignoring the mutual information in the calculation of entropy, which yields the thermodynamic entropy (divided by Boltzmann's constant).
* In [[statistical mechanics]], [[Loschmidt's paradox]] may be expressed in terms of mutual information.<ref name=everett56>[[Hugh Everett]] [https://www.pbs.org/wgbh/nova/manyworlds/pdf/dissertation.pdf Theory of the Universal Wavefunction], Thesis, Princeton University, (1956, 1973), pp 1–140 (page 30)</ref><ref name=everett57>{{cite journal | last1 = Everett | first1 = Hugh | authorlink = Hugh Everett | year = 1957 | title = Relative State Formulation of Quantum Mechanics | url = http://www.univer.omsk.su/omsk/Sci/Everett/paper1957.html | journal = Reviews of Modern Physics | volume = 29 | issue = 3 | pages = 454–462 | doi = 10.1103/revmodphys.29.454 | bibcode = 1957RvMP...29..454E | access-date = 2012-07-16 | archive-url = https://web.archive.org/web/20111027191052/http://www.univer.omsk.su/omsk/Sci/Everett/paper1957.html | archive-date = 2011-10-27 | url-status = dead }}</ref> Loschmidt noted that it must be impossible to determine a physical law which lacks [[time reversal symmetry]] (e.g. the [[second law of thermodynamics]]) only from physical laws which have this symmetry. He pointed out that the [[H-theorem]] of [[Boltzmann]] made the assumption that the velocities of particles in a gas were permanently uncorrelated, which removed the time symmetry inherent in the H-theorem. It can be shown that if a system is described by a probability density in [[phase space]], then [[Liouville's theorem (Hamiltonian)|Liouville's theorem]] implies that the joint information (negative of the joint entropy) of the distribution remains constant in time. The joint information is equal to the mutual information plus the sum of all the marginal information (negative of the marginal entropies) for each particle coordinate. Boltzmann's assumption amounts to ignoring the mutual information in the calculation of entropy, which yields the thermodynamic entropy (divided by Boltzmann's constant).
In statistical mechanics, Loschmidt's paradox may be expressed in terms of mutual information.[27][28] Loschmidt noted that it must be impossible to determine a physical law which lacks time reversal symmetry (e.g. the second law of thermodynamics) only from physical laws which have this symmetry. He pointed out that the H-theorem of Boltzmann made the assumption that the velocities of particles in a gas were permanently uncorrelated, which removed the time symmetry inherent in the H-theorem. It can be shown that if a system is described by a probability density in phase space, then Liouville's theorem implies that the joint information (negative of the joint entropy) of the distribution remains constant in time. The joint information is equal to the mutual information plus the sum of all the marginal information (negative of the marginal entropies) for each particle coordinate. Boltzmann's assumption amounts to ignoring the mutual information in the calculation of entropy, which yields the thermodynamic entropy (divided by Boltzmann's constant).
在统计力学中，Loschmidt悖论可以用相互信息来表示。Loschmidt指出，只有从具有这种对称性的物理定律中确定缺乏时间反转对称性的物理定律（例如热力学第二定律）是不可能的。他指出，玻尔兹曼的H-定理假设气体中粒子的速度是永久不相关的，这就消除了H-定理固有的时间对称性。可以证明，如果系统在相空间中用概率密度来描述，那么Liouville定理意味着分布的联合信息（联合熵的负）在时间上保持不变。关节信息等于互信息加上每个粒子坐标的所有边缘信息（负的边缘熵）之和。玻尔兹曼的假设相当于在熵的计算中忽略了相互信息，从而得到了热力学熵（除以玻尔兹曼常数）。
* The mutual information is used to learn the structure of [[Bayesian network]]s/[[dynamic Bayesian network]]s, which is thought to explain the causal relationship between random variables, as exemplified by the GlobalMIT toolkit:<ref>{{Google Code|globalmit|GlobalMIT}}</ref> learning the globally optimal dynamic Bayesian network with the Mutual Information Test criterion.
* The mutual information is used to learn the structure of [[Bayesian network]]s/[[dynamic Bayesian network]]s, which is thought to explain the causal relationship between random variables, as exemplified by the GlobalMIT toolkit:<ref>{{Google Code|globalmit|GlobalMIT}}</ref> learning the globally optimal dynamic Bayesian network with the Mutual Information Test criterion.
The mutual information is used to learn the structure of Bayesian networks/dynamic Bayesian networks, which is thought to explain the causal relationship between random variables, as exemplified by the GlobalMIT toolkit:[29] learning the globally optimal dynamic Bayesian network with the Mutual Information Test criterion.
互信息用于学习贝叶斯网络/动态贝叶斯网络的结构，被认为是用来解释随机变量之间的因果关系，如GlobalMIT工具包[29]用互信息检验准则学习全局最优动态贝叶斯网络。
* Popular cost function in [[decision tree learning]].
* Popular cost function in [[decision tree learning]].
Popular cost function in decision tree learning.
作为决策树学习中常用的代价函数。
* The mutual information is used in [[cosmology]] to test the influence of large-scale environments on galaxy properties in the [[Galaxy Zoo]].
* The mutual information is used in [[cosmology]] to test the influence of large-scale environments on galaxy properties in the [[Galaxy Zoo]].
The mutual information is used in cosmology to test the influence of large-scale environments on galaxy properties in the Galaxy Zoo.
在'''<font color="#32CD32">星系 Galaxy Zoo</font>'''中，利用互信息在宇宙学中测试大尺度环境对星系性质的影响。
* The mutual information was used in [[Solar Physics]] to derive the solar [[differential rotation]] profile, a travel-time deviation map for sunspots, and a time–distance diagram from quiet-Sun measurements<ref>{{cite journal|last1=Keys|first1=Dustin|last2=Kholikov|first2=Shukur|last3=Pevtsov|first3=Alexei A.|title=Application of Mutual Information Methods in Time Distance Helioseismology|journal=Solar Physics|date=February 2015|volume=290|issue=3|pages=659–671|doi=10.1007/s11207-015-0650-y|arxiv=1501.05597|bibcode=2015SoPh..290..659K}}</ref>
* The mutual information was used in [[Solar Physics]] to derive the solar [[differential rotation]] profile, a travel-time deviation map for sunspots, and a time–distance diagram from quiet-Sun measurements<ref>{{cite journal|last1=Keys|first1=Dustin|last2=Kholikov|first2=Shukur|last3=Pevtsov|first3=Alexei A.|title=Application of Mutual Information Methods in Time Distance Helioseismology|journal=Solar Physics|date=February 2015|volume=290|issue=3|pages=659–671|doi=10.1007/s11207-015-0650-y|arxiv=1501.05597|bibcode=2015SoPh..290..659K}}</ref>
The mutual information was used in Solar Physics to derive the solar differential rotation profile, a travel-time deviation map for sunspots, and a time–distance diagram from quiet-Sun measurements.
在太阳物理学中，相互信息被用于推导太阳差分自转剖面图、太阳黑子的旅行时间偏差图和安静太阳测量的时间-距离图。
* Used in Invariant Information Clustering to automatically train neural network classifiers and image segmenters given no labelled data.<ref name=iic>[https://arxiv.org/abs/1807.06653 Invariant Information Clustering for Unsupervised Image Classification and Segmentation] by Xu Ji, Joao Henriques and Andrea Vedaldi</ref>
* Used in Invariant Information Clustering to automatically train neural network classifiers and image segmenters given no labelled data.<ref name=iic>[https://arxiv.org/abs/1807.06653 Invariant Information Clustering for Unsupervised Image Classification and Segmentation] by Xu Ji, Joao Henriques and Andrea Vedaldi</ref>
Used in Invariant Information Clustering to automatically train neural network classifiers and image segmenters given no labelled data.
用于不变信息聚类，在没有标记数据的情况下自动训练神经网络分类器和图像分割器。
== 参见 See also ==
== 参见 See also ==