更改

[[文件:MI pic4.png|居中|缩略图|MI_pic4]]

[[文件:MI pic4.png|居中|缩略图]]

where <math>\Eta(X)</math> and <math>\Eta(Y)</math> are the marginal entropies, <math>\Eta(X|Y)</math> and <math>\Eta(Y|X)</math> are the conditional entropies, and <math>\Eta(X,Y)</math> is the joint entropy of <math>X</math> and <math>Y</math>.

where <math>\Eta(X)</math> and <math>\Eta(Y)</math> are the marginal entropies, <math>\Eta(X|Y)</math> and <math>\Eta(Y|X)</math> are the conditional entropies, and <math>\Eta(X,Y)</math> is the joint entropy of <math>X</math> and <math>Y</math>.

其中 math  Eta (x) / math  Eta (y) / math 是边际熵，math  Eta (x | y) / math  Eta (y | x) / math  Eta (y | x) / math 是条件熵，math Eta (x，y) / math 是数学 x / math 和数学 y / math 的联合熵。

其中H（X）和H（Y）是边际熵，H（X | Y）和H（Y | X）是条件熵，H（X，Y）是<math>X</math>和<math>Y</math>的联合熵。

Notice the analogy to the union, difference, and intersection of two sets: in this respect, all the formulas given above are apparent from the Venn diagram reported at the beginning of the article.

Notice the analogy to the union, difference, and intersection of two sets: in this respect, all the formulas given above are apparent from the Venn diagram reported at the beginning of the article.

 

In terms of a communication channel in which the output <math>Y</math> is a noisy version of the input <math>X</math>, these relations are summarised in the figure:

In terms of a communication channel in which the output <math>Y</math> is a noisy version of the input <math>X</math>, these relations are summarised in the figure:

在通信通道中，输出数学 y / math 是输入数学 x / math 的噪声版本，这些关系如图所示:

就输出<math>Y</math>是输入<math>X</math>的噪声版本的通信信道而言，这些关系如图中总结所示：

 

 

 

Because <math>\operatorname{I}(X;Y)</math> is non-negative, consequently, <math>\Eta(X) \ge \Eta(X|Y)</math>. Here we give the detailed deduction of <math>\operatorname{I}(X;Y)=\Eta(Y)-\Eta(Y|X)</math> for the case of jointly discrete random variables:

Because <math>\operatorname{I}(X;Y)</math> is non-negative, consequently, <math>\Eta(X) \ge \Eta(X|Y)</math>. Here we give the detailed deduction of <math>\operatorname{I}(X;Y)=\Eta(Y)-\Eta(Y|X)</math> for the case of jointly discrete random variables:

因为I（X；Y）是非负的，因此H（X）>=H（X|Y）。这里我们给出了联合离散随机变量情形下结论I（X;Y）=H（Y）-H（Y | X）的详细推导过程：

 

 

 

 

 

:<math>

 

 

 

 

 

 

\operatorname{I}(X;Y) & {} = \sum_{x \in \mathcal{X},y \in \mathcal{Y}} p_{(X,Y)}(x,y) \log \frac{p_{(X,Y)}(x,y)}{p_X(x)p_Y(y)}\\

 

 

运算符名称{ i }(x; y) & { sum { in mathcal { x，y } p {(x，y)}(x，y) log frac { p (x，y)}(x，y)}{ p x (x) p (x) y (y)}

 

 

 

 

 

 

 

 

 

 

 

& {} = \sum_{x \in \mathcal{X}} p_X(x) \left(\sum_{y \in \mathcal{Y}} p_{Y|X=x}(y) \log p_{Y|X=x}(y)\right) - \sum_{y \in \mathcal{Y}} \left(\sum_x p_{(X,Y)}(x,y)\right) \log p_Y(y) \\

 

 

 

 

 

 

 

 

 

 

 

埃塔(y)-埃塔(y | x)。\\

 

 

The proofs of the other identities above are similar. The proof of the general case (not just discrete) is similar, with integrals replacing sums.

The proofs of the other identities above are similar. The proof of the general case (not just discrete) is similar, with integrals replacing sums.

The proofs of the other identities above are similar. The proof of the general case (not just discrete) is similar, with integrals replacing sums.

The proofs of the other identities above are similar. The proof of the general case (not just discrete) is similar, with integrals replacing sums.

上述其他恒等式的证明是相似的。一般情况(不仅仅是离散情况)的证明是类似的，用积分代替和。

同理，上述其他恒等式的证明方法都是是相似的。一般情况(不仅仅是离散情况)的证明是类似的，用积分代替求。

Intuitively, if entropy <math>\Eta(Y)</math> is regarded as a measure of uncertainty about a random variable, then <math>\Eta(Y|X)</math> is a measure of what <math>X</math> does not say about <math>Y</math>. This is "the amount of uncertainty remaining about <math>Y</math> after <math>X</math> is known", and thus the right side of the second of these equalities can be read as "the amount of uncertainty in <math>Y</math>, minus the amount of uncertainty in <math>Y</math> which remains after <math>X</math> is known", which is equivalent to "the amount of uncertainty in <math>Y</math> which is removed by knowing <math>X</math>". This corroborates the intuitive meaning of mutual information as the amount of information (that is, reduction in uncertainty) that knowing either variable provides about the other.

Intuitively, if entropy <math>\Eta(Y)</math> is regarded as a measure of uncertainty about a random variable, then <math>\Eta(Y|X)</math> is a measure of what <math>X</math> does not say about <math>Y</math>. This is "the amount of uncertainty remaining about <math>Y</math> after <math>X</math> is known", and thus the right side of the second of these equalities can be read as "the amount of uncertainty in <math>Y</math>, minus the amount of uncertainty in <math>Y</math> which remains after <math>X</math> is known", which is equivalent to "the amount of uncertainty in <math>Y</math> which is removed by knowing <math>X</math>". This corroborates the intuitive meaning of mutual information as the amount of information (that is, reduction in uncertainty) that knowing either variable provides about the other.

直观地说，如果熵的数学 Eta (y) / 数学被看作是对一个随机变量的不确定性的度量，那么 math (y | x) / math 是对数学 x / math 没有对数学 y / math 进行说明的度量。这是“数学 x / 数学已知后数学 y / 数学剩余的不确定性量” ，因此，第二个等式的右边可以被解读为“数学 y / 数学中的不确定性量，减去数学 y / 数学中的不确定性量，在数学 x / 数学已知后仍然存在的不确定性量” ，这相当于“数学 y / 数学中的不确定性量，通过知道数学 x / 数学而去除”。这证实了互信息的直观含义，即知道任何一个变量提供的关于另一个变量的信息量(即不确定性的减少)。

直观地说，如果熵H (Y)被看作是对一个随机变量的不确定性的度量，那么<math>H(Y|X)</math>是对 <math>X</math>没有对<math>Y</math>进行说明的度量。这是“<math>X</math>已知后<math>Y</math>剩余的不确定性量” ，因此，第二个等式的右边可以被解读为“数学 y / 数学中的不确定性量，减去数学 y / 数学中的不确定性量，在数学 x / 数学已知后仍然存在的不确定性量” ，这相当于“数学 y / 数学中的不确定性量，通过知道数学 x / 数学而去除”。这证实了互信息的直观含义，即知道任何一个变量提供的关于另一个变量的信息量(即不确定性的减少)。