更改

Notice, as per property of the [[Kullback–Leibler divergence]], that <math>I(X;Y)</math> is equal to zero precisely when the joint distribution coincides with the product of the marginals, i.e. when <math>X</math> and <math>Y</math> are independent (and hence observing <math>Y</math> tells you nothing about <math>X</math>). '''<font color="#32CD32">In general <math>I(X;Y)</math> is non-negative, it is a measure of the price for encoding <math>(X,Y)</math> as a pair of independent random variables, when in reality they are not.</font>'''

Notice, as per property of the [[Kullback–Leibler divergence]], that <math>I(X;Y)</math> is equal to zero precisely when the joint distribution coincides with the product of the marginals, i.e. when <math>X</math> and <math>Y</math> are independent (and hence observing <math>Y</math> tells you nothing about <math>X</math>). In general <math>I(X;Y)</math> is non-negative, it is a measure of the price for encoding <math>(X,Y)</math> as a pair of independent random variables, when in reality they are not.

需要注意的是，根据KL散度的性质，当两个随机变量的联合分布与其分别的边缘分布的乘积相等时，如当<math>X</math>和<math>Y</math>是相互独立时（因此观察y不会得到x的信息），<math>I(X;Y)</math>等于零（因此已知<math>Y</math>的信息并不能得到任何关于<math>X</math>的信息）。'''<font color="#32CD32">一般来说，<math>I(X;Y)</math>是非负的，因为它是将<math>(X,Y)</math>作为一对独立随机变量来编码进而进行价值度量的，但实际上它们并不一定是非负的。</font>'''

需要注意的是，根据KL散度的性质，当两个随机变量的联合分布与其分别的边缘分布的乘积相等时，如当<math>X</math>和<math>Y</math>是相互独立时（因此观察y不会得到x的信息），<math>I(X;Y)</math>等于零（因此已知<math>Y</math>的信息并不能得到任何关于<math>X</math>的信息）。一般来说，<math>I(X;Y)</math>是非负的，因为它是将<math>(X,Y)</math>作为一对独立随机变量来编码进而进行价值度量的，但实际上它们并不一定是非负的。

== 关于离散分布的PMF In terms of PMFs for discrete distributions ==

== 关于离散分布的PMF In terms of PMFs for discrete distributions ==