更改

注意，在离散情况下，<math>H(X|X) = 0</math>，因此<math>H(X) = \operatorname{I}(X;X)</math>。所以，<math>\operatorname{I}(X; X) \ge \operatorname{I}(X; Y)</math>，据此我们可以得到一个基本原则，那就是一个变量至少包含与任何其他变量所能提供的关于自身的信息量的这么多信息。

注意，在离散情况下，<math>H(X|X) = 0</math>，因此<math>H(X) = \operatorname{I}(X;X)</math>。所以，<math>\operatorname{I}(X; X) \ge \operatorname{I}(X; Y)</math>，据此我们可以得到一个基本原则，那就是一个变量至少包含与任何其他变量所能提供的关于自身的信息量的这么多信息。

=== 与Kullback-Leibler散度的关系 Relation to Kullback–Leibler divergence ===

=== 与相对熵的关系 Relation to Kullback–Leibler divergence ===

For jointly discrete or jointly continuous pairs <math>(X,Y)</math>,  

For jointly discrete or jointly continuous pairs <math>(X,Y)</math>,  

For jointly discrete or jointly continuous pairs <math>(X,Y)</math>,  

For jointly discrete or jointly continuous pairs <math>(X,Y)</math>,  

对于联合离散或联合连续变量对（𝑋，𝑌）,

对于联合的离散或连续分布的变量对<math>(X,Y)</math>,

mutual information is the [[Kullback–Leibler divergence]] of the product of the [[marginal distribution]]s, <math>p_X \cdot p_Y</math>, from the [[joint distribution]] <math>p_{(X,Y)}</math>, that is,

mutual information is the [[Kullback–Leibler divergence]] of the product of the [[marginal distribution]]s, <math>p_X \cdot p_Y</math>, from the [[joint distribution]] <math>p_{(X,Y)}</math>, that is,