463
个编辑
更改
→动机 Motivation
Mutual information is a measure of the inherent dependence expressed in the joint distribution of 𝑋 and 𝑌 relative to the joint distribution of 𝑋 and 𝑌 under the assumption of independence. Mutual information therefore measures dependence in the following sense: I(𝑋;𝑌)=0 if and only if 𝑋 and 𝑌 are independent random variables. This is easy to see in one direction: if 𝑋 and 𝑌 are independent, then 𝑝(𝑋,𝑌)(𝑥,𝑦)=𝑝𝑋(𝑥)⋅𝑝𝑌(𝑦), and therefore:
Mutual information is a measure of the inherent dependence expressed in the joint distribution of 𝑋 and 𝑌 relative to the joint distribution of 𝑋 and 𝑌 under the assumption of independence. Mutual information therefore measures dependence in the following sense: I(𝑋;𝑌)=0 if and only if 𝑋 and 𝑌 are independent random variables. This is easy to see in one direction: if 𝑋 and 𝑌 are independent, then 𝑝(𝑋,𝑌)(𝑥,𝑦)=𝑝𝑋(𝑥)⋅𝑝𝑌(𝑦), and therefore:
互信息是在独立假设下，<math>X</math和<math>Y</math>的联合分布相对于<math>X</math和<math>Y</math>的联合分布表示的内在相关性的度量。因此互信息在以下意义上衡量相关性：<math>\operatorname{I}(X;Y)=0</math>当且仅当<math>X</math和<math>Y</math>是独立随机变量时。这很容易从一个方向看出：如果<math>X</math和<math>Y</math>是独立的，那么<math>p_{(X,Y)}(x,y)=p_X(x) \cdot p_Y(y)</math>，因此：
互信息是在独立假设下，<math>X</math和<math>Y</math>的联合分布相对于其内在相关性的度量。因此互信息是在以下条件下定义相关性的：<math>\operatorname{I}(X;Y)=0</math>当且仅当<math>X</math和<math>Y</math>是独立随机变量时。这很容易从一个方向看出：如果<math>X</math和<math>Y</math>是独立的，那么<math>p_{(X,Y)}(x,y)=p_X(x) \cdot p_Y(y)</math>，因此：
Moreover, mutual information is nonnegative (i.e. <math>\operatorname{I}(X;Y) \ge 0</math> see below) and symmetric (i.e. <math>\operatorname{I}(X;Y) = \operatorname{I}(Y;X)</math> see below).
Moreover, mutual information is nonnegative (i.e. <math>\operatorname{I}(X;Y) \ge 0</math> see below) and symmetric (i.e. <math>\operatorname{I}(X;Y) = \operatorname{I}(Y;X)</math> see below).
此外，互信息是非负的（例如：（<math>\operatorname{I}(X;Y) \ge 0</math>，见下文）和对称的（即<math>\operatorname{I}(X;Y) = \operatorname{I}(Y;X)</math>，见下文）。
== 与其他量的关系 Relation to other quantities ==
== 与其他量的关系 Relation to other quantities ==