463
个编辑
更改
→条件互信息 Conditional mutual information
{{Equation box 1
{{Equation box 1
|indent =
|indent =
|title=
|title=
|equation =
|equation =
<math>
<math>
\operatorname{I}(X;Y|Z) = \mathbb{E}_Z [D_{\mathrm{KL}}( P_{(X,Y)|Z} \| P_{X|Z} \otimes P_{Y|Z} )]
\operatorname{I}(X;Y|Z) = \mathbb{E}_Z [D_{\mathrm{KL}}( P_{(X,Y)|Z} \| P_{X|Z} \otimes P_{Y|Z} )]
</math>
</math>
|cellpadding= 1
|cellpadding= 1
|border
|border
|border colour = #0073CF
|border colour = #0073CF
|background colour=#F5FFFA}}
|background colour=#F5FFFA}}
:<math>
:<math>
\operatorname{I}(X;Y|Z) = \sum_{z\in \mathcal{Z}} \sum_{y\in \mathcal{Y}} \sum_{x\in \mathcal{X}}
\operatorname{I}(X;Y|Z) = \sum_{z\in \mathcal{Z}} \sum_{y\in \mathcal{Y}} \sum_{x\in \mathcal{X}}
{p_Z(z)\, p_{X,Y|Z}(x,y|z)
{p_Z(z)\, p_{X,Y|Z}(x,y|z)
\log\left[\frac{p_{X,Y|Z}(x,y|z)}{p_{X|Z}\,(x|z)p_{Y|Z}(y|z)}\right]},
\log\left[\frac{p_{X,Y|Z}(x,y|z)}{p_{X|Z}\,(x|z)p_{Y|Z}(y|z)}\right]},
</math>
</math>
which can be simplified as
which can be simplified as
:<math>
:<math>
\operatorname{I}(X;Y|Z) = \sum_{z\in \mathcal{Z}} \sum_{y\in \mathcal{Y}} \sum_{x\in \mathcal{X}}
\operatorname{I}(X;Y|Z) = \sum_{z\in \mathcal{Z}} \sum_{y\in \mathcal{Y}} \sum_{x\in \mathcal{X}}
p_{X,Y,Z}(x,y,z) \log \frac{p_{X,Y,Z}(x,y,z)p_{Z}(z)}{p_{X,Z}(x,z)p_{Y,Z}(y,z)}.
p_{X,Y,Z}(x,y,z) \log \frac{p_{X,Y,Z}(x,y,z)p_{Z}(z)}{p_{X,Z}(x,z)p_{Y,Z}(y,z)}.
</math>
</math>
:<math>
:<math>
\operatorname{I}(X;Y|Z) = \int_{\mathcal{Z}} \int_{\mathcal{Y}} \int_{\mathcal{X}}
\operatorname{I}(X;Y|Z) = \int_{\mathcal{Z}} \int_{\mathcal{Y}} \int_{\mathcal{X}}
{p_Z(z)\, p_{X,Y|Z}(x,y|z)
{p_Z(z)\, p_{X,Y|Z}(x,y|z)
\log\left[\frac{p_{X,Y|Z}(x,y|z)}{p_{X|Z}\,(x|z)p_{Y|Z}(y|z)}\right]} dx dy dz,
\log\left[\frac{p_{X,Y|Z}(x,y|z)}{p_{X|Z}\,(x|z)p_{Y|Z}(y|z)}\right]} dx dy dz,
</math>
</math>
which can be simplified as
which can be simplified as
:<math>
:<math>
\operatorname{I}(X;Y|Z) = \int_{\mathcal{Z}} \int_{\mathcal{Y}} \int_{\mathcal{X}}
\operatorname{I}(X;Y|Z) = \int_{\mathcal{Z}} \int_{\mathcal{Y}} \int_{\mathcal{X}}
p_{X,Y,Z}(x,y,z) \log \frac{p_{X,Y,Z}(x,y,z)p_{Z}(z)}{p_{X,Z}(x,z)p_{Y,Z}(y,z)} dx dy dz.
p_{X,Y,Z}(x,y,z) \log \frac{p_{X,Y,Z}(x,y,z)p_{Z}(z)}{p_{X,Z}(x,z)p_{Y,Z}(y,z)} dx dy dz.
</math>
</math>
:<math>\operatorname{I}(X;Y|Z) \ge 0</math>
:<math>\operatorname{I}(X;Y|Z) \ge 0</math>
for discrete, jointly distributed random variables <math>X,Y,Z</math>. This result has been used as a basic building block for proving other [[inequalities in information theory]].
for discrete, jointly distributed random variables <math>X,Y,Z</math>. This result has been used as a basic building block for proving other [[inequalities in information theory]].
为离散，联合分布的随机变量数学 x，y，z / math。这个结果已被用作证明信息论中其他不等式的基本构件。
为离散，联合分布的随机变量数学 x，y，z / math。这个结果已被用作证明信息论中其他不等式的基本构件。
=== 多元互信息 Multivariate mutual information ===
=== 多元互信息 Multivariate mutual information ===