2,323
个编辑
更改
跳到导航
跳到搜索
第831行：
第831行： − + − 可逆神经网络下的数值解以及随机迭代系统的解析解都可以给出计算方法。+ − 对于一般的神经网络，它可看做是从输入X到输出Y的一种不确定的函数映射： <math>+ − 其中x的维度为input_size，y的维度为output_size，[math]\xi[/math]为一高斯分布，其协方差矩阵为：[math]\Sigma=\mathrm{diag}(\sigma_1,\sigma_2,\cdots,\sigma_n)[/math]，这里，[math]\sigma_i[/math]为该神经网络第i个维度的均方误差（MSE）。该矩阵的逆记为sigmas_matrix。映射函数f记为func，则可以用下面的代码计算该神经网络的EI。该算法的基本思想是利用蒙特卡洛方法计算公式{{EquationNote|6}}的积分。+ + + + + − + + +
无编辑摘要
Note that the above conclusions all require: <math>\partial_{x}f(x)</math> is not 0, but for all <math>x</math>, when <math>\partial_{x}f(x)</math> is 0 everywhere, we obtain: <math>\begin{gathered}EI(f)\approx\end{gathered}0</math>. For more general cases, it is necessary to consider the situation where the matrix is not of rank, please refer to the effective information of neural networks.
Note that the above conclusions all require: <math>\partial_{x}f(x)</math> is not 0, but for all <math>x</math>, when <math>\partial_{x}f(x)</math> is 0 everywhere, we obtain: <math>\begin{gathered}EI(f)\approx\end{gathered}0</math>. For more general cases, it is necessary to consider the situation where the matrix is not of rank, please refer to the effective information of neural networks.
==连续系统EI的源代码==
==Source Code for EI in Continuous Systems==
The numerical solution under reversible neural networks and the analytical solution for stochastic iterative systems both provide methods for calculating the Effective Information (EI). For general neural networks, they can be considered uncertain function mappings from input X to output Y: y=f(x)+ξ
<math>
y=f(x)+\xi
y=f(x)+\xi
</math>
</math>
Where x has a dimension of <code>input_size</code>, y has a dimension of <code>output_size</code>, and ξ is a Gaussian distribution with a covariance matrix:
Σ=diag(σ1,σ2,⋯,σn)
Here, σi represents the mean squared error (MSE) for the i-th dimension of the neural network. The inverse of this matrix is denoted as <code>sigmas_matrix</code>. The mapping function f is represented by <code>func</code>. The following code can be used to calculate the EI for this neural network. The basic idea of the algorithm is to use the Monte Carlo method to calculate the integral in Equation 6.
*输入变量：
*输入变量：
神经网络的输入维度（input_size）输出维数（output_size）、输出变量视为高斯分布后的协方差矩阵的逆（sigmas_matrix）、映射函数（func）、干预区间大小（L的取值）、以及蒙特卡罗积分的样本数（num_samples）。
神经网络的输入维度（input_size）输出维数（output_size）、输出变量视为高斯分布后的协方差矩阵的逆（sigmas_matrix）、映射函数（func）、干预区间大小（L的取值）、以及蒙特卡罗积分的样本数（num_samples）。
*输出变量：
*输出变量：
维度平均EI（d_EI）、EI系数（eff）、EI值（EI）、确定性（term1）、简并性（term2）和[math]\ln L[/math]（-np.log(rho)）。<syntaxhighlight lang="python3">
维度平均EI（d_EI）、EI系数（eff）、EI值（EI）、确定性（term1）、简并性（term2）和[math]\ln L[/math]（-np.log(rho)）。
<syntaxhighlight lang="python3">
def approx_ei(input_size, output_size, sigmas_matrix, func, num_samples, L, easy=True, device=None):
def approx_ei(input_size, output_size, sigmas_matrix, func, num_samples, L, easy=True, device=None):