更改

Transfer entropy is a non-parametric statistic measuring the amount of directed (time-asymmetric) transfer of information between two random processes. Transfer entropy from a process X to another process Y is the amount of uncertainty reduced in future values of Y  by knowing the past values of X given past values of Y. More specifically, if  <math> X_t </math>  and  <math> Y_t </math>  for  <math> t\in \mathbb{N} </math>  denote two random processes and the amount of information is measured using Shannon's entropy, the transfer entropy can be written as:

Transfer entropy is a non-parametric statistic measuring the amount of directed (time-asymmetric) transfer of information between two random processes. Transfer entropy from a process X to another process Y is the amount of uncertainty reduced in future values of Y  by knowing the past values of X given past values of Y. More specifically, if  <math> X_t </math>  and  <math> Y_t </math>  for  <math> t\in \mathbb{N} </math>  denote two random processes and the amount of information is measured using Shannon's entropy, the transfer entropy can be written as:

<font color="#ff8000"> 转移熵 Transfer entropy</font>（也可译为<font color="#ff8000">传递熵</font>）是衡量两个随机过程之间有向（时间不对称）信息传递量的非参数统计量。过程X到过程Y的转移熵是指在给定过去值Y得到过去值X时，Y值不确定性的减少量。更具体地，如果Xt和Yt（t∈N）表示两个随机过程，且信息量用<font color="#ff8000"> 香农熵 Shannon entropy</font>测量，则转移熵可以写为：  

<font color="#ff8000"> 转移熵 Transfer entropy</font>（也可译为<font color="#ff8000">传递熵</font>）是衡量两个随机过程之间有向（时间不对称）信息传递量的非参数统计量。<ref>{{cite journal|last=Schreiber|first=Thomas|title=Measuring information transfer|journal=Physical Review Letters|date=1 July 2000|volume=85|issue=2|pages=461–464|doi=10.1103/PhysRevLett.85.461|pmid=10991308|arxiv=nlin/0001042|bibcode=2000PhRvL..85..461S}}</ref><ref name=Scholarpedia >{{cite encyclopedia |year= 2007 |title = Granger causality |volume = 2 |issue = 7 |pages = 1667 |last= Seth |first=Anil|encyclopedia=[[Scholarpedia]] |url=http://www.scholarpedia.org/article/Granger_causality|doi=10.4249/scholarpedia.1667 |bibcode=2007SchpJ...2.1667S|doi-access= free }}</ref><ref name=Schindler07>{{cite journal|last=Hlaváčková-Schindler|first=Katerina|author2=Palus, M |author3=Vejmelka, M |author4= Bhattacharya, J |title=Causality detection based on information-theoretic approaches in time series analysis|journal=Physics Reports|date=1 March 2007|volume=441|issue=1|pages=1–46|doi=10.1016/j.physrep.2006.12.004|bibcode=2007PhR...441....1H|citeseerx=10.1.1.183.1617}}</ref>过程X到过程Y的转移熵是指在给定过去值Y得到过去值X时，Y值不确定性的减少量。更具体地，如果Xt和Yt（t∈N）表示两个随机过程，且信息量用<font color="#ff8000"> 香农熵 Shannon entropy</font>测量，则转移熵可以写为：  

where H(X) is Shannon entropy of X. The above definition of transfer entropy has been extended by other types of entropy measures such as Rényi entropy.

where H(X) is Shannon entropy of X. The above definition of transfer entropy has been extended by other types of entropy measures such as Rényi entropy.

其中 H (x)是 x 的香农熵。上述转移熵的定义已被其他类型的熵测度(如Rényi熵)所扩展。

其中 H (x)是 x 的香农熵。上述转移熵的定义已被其他类型的熵测度(如<font color="#ff8000"> Rényi熵 Rényi entropy</font>)所扩展。<ref name ="  Schindler07"/><ref>{{Cite journal|last=Jizba|first=Petr|last2=Kleinert|first2=Hagen|last3=Shefaat|first3=Mohammad|date=2012-05-15|title=Rényi's information transfer between financial time series|journal=Physica A: Statistical Mechanics and Its Applications|language=en|volume=391|issue=10|pages=2971–2989|doi=10.1016/j.physa.2011.12.064|issn=0378-4371|arxiv=1106.5913|bibcode=2012PhyA..391.2971J}}</ref>

Transfer entropy is [[conditional mutual information]],<ref name = Wyner1978>{{cite journal|last=Wyner|first=A. D. |title=A definition of conditional mutual information for arbitrary ensembles|journal=Information and Control|year=1978|volume=38|issue=1|pages=51–59|doi=10.1016/s0019-9958(78)90026-8|doi-access=free}}</ref><ref name = Dobrushin1959>{{cite journal|last=Dobrushin|first=R. L. |title=General formulation of Shannon's main theorem in information theory|journal=Uspekhi Mat. Nauk|year=1959|volume=14|pages=3–104}}</ref> with the history of the influenced variable <math>Y_{t-1:t-L}</math> in the condition:

Transfer entropy is [[conditional mutual information]],<ref name = Wyner1978>{{cite journal|last=Wyner|first=A. D. |title=A definition of conditional mutual information for arbitrary ensembles|journal=Information and Control|year=1978|volume=38|issue=1|pages=51–59|doi=10.1016/s0019-9958(78)90026-8|doi-access=free}}</ref><ref name = Dobrushin1959>{{cite journal|last=Dobrushin|first=R. L. |title=General formulation of Shannon's main theorem in information theory|journal=Uspekhi Mat. Nauk|year=1959|volume=14|pages=3–104}}</ref> with the history of the influenced variable <math>Y_{t-1:t-L}</math> in the condition:

Transfer entropy is conditional mutual information, with the history of the influenced variable <math>Y_{t-1:t-L}</math> in the condition:

Transfer entropy is conditional mutual information,<ref name = Wyner1978>{{cite journal|last=Wyner|first=A. D. |title=A definition of conditional mutual information for arbitrary ensembles|journal=Information and Control|year=1978|volume=38|issue=1|pages=51–59|doi=10.1016/s0019-9958(78)90026-8|doi-access=free}}</ref><ref name = Dobrushin1959>{{cite journal|last=Dobrushin|first=R. L. |title=General formulation of Shannon's main theorem in information theory|journal=Uspekhi Mat. Nauk|year=1959|volume=14|pages=3–104}}</ref> with the history of the influenced variable <math>Y_{t-1:t-L}</math> in the condition:

转移熵是条件互信息，其历史变量为 Yt−1:t−L:

转移熵是<font color="#ff8000">条件互信息 conditional mutual information</font>，其历史变量为 Yt−1:t−L:

Transfer entropy reduces to Granger causality for vector auto-regressive processes. Hence, it is advantageous when the model assumption of Granger causality doesn't hold, for example, analysis of non-linear signals. However, it usually requires more samples for accurate estimation.

Transfer entropy reduces to Granger causality for vector auto-regressive processes. Hence, it is advantageous when the model assumption of Granger causality doesn't hold, for example, analysis of non-linear signals. However, it usually requires more samples for accurate estimation.

对于向量自回归过程，转移熵简化为<font color="#ff8000"> 格兰杰因果关系 Granger causality</font>。因此，当格兰杰因果关系的模型假设不成立时，例如对非线性信号的分析时，转移熵就更具优势。然而，它通常需要更多的样本才能进行准确估计 。

对于<font color="#ff8000">向量自回归过程 vector auto-regressive processes</font>，转移熵简化为<font color="#ff8000"> 格兰杰因果关系 Granger causality</font>。<ref name=Equal>{{cite journal|last=Barnett|first=Lionel|title=Granger Causality and Transfer Entropy Are Equivalent for Gaussian Variables|journal=Physical Review Letters|date=1 December 2009|volume=103|issue=23|doi=10.1103/PhysRevLett.103.238701|bibcode=2009PhRvL.103w8701B|pmid=20366183|page=238701|arxiv=0910.4514}}</ref>因此，当格兰杰因果关系的模型假设不成立时，例如对非线性信号的分析时，转移熵就更具优势。<ref name=Greg/><ref>{{cite journal|last=Lungarella|first=M.|author2=Ishiguro, K. |author3=Kuniyoshi, Y. |author4= Otsu, N. |title=Methods for quantifying the causal structure of bivariate time series|journal=International Journal of Bifurcation and Chaos|date=1 March 2007|volume=17|issue=3|pages=903–921|doi=10.1142/S0218127407017628|bibcode=2007IJBC...17..903L|citeseerx=10.1.1.67.3585}}</ref>然而，它通常需要更多的样本才能进行准确估计 。

The probabilities in the entropy formula can be estimated using different approaches (binning, nearest neighbors) or, in order to reduce complexity, using a non-uniform embedding.<ref>{{cite journal|last=Montalto|first=A|author2=Faes, L |author3=Marinazzo, D |title=MuTE: A MATLAB Toolbox to Compare Established and Novel Estimators of the Multivariate Transfer Entropy.|journal=PLOS ONE|date=Oct 2014|pmid=25314003|doi=10.1371/journal.pone.0109462|volume=9|issue=10|pmc=4196918|page=e109462|bibcode=2014PLoSO...9j9462M}}</ref>

The probabilities in the entropy formula can be estimated using different approaches (binning, nearest neighbors) or, in order to reduce complexity, using a non-uniform embedding.<ref>{{cite journal|last=Montalto|first=A|author2=Faes, L |author3=Marinazzo, D |title=MuTE: A MATLAB Toolbox to Compare Established and Novel Estimators of the Multivariate Transfer Entropy.|journal=PLOS ONE|date=Oct 2014|pmid=25314003|doi=10.1371/journal.pone.0109462|volume=9|issue=10|pmc=4196918|page=e109462|bibcode=2014PLoSO...9j9462M}}</ref>

The probabilities in the entropy formula can be estimated using different approaches (binning, nearest neighbors) or, in order to reduce complexity, using a non-uniform embedding.

The probabilities in the entropy formula can be estimated using different approaches (binning, nearest neighbors) or, in order to reduce complexity, using a non-uniform embedding.

While it was originally defined for [[bivariate analysis]], transfer entropy has been extended to [[Multivariate analysis|multivariate]] forms, either conditioning on other potential source variables<ref>{{cite journal|last=Lizier|first=Joseph|author2=Prokopenko, Mikhail |author3=Zomaya, Albert |title=Local information transfer as a spatiotemporal filter for complex systems|journal=Physical Review E|year=2008|volume=77|issue=2|pages=026110|doi=10.1103/PhysRevE.77.026110|pmid=18352093|arxiv=0809.3275|bibcode=2008PhRvE..77b6110L}}</ref> or considering transfer from a collection of sources,<ref name = Lizier2011>{{cite journal|last=Lizier|first=Joseph|author2=Heinzle, Jakob |author3=Horstmann, Annette |author4=Haynes, John-Dylan |author5= Prokopenko, Mikhail |title=Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity|journal=Journal of Computational Neuroscience|year=2011|volume=30|issue=1|pages=85–107|doi=10.1007/s10827-010-0271-2|pmid=20799057}}</ref> although these forms require more samples again.

While it was originally defined for [[bivariate analysis]], transfer entropy has been extended to [[Multivariate analysis|multivariate]] forms, either conditioning on other potential source variables<ref>{{cite journal|last=Lizier|first=Joseph|author2=Prokopenko, Mikhail |author3=Zomaya, Albert |title=Local information transfer as a spatiotemporal filter for complex systems|journal=Physical Review E|year=2008|volume=77|issue=2|pages=026110|doi=10.1103/PhysRevE.77.026110|pmid=18352093|arxiv=0809.3275|bibcode=2008PhRvE..77b6110L}}</ref> or considering transfer from a collection of sources,<ref name = Lizier2011>{{cite journal|last=Lizier|first=Joseph|author2=Heinzle, Jakob |author3=Horstmann, Annette |author4=Haynes, John-Dylan |author5= Prokopenko, Mikhail |title=Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity|journal=Journal of Computational Neuroscience|year=2011|volume=30|issue=1|pages=85–107|doi=10.1007/s10827-010-0271-2|pmid=20799057}}</ref> although these forms require more samples again.

While it was originally defined for bivariate analysis, transfer entropy has been extended to multivariate forms, either conditioning on other potential source variables or considering transfer from a collection of sources, although these forms require more samples again.

While it was originally defined for bivariate analysis, transfer entropy has been extended to multivariate forms, either conditioning on other potential source variables or considering transfer from a collection of sources, although these forms require more samples again.

Transfer entropy has been used for estimation of functional connectivity of neurons and social influence in social networks.

Transfer entropy has been used for estimation of functional connectivity of neurons and social influence in social networks.

Transfer entropy is a finite version of the  [[Directed Information]] which was defined in 1990 by [[James Massey]] <ref>{{cite journal|last1=Massey|first1=James|title=Causality, Feedback And Directed Information|date=1990|issue=ISITA|citeseerx=10.1.1.36.5688}}</ref> as  

Transfer entropy is a finite version of the  [[Directed Information]] which was defined in 1990 by [[James Massey]] <ref>{{cite journal|last1=Massey|first1=James|title=Causality, Feedback And Directed Information|date=1990|issue=ISITA|citeseerx=10.1.1.36.5688}}</ref> as  

Transfer entropy is a finite version of the  Directed Information which was defined in 1990 by James Massey  as  

Transfer entropy is a finite version of the  Directed Information which was defined in 1990 by James Massey  as  

转移熵是有向信息的有限形式，1990年由詹姆斯·梅西 James Massey定义为

转移熵是有向信息的有限形式，1990年由詹姆斯·梅西 James Massey<ref>{{cite journal|last1=Massey|first1=James|title=Causality, Feedback And Directed Information|date=1990|issue=ISITA|citeseerx=10.1.1.36.5688}}</ref>定义为

<math>I(X^n\to Y^n) =\sum_{i=1}^n I(X^i;Y_i|Y^{i-1})</math>,  where <math>X^n</math> denotes the vector <math>X_1,X_2,...,X_n</math> and <math>Y^n</math> denotes <math>Y_1,Y_2,...,Y_n</math>. The [[directed information]] places an important role in characterizing the fundamental limits ([[channel capacity]]) of communication channels with or without feedback <ref>{{cite journal|last1=Permuter|first1=Haim Henry|last2=Weissman|first2=Tsachy|last3=Goldsmith|first3=Andrea J.|title=Finite State Channels With Time-Invariant Deterministic Feedback|journal=IEEE Transactions on Information Theory|date=February 2009|volume=55|issue=2|pages=644–662|doi=10.1109/TIT.2008.2009849|arxiv=cs/0608070}}</ref>  

<math>I(X^n\to Y^n) =\sum_{i=1}^n I(X^i;Y_i|Y^{i-1})</math>,  where <math>X^n</math> denotes the vector <math>X_1,X_2,...,X_n</math> and <math>Y^n</math> denotes <math>Y_1,Y_2,...,Y_n</math>. The [[directed information]] places an important role in characterizing the fundamental limits ([[channel capacity]]) of communication channels with or without feedback <ref>{{cite journal|last1=Permuter|first1=Haim Henry|last2=Weissman|first2=Tsachy|last3=Goldsmith|first3=Andrea J.|title=Finite State Channels With Time-Invariant Deterministic Feedback|journal=IEEE Transactions on Information Theory|date=February 2009|volume=55|issue=2|pages=644–662|doi=10.1109/TIT.2008.2009849|arxiv=cs/0608070}}</ref>  

<math>I(X^n\to Y^n) =\sum_{i=1}^n I(X^i;Y_i|Y^{i-1})</math>,  where <math>X^n</math> denotes the vector<math>X_1,X_2,...,X_n</math>and <math>Y^n</math> denotes <math>Y_1,Y_2,...,Y_n</math>. The directed information places an important role in characterizing the fundamental limits (channel capacity) of communication channels with or without feedback

<math>I(X^n\to Y^n) =\sum_{i=1}^n I(X^i;Y_i|Y^{i-1})</math>,  where <math>X^n</math> denotes the vector<math>X_1,X_2,...,X_n</math>and <math>Y^n</math> denotes <math>Y_1,Y_2,...,Y_n</math>. The directed information places an important role in characterizing the fundamental limits (channel capacity) of communication channels with or without feedback and gambling with causal side information,

I(Xn→Yn)=∑ni=1I(Xi;Yi|Yi−1)，其中 Xn表示向量X1,X2,...,Xn和Yn表示 Y1,Y2,...,Yn。有向信息在描述有无反馈信道的基本限制(信道容量)方面起着重要作用。

I(Xn→Yn)=∑ni=1I(Xi;Yi|Yi−1)，其中 Xn表示向量X1,X2,...,Xn和Yn表示 Y1,Y2,...,Yn。有向信息在描述有无反馈<ref>{{cite journal|last1=Permuter|first1=Haim Henry|last2=Weissman|first2=Tsachy|last3=Goldsmith|first3=Andrea J.|title=Finite State Channels With Time-Invariant Deterministic Feedback|journal=IEEE Transactions on Information Theory|date=February 2009|volume=55|issue=2|pages=644–662|doi=10.1109/TIT.2008.2009849|arxiv=cs/0608070}}</ref> <ref>{{cite journal|last1=Kramer|first1=G.|title=Capacity results for the discrete memoryless network|journal=IEEE Transactions on Information Theory|date=January 2003|volume=49|issue=1|pages=4–21|doi=10.1109/TIT.2002.806135}}</ref>信道的基本限制(信道容量)与基于因果信息赌博<ref>{{cite journal|last1=Permuter|first1=Haim H.|last2=Kim|first2=Young-Han|last3=Weissman|first3=Tsachy|title=Interpretations of Directed Information in Portfolio Theory, Data Compression, and Hypothesis Testing|journal=IEEE Transactions on Information Theory|date=June 2011|volume=57|issue=6|pages=3248–3259|doi=10.1109/TIT.2011.2136270|arxiv=0912.4872}}</ref>方面起着重要作用。

<ref>{{cite journal|last1=Kramer|first1=G.|title=Capacity results for the discrete memoryless network|journal=IEEE Transactions on Information Theory|date=January 2003|volume=49|issue=1|pages=4–21|doi=10.1109/TIT.2002.806135}}</ref> and [[gambling]] with causal side information,<ref>{{cite journal|last1=Permuter|first1=Haim H.|last2=Kim|first2=Young-Han|last3=Weissman|first3=Tsachy|title=Interpretations of Directed Information in Portfolio Theory, Data Compression, and Hypothesis Testing|journal=IEEE Transactions on Information Theory|date=June 2011|volume=57|issue=6|pages=3248–3259|doi=10.1109/TIT.2011.2136270|arxiv=0912.4872}}</ref>

<ref>{{cite journal|last1=Kramer|first1=G.|title=Capacity results for the discrete memoryless network|journal=IEEE Transactions on Information Theory|date=January 2003|volume=49|issue=1|pages=4–21|doi=10.1109/TIT.2002.806135}}</ref> and [[gambling]] with causal side information,<ref>{{cite journal|last1=Permuter|first1=Haim H.|last2=Kim|first2=Young-Han|last3=Weissman|first3=Tsachy|title=Interpretations of Directed Information in Portfolio Theory, Data Compression, and Hypothesis Testing|journal=IEEE Transactions on Information Theory|date=June 2011|volume=57|issue=6|pages=3248–3259|doi=10.1109/TIT.2011.2136270|arxiv=0912.4872}}</ref>