更改

Causal Emergence (查看源代码)

2025年3月8日 (六) 16:32的版本

删除51字节、 2025年3月8日 (六) 16:32

→‎Quantification of causal emergence

第89行：第89行：

== Quantification of causal emergence ==

This section introduces studies that use measures of causal effects to quantify emergence phenomena.

−

=== Several causal emergence theories ===

Defining causal emergence is a key issue. Several representative works include the method based on EI by Hoel et al. <ref name=":0" /><ref name=":1" />, and the [[information decomposition]] method by Rosas et al. <ref name=":5">Rosas F E, Mediano P A, Jensen H J, et al. Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data[J]. PLoS computational biology, 2020, 16(12): e1008289.</ref>. Zhang Jiang et al. <ref name=":2" />, also proposed a new causal emergence theory based on [[singular value decomposition]].

−

==== Erik Hoel's causal emergence theory ====

Hoel et al. (2013) <ref name=":0" /><ref name=":1" /> proposed the theory of causal emergence. The following figure is an abstract framework for this theory. The horizontal axis represents time and the vertical axis represents scale. This framework describes the same dynamical system at both microscopic and macroscopic scales. Among them, [math]f_m[/math] represents microscopic dynamics, [math]f_M[/math] represents macroscopic dynamics, and the two are connected by a coarse-graining function [math]\phi[/math]. In a discrete-state Markov dynamical system, both [math]f_m[/math] and [math]f_M[/math] are Markov chains. By performing [[coarse-graining of the Markov chain]] on [math]f_m[/math], [math]f_M[/math] can be obtained. <math> EI </math> is a measure of EI. Since the microscopic state may have greater randomness, which leads to relatively weak [[causality]] of microscopic dynamics, by performing reasonable coarse-graining on the microscopic state at each moment, it is possible to obtain a macroscopic state with stronger causality. Causal emergence refers to the phenomenon where coarse-graining a microscopic state increases the EI of macroscopic dynamics. The difference in EI between the macroscopic and the microscopic states is defined as the strength of causal emergence.

−

[[文件:CE.png|替代=因果涌现示意图|500x500像素]]

−

===== Effective information =====

[[Tononi]] et al. first introduced [[Effective Information]] (<math> EI </math>) in their study on [[integrated information theory]] <ref>Tononi G, Sporns O. Measuring information integration[J]. BMC neuroscience, 2003, 41-20.</ref>. In causal emergence research, [[Erik Hoel]] and colleagues use this [[measures of causal effects]] index to quantify the causal strength of a [[causal mechanism]].

−

Specifically, <math> EI </math> is calculated by [[intervening]] on the independent variable and measuing the [[mutual information]] between the cause and effect variables. This mutual information represents EI, quantifying the causal effect of the mechanism.

−

In a [[Markov chain]], the state variable [math]X_{t}[/math] represents the cause, while [math]X_{t + 1}[/math] represents the effect. Thus, the [math]X_{t + 1}[/math] at the next time can be regarded as the result. Thus, the [[state transition matrix]] of a [[Markov chain]] represents its [[causal mechanism]]. Therefore, the calculation formula for <math>EI</math> for a [[Markov chain]] is as follows:

−

<math>

第118行：第111行：

\end{aligned}

</math>

−

<math>f</math> is the state transition matrix of a Markov chain, [math]U(\mathcal{X})[/math] represents the uniform distribution over the value space [math]\mathcal{X}[/math] of the state variable [math]X_{t}[/math]. [math]X_{t}[/math], [math]X_{t+1}[/math] are the states at two consecutive moments following the [[intervention]] of [math]X_{t}[/math] at time <math>t</math> into a [[uniform distribution]]. <math>p_{ij}</math> denots the transition probability from state <math>i</math> to state <math>j</math>. The formula shows that <math> EI </math> depends solely on the probability transition matrix [math]f[/math]. The intervention operation is ensures that EI measures the causal effects of the dynamics objectively. This approach avoids influence from the original data distribution.

−

EI can be decomposed into two parts: '''determinism''' and '''degeneracy'''. It is also possible to eliminate the influence of the size of the state space by introducing normalization. For more detailed information about EI, please refer to the entry: [[Effective Information]].

−

The effective coefficient is introduced to eliminate the influence of the number of states on EI. The formula is shown as follows:

−

<math>

第134行：第123行：

\end{aligned}

</math>

−

where <math>N</math> represents the number of states.

第140行：第128行：

=====Causal emergence measurement=====

Causal emergence can be defined by comparing the EI of macroscopic and microscopic dynamics in the system:

−

<math>

CE = EI\left ( f_M \right ) - EI\left (f_m \right )

</math>

−

Here <math>CE</math> is the strength of causal emergence. Macroscopic dynamics exhibit causal emergence if their EI exceeds that of microscopic dynamics <math>CE>0</math>.

−

Furthermore, <math>CE</math> can be decomposed into the sum of two terms, <math>CE = \Delta I_{Eff} + \Delta I_{Size}</math>, where <math>\Delta I_{Eff} = (Eff(f_M) - Eff(f_m))\cdot \log_2(M)</math>, <math>\Delta I_{Size} = Eff(f_m)\cdot(\log_2(M) - \log_2(m))</math>. Here, <math>M</math> and <math>m</math> represent the sizes of the macroscopic and microscopic states respectively. Because coarse-graining reduces the macroscopic state, <math>\Delta I_{Size}</math> is necessarily less than 0. Causal emergence occurs if the increase in <math>\Delta I_{Eff}</math> exceeds the decrease in <math>\Delta I_{Size}</math>.

第154行：第139行：

=====Markov chain example=====

In the literature <ref name=":0"/>, Hoel gives an example of a state transition matrix ([math]f_m[/math]) of a Markov chain with 8 states, as shown in the left figure below. Among them, the first 7 states transfer with equal probability, and the last state is independent and can only transition to its own state.

−

First, merge the first 7 states into one macroscopic state, tentatively called A, and then sum the probability values in the first 7 rows and the first 7 columns of [math]f_m[/math] and divide by 7, thereby obtaining the state transition probability from macroscopic state A to A, while leaving the other values of the [math]f_m[/math] matrix unchanged. The new probability transition matrix after merging is shown in the right figure, denoted as [math]f_M[/math]. This definite macroscopic Markov transition matrix ensures that the future state is entirely determined by the current state. At this time <math>EI(f_M)>EI(f_m)</math>, and causal emergence occurs in the system.

−

[[文件:Causal emergence in the state space3.png|居左|600x600像素]]

−

However, for more general Markov chains and more general state groupings, this simple operation of averaging probabilities is not always feasible. This occurs when the merged transition matrix does not meet Markov chain conditions, such as row normalization or element values within [0, 1]. Refer to the “Reduction of Markov Chains” section or the “[[Coarse-graining of Markov Chains]]” entry for details on feasible macroscopic Markov chains.

第166行：第148行：

=====Boolean network example=====

Another instance of causal emergence in the literature <ref name=":0"/> involves a [[Boolean network]]. The figure illustrates a Boolean network with 4 nodes. Each node can exist in one of two states, 0 or 1. Each node is connected to two other nodes and operates according to identical microscopic dynamics (figure a). Therefore, this system contains a total of sixteen microscopic states, and its dynamics can be represented by a state transition matrix (figure c).

−

The coarse-graining operation of this system is divided into two steps. The first step is to cluster the nodes in the Boolean network. As shown in figure b below, node A and B are merged to form the macroscopic node [math]\alpha[/math], and node C and D are merged to obtain the macroscopic node [math]\beta[/math]. The second step is to map the microscopic node states in each group to the merged macroscopic node states. This mapping function is shown in figure d below. All microscopic node states containing 0 are transformed into the "off" state of the macroscopic node, while the microscopic 11 state is transformed into the on state of the macroscopic. In this way, we can obtain a new macroscopic Boolean network, and derive the dynamic mechanism of the macroscopic Boolean network according to the dynamic mechanism of the microscopic nodes. According to this mechanism, the state transition matrix of the macroscopic network can be obtained (as shown in figure e).

−

A comparison reveals that the EI of macroscopic dynamics exceeds that of microscopic dynamics <math>EI(f_M\ )>EI(f_m\ ) </math>. This demonstrates the occurrence of causal emergence in this system.

−

[[文件:含有4个节点的布尔网络.png|居左|700x700像素|离散布尔网络上的因果涌现|替代=含有4个节点布尔网络的因果涌现]]

第181行：第160行：

====Rosas's causal emergence theory====

Rosas et al. <ref name=":5" /> propose from the perspective of [[information decomposition]] theory a method for defining causal emergence based on [[integrated information decomposition]], and further divide causal emergence into two parts: [[causal decoupling]] (Causal Decoupling) and [[downward causation]] (Downward Causation). Causal decoupling represents the causal effect of the macroscopic state at the current moment on the macroscopic state at the next moment. Downward causation represents the causal effect of the macroscopic state at the previous moment on the microscopic state at the next moment. The schematic diagrams of causal decoupling and downward causation are shown in the following figure. The microscopic state input is <math>X_t\ (X_t^1,X_t^2,…,X_t^n ) </math>, while the macroscopic state, represented as <math>V_t </math>, is derived through the coarse-graining of the microscopic state variable <math>X_t </math>. Consequently, it serves as a supervenient feature of <math>X_t </math>. Additionally, <math>X_{t + 1} </math> and <math>V_{t + 1} </math> denote the microscopic and macroscopic states at the subsequent moment, respectively.

−

[[文件:Causal_Emergence_Relationship_Diagram.png|替代=|无框|400x300像素]]

−

=====Partial information decomposition=====

This method is based on the nonnegative decomposition of multivariate information theory proposed by Williams and Beer <ref name=":16" />. This paper uses [[partial information decomposition]] (PID) to decompose the [[mutual information]] between microstates and macrostates.

−

Without loss of generality, assume that our microstate is <math>X(X^1,X^2)</math>, meaning it is a two-dimensional variable, and the macrostate is V. Then the [[mutual information]] between them can be decomposed into four parts:

−

−

Among them, <math>Red(X^1,X^2;V)</math> represents [[redundant information]], which refers to the information repeatedly provided by two microstates <math>X^1</math> and <math>X^2</math> to the macrostate <math>V</math>; <math>Un(X^1;V│X^2)</math> and <math>Un(X^2;V│X^1)</math> represent [[unique information]], which refers to the information provided by each microstate variable alone to the macrostate; <math>Syn(X^1,X^2;V)</math> represents [[synergistic information]], which refers to the information provided by all microstates <math>X</math> jointly to the macrostate <math>V</math>.

第200行：第174行：

=====Definition of causal emergence=====

However, the PID framework is limited to decomposing the mutual information between multiple source variables and one target variable. Rosas extended this framework and proposed the integrated information decomposition method <math>\Phi ID</math><ref name=":18" /> to handle the mutual information between multiple source variables and multiple target variables. It can also be used to decompose the mutual information between different moments. Based on the decomposed information, the author proposed two methods for defining causal emergence:

−

1) When the [[unique information]] <math>Un(V_t;X_{t+1}| X_t^1,\ldots,X_t^n\ )>0 </math>, it means that the macroscopic state <math>V_{t}</math> at the current moment contains more predictive information about the overall system <math>X_{t + 1}</math> at the next moment than the microscopic state <math>X_t</math> at the current moment. In this scenario, there exists causal emergence in the system;

−

2) The second method bypasses the selection of a specific macroscopic state <math>V_t</math>. It defines causal emergence solely based on the [[synergistic information]] between the microscopic state <math>X_t</math> and the microscopic state <math>X_{t + 1}</math> at the next moment of the system. When the synergistic information <math>Syn(X_t^1,…,X_t^n;X_{t + 1}^1,…,X_{t + 1}^n)>0</math>, causal emergence occurs in the system.

−

It should be noted that for the first method to judge the occurrence of causal emergence, its accuracy depends on the selection of the macroscopic state <math>V_t</math>. The first method is the lower bound of the second method. This is because <math>Syn(X_t;X_{t+1}\ ) ≥ Un(V_t;X_{t+1}| X_t\ )</math> always holds. So, if <math>Un(V_t;X_{t + 1}|X_t)</math> is greater than 0, then causal emergence occurs in the system. However, the selection of Vt often requires predefining a coarse-graining function, so the limitations of the [[Erik Hoel causal emergence theory]] cannot be avoided. Another natural idea is to use the second method to judge the occurrence of causal emergence with the help of synergistic information. However, the calculation of synergistic information is very difficult and there is a combinatorial explosion problem. Therefore, the calculation based on synergistic information in the second method is often infeasible. In short, both quantitative characterization methods of causal emergence have some weaknesses, so a more reasonable quantification method needs to be proposed.

第212行：第183行：

=====Specific example=====

[[文件:Examples of causal decoupling and downward causation.png|居左]]

−

The author of the paper <ref name=":5" /> lists a specific example (as above), to demonstrate the condition under which [[causal decoupling]], [[downward causation]] and [[causal emergence]] take place. This example is a special Markov process. Here, <math>p_{X_{t + 1}|X_t}(x_{t + 1}|x_t)</math> represents the dynamic relationship, and <math>X_t=(x_t^1,…,x_t^n)\in\{0,1\}^n</math> is the microstate. The definition of this process is to determine the probability of taking different values of the state <math>x_{t + 1}</math> at the next moment by checking the values of the variables <math>x_t</math> and <math>x_{t + 1}</math> at two consecutive moments. This involves determine if the sum modulo 2 of all dimensions of <math>x_t</math> is the same as the first dimension of <math>x_{t + 1}</math>: if they are different, the probability is 0; otherwise, judge whether <math>x_t,x_{t + 1}</math> have the same sum modulo 2 value in all dimensions. If both conditions are satisfied, the value probability is <math>\gamma/2^{n - 2}</math>, otherwise the value probability is <math>(1-\gamma)/2^{n - 2}</math>. Here <math>\gamma</math> is a parameter and <math>n</math> is the total dimension of x.

−

In fact, if <math>\sum_{j = 1}^n x^j_t</math> is even or 0, then <math>\oplus^n_{j = 1} x^j_t:=1</math>, otherwise <math>\oplus^n_{j = 1} x^j_t:=0</math>. Therefore, the result of <math>\oplus^n_{j = 1} x^j_t</math> is the parity of the entire X sequence, and the first dimension can be regarded as a parity check bit. <math>\gamma</math> actually represents the probability that a mutation occurs in two bits of the X sequence. This mutation can ensure that the parity of the entire sequence remains unchanged, and the parity check bit of the sequence also conforms to the actual parity of the entire sequence.

−

Therefore, the macroscopic state of this process can be regarded as the parity of the overall parity of the sequence, and the probability distribution of this parity is the result of the XOR calculation of the microstate. <math>x_{t + 1}^1</math> is a special microstate that always remains consistent with the macroscopic state of the sequence at the previous moment. Therefore, when only the first item in the second judgment condition is satisfied, the downward causation of the system occurs. When only the second item is satisfied, the causal decoupling of the system occurs. When both items are satisfied simultaneously, it is said that causal emergence occurs in the system.

第224行：第192行：

====Causal emergence theory based on singular value decomposition====

[[Erik Hoel's causal emergence theory]] needs to specify a coarse-graining strategy in advance. Rosas' information decomposition theory does not completely solve this problem. Therefore, [[Zhang Jiang]] et al.<ref name=":2">Zhang J, Tao R, Yuan B. Dynamical Reversibility and A New Theory of Causal Emergence. arXiv preprint arXiv:2402.15054. 2024 Feb 23.</ref> further proposed the [[causal emergence theory based on singular value decomposition]].

−

=====Singular value decomposition of Markov chains=====

第231行：第198行：

=====Approximate dynamical reversibility and effective Information=====

We can define the sum of the <math>\alpha</math> powers of the singular values (also known as the [math]\alpha[/math]-order [[Schatten norm]] of the matrix) as a measure of the [[approximate dynamical reversibility]] of the Markov chain, that is:

−

<math>

\Gamma_{\alpha}\equiv \sum_{i = 1}^N\sigma_i^{\alpha}

</math>

−

Here, [math]\alpha\in(0,2)[/math] is a specified parameter that acts as a weight or tendency to make [math]\Gamma_{\alpha}[/math] reflect [[determinism]] or [[degeneracy]] more. Under normal circumstances, we take [math]\alpha = 1[/math], which can make [math]\Gamma_{\alpha}[/math] achieve a balance between determinism and degeneracy.

−

In addition, the authors in the literature prove that there is an approximate relationship between <math>EI</math> and [math]\Gamma_{\alpha}[/math]:

−

<math>

EI\sim \log\Gamma_{\alpha}

</math>

−

Moreover, to a certain extent, [math]\Gamma_{\alpha}[/math] can be used instead of <math>EI</math> to measure the extent of causal effect of Markov chains. Therefore, the so-called causal emergence can also be understood as an '''emergence of dynamical reversibility'''.

第253行：第215行：

=====Quantification of causal emergence without coarse-graining=====

However, the greatest value of this theory lies in the fact that emergence can be directly quantified without a coarse-graining strategy. If the rank of <math>P</math> is <math>r</math>, that is, starting from the <math>r + 1</math>th singular value, all singular values are 0, then we say that the dynamics <math>P</math> has '''clear causal emergence''', and the numerical value of causal emergence is:

−

<math>

\Delta \Gamma_{\alpha} = \Gamma_{\alpha}(1/r - 1/N)

</math>

−

If the matrix <math>P</math> is full rank, but for any given small number <math>\epsilon</math>, there exists <math>r_{\epsilon}</math> such that starting from <math>r_{\epsilon}+1</math>, all singular values are less than <math>\epsilon</math>, then it is said that the system has a degree of '''vague causal emergence''', and the numerical value of causal emergence is:

−

<math>\Delta \Gamma_{\alpha}(\epsilon) = \frac{\sum_{i = 1}^{r} \sigma_{i}^{\alpha}}{r} - \frac{\sum_{i = 1}^{N} \sigma_{i}^{\alpha}}{N} </math>

−

In summary, the advantage of this method for quantifying causal emergence is that it can quantify causal emergence more objectively without relying on a specific coarse-graining strategy. The disadvantage of this method is that to calculate [math]\Gamma_{\alpha}[/math], it is necessary to perform [[SVD decomposition]] on <math>P</math> in advance. As a result, the computational complexity is [math]O(N^3)[/math], which is higher than the computational complexity of <math>EI</math>. Moreover, [math]\Gamma_{\alpha}[/math] cannot be explicitly decomposed into two components: determinism and degeneracy.

=====Specific example=====

−

[[文件:Gamma例子.png|居左|500x500像素|<math>EI</math>与<math>\Gamma</math>对比]]

−

The author gives four specific examples of Markov chains. The state transition matrix of this Markov chain is shown in the figure. We can compare the <math>EI</math> and [[approximate dynamical reversibility]] (the <math>\Gamma</math> in the figure, that is, <math>\Gamma_{\alpha = 1}</math>) of this Markov chain. Comparing figures a and b, we find that for different state transition matrices, when <math>EI</math> decreases, <math>\Gamma</math> also decreases simultaneously. Further, figures c and d are comparisons of the effects before and after coarse-graining. In figure d, the state transition matrix of figure c is coarse-grained by merging the first three states into a macroscopic state. Since the macroscopic state transition matrix in figure d is a [[deterministic system]], the normalized <math>EI</math>, <math>eff\equiv EI/\log N</math> and the normalized [math]\Gamma[/math]: <math>\gamma\equiv \Gamma/N</math> all reach the maximum value of 1.

第278行：第234行：

====Dynamic independence====

[[Dynamic independence]] is a method to characterize the macroscopic dynamical state after coarse-graining such that it is independent of the microscopic dynamical state <ref name=":6">Barnett L, Seth AK. Dynamical independence: discovering emergent macroscopic processes in complex dynamical systems. Physical Review E. 2023 Jul;108(1):014304.</ref>. The core idea is that although macroscopic variables are composed of microscopic variables, when predicting the future state of macroscopic variables, only the historical information of macroscopic variables is needed, and no additional information from microscopic history is needed. This phenomenon is called [[dynamic independence]] by the author. It is another means of quantifying emergence. The macroscopic dynamics at this time is called emergent dynamics. The independence, causal dependence, etc. in the concept of dynamic independence can be quantified by [[transfer entropy]].

−

=====Quantification of dynamic independence=====

[[Transfer entropy]] is a non-parametric statistic that measures the amount of directed (time-asymmetric) information transfer between two stochastic processes. The transfer entropy from process <math>X</math> to another process <math>Y</math> can be defined as the degree to which knowing the past values of <math>X</math> can reduce the uncertainty about the future value of <math>Y</math> given the past values of <math>Y</math>. The formula is as follows:

−

−

Here, <math>Y_t</math> represents the macroscopic variable at time <math>t</math>, and <math>X^-_t</math> and <math>Y^-_t</math> represent the microscopic and macroscopic variables before time <math>t</math> respectively. [math]I[/math] is mutual information and [math]H[/math] is Shannon entropy. <math>Y</math> is dynamically decoupled with respect to <math>X</math> if and only if the transfer entropy from <math>X</math> to <math>Y</math> at time <math>t</math> is <math>T_t(X \to Y)=0</math>.

−

The concept of dynamic independence can be widely applied to a variety of complex dynamical systems, including neural systems, economic processes, and evolutionary processes. Through the coarse-graining method, the high-dimensional microscopic system can be simplified into a low-dimensional macroscopic system, thereby revealing the emergent structure in complex systems.

−

In the paper, the author conducts experimental verification in a [[linear system]]. The experimental process is: 1) Use the linear system to generate parameters and laws; 2) Set the coarse-graining function; 3) Obtain the expression of transfer entropy; 4) Optimize and solve the coarse-graining method of maximum decoupling (corresponding to minimum transfer entropy). Here, the optimization algorithm can use transfer entropy as the optimization goal, and then use the [[gradient descent algorithm]] to solve the coarse-graining function, or use the [[genetic algorithm]] for optimization.

−

=====Example=====

第301行：第251行：

===Comparison of several causal emergence theories===

We can compare the above four different quantitative causal emergence theories from several different dimensions such as whether causality is considered, whether a coarse-graining function needs to be specified, the applicable dynamical systems, and quantitative indicators. The comparison is summarized in the following table:

−

{| class="wikitable"

Complexivist Ran

453

个编辑