Causal Emergence - 版本历史

2025年3月15日 (六) 01:24 Jake

2025-03-15T01:24:28Z

Jake：/* Chinese classical philosophy */

2025-03-15T01:19:28Z

Chinese classical philosophy

Jake：/* Causal representation learning */

2025-03-15T01:07:04Z

Causal representation learning

Jake：/* Application on the brain nervous system */

2025-03-15T01:03:21Z

Application on the brain nervous system

Jake：/* Causal emergence in complex networks */

2025-03-15T00:57:45Z

Causal emergence in complex networks

Jake：/* NIS+ */

2025-03-15T00:53:41Z

NIS+

Jake：/* NIS */

2025-03-15T00:43:04Z

NIS

Jake：/* Neural information squeezer */

2025-03-15T00:32:15Z

Neural information squeezer

Jake：/* Machine learning-based method */

2025-03-15T00:25:02Z

Machine learning-based method

Jake：/* Method based on mutual information approximation */

2025-03-15T00:20:39Z

Method based on mutual information approximation

←上一版本		2025年3月15日 (六) 01:24的版本
第1行：		第1行：
		+	{{#seo:
		+	\|keywords=Causal Emergence, Emergence, Causality, Causal Representation Learning, World Models, Complex Systems
		+	\|description=This is a comprehensive entry introducing causal emergence theory, including various quantitative methods for characterizing causal emergence, and providing a thorough comparison between causal emergence theory and causal representation learning in artificial intelligence, among other theories.
		+	}}
	Causal emergence refers to a specific type of [[emergence]] phenomenon in dynamical systems, where stronger [[causal effects]] emerge at the system's macroscopic scale. In particular, for a class of [[Markov dynamical systems]], after appropriate [[coarse-graining]] of the state space, the resulting macroscopic dynamics demonstrate stronger [[causal effects]] than their microscopic counterparts. This is referred to as causal emergence <ref name=":0">Hoel E P, Albantakis L, Tononi G. Quantifying causal emergence shows that macro can beat micro[J]. Proceedings of the National Academy of Sciences, 2013, 110(49): 19790-19795.</ref><ref name=":1">Hoel E P. When the map is better than the territory[J]. Entropy, 2017, 19(5): 188.</ref>. Furthermore, the theory of causal emergence uses measures of causal effects to quantify emergence phenomena in complex systems.		Causal emergence refers to a specific type of [[emergence]] phenomenon in dynamical systems, where stronger [[causal effects]] emerge at the system's macroscopic scale. In particular, for a class of [[Markov dynamical systems]], after appropriate [[coarse-graining]] of the state space, the resulting macroscopic dynamics demonstrate stronger [[causal effects]] than their microscopic counterparts. This is referred to as causal emergence <ref name=":0">Hoel E P, Albantakis L, Tononi G. Quantifying causal emergence shows that macro can beat micro[J]. Proceedings of the National Academy of Sciences, 2013, 110(49): 19790-19795.</ref><ref name=":1">Hoel E P. When the map is better than the territory[J]. Entropy, 2017, 19(5): 188.</ref>. Furthermore, the theory of causal emergence uses measures of causal effects to quantify emergence phenomena in complex systems.

@@ 第537行： / 第537行： @@
 According to the causal emergence theory, the quality of a theory depends on the strength of causality, that is, the size of <math>EI</math>. Different coarse-graining schemes result in distinct macroscopic theories (macroscopic dynamics). It is very likely that when facing the same research object of complex systems, the Western philosophical and scientific system gives a set of relatively specific and microscopic causal mechanisms (dynamics), while Eastern philosophy gives a set of more coarsely grained macroscopic causal mechanisms. According to the causal emergence theory or the [[Causal Equivalence Principle]] proposed by Yurchenko, the two are completely likely to be compatible with each other. That is to say, for the same set of phenomena, the East and the West can make correct predictions and even intervention methods according to two different sets of causal mechanisms. It is also possible that in certain types of problems or phenomena, a more macroscopic causal mechanism is more explanatory or leads to a good solution. For some problems or phenomena, a more microscopic causal mechanism is more favorable.
-For example, in Eastern philosophy, the [[five elements]] can be interpreted as  macroscopic states, with their mutual generation and restraint representing a macroscopic causal mechanism. The process of identifying these five states from all phenomena is a coarse-graining process dependent on the observer's ability to analogize. Therefore, the theory of five elements can be regarded as an abstract causal emergence theory for everything. Similarly, the concept of causal emergence can be extended to fields like traditional Chinese medicine, divination, and feng shui. The common point of these applications will be that its causal mechanism is simpler and possibly has stronger causality compared to Western science. However, the process of obtaining such an abstract coarse-graining is more complex and more dependent on experienced abstractors. This explains why Eastern philosophies all emphasize the self-cultivation of practitioners. This is because Eastern philosophical theories place significant complexity and computational demands on '''analogical thinking'''.
+For example, in Eastern philosophy, the [[five elements]] can be interpreted as  macroscopic states, with their mutual generation and restraint representing a macroscopic causal mechanism. The process of identifying these five states from all phenomena is a coarse-graining process dependent on the observer's ability to analogize. Therefore, the theory of five elements can be regarded as an abstract causal emergence theory for everything. Similarly, the concept of causal emergence can be extended to fields like traditional Chinese medicine, divination, and feng shui. The common point of these applications will be that its causal mechanism is simpler and possibly has stronger causality compared to Western science. However, the process of obtaining such an abstract coarse-graining is more complex and more dependent on experienced abstractors. This explains why Eastern philosophies all emphasize the self-cultivation of practitioners. This is because Eastern philosophical theories place significant complexity and computational demands on '''analogical thinking''', i.e., the coarse-graining process.
 ==Critique==

@@ 第481行： / 第481行： @@
 ====Causal representation learning====
-[[Causal representation learning]] is an emerging field in artificial intelligence. It attempts to combine two important fields in machine learning: [[representation learning]] and [[causal inference]]. By integrating their respective advantages, it aims to automatically extract important features and uncover causal relationships hidden within the data <ref>B. Sch ̈olkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, Y. Bengio, Toward causal representation learning, Proceedings of the IEEE 109 (5) (2021) 612–634.</ref>.Causal emergence identification based on EI can be equivalent to a causal representation learning task. Identifying the emergence of causal relationships from data is equivalent to learning the underlying causal relationships and causal mechanisms behind the data. Specifically, we can regard the macroscopic state as a causal variable and the macroscopic dynamics as a causal mechanism by analogy. The coarse-graining strategy can be seen as an encoding process or representation from the original data to the causal variable, while the EI serves as a measure of the strength of causal effect of the mechanism.
+[[Causal representation learning]] is an emerging field in artificial intelligence. It attempts to combine two important fields in machine learning: [[representation learning]] and [[causal inference]]. By integrating their respective advantages, it aims to automatically extract important features and uncover causal relationships hidden within the data <ref>B. Sch ̈olkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, Y. Bengio, Toward causal representation learning, Proceedings of the IEEE 109 (5) (2021) 612–634.</ref>. Causal emergence identification based on EI can be equivalent to a causal representation learning task. Identifying the emergence of causal relationships from data is equivalent to learning the underlying causal relationships and causal mechanisms behind the data. Specifically, we can regard the macroscopic state as a causal variable and the macroscopic dynamics as a causal mechanism by analogy. The coarse-graining strategy can be seen as an encoding process or representation from the original data to the causal variable, while the EI serves as a measure of the strength of causal effect of the mechanism.
 Since there are many similarities between the two, the techniques and concepts of the two fields can learn from each other. For example, [[causal representation learning]] technology can be applied to [[causal emergence identification]]. In turn, the learned abstract causal representation can be interpreted as a macroscopic state, thereby enhancing the interpretability of causal representation learning. However, there are also significant differences between the two, mainly including two points: 1) Causal representation learning assumes that there is a real [[causal mechanism]] behind it, and the data is generated by this causal mechanism. However, there may not be a "true causal relationship" between the emergent macroscopic states and dynamics; 2) The macroscopic state after coarse-graining in causal emergence is a low-dimensional description, but there is no such requirement in causal representation learning. From an epistemological perspective, there is no difference between the two, because what both do is to extract EI from observational data, so as to obtain a representation with stronger causal effect.

@@ 第474行： / 第474行： @@
 Observing EI, including [[sensitivity]] and [[degeneracy]] changes during model training, reveals the model's generalization ability and aids scholars in understanding neural networks.
-===Application on the brain nervous system===
+===Application on brain nervous system===
 The brain nervous system is an emergent multi-scale [[complex system]]. Luppi et al. <ref>Luppi AI, Mediano PA, Rosas FE, Allanson J, Pickard JD, Carhart-Harris RL, Williams GB, Craig MM, Finoia P, Owen AM, Naci L. A synergistic workspace for human consciousness revealed by integrated information decomposition. BioRxiv. 2020 Nov 26:2020-11.</ref> used [[integrated information decomposition]] to reveal the synergistic workspace of human [[consciousness]]. The authors constructed a three-layer architecture of brain cognition, including: external environment, specific modules and synergistic global space. The working principle of the brain mainly includes three stages: the first stage is responsible for collecting information from multiple different modules into the workspace; the second stage involves integrating the collected information within the workspace; the third stage focuses on broadcasting global information to other parts of the brain. The authors conducted experiments on three types of fMRI data in different resting states, including 100 normal people, 15 subjects participating in anesthesia experiments (including three different states before anesthesia, during anesthesia and recovery), and 22 subjects with chronic disorders of consciousness (DOC). This article uses [[integrated information decomposition]] to obtain [[synergistic information]] and [[redundant information]] It then employs the revised [[integrated information value]] <math>\Phi_R</math> to calculate the synergy and redundancy values between each two brain regions, determining whether each brain region plays a greater role is synergy or redundancy. By comparing the data of conscious individuals, they found that regions with significantly reduced [[integrated information]] in unconscious individuals corresponded to brain regions dominated by [[synergistic information]]. Meanwhile, these regions all belonged to functional regions such as DMN (Default Mode Network), allowing the authors to locate brain regions with a significant effect on the occurrence of consciousness.

@@ 第438行： / 第438行： @@
 In 2020, Klein and Hoel improved the method of quantifying causal emergence on Markov chains to apply to [[complex networks]] <ref>Klein B, Hoel E. The emergence of informative higher scales in complex networks[J]. Complexity, 2020, 20201-12.</ref>. The authors defined the [[Markov chain]] in the network with the help of [[random walkers]]. Placing random walkers on nodes is equivalent to intervening on nodes. They then defined the transition probability matrix between nodes based on the random walk probability. At the same time, the authors establish a connection between EI and the connectivity of the network. Connectivity is characterized by the uncertainty in the weights of outgoing and incoming edges of the nodes. Based on this, the EI in complex networks is defined. For detailed methods, refer to [[causal emergence in complex networks]].
-The authors conducted experimental comparisons in artificial networks such as [[random network]] (ER), the [[preferential attachment network model]] (PA), and four types of real networks. They found that, for ER networks, the degree of EI only depends on the connection probability <math>p</math>, and as the network size increases, it will converge to the value <math>-\log_2p</math>. At the same time, a key finding shows that there is a phase transition point in the EI value, which approximately appears at the position where the [[average degree]] (<math><k></math>) of the network is equal to <math>\log_2N</math>. This also corresponds to the phase transition point where the random network structure does not contain more information as its scale increases with the increase of the connection probability. For preferential attachment model networks, the EI depends on the power-law exponent α of the network's degree distribution. When <math>\alpha<1.0</math>, the degree of EI increases as the network size grows. Conversely, when <math>\alpha>1.0</math>, the degree of EI decreases with network size. The case of <math>\alpha = 1.0</math> corresponds to the scale-free network, making the critical boundary for network growth. For real networks, the authors found that biological networks have the lowest EI because they exhibit significant noice. However, we can remove this noise through effective coarse-graining, which makes biological networks show more significant causal emergence phenomena than other types of networks. On the other hand, technical type networks are sparser and non-degenerate, resulting in higher average efficiency, more specific node relationships, and the highest EI. However, it is challenging to increase the degree of causal emergence in these networks through coarse-graining.
+The authors conducted experimental comparisons in artificial networks such as [[random network]] (ER), the [[preferential attachment network model]] (PA), and four types of real networks. They found that, for ER networks, the degree of EI only depends on the connection probability <math>p</math>, and as the network size increases, it will converge to the value <math>-\log_2p</math>. At the same time, a key finding shows that there is a phase transition point in the EI value, which approximately appears at the position where the [[average degree]] (<math><k></math>) of the network is equal to <math>\log_2N</math>. This also corresponds to the phase transition point where the random network structure does not contain more information as its scale increases with the increase of the connection probability. For preferential attachment model networks, the EI depends on the power-law exponent α of the network's degree distribution. When <math>\alpha<1.0</math>, the degree of EI increases as the network size grows. Conversely, when <math>\alpha>1.0</math>, the degree of EI decreases with network size. The case of <math>\alpha = 1.0</math> corresponds to the scale-free network, making the critical boundary for network growth. For real networks, the authors found that biological networks have the lowest EI because they exhibit significant noise. However, we can remove this noise through effective coarse-graining, which makes biological networks show more significant causal emergence phenomena than other types of networks. On the other hand, technical type networks are sparser and non-degenerate, resulting in higher average efficiency, more specific node relationships, and the highest EI. However, it is challenging to increase the degree of causal emergence in these networks through coarse-graining.
 In this article, the authors use the [[greedy algorithm]] to coarse-grain the network. However, for large-scale networks, this algorithm is very inefficient. Subsequently, Griebenow et al. <ref>Griebenow R, Klein B, Hoel E. Finding the right scale of a network: efficient identification of causal emergence through spectral clustering[J]. arXiv preprint arXiv:190807565, 2019.</ref> proposed a method based on [[spectral clustering]] to identify causal emergence in [[preferential attachment networks]]. Compared with the [[greedy algorithm]] and the [[gradient descent algorithm]], the [[spectral clustering]] algorithm has less computation time and the causal emergence of the found macroscopic network is also more significant.

@@ 第397行： / 第397行： @@
 =====NIS+=====
-Although NIS took the lead in proposing a scheme to optimize EI to identify causal emergence in data, this method has some shortcomings: the author divides the optimization process into two stages, but does not truly maximize the EI, that is, Equation {{EquationNote|1}}. Therefore, Yang et al. <ref name=":6" /> further improved this method and proposed the [[NIS+]] scheme. By introducing reverse dynamics and [[reweighting technique]], the original maximization of EI is transformed into maximizing its variational lower bound. This is achieved using [[variational inequality]], enabling the direct optimization of the objective function.
+NIS, although it was the first to propose a framework to optimize EI to identify causal emergence in data, has some shortcomings: the authors divide the optimization process into two stages, but does not truly maximize the EI directly, that is, Equation {{EquationNote|1}}. Therefore, Yang et al. <ref name=":6" /> further improved this method and proposed the [[NIS+]] scheme. By introducing reverse dynamics and [[reweighting technique]], the original maximization of EI is transformed into maximizing its variational lower bound. This is achieved using [[variational inequality]], enabling the direct optimization of the objective function.
 ======Mathematical principles======
@@ 第410行： / 第410行： @@
 </math>
-Here <math>\tilde{p}(\boldsymbol{y}_{t})</math> is the target distribution and <math>p(\boldsymbol{y}_{t})</math> is the original distribution of the data.
+Here <math>\tilde{p}(\boldsymbol{y}_{t})</math> is the target distribution and <math>p(\boldsymbol{y}_{t})</math> is the original distribution of the data, [math]\phi[/math] is the coarse-graining function.
 ======Workflow and model architecture======
-The following figure shows the entire model framework of NIS+. Figure a is the input of the model: time series data, such as trajectory sequence, continuous image sequence, and EEG time series data; Figure c is the output of the model, which includes the degree of causal emergence, macroscopic dynamics, emergent patterns, and coarse-graining strategies; Figure b is the specific model architecture. Different from the NIS method, two parts of reverse dynamics and reweighting technology are added.
+The following figure shows the entire model framework of NIS+. Figure a is the input of the model: time series data, such as trajectory sequence, continuous image sequence, and EEG time series data; Figure c is the output of the model, which includes the degree of causal emergence, macroscopic dynamics, emergent patterns, and coarse-graining strategies; Figure b is the specific model architecture. Different from the NIS method, the reverse dynamics training and the reweighting technology are added.
 [[文件:NIS+.png|居左|600x600像素|替代=NIS模型框架图|NIS+模型框架图]]
@@ 第421行： / 第421行： @@
 The following figure shows the experimental results of NIS+ learning the flocking behavior of the Boids model. (a) and (e) give the actual and predicted trajectories of bird flocks under different conditions. Specifically, the author divides the bird flock into two groups and compares the multi-step prediction results under different noise levels (<math>\alpha</math> is 0.001 and 0.4 respectively). The prediction is good when the noise is relatively small, and the prediction curve will diverge when the noise is relatively large. (b) shows that the mean absolute error (MAE) of multi-step prediction gradually increases as the radius r increases. (c) shows the change of the degree of [[causal emergence]] <math>\Delta J</math> and prediction error (MAE) under different dimensions (q) with the change of training epoch. The author finds that causal emergence is most significant when the macroscopic state dimension q = 8. (d) is the attribution analysis of macroscopic variables to microscopic variables, and the obtained significance map intuitively describes the learned coarse-graining function. Here, each macroscopic dimension can correspond to the spatial coordinate (microscopic dimension) of each bird. The darker the color, the higher the correlation. Here, the microscopic coordinates corresponding to the maximum correlation of each macroscopic state dimension are highlighted with orange dots. These attribution significance values are obtained by using the [[Integrated Gradient]] (referred to as IG) method. The horizontal axis represents the x and y coordinates of 16 birds in the microscopic state, and the vertical axis represents 8 macroscopic dimensions. The light blue dotted line distinguishes the coordinates of different individual Boids, and the blue solid line separates the two bird flocks. (f) and (g) represent the changing trends of the degree of causal emergence ΔJ and normalized error MAE under different noise levels. (f) represents the influence of changes in external noise (that is, adding observation noise to microscopic data) on causal emergence. (g) represents the influence of internal noise (represented by α, added by modifying the dynamics of the Boids model) on causal emergence. In (f) and (g), the horizontal line represents the threshold that violates the error constraint in Equation {{EquationNote|1}}. When the normalized MAE is greater than the threshold of 0.3, the constraint is violated and the result is unreliable.
 This set of experiments shows that [[NIS+]] can learn macroscopic states and coarse-graining strategies by maximizing EI. This maximization improves the model's generalization ability to situations outside the training data range. The learned macroscopic state effectively identifies the average [[group behavior]] and can be attributed to individual positions using the gradient integration method. The degree of causal emergence increases with external noise but decreases with increase of internal noise. This result indicates that the model reduces external noise through coarse-graining but cannot mitigate internal noise.
-[[文件:NIS+ boids.png|居左|700x700像素|鸟群中的因果涌现]]
+The brain experiment uses fMRI data collected from two experiments involving 830 human subjects (the results are shown in the following picture). In the first group, subjects performed a visual task by watching a short movie clip, during which recordings were made. In the second group of experiments, they were asked to be in a resting state and the recording was completed. Since the original dimension is relatively high, the authors reduced the original 14000-dimensional data to 100 dimensions using the [[Schaefer atlas]] method, with each dimension corresponding to a brain region. After that, the authors learned these data through NIS+ and extracted the dynamics at six different macroscopic scales. Figure a shows the multi-step prediction error results at different scales. Figure b shows the comparison of EI of NIS and NIS+ methods on different macroscopic dimensions in the resting state and the visual task of watching movies. The authors found that in the visual task, causal emergence is most significant when the macroscopic state dimension is q = 1. Through attribution analysis, it is found that the visual area plays the largest role (Figure c), which is consistent with the real scene. Figure d shows different perspective views of brain region attribution. In the resting state, one macroscopic dimension is not enough to predict the microscopic time series data. The dimension with the largest causal emergence is between 3 and 7 dimensions.
 [[文件:NIS+ 脑数据.png|居左|700x700像素|脑神经系统中的因果涌现]]

@@ 第364行： / 第364行： @@
 </math>
-Here [math]z\sim\mathcal{Ν}\left (0,I_{p - q}\right )[/math] is a [math]p-q[/math]-dimensional random vector that obeys the standard normal distribution.
+Here [math]z\sim\mathcal{Ν}\left (0,I_{p - q}\right )[/math] is a [math]p-q[/math]-dimensional gaussian random vector.
-However, if we directly optimize the dimension-averaged EI, there will be certain difficulties. The article <ref name="NIS" /> does not directly optimize Equation {{EquationNote|1}}, but adopts a clever method. To solve this problem, the author divides the optimization process into two stages. The first stage is to minimize the microscopic state prediction error under the condition of a given macroscopic scale <math>q</math>, that is, <math>\min _{\phi, f_q, \phi^{\dagger}}\left\|\phi^{\dagger}(Y(t + 1)) - X_{t + 1}\right\|<\epsilon</math> and obtain the optimal macroscopic state dynamics [math]f_q^\ast[/math]; the second stage is to search for the hyperparameter <math>q</math> to maximize the EI ([math]\mathcal{J}[/math]), that is, <math>\max_{q}\mathcal{J}(f_{q}^\ast)</math>. Practice has proved that this method can effectively find macroscopic dynamics and coarse-graining functions, but it cannot truly maximize EI in advance.
+However, if we directly optimize the dimension-averaged EI, there will be certain difficulties. The article <ref name="NIS" /> does not directly optimize Equation {{EquationNote|1}} directly, but adopts a trick. To solve this problem, the authors separate the optimization process into two stages. The first stage is to minimize the microscopic state prediction error under the condition of a given macroscopic scale <math>q</math>, that is, <math>\min _{\phi, f_q, \phi^{\dagger}}\left\|\phi^{\dagger}(Y(t + 1)) - X_{t + 1}\right\|<\epsilon</math> and obtain the optimal macroscopic state dynamics [math]f_q^\ast[/math]; the second stage is to search for the hyperparameter <math>q</math> to maximize the EI ([math]\mathcal{J}[/math]), that is, <math>\max_{q}\mathcal{J}(f_{q}^\ast)</math>. It has been proved that this method can effectively find macroscopic dynamics and coarse-graining functions, but it cannot truly maximize EI in advance.
 In addition to being able to automatically identify causal emergence based on time series data, this framework also has good theoretical properties. There are two important theorems:
-'''Theorem 1''': The [[information bottleneck]] of the neural information squeezer. That is, for any bijection <math>\mathrm{\psi}</math>, projection <math>\chi</math>, macroscopic dynamics <math>f</math> and Gaussian noise <math>z_{p - q}\sim\mathcal{Ν}\left (0,I_{p - q}\right )</math>,
+'''Theorem 1''': The [[information bottleneck]] of the neural information squeezer, that is, for any bijection <math>\mathrm{\psi}</math>, projection <math>\chi</math>, macroscopic dynamics <math>f</math> and Gaussian noise <math>z_{p - q}\sim\mathcal{Ν}\left (0,I_{p - q}\right )</math>,
 <math>
@@ 第392行： / 第392行： @@
 ======Computational examples======
-The author of NIS conducted experiments in the [[spring oscillator model]], and the results are shown in the following figure. Figure a demostrates that the results for the next moment align linearly with the iterative macroscopic dynamics, validating the model's effectiveness. Figure b illustrates that the learned and real dynamics align, further validating the model. Figure c illustrates the model's accuracy in multi-step predictions, as the predicted and real curves are closely aligned. Figure d shows the degree of causal emergence at different scales. Causal emergence is most significant at scale is 2, consistent with the real spring oscillator model, where only two states (position and velocity) are needed to describe the system.
+The authors of NIS conducted experiments in the [[spring oscillator model]], and the results are shown in the following figure. Figure a demostrates that the results for the next moment align linearly with the iterative macroscopic dynamics, validating the model's effectiveness. Figure b illustrates that the learned and real dynamics align, further validating the model. Figure c illustrates the model's accuracy in multi-step predictions, as the predicted and real curves are closely aligned. Figure d shows the degree of causal emergence at different scales. Causal emergence is most significant at scale is 2, consistent with the real spring oscillator model, where only two states (position and velocity) are needed to describe the system.
 [[文件:弹簧振子模型1.png|居左|600x600像素|替代=弹簧振子模型1|弹簧振子模型]]

@@ 第321行： / 第321行： @@
 ====Neural information squeezer====
-In recent years, emerging [[artificial intelligence]] technologies have overcome a series of major problems. Machine learning methods incorporate carefully designed [[neural network]] structures and [[automatic differentiation]] technologies. These technologies enable the approximation of any function in a vast function space. Therefore, Zhang et al. proposed a data-driven method using neural networks to identify causal emergence from time series data <ref name="NIS">Zhang J, Liu K. Neural information squeezer for causal emergence[J]. Entropy, 2022, 25(1): 26.</ref><ref name=":6" />. This method can automatically extract effective coarse-graining strategies and macroscopic dynamics, overcoming various deficiencies of the Rosas's method <ref name=":5" />.
+In recent years, emerging [[artificial intelligence]] technologies have overcome a series of major problems. Machine learning methods incorporate carefully designed [[neural network]] structures and [[automatic differentiation]] technologies. These technologies enable the approximation of any function in a vast function space. Therefore, Zhang et al. proposed a data-driven method using neural networks to identify causal emergence from time series data <ref name="NIS">Zhang J, Liu K. Neural information squeezer for causal emergence[J]. Entropy, 2022, 25(1): 26.</ref><ref name=":6" />. This method can automatically extract effective coarse-graining strategies and macroscopic dynamics, overcoming various deficiencies of the Rosas's method <ref name=":5" />. This framework is based on Erik Hoel's theory of causal emergence.
 In this work, the input consists of time series data <math>(X_1,X_2,...,X_T )</math>, where <math>X_t\equiv (X_t^1,X_t^2,…,X_t^p )</math> and <math>p</math> denotes the dimensionality of the input data. The author assumes that this set of data is generated by a general [[stochastic dynamical system]]:
@@ 第337行： / 第337行： @@
 |{{EquationRef|2}}}}
-[math]\mathcal{J}[/math] represents the dimension-averaged <math>EI</math> (see the entry [[effective information]]), and <math>\mathrm{\phi}</math> is the coarse-graining strategy function. <math>f_{q}</math> denotes the macroscopic dynamics, and <math>q</math> represents the dimension of the coarsened macroscopic state. [math]\hat{X}_{t + 1}[/math] is the prediction of the microscopic state at time <math>t + 1</math> by the entire framework. This prediction is obtained by applying the inverse coarse-graining function ([math]\phi^{\dagger}[/math]) to the macroscopic state prediction of the macroscopic state <math>\hat{Y}_{t + 1}</math> at time <math>t + 1</math>. Here [math]\hat{Y}_{t + 1}\equiv f_q(Y_t)[/math] represents the prediction of the macroscopic state at time <math>t + 1</math> by the dynamics learner, based on the macroscopic state [math]Y_t[/math] at time <math>t</math>. The macroscopic state [math]Y_t\equiv \phi(X_t)[/math] at time <math>t</math> is obtained by coarse-graining the microscopic state [math]X_t[/math] using the function [math]\phi[/math]. Finally, the difference between [math]\hat{X}_{t + 1}[/math] and the real microscopic state data [math]X_{t + 1}[/math] is compared to obtain the microscopic prediction error.
+[math]\mathcal{J}[/math] represents the dimension-averaged <math>EI</math> (see the entry [[effective information]]), and <math>\mathrm{\phi}</math> is the coarse-graining strategy function. <math>f_{q}</math> denotes the macroscopic dynamics, and <math>q</math> represents the dimension of the coarsened macroscopic state. [math]\hat{X}_{t + 1}[/math] is the prediction of the microscopic state at time <math>t + 1</math> by the entire framework. This prediction is obtained by applying the inverse coarse-graining function ([math]\phi^{\dagger}[/math]) on the macroscopic state <math>\hat{Y}_{t + 1}</math> at time <math>t + 1</math>. Here [math]\hat{Y}_{t + 1}\equiv f_q(Y_t)[/math] represents the prediction of the macroscopic state at time <math>t + 1</math> by the dynamics learner, based on the macroscopic state [math]Y_t[/math] at time <math>t</math>. The macroscopic state [math]Y_t\equiv \phi(X_t)[/math] at time <math>t</math> is obtained by coarse-graining the microscopic state [math]X_t[/math] using the function [math]\phi[/math]. Finally, the difference between [math]\hat{X}_{t + 1}[/math] and the real microscopic state data [math]X_{t + 1}[/math] is compared to obtain the microscopic prediction error.
 The entire optimization framework is shown below:

@@ 第312行： / 第312行： @@
 =====Machine learning-based method=====
-Kaplanis et al. <ref name=":2" /> use a [[representation learning]] to learn the macroscopic state variable <math>V</math> by maximizing <math>\mathrm{\Psi}</math> (i.e., Equation {{EquationNote|1}}). Specifically, the authors use a neural network <math>f_{\theta}</math> to learn the representation function that coarsens the microscopic input <math>X_t</math> into the macroscopic output <math>V_t</math>. At the same time, they use neural networks <math>g_{\phi}</math> and <math>h_{\xi}</math> to learn the calculation of mutual information, such as <math>I(V_t;V_{t + 1})</math> and <math>\sum_i(I(V_{t + 1};X_{t}^i))</math>, respectively. Finally, this method optimizes the neural network by maximizing the difference between the two (i.e., <math>\mathrm{\Psi}</math>). The architecture diagram of this neural network system is shown in Figure a below.
+Kaplanis et al. <ref name=":2" /> use a [[representation learning]] method to learn the macroscopic state variable <math>V</math> by maximizing <math>\mathrm{\Psi}</math> (i.e., Equation {{EquationNote|1}}). Specifically, the authors use a neural network <math>f_{\theta}</math> to learn the representation function that coarsens the microscopic input <math>X_t</math> into the macroscopic output <math>V_t</math>. At the same time, they use neural networks <math>g_{\phi}</math> and <math>h_{\xi}</math> to learn the calculation of mutual information, such as <math>I(V_t;V_{t + 1})</math> and <math>\sum_i(I(V_{t + 1};X_{t}^i))</math>, respectively. Finally, this method optimizes the neural network by maximizing the difference between the two terms (i.e., <math>\mathrm{\Psi}</math>). The architecture diagram of this neural network system is shown in Figure a below.
 [[文件:Architectures for learning causal emergent representations1.png|居左|替代=|700x600像素]]

@@ 第282行： / 第282行： @@
 The authors proposed three new indicators based on [[mutual information]], <math>\mathrm{\Psi}</math>, <math>\mathrm{\Delta}</math> and <math>\mathrm{\Gamma}</math>, which can be used to identify causal emergence, [[causal decoupling]] and [[downward causation]] in the system respectively. The specific calculation formulas of the three indicators are as follows:
-* Indicator for judging causal emergence:
+* Indicator for the judgement of causal emergence:
 {{NumBlk|:|
 <math>\Psi_{t, t + 1}(V):=I\left(V_t ; V_{t + 1}\right)-\sum_j I\left(X_t^j ; V_{t + 1}\right)</math>
 |{{EquationRef|1}}}}
-Here <math>X_t^j</math> represents the microscopic variable at time t in the j-th dimension, and <math>V_t ; V_{t + 1}</math> respectively represent macroscopic variables at two consecutive times. Rosas et al. defined that when <math>\mathrm{\Psi}>0</math>, emergence occurs in the system; but when <math>\mathrm{\Psi}<0</math>, we cannot determine whether the system has emergence, because this condition is only a sufficient condition for the occurrence of causal emergence.
+Here <math>X_t^j</math> represents the microscopic variable at time t in the j-th dimension, and <math>V_t ; V_{t + 1}</math> respectively represent macroscopic variables at two consecutive times. Rosas et al. defined that when <math>\mathrm{\Psi}>0</math>, emergence occurs in the system; but when <math>\mathrm{\Psi}<0</math>, we cannot determine whether emergence occurs, because this condition is only a sufficient condition for the occurrence of causal emergence.
-* Indicator for judging [[downward causation]]:
+* Indicator for [[downward causation]]:
 <math>\Delta_{t, t + 1}(V):=\max _j\left(I\left(V_t ; X_{t + 1}^j\right)-\sum_i I\left(X_t^i ; X_{t + 1}^j\right)\right)</math>
@@ 第295行： / 第295行： @@
 When <math>\mathrm{\Delta}>0</math>, there exists [[downward causation]] from macroscopic variable <math>V</math> to microscopic variable <math>X</math>.
-* Indicator for judging [[causal decoupling]]:
+* Indicator for [[causal decoupling]]:
 <math>\Gamma_{t, t + 1}(V):=\max _j I\left(V_t ; X_{t + 1}^j\right)</math>
@@ 第307行： / 第307行： @@
 Since <math>Red(V_t, V_{t + 1};X_t)</math> is non-negative, we can thus propose a sufficient but not necessary condition: when <math>\Psi_{t, t + 1}(V)>0</math>.
-In summary, this method is relatively convenient to calculate because it is based on mutual information, and there is no assumption or requirement of Markov property for the dynamics of the system. However, this theory also has many shortcomings: 1) The indicators <math>\mathrm{\Psi}</math>, <math>\mathrm{\Delta}</math> and <math>\mathrm{\Gamma}</math> based only on [[mutual information]] and do not account for causality; 2) The method provides only a sufficient condition for causal emergence; 3) Results depend heavily on the selection of macroscopic variables, with different choices leading to significantly different outcomes; 4) For systems with large amounts of redundant information or many variables, the [[computational complexity]] becomes very high. At the same time, since <math>\Psi</math> is an approximate calculation, there will be a very large error in high-dimensional systems, and it is also very easy to obtain negative values, making it difficult to reliably determine the occurrence of causal emergence.
+In summary, this method is relatively convenient to calculate because it is based on mutual information, and there is no assumption or requirement of Markov property for the dynamics of the system. However, this theory also has many shortcomings: 1) The indicators <math>\mathrm{\Psi}</math>, <math>\mathrm{\Delta}</math> and <math>\mathrm{\Gamma}</math> based only on [[mutual information]] and do not account for causality; 2) The method provides only a sufficient condition for causal emergence; 3) The results depend heavily on the selection of macroscopic variables, with different choices leading to significantly different outcomes; 4) For systems with large amounts of redundant information or many variables, the [[computational complexity]] becomes very high. At the same time, since <math>\Psi</math> is an approximate calculation, there will be a very large error in high-dimensional systems, and it is also very easy to obtain negative values, making it difficult to reliably determine the occurrence of causal emergence.
 To verify that the information related to macaque movement is an emergent feature of its cortical activity, Rosas et al. conducted an experiment using the electrocorticogram (ECoG) of macaques as the observational data of microscopic dynamics. To obtain the macroscopic state variable <math>V</math>, the authors chose the time series data of the limb movement trajectory of macaques obtained by motion capture (MoCap). The ECoG consisted of 64 channels, while the MoCap data consisted of 3 channels. Since the original MoCap data does not satisfy the conditional independence assumption of the supervenience feature, they used [[partial least squares]] and [[support vector machine]] algorithms to infer the part of neural activity encoded in the ECoG signal related to predicting macaque behavior. They then speculated that this information is an emergent feature of potential neural activity. Finally, based on the microscopic state and the calculated macroscopic features, the authors verified the existence of causal emergence.