更改

===Causal emergence in complex networks===

===Causal emergence in complex networks===

In 2020, Klein and Hoel improved the method of quantifying causal emergence on Markov chains to be applied to [[complex networks]] <ref>Klein B, Hoel E. The emergence of informative higher scales in complex networks[J]. Complexity, 2020, 20201-12.</ref>. The authors defined the [[Markov chain]] in the network with the help of [[random walkers]]. Placing random walkers on nodes is equivalent to intervening on nodes. They then defined the transition probability matrix between nodes based on the random walk probability. At the same time, the authors establish a connection between [[effective information]] and the connectivity of the network. Connectivity is characterized by the uncertainty in the weights of outgoing and incoming node edges. Based on this, the effective information in complex networks is defined. For detailed methods, refer to [[causal emergence in complex networks]].

In 2020, Klein and Hoel improved the method of quantifying causal emergence on Markov chains to be applied to [[complex networks]] <ref>Klein B, Hoel E. The emergence of informative higher scales in complex networks[J]. Complexity, 2020, 20201-12.</ref>. The authors defined the [[Markov chain]] in the network with the help of [[random walkers]]. Placing random walkers on nodes is equivalent to intervening on nodes. They then defined the transition probability matrix between nodes based on the random walk probability. At the same time, the authors establish a connection between EI and the connectivity of the network. Connectivity is characterized by the uncertainty in the weights of outgoing and incoming node edges. Based on this, the EI in complex networks is defined. For detailed methods, refer to [[causal emergence in complex networks]].

The authors conducted experimental comparisons in artificial networks such as [[random network]] (ER), the [[preferential attachment network model]] (PA), and four types of real networks. They found that, for ER networks, the magnitude of effective information only depends on the connection probability <math>p</math>, and as the network size increases, it will converge to the value <math>-\log_2p</math>. At the same time, a key finding shows that there is a phase transition point in the EI value, which approximately appears at the position where the [[average degree]] (<math><k></math>) of the network is equal to <math>\log_2N</math>. This also corresponds to the phase transition point where the random network structure does not contain more information as its scale increases with the increase of the connection probability. For preferential attachment model networks, the effective information depends on the power-law exponent α of the network's degree distribution. When <math>\alpha<1.0</math>, the magnitude of effective information increases as the network size grows. Conversely, when <math>\alpha>1.0</math>, the magnitude of effective information decreases with network size. The case of <math>\alpha = 1.0</math> corresponds to the scale-free network, making the critical boundary for network growth. For real networks, the authors found that biological networks have the lowest effective information because they exhibit significant noice. However, we can remove this noise through effective coarse-graining, which makes biological networks show more significant causal emergence phenomena than other types of networks. On the other hand, technical type networks are sparser and non-degenerate, resulting in higher average efficiency, more specific node relationships, and the highest effective information. However, it is challenging to increase the causal emergence measure in these networks through coarse-graining.

The authors conducted experimental comparisons in artificial networks such as [[random network]] (ER), the [[preferential attachment network model]] (PA), and four types of real networks. They found that, for ER networks, the magnitude of EI only depends on the connection probability <math>p</math>, and as the network size increases, it will converge to the value <math>-\log_2p</math>. At the same time, a key finding shows that there is a phase transition point in the EI value, which approximately appears at the position where the [[average degree]] (<math><k></math>) of the network is equal to <math>\log_2N</math>. This also corresponds to the phase transition point where the random network structure does not contain more information as its scale increases with the increase of the connection probability. For preferential attachment model networks, the EI depends on the power-law exponent α of the network's degree distribution. When <math>\alpha<1.0</math>, the magnitude of EI increases as the network size grows. Conversely, when <math>\alpha>1.0</math>, the magnitude of EI decreases with network size. The case of <math>\alpha = 1.0</math> corresponds to the scale-free network, making the critical boundary for network growth. For real networks, the authors found that biological networks have the lowest EI because they exhibit significant noice. However, we can remove this noise through effective coarse-graining, which makes biological networks show more significant causal emergence phenomena than other types of networks. On the other hand, technical type networks are sparser and non-degenerate, resulting in higher average efficiency, more specific node relationships, and the highest EI. However, it is challenging to increase the causal emergence measure in these networks through coarse-graining.

Hoel et al. in the article <ref>Hoel E, Levin M. Emergence of informative higher scales in biological systems: a computational toolkit for optimal prediction and control[J]. Communicative & Integrative Biology, 2020, 13(1): 108-118.</ref> further studied causal emergence in biological systems with the help of [[effective information]] theory. The author applied effective information to [[gene regulatory networks]] to identify the most informative heart development model to control the heart development of mammals. By quantifying the causal emergence in the [[largest connected component]] of the Saccharomyces cerevisiae gene network, the article reveals that informative macroscopic scales are ubiquitous in biology. It also shows that life mechanisms themselves often operate on macroscopic scales. This article also provides biologists with a computable tool to identify the most informative macroscopic scale, and can model, predict, control and understand complex biological systems on this basis.

Hoel et al. in the article <ref>Hoel E, Levin M. Emergence of informative higher scales in biological systems: a computational toolkit for optimal prediction and control[J]. Communicative & Integrative Biology, 2020, 13(1): 108-118.</ref> further studied causal emergence in biological systems with the help of EI theory. The author applied EI to [[gene regulatory networks]] to identify the most informative heart development model to control the heart development of mammals. By quantifying the causal emergence in the [[largest connected component]] of the Saccharomyces cerevisiae gene network, the article reveals that informative macroscopic scales are ubiquitous in biology. It also shows that life mechanisms themselves often operate on macroscopic scales. This article also provides biologists with a computable tool to identify the most informative macroscopic scale, and can model, predict, control and understand complex biological systems on this basis.

Swain et al. in the article <ref>Swain A, Williams S D, Di Felice L J, et al. Interactions and information: exploring task allocation in ant colonies using network analysis[J]. Animal Behaviour, 2022, 18969-81.</ref> explored the influence of the interaction history of ant colonies on task allocation and task switching, and used effective information to study how noise spreads among ants. The results indicate that historical interactions between ant colonies affects task allocation, while specific interaction types determine noise levels. In addition, even when ants switch functional groups, the emergent cohesion of ant colonies can ensure the stability of the colony. At the same time, different functional ant colonies also play different roles in maintaining the cohesion of the colony.

Swain et al. in the article <ref>Swain A, Williams S D, Di Felice L J, et al. Interactions and information: exploring task allocation in ant colonies using network analysis[J]. Animal Behaviour, 2022, 18969-81.</ref> explored the influence of the interaction history of ant colonies on task allocation and task switching, and used EI to study how noise spreads among ants. The results indicate that historical interactions between ant colonies affects task allocation, while specific interaction types determine noise levels. In addition, even when ants switch functional groups, the emergent cohesion of ant colonies can ensure the stability of the colony. At the same time, different functional ant colonies also play different roles in maintaining the cohesion of the colony.

===Application on artificial neural networks===

===Application on artificial neural networks===

Marrow et al. <ref>Marrow S, Michaud E J, Hoel E. Examining the Causal Structures of Deep Neural Networks Using Information Theory[J]. Entropy, 2020, 22(12): 1429.</ref> introduced [[effective information]] into [[neural networks]] to quantify and track the changes in the [[causal structure]] during training. Here, [[effective information]] is used to evaluate the degree of causal influence of nodes and edges on downstream targets of each layer. Effective information (EI) for each neural network layer is defined as:

Marrow et al. <ref>Marrow S, Michaud E J, Hoel E. Examining the Causal Structures of Deep Neural Networks Using Information Theory[J]. Entropy, 2020, 22(12): 1429.</ref> introduced EI into [[neural networks]] to quantify and track the changes in the [[causal structure]] during training. Here, EI is used to evaluate the degree of causal influence of nodes and edges on downstream targets of each layer. EI for each neural network layer is defined as:

Here, <math>L_1</math> and <math>L_2</math> respectively represent the input and output layers connecting the neural network. Here, the input layer is do-intervened as a uniform distribution, and the mutual information between cause and effect is then calculated. [[Effective information]] can be decomposed into sensitivity and degeneracy. Here, sensitivity is defined as:

Here, <math>L_1</math> and <math>L_2</math> respectively represent the input and output layers connecting the neural network. Here, the input layer is do-intervened as a uniform distribution, and the mutual information between cause and effect is then calculated. EI can be decomposed into sensitivity and degeneracy. Here, sensitivity is defined as:

Here, it should be distinguished from the definition of effective information. Here, each neuron in the input layer is do-intervened separately, and then the mutual information calculated by each two neurons is accumulated as the definition of sensitivity. Degeneracy is obtained by the difference between effective information and sensitivity and is defined as:

Here, it should be distinguished from the definition of EI. Here, each neuron in the input layer is do-intervened separately, and then the mutual information calculated by each two neurons is accumulated as the definition of sensitivity. Degeneracy is obtained by the difference between EI and sensitivity and is defined as:

Observing effective information, including [[sensitivity]] and [[degeneracy]] changes during model training, reveals the model's generalization ability and aids scholars in understanding neural networks.

Observing EI, including [[sensitivity]] and [[degeneracy]] changes during model training, reveals the model's generalization ability and aids scholars in understanding neural networks.

===Application in artificial intelligence systems===

===Application in artificial intelligence systems===

Causal emergence theory has significant connections to [[artificial intelligence]]. These connections are evident in the following ways. First, the machine learning solution to the causal emergence identification problem is actually an application of [[causal representation learning]]. Second, technologies like maximizing [[effective information]] are anticipated to have applications in [[causal machine learning]].

Causal emergence theory has significant connections to [[artificial intelligence]]. These connections are evident in the following ways. First, the machine learning solution to the causal emergence identification problem is actually an application of [[causal representation learning]]. Second, technologies like maximizing EI are anticipated to have applications in [[causal machine learning]].

====Causal representation learning====

====Causal representation learning====

[[Causal representation learning]] is an emerging field in artificial intelligence. It attempts to combine two important fields in machine learning: [[representation learning]] and [[causal inference]]. By integrating their respective advantages, it aims to automatically extract important features and uncover causal relationships hidden within the data <ref>B. Sch ̈olkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, Y. Bengio, Toward causal representation learning, Proceedings of the IEEE 109 (5) (2021) 612–634.</ref>.Causal emergence identification based on effective information can be equivalent to a causal representation learning task. Identifying causal relationships in data is involves learning the underlying causal relationships and mechanisms. Specifically, we can regard the macroscopic state as a causal variable and the macroscopic dynamics as a causal mechanism by analogy. The coarse-graining strategy can be seen as an encoding process or representation from the original data to the causal variable, while the effective information serves as a measure of the causal effect strength of the mechanism.

[[Causal representation learning]] is an emerging field in artificial intelligence. It attempts to combine two important fields in machine learning: [[representation learning]] and [[causal inference]]. By integrating their respective advantages, it aims to automatically extract important features and uncover causal relationships hidden within the data <ref>B. Sch ̈olkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, Y. Bengio, Toward causal representation learning, Proceedings of the IEEE 109 (5) (2021) 612–634.</ref>.Causal emergence identification based on EI can be equivalent to a causal representation learning task. Identifying causal relationships in data is involves learning the underlying causal relationships and mechanisms. Specifically, we can regard the macroscopic state as a causal variable and the macroscopic dynamics as a causal mechanism by analogy. The coarse-graining strategy can be seen as an encoding process or representation from the original data to the causal variable, while the EI serves as a measure of the causal effect strength of the mechanism.

Since there are many similarities between the two, the techniques and concepts of the two fields can learn from each other. For example, [[causal representation learning]] technology can be applied to [[causal emergence identification]]. In turn, the learned abstract causal representation can be interpreted as a macroscopic state, thereby enhancing the interpretability of causal representation learning. However, there are also significant differences between the two, mainly including two points: 1) Causal representation learning assumes that there is a real [[causal mechanism]] behind it, and the data is generated by this causal mechanism. However, there may not be a "true causal relationship" between the macroscopic states and dynamics emerging; 2) The macroscopic state after coarse-graining in causal emergence is a low-dimensional description, but there is no such requirement in causal representation learning. From an epistemological perspective, there is no difference between the two, because both are extracting effective information from observational data to obtain representations with stronger causal effects.

Since there are many similarities between the two, the techniques and concepts of the two fields can learn from each other. For example, [[causal representation learning]] technology can be applied to [[causal emergence identification]]. In turn, the learned abstract causal representation can be interpreted as a macroscopic state, thereby enhancing the interpretability of causal representation learning. However, there are also significant differences between the two, mainly including two points: 1) Causal representation learning assumes that there is a real [[causal mechanism]] behind it, and the data is generated by this causal mechanism. However, there may not be a "true causal relationship" between the macroscopic states and dynamics emerging; 2) The macroscopic state after coarse-graining in causal emergence is a low-dimensional description, but there is no such requirement in causal representation learning. From an epistemological perspective, there is no difference between the two, because both are extracting EI from observational data to obtain representations with stronger causal effects.

====Application of effective information in causal machine learning====

====Application of effective information in causal machine learning====

Causal emergence can enhance the performance of [[machine learning]] in out-of-distribution scenarios. The do-intervention introduced in <math>EI</math> captures the causal dependence in the data generation process and suppresses spurious correlations. This approach supplements machine learning algorithms based on associations and establishing a connection between <math>EI</math> and [[out-of-distribution generalization]] (Out Of Distribution, abbreviated as OOD) <ref name="Emergence_and_causality_in_complex_systems">{{cite journal|author1=Yuan, B|author2=Zhang, J|author3=Lyu, A|author4=Wu, J|author5=Wang, Z|author6=Yang, M|author7=Liu, K|author8=Mou, M|author9=Cui, P|title=Emergence and causality in complex systems: A survey of causal emergence and related quantitative studies|journal=Entropy|year=2024|volume=26|issue=2|page=108|url=https://www.mdpi.com/1099-4300/26/2/108}}</ref>. Due to the universality of [[effective information]], causal emergence can be applied to supervised machine learning to evaluate the strength of the causal relationship between the feature space <math>X</math> and the target space <math>Y</math>. This application improves the prediction accuracy from cause (feature) to result (target). It is worth noting that direct fitting of observations from <math>X</math> to <math>Y</math> works well for standard prediction tasks with the i.i.d. assumption, which means that the training data and test data are [[independently and identically distributed]]. However, if samples are drawn from outside the training distribution, a generalization representation space from training to test environments must be learned. It is generally believed that causality generalizes better than [[statistical correlation]] <ref>Arjovsky, M.; Bottou, L.; Gulrajani, I.; Lopez-Paz, D. Invariant risk minimization. arXiv 2019, arXiv:1907.02893.</ref>; thus, causal emergence theory can serve as a standard for embedding causal relationships in the representation space. The occurrence of causal emergence reveals the potential causal factors of the target, thereby producing a robust representation space for out-of-distribution generalization. Causal emergence may provide a unified representation measure for out-of-distribution generalization based on causal theory. <math>EI</math> can also be regarded as an information-theoretic abstraction of the out-of-distribution generalization's reweighting-based debiasing technique. In addition, we conjecture that out-of-distribution generalization can be achieved while maximizing <math>EI</math>. <math>EI</math> may reach its peak at the intermediate stage of the original feature abstraction, which aligns with the idea of OOD generalization—less is more. Ideally, when causal emergence occurs at the peak of <math>EI</math>, all non-causal features are excluded and causal features are revealed, resulting in the most informative representation.

Causal emergence can enhance the performance of [[machine learning]] in out-of-distribution scenarios. The do-intervention introduced in <math>EI</math> captures the causal dependence in the data generation process and suppresses spurious correlations. This approach supplements machine learning algorithms based on associations and establishing a connection between <math>EI</math> and [[out-of-distribution generalization]] (Out Of Distribution, abbreviated as OOD) <ref name="Emergence_and_causality_in_complex_systems">{{cite journal|author1=Yuan, B|author2=Zhang, J|author3=Lyu, A|author4=Wu, J|author5=Wang, Z|author6=Yang, M|author7=Liu, K|author8=Mou, M|author9=Cui, P|title=Emergence and causality in complex systems: A survey of causal emergence and related quantitative studies|journal=Entropy|year=2024|volume=26|issue=2|page=108|url=https://www.mdpi.com/1099-4300/26/2/108}}</ref>. Due to the universality of EI, causal emergence can be applied to supervised machine learning to evaluate the strength of the causal relationship between the feature space <math>X</math> and the target space <math>Y</math>. This application improves the prediction accuracy from cause (feature) to result (target). It is worth noting that direct fitting of observations from <math>X</math> to <math>Y</math> works well for standard prediction tasks with the i.i.d. assumption, which means that the training data and test data are [[independently and identically distributed]]. However, if samples are drawn from outside the training distribution, a generalization representation space from training to test environments must be learned. It is generally believed that causality generalizes better than [[statistical correlation]] <ref>Arjovsky, M.; Bottou, L.; Gulrajani, I.; Lopez-Paz, D. Invariant risk minimization. arXiv 2019, arXiv:1907.02893.</ref>; thus, causal emergence theory can serve as a standard for embedding causal relationships in the representation space. The occurrence of causal emergence reveals the potential causal factors of the target, thereby producing a robust representation space for out-of-distribution generalization. Causal emergence may provide a unified representation measure for out-of-distribution generalization based on causal theory. <math>EI</math> can also be regarded as an information-theoretic abstraction of the out-of-distribution generalization's reweighting-based debiasing technique. In addition, we conjecture that out-of-distribution generalization can be achieved while maximizing <math>EI</math>. <math>EI</math> may reach its peak at the intermediate stage of the original feature abstraction, which aligns with the idea of OOD generalization—less is more. Ideally, when causal emergence occurs at the peak of <math>EI</math>, all non-causal features are excluded and causal features are revealed, resulting in the most informative representation.

The similarities and common features between the two fields can help us borrow ideas and techniques from one field to another. For example, an agent with a world model can interact with a complex system as a whole and obtain emergent causal laws from the interaction, thereby better helping us with the task of causal emergence identification. Maximizing effective information technology can enhance the causal characteristics of the world model in reinforcement learning.

The similarities and common features between the two fields can help us borrow ideas and techniques from one field to another. For example, an agent with a world model can interact with a complex system as a whole and obtain emergent causal laws from the interaction, thereby better helping us with the task of causal emergence identification. Maximizing EI technology can enhance the causal characteristics of the world model in reinforcement learning.

====Consciousness research====

====Consciousness research====

First of all, the proposal of the causal emergence theory is closely related to consciousness science. This is because the core indicator of the causal emergence theory, [[effective information]], was first proposed by [[Tononi]] in the quantitative theory of consciousness research, [[integrated information theory]]. After being modified, it was applied to Markov chains by [[Erik Hoel]] and the concept of causal emergence was proposed. Therefore, effective information can be seen as a by-product of quantitative consciousness science.

First of all, the proposal of the causal emergence theory is closely related to consciousness science. This is because the core indicator of the causal emergence theory, EI, was first proposed by [[Tononi]] in the quantitative theory of consciousness research, [[integrated information theory]]. After being modified, it was applied to Markov chains by [[Erik Hoel]] and the concept of causal emergence was proposed. Therefore, EI can be seen as a by-product of quantitative consciousness science.

For example, in Eastern philosophy, the [[five elements]] can be interpreted as  macroscopic states, with their mutual generation and restraint representing a macroscopic causal mechanism. The process of identifying these five states from all phenomena is a coarse-graining process dependent on the observer's ability to analogize. Therefore, the theory of five elements can be regarded as an abstract causal emergence theory for everything. Similarly, the concept of causal emergence can be extended to fields like traditional Chinese medicine, divination, and feng shui. The common point of these applications will be that its causal mechanism is simpler and possibly has stronger causality compared to Western science. However, the process of obtaining such an abstract coarse-graining is more complex and more dependent on experienced abstractors. This explains why Eastern philosophies all emphasize the self-cultivation of practitioners. This is because Eastern philosophical theories place significant complexity and computational demands on '''analogical thinking'''.

For example, in Eastern philosophy, the [[five elements]] can be interpreted as  macroscopic states, with their mutual generation and restraint representing a macroscopic causal mechanism. The process of identifying these five states from all phenomena is a coarse-graining process dependent on the observer's ability to analogize. Therefore, the theory of five elements can be regarded as an abstract causal emergence theory for everything. Similarly, the concept of causal emergence can be extended to fields like traditional Chinese medicine, divination, and feng shui. The common point of these applications will be that its causal mechanism is simpler and possibly has stronger causality compared to Western science. However, the process of obtaining such an abstract coarse-graining is more complex and more dependent on experienced abstractors. This explains why Eastern philosophies all emphasize the self-cultivation of practitioners. This is because Eastern philosophical theories place significant complexity and computational demands on '''analogical thinking'''.

==Critique==

==Critique==