更改

Causal Emergence (查看源代码)

2025年3月8日 (六) 16:30的版本

删除33字节、 2025年3月8日 (六) 16:30

第480行：第480行：

==Applications==

This subsection explores the potential applications of causal emergence in various [[complex systems]]. These include biological systems, [[neural networks]], brain nervous systems, and [[artificial intelligence]] (such as [[causal representation learning]], [[reinforcement learning based on world models]], and [[causal model abstraction]]). Additional applications include consciousness research and Chinese classical philosophy.

−

===Causal emergence in complex networks===

In 2020, Klein and Hoel improved the method of quantifying causal emergence on Markov chains to be applied to [[complex networks]] <ref>Klein B, Hoel E. The emergence of informative higher scales in complex networks[J]. Complexity, 2020, 20201-12.</ref>. The authors defined the [[Markov chain]] in the network with the help of [[random walkers]]. Placing random walkers on nodes is equivalent to intervening on nodes. They then defined the transition probability matrix between nodes based on the random walk probability. At the same time, the authors establish a connection between EI and the connectivity of the network. Connectivity is characterized by the uncertainty in the weights of outgoing and incoming node edges. Based on this, the EI in complex networks is defined. For detailed methods, refer to [[causal emergence in complex networks]].

−

The authors conducted experimental comparisons in artificial networks such as [[random network]] (ER), the [[preferential attachment network model]] (PA), and four types of real networks. They found that, for ER networks, the magnitude of EI only depends on the connection probability <math>p</math>, and as the network size increases, it will converge to the value <math>-\log_2p</math>. At the same time, a key finding shows that there is a phase transition point in the EI value, which approximately appears at the position where the [[average degree]] (<math><k></math>) of the network is equal to <math>\log_2N</math>. This also corresponds to the phase transition point where the random network structure does not contain more information as its scale increases with the increase of the connection probability. For preferential attachment model networks, the EI depends on the power-law exponent α of the network's degree distribution. When <math>\alpha<1.0</math>, the magnitude of EI increases as the network size grows. Conversely, when <math>\alpha>1.0</math>, the magnitude of EI decreases with network size. The case of <math>\alpha = 1.0</math> corresponds to the scale-free network, making the critical boundary for network growth. For real networks, the authors found that biological networks have the lowest EI because they exhibit significant noice. However, we can remove this noise through effective coarse-graining, which makes biological networks show more significant causal emergence phenomena than other types of networks. On the other hand, technical type networks are sparser and non-degenerate, resulting in higher average efficiency, more specific node relationships, and the highest EI. However, it is challenging to increase the causal emergence measure in these networks through coarse-graining.

−

In this article, the authors use the [[greedy algorithm]] to coarse-grain the network. However, for large-scale networks, this algorithm is very inefficient. Subsequently, Griebenow et al. <ref>Griebenow R, Klein B, Hoel E. Finding the right scale of a network: efficient identification of causal emergence through spectral clustering[J]. arXiv preprint arXiv:190807565, 2019.</ref> proposed a method based on [[spectral clustering]] to identify causal emergence in [[preferential attachment networks]]. Compared with the [[greedy algorithm]] and the [[gradient descent algorithm]], the [[spectral clustering]] algorithm has less computation time and the causal emergence of the found macroscopic network is also more significant.

第493行：第490行：

===Application on biological networks===

Furthermore, Klein et al. extended the method of [[causal emergence in complex networks]] to more [[biological networks]]. As mentioned earlier, biological networks have more noise, which makes it difficult for us to understand their internal operating principles. This noise arises both from inherent system noise and measurement or observation errors. Klein et al. <ref>Klein B, Swain A, Byrum T, et al. Exploring noise, degeneracy and determinism in biological networks with the einet package[J]. Methods in Ecology and Evolution, 2022, 13(4): 799-804.</ref> further explored the relationship and specific meanings among noise, [[degeneracy]] and [[determinism]] in biological networks, and drew some interesting conclusions.

−

For example, high [[determinism]] in gene regulatory networks (GRN) can be understood as one gene almost certainly leading to the expression of another gene. High [[degeneracy]] is also prevalent in biological systems throughout evolution. These two factors make it unclear at what scale biological systems should be analyzed to better understand their functions. Klein et al. <ref>Klein B, Hoel E, Swain A, et al. Evolution and emergence: higher order information structure in protein interactomes across the tree of life[J]. Integrative Biology, 2021, 13(12): 283-294.</ref> analyzed [[protein interaction networks]] of more than 1800 species and found that networks at macroscopic scales have less noise and [[degeneracy]]. At the same time, compared with nodes that do not participate in macroscopic scales, nodes in macroscopic scale networks are more resilient. Therefore, in order to meet the requirements of evolution, biological networks need to evolve macroscopic scales to increase certainty to enhance [[network resilience]] and improve the effectiveness of information transmission.

−

Hoel et al. in the article <ref>Hoel E, Levin M. Emergence of informative higher scales in biological systems: a computational toolkit for optimal prediction and control[J]. Communicative & Integrative Biology, 2020, 13(1): 108-118.</ref> further studied causal emergence in biological systems with the help of EI theory. The author applied EI to [[gene regulatory networks]] to identify the most informative heart development model to control the heart development of mammals. By quantifying the causal emergence in the [[largest connected component]] of the Saccharomyces cerevisiae gene network, the article reveals that informative macroscopic scales are ubiquitous in biology. It also shows that life mechanisms themselves often operate on macroscopic scales. This article also provides biologists with a computable tool to identify the most informative macroscopic scale, and can model, predict, control and understand complex biological systems on this basis.

−

Swain et al. in the article <ref>Swain A, Williams S D, Di Felice L J, et al. Interactions and information: exploring task allocation in ant colonies using network analysis[J]. Animal Behaviour, 2022, 18969-81.</ref> explored the influence of the interaction history of ant colonies on task allocation and task switching, and used EI to study how noise spreads among ants. The results indicate that historical interactions between ant colonies affects task allocation, while specific interaction types determine noise levels. In addition, even when ants switch functional groups, the emergent cohesion of ant colonies can ensure the stability of the colony. At the same time, different functional ant colonies also play different roles in maintaining the cohesion of the colony.

第505行：第499行：

===Application on artificial neural networks===

Marrow et al. <ref>Marrow S, Michaud E J, Hoel E. Examining the Causal Structures of Deep Neural Networks Using Information Theory[J]. Entropy, 2020, 22(12): 1429.</ref> introduced EI into [[neural networks]] to quantify and track the changes in the [[causal structure]] during training. Here, EI is used to evaluate the degree of causal influence of nodes and edges on downstream targets of each layer. EI for each neural network layer is defined as:

−

<math>

I(L_1;L_2|do(L_1=H^{max}))

</math>

−

Here, <math>L_1</math> and <math>L_2</math> respectively represent the input and output layers connecting the neural network. Here, the input layer is do-intervened as a uniform distribution, and the mutual information between cause and effect is then calculated. EI can be decomposed into sensitivity and degeneracy. Here, sensitivity is defined as:

−

<math>

\sum_{(i \in L_1,j \in L_2)}I(t_i;t_j|do(i=H^{max}))

</math>

−

Here, i and j respectively represent any neuron combination in the input layer and output layer. <math>t_i</math> and <math>t_j</math> respectively represent the state combinations of neurons in the input and output layers. These states are observed after intervening i as the maximum entropy distribution, under the condition that the neural network mechanism remains unchanged. In other words, intervening input neuron i to follow a uniform distribution causes a change in the output neuron. Then this value measures the mutual information between the two.

−

Here, it should be distinguished from the definition of EI. Here, each neuron in the input layer is do-intervened separately, and then the mutual information calculated by each two neurons is accumulated as the definition of sensitivity. Degeneracy is obtained by the difference between EI and sensitivity and is defined as:

−

<math>

I(L_1;L_2|do(L_1=H^{max}))-\sum_{(i \in L_1,j \in L_2)}I(t_i;t_j|do(i=H^{max}))

</math>.

−

Observing EI, including [[sensitivity]] and [[degeneracy]] changes during model training, reveals the model's generalization ability and aids scholars in understanding neural networks.

−

===Application on the brain nervous system===

The brain nervous system is an emergent multi-scale [[complex system]]. Luppi et al. <ref>Luppi AI, Mediano PA, Rosas FE, Allanson J, Pickard JD, Carhart-Harris RL, Williams GB, Craig MM, Finoia P, Owen AM, Naci L. A synergistic workspace for human consciousness revealed by integrated information decomposition. BioRxiv. 2020 Nov 26:2020-11.</ref> used [[integrated information decomposition]] to reveal the synergistic workspace of human [[consciousness]]. The authors constructed a three-layer architecture of brain cognition, including: external environment, specific modules and synergistic global space. The working principle of the brain mainly includes three stages: the first stage is responsible for collecting information from multiple different modules into the workspace; the second stage involves integrating the collected information within the workspace; the third stage focuses on broadcasting global information to other parts of the brain. The authors conducted experiments on three types of fMRI data in different resting states, including 100 normal people, 15 subjects participating in anesthesia experiments (including three different states before anesthesia, during anesthesia and recovery), and 22 subjects with chronic disorders of consciousness (DOC). This article uses [[integrated information decomposition]] to obtain [[synergistic information]] and [[redundant information]] It then employs the revised [[integrated information value]] <math>\Phi_R</math> to calculate the synergy and redundancy values between each two brain regions, determining whether each brain region plays a greater role is synergy or redundancy. By comparing the data of conscious individuals, they found that regions with significantly reduced [[integrated information]] in unconscious individuals corresponded to brain regions dominated by [[synergistic information]]. Meanwhile, these regions all belonged to functional regions such as DMN (Default Mode Network), allowing the authors to locate brain regions with a significant effect on the occurrence of consciousness.

−

===Application in artificial intelligence systems===

Causal emergence theory has significant connections to [[artificial intelligence]]. These connections are evident in the following ways. First, the machine learning solution to the causal emergence identification problem is actually an application of [[causal representation learning]]. Second, technologies like maximizing EI are anticipated to have applications in [[causal machine learning]].

−

====Causal representation learning====

[[Causal representation learning]] is an emerging field in artificial intelligence. It attempts to combine two important fields in machine learning: [[representation learning]] and [[causal inference]]. By integrating their respective advantages, it aims to automatically extract important features and uncover causal relationships hidden within the data <ref>B. Sch ̈olkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, Y. Bengio, Toward causal representation learning, Proceedings of the IEEE 109 (5) (2021) 612–634.</ref>.Causal emergence identification based on EI can be equivalent to a causal representation learning task. Identifying causal relationships in data is involves learning the underlying causal relationships and mechanisms. Specifically, we can regard the macroscopic state as a causal variable and the macroscopic dynamics as a causal mechanism by analogy. The coarse-graining strategy can be seen as an encoding process or representation from the original data to the causal variable, while the EI serves as a measure of the causal effect strength of the mechanism.

−

Since there are many similarities between the two, the techniques and concepts of the two fields can learn from each other. For example, [[causal representation learning]] technology can be applied to [[causal emergence identification]]. In turn, the learned abstract causal representation can be interpreted as a macroscopic state, thereby enhancing the interpretability of causal representation learning. However, there are also significant differences between the two, mainly including two points: 1) Causal representation learning assumes that there is a real [[causal mechanism]] behind it, and the data is generated by this causal mechanism. However, there may not be a "true causal relationship" between the macroscopic states and dynamics emerging; 2) The macroscopic state after coarse-graining in causal emergence is a low-dimensional description, but there is no such requirement in causal representation learning. From an epistemological perspective, there is no difference between the two, because both are extracting EI from observational data to obtain representations with stronger causal effects.

−

To better compare causal representation learning and causal emergence identification tasks, we list the following table:

−

{| class="wikitable" style="text-align:center;"

第569行：第550行：

|'''Goal'''||Finding the optimal representation of the original data to ensure that an independent causal mechanism can be achieved through the representation||Finding an effective coarse-graining strategy and macroscopic dynamics with strong causal effects

|}

−

====Application of effective information in causal machine learning====

Causal emergence can enhance the performance of [[machine learning]] in out-of-distribution scenarios. The do-intervention introduced in <math>EI</math> captures the causal dependence in the data generation process and suppresses spurious correlations. This approach supplements machine learning algorithms based on associations and establishing a connection between <math>EI</math> and [[out-of-distribution generalization]] (Out Of Distribution, abbreviated as OOD) <ref name="Emergence_and_causality_in_complex_systems">{{cite journal|author1=Yuan, B|author2=Zhang, J|author3=Lyu, A|author4=Wu, J|author5=Wang, Z|author6=Yang, M|author7=Liu, K|author8=Mou, M|author9=Cui, P|title=Emergence and causality in complex systems: A survey of causal emergence and related quantitative studies|journal=Entropy|year=2024|volume=26|issue=2|page=108|url=https://www.mdpi.com/1099-4300/26/2/108}}</ref>. Due to the universality of EI, causal emergence can be applied to supervised machine learning to evaluate the strength of the causal relationship between the feature space <math>X</math> and the target space <math>Y</math>. This application improves the prediction accuracy from cause (feature) to result (target). It is worth noting that direct fitting of observations from <math>X</math> to <math>Y</math> works well for standard prediction tasks with the i.i.d. assumption, which means that the training data and test data are [[independently and identically distributed]]. However, if samples are drawn from outside the training distribution, a generalization representation space from training to test environments must be learned. It is generally believed that causality generalizes better than [[statistical correlation]] <ref>Arjovsky, M.; Bottou, L.; Gulrajani, I.; Lopez-Paz, D. Invariant risk minimization. arXiv 2019, arXiv:1907.02893.</ref>; thus, causal emergence theory can serve as a standard for embedding causal relationships in the representation space. The occurrence of causal emergence reveals the potential causal factors of the target, thereby producing a robust representation space for out-of-distribution generalization. Causal emergence may provide a unified representation measure for out-of-distribution generalization based on causal theory. <math>EI</math> can also be regarded as an information-theoretic abstraction of the out-of-distribution generalization's reweighting-based debiasing technique. In addition, we conjecture that out-of-distribution generalization can be achieved while maximizing <math>EI</math>. <math>EI</math> may reach its peak at the intermediate stage of the original feature abstraction, which aligns with the idea of OOD generalization—less is more. Ideally, when causal emergence occurs at the peak of <math>EI</math>, all non-causal features are excluded and causal features are revealed, resulting in the most informative representation.

−

=====Causal model abstraction=====

In complex systems, noisy microscopic states are coarse-grained to produce macroscopic states with less noise, strengthening the causality of macroscopic dynamics. The same is true for causal models that explain various types of data. Due to the excessive complexity of the original model or limited computing resources, people often need to obtain a more abstract causal model. It is crucial to ensure that the abstract model maintains the [[causal mechanism]] of the original model as much as possible. This process is referred to as [[causal model abstraction]].

−

[[Causal model abstraction]] belongs to a subfield of artificial intelligence and plays a particularly important role in causal inference and model interpretability. This abstraction can help us better understand the hidden causal mechanisms in the data and the interactions between variables. Causal model abstraction involves optimizing a high-level model to simulate the causal effects of a low-level model <ref>Beckers, Sander, and Joseph Y. Halpern. "Abstracting causal models." Proceedings of the aaai conference on artificial intelligence. Vol. 33. No. 01. 2019.</ref>. A high-level model that generalizes the causal effects of a low-level model is referred to as a causal abstraction of the low-level model.

−

Causal model abstraction examines the interaction between causal relationships and model abstraction, which can be viewed as a coarse-graining process <ref>S. Beckers, F. Eberhardt, J. Y. Halpern, Approximate causal abstractions, in: Uncertainty in artificial intelligence, PMLR, 2020, pp. 606–615.</ref>. Therefore, causal emergence identification and causal model abstraction have many similarities. The original causal mechanism can be understood as microscopic dynamics, and the abstracted mechanism can be understood as macroscopic dynamics. In the [[neural information squeezer]] (NIS), researchers place restrictions on coarse-graining strategies and macroscopic dynamics. They require that the microscopic prediction error of macroscopic dynamics be small enough to exclude trivial solutions. This requirement aligns with causal model abstraction's goal of enrusing the abstracted causal model closely resembles the original model. However, there are also some differences between the two: 1) Causal emergence identification is to coarse-grain states or data, while causal model abstraction is to perform coarse-graining operations on models; 2) Causal model abstraction considers confounding factors, but this point is ignored in the discussion of causal emergence identification.

−

=====Reinforcement learning based on world models=====

[[Reinforcement learning]] based on [[world models]] assumes that the agent contains a world model to simulate the dynamics of its environment <ref>D. Ha, J. Schmidhuber, World models, arXiv preprint arXiv:1803.10122 (2018).</ref>. The dynamics of the world model can be learned through the interaction between the agent and the environment, thereby helping the [[agent]] to plan and make decisions in an uncertain environment. At the same time, in order to represent a complex environment, the world model must be a coarse-grained description of the environment. A typical world model architecture always contains an encoder and a decoder.

−

Reinforcement learning based on world models also has many similarities with causal emergence identification. The world model can also be regarded as a macroscopic dynamics. All states in the environment can be regarded as macroscopic states. These can be regarded as compressed states that ignore irrelevant information and can capture the most important causal features in the environment for the agent to make better decisions. In the planning process, the agent can also use the world model to simulate the dynamics of the real world.

−

The similarities and common features between the two fields can help us borrow ideas and techniques from one field to another. For example, an agent with a world model can interact with a complex system as a whole and obtain emergent causal laws from the interaction, thereby better helping us with the task of causal emergence identification. Maximizing EI technology can enhance the causal characteristics of the world model in reinforcement learning.

−

===Other potential applications===

In addition to the above application fields, the causal emergence theory may have 'significant potential applications for other important issues. For example, it has certain prospects in the research of consciousness issues and the modern scientific interpretation of Chinese classical philosophy.

−

====Consciousness research====

First of all, the proposal of the causal emergence theory is closely related to consciousness science. This is because the core indicator of the causal emergence theory, EI, was first proposed by [[Tononi]] in the quantitative theory of consciousness research, [[integrated information theory]]. After being modified, it was applied to Markov chains by [[Erik Hoel]] and the concept of causal emergence was proposed. Therefore, EI can be seen as a by-product of quantitative consciousness science.

−

Secondly, causal emergence, as an important concept in complex systems, also plays an important role in the research of consciousness science. A core question in consciousness research is whether consciousness is a macroscopic or a microscopic phenomenon? So far, there is no direct evidence to show the scale at which consciousness occurs. In-depth research on causal emergence, especially combined with experimental data of the brain nerve, may answer the scale at which consciousness occurs.

−

Thirdly, causal emergence may answer the question of free will. Questions about whether free will is real or an illusion are central to this discussion. In fact, if we accept the concept of causal emergence and admit that macroscopic variables will have [[causal force]] on microscopic variables, then all our decisions are actually made spontaneously by the brain system. Consciousness, in this view, is merely a certain level of explanation of this complex decision-making process. Therefore, free will is an emergent [[downward causation]]. The answers to these questions await further research of the causal emergence theory.

−

====Chinese classical philosophy====

Different from Western science and philosophy, [[Chinese classical philosophy]] retains a complete and different theoretical framework for explaining the universe. This framework includes [[yin and yang]], the [[five elements]], the [[eight trigrams]], as well as practices like [[divination]], [[feng shui]], and [[traditional Chinese medicine]], offering independent explanations for various phenomena in the universe. For a long time, the two sets of philosophies in the East and the West have always been difficult to integrate. The idea of causal emergence may provide a new explanation to bridge the conflict between Eastern and Western philosophies.

−

According to the causal emergence theory, the quality of a theory depends on the strength of causality, that is, the size of <math>EI</math>. Different coarse-graining schemes result in distinct macroscopic theories (macroscopic dynamics). It is very likely that when facing the same research object of complex systems, the Western philosophical and scientific system gives a set of relatively specific and microscopic causal mechanisms (dynamics), while Eastern philosophy gives a set of more coarsely grained macroscopic causal mechanisms. According to the causal emergence theory or the [[Causal Equivalence Principle]] proposed by Yurchenko, the two are completely likely to be compatible with each other. That is to say, for the same set of phenomena, the East and the West can make correct predictions and even intervention methods according to two different sets of causal mechanisms. It is also possible that in certain types of problems or phenomena, a more macroscopic causal mechanism is more explanatory or leads to a good solution. For some problems or phenomena, a more microscopic causal mechanism is more favorable.

−

For example, in Eastern philosophy, the [[five elements]] can be interpreted as macroscopic states, with their mutual generation and restraint representing a macroscopic causal mechanism. The process of identifying these five states from all phenomena is a coarse-graining process dependent on the observer's ability to analogize. Therefore, the theory of five elements can be regarded as an abstract causal emergence theory for everything. Similarly, the concept of causal emergence can be extended to fields like traditional Chinese medicine, divination, and feng shui. The common point of these applications will be that its causal mechanism is simpler and possibly has stronger causality compared to Western science. However, the process of obtaining such an abstract coarse-graining is more complex and more dependent on experienced abstractors. This explains why Eastern philosophies all emphasize the self-cultivation of practitioners. This is because Eastern philosophical theories place significant complexity and computational demands on '''analogical thinking'''.

Complexivist Ran

453

个编辑

更改

Causal Emergence (查看源代码)

2025年3月8日 (六) 16:30的版本

导航菜单

搜索