更改

====Application of effective information in causal machine learning====

====Application of effective information in causal machine learning====

Causal emergence can enhance the performance of [[machine learning]] in out-of-distribution scenarios. The do-intervention introduced in <math>EI</math> captures the causal dependence in the data generation process and suppresses spurious correlations. This approach supplements machine learning algorithms based on associations and establishing a connection between <math>EI</math> and [[out-of-distribution generalization]] (Out Of Distribution, abbreviated as OOD) <ref name="Emergence_and_causality_in_complex_systems">{{cite journal|author1=Yuan, B|author2=Zhang, J|author3=Lyu, A|author4=Wu, J|author5=Wang, Z|author6=Yang, M|author7=Liu, K|author8=Mou, M|author9=Cui, P|title=Emergence and causality in complex systems: A survey of causal emergence and related quantitative studies|journal=Entropy|year=2024|volume=26|issue=2|page=108|url=https://www.mdpi.com/1099-4300/26/2/108}}</ref>. Due to the universality of EI, causal emergence can be applied to supervised machine learning to evaluate the strength of the causal relationship between the feature space <math>X</math> and the target space <math>Y</math>. This application improves the prediction accuracy from cause (feature) to effect (target). It is worth noting that direct fitting of observations from <math>X</math> to <math>Y</math> works well for standard prediction tasks with the i.i.d. assumption, which means that the training data and test data are [[independently and identically distributed]]. However, if samples are drawn from outside the training distribution, a generalization representation space from training to test environments must be learned. It is generally believed that causality generalizes better than [[statistical correlation]] <ref>Arjovsky, M.; Bottou, L.; Gulrajani, I.; Lopez-Paz, D. Invariant risk minimization. arXiv 2019, arXiv:1907.02893.</ref>; thus, causal emergence theory can serve as a standard for embedding causal relationships in the representation space. The occurrence of causal emergence reveals the potential causal factors of the target, thereby producing a robust representation space for out-of-distribution generalization. Causal emergence may provide a unified representation measure for out-of-distribution generalization based on causal theory. <math>EI</math> can also be regarded as an information-theoretic abstraction of the out-of-distribution generalization's reweighting-based debiasing technique. In addition, we conjecture that out-of-distribution generalization can be achieved while maximizing <math>EI</math>. <math>EI</math> may reach its peak at the intermediate stage of the original feature abstraction, which aligns with the idea of OOD generalization—less is more. Ideally, when causal emergence occurs at the peak of <math>EI</math>, all non-causal features are excluded and causal features are revealed, resulting in the most informative representation.

Causal emergence can enhance the performance of [[machine learning]] in out-of-distribution scenarios. The do-intervention introduced in <math>EI</math> captures the causal dependence in the data generation process and suppresses spurious correlations. This approach supplements machine learning algorithms based on associations and establishing a connection between <math>EI</math> and [[out-of-distribution generalization]] (Out Of Distribution, abbreviated as OOD) <ref name="Emergence_and_causality_in_complex_systems">{{cite journal|author1=Yuan, B|author2=Zhang, J|author3=Lyu, A|author4=Wu, J|author5=Wang, Z|author6=Yang, M|author7=Liu, K|author8=Mou, M|author9=Cui, P|title=Emergence and causality in complex systems: A survey of causal emergence and related quantitative studies|journal=Entropy|year=2024|volume=26|issue=2|page=108|url=https://www.mdpi.com/1099-4300/26/2/108}}</ref>. Due to the universality of EI, causal emergence can be applied to supervised machine learning to evaluate the strength of the causal relationship between the feature space <math>X</math> and the target space <math>Y</math>. This application improves the prediction accuracy from cause (feature) to effect (target). It is worth noting that direct fitting of observations from <math>X</math> to <math>Y</math> works well for standard prediction tasks with the i.i.d. assumption, which means that the training data and test data are [[independent and identically distributed]]. However, if samples are drawn from outside the training distribution, a generalization representation space from training to test environments must be learned. It is generally believed that causality generalizes better than [[statistical correlation]] <ref>Arjovsky, M.; Bottou, L.; Gulrajani, I.; Lopez-Paz, D. Invariant risk minimization. arXiv 2019, arXiv:1907.02893.</ref>; thus, causal emergence theory can serve as a standard for embedding causal relationships in the representation space. The occurrence of causal emergence reveals the potential causal factors of the target, thereby producing a robust representation space for out-of-distribution generalization. Causal emergence may provide a unified representation measure for out-of-distribution generalization based on causal theory. <math>EI</math> can also be regarded as an information-theoretic abstraction of the out-of-distribution generalization's reweighting-based debiasing technique. In addition, we conjecture that out-of-distribution generalization can be achieved while maximizing <math>EI</math>. <math>EI</math> may reach its peak at the intermediate stage of the original feature abstraction, which aligns with the idea of OOD generalization—less is more. Ideally, when causal emergence occurs at the peak of <math>EI</math>, all non-causal features are excluded and causal features are revealed, resulting in the most informative representation.

=====Causal model abstraction=====

=====Causal model abstraction=====