更改

A simple way of quantifying complexity on a structural basis would be to count the number of components and/or interactions within a system.  An examination of the number of structural parts (McShea, 1996) and functional behaviors of organisms across evolution demonstrates that these measures increase over time, a finding that contributes to the ongoing debate over whether complexity grows as a result of natural selection.  However, numerosity alone may only be an indicator of complicatedness, but not necessarily of complexity.  Large and highly coupled systems may not be more complex than those that are smaller and less coupled.  For example, a very large system that is fully connected can be described in a compact manner and may tend to generate uniform behavior, while the description of a smaller but more heterogeneous system may be less compressible and its behavior may be more differentiated.

A simple way of quantifying complexity on a structural basis would be to count the number of components and/or interactions within a system.  An examination of the number of structural parts (McShea, 1996) and functional behaviors of organisms across evolution demonstrates that these measures increase over time, a finding that contributes to the ongoing debate over whether complexity grows as a result of natural selection.  However, numerosity alone may only be an indicator of complicatedness, but not necessarily of complexity.  Large and highly coupled systems may not be more complex than those that are smaller and less coupled.  For example, a very large system that is fully connected can be described in a compact manner and may tend to generate uniform behavior, while the description of a smaller but more heterogeneous system may be less compressible and its behavior may be more differentiated.

Effective complexity (Gell-Mann, 1995) measures the minimal description length of a system’s regularities.  As such, this measure is related to AIC, but it attempts to distinguish regular features from random or incidental ones and therefore belongs within the family of complexity measures that aim at capturing how much structure a system contains. The separation of regular features from random ones may be difficult for any given empirical system, and it may crucially depend on criteria supplied by an external observer.

Effective complexity (Gell-Mann, 1995) measures the minimal description length of a system’s regularities.  As such, this measure is related to AIC, but it attempts to distinguish regular features from random or incidental ones and therefore belongs within the family of complexity measures that aim at capturing how much structure a system contains. The separation of regular features from random ones may be difficult for any given empirical system, and it may crucially depend on criteria supplied by an external observer.

Physical complexity (Adami and Cerf, 2000) is related to effective complexity and is designed to estimate the complexity of any sequence of symbols that is about a physical world or environment.  As such the measure is particularly useful when applied to biological systems.  It is defined as the Kolmogorov complexity (AIC) that is shared between a sequence of symbols (such as a genome) and some description of the environment in which that sequence has meaning (such as an ecological niche).  Since the Kolmogorov complexity is not computable, neither is the physical complexity.  However, the average physical complexity of an ensemble of sequences (e.g. the set of genomes of an entire population of organisms) can be approximated by the mutual information between the ensembles of sequences (genomes) and their environment (ecology).  Experiments conducted in a digital ecology (Adami, 2002) demonstrated that the mutual information between self-replicating genomes and their environment increased along evolutionary time.  Physical complexity has also been used to estimate the complexity of biomolecules.  The structural and functional complexity of a set of RNA molecules were shown to positively correlate with physical complexity (Carothers et al., 2004), indicating a possible link between functional capacities of evolved molecular structures and the amount of information they encode.

Physical complexity (Adami and Cerf, 2000) is related to effective complexity and is designed to estimate the complexity of any sequence of symbols that is about a physical world or environment.  As such the measure is particularly useful when applied to biological systems.  It is defined as the Kolmogorov complexity (AIC) that is shared between a sequence of symbols (such as a genome) and some description of the environment in which that sequence has meaning (such as an ecological niche).  Since the Kolmogorov complexity is not computable, neither is the physical complexity.  However, the average physical complexity of an ensemble of sequences (e.g. the set of genomes of an entire population of organisms) can be approximated by the mutual information between the ensembles of sequences (genomes) and their environment (ecology).  Experiments conducted in a digital ecology (Adami, 2002) demonstrated that the mutual information between self-replicating genomes and their environment increased along evolutionary time.  Physical complexity has also been used to estimate the complexity of biomolecules.  The structural and functional complexity of a set of RNA molecules were shown to positively correlate with physical complexity (Carothers et al., 2004), indicating a possible link between functional capacities of evolved molecular structures and the amount of information they encode.

Statistical complexity (Crutchfield and Young, 1989) is a component of a broader theoretic framework known as computational mechanics, and can be calculated directly from empirical data.  To calculate the statistical complexity, each point in the time series is mapped to a corresponding symbol according to some partitioning scheme, so that the raw data is now a stream of consecutive symbols.  The symbol sequences are then clustered into causal states according to the following rule: two symbol sequences (i.e. histories of the dynamics) are contained in the same causal state if the conditional probability of any future symbol is identical for these two histories.  In other words, two symbol sequences are considered to be the same if, on average, they predict the same distribution of future dynamics. Once these causal states have been identified, the transition probabilities between causal states can be extracted from the data, and the long-run probability distribution over all causal states can be calculated. The statistical complexity is then defined as the Shannon entropy of this distribution over causal states.  Statistical complexity can be calculated analytically for abstract systems such as the logistic map, cellular automata and many basic Markov processes, and computational methods for constructing appropriate causal states in real systems, while taxing, exist and have been applied in a variety of contexts.

Statistical complexity (Crutchfield and Young, 1989) is a component of a broader theoretic framework known as computational mechanics, and can be calculated directly from empirical data.  To calculate the statistical complexity, each point in the time series is mapped to a corresponding symbol according to some partitioning scheme, so that the raw data is now a stream of consecutive symbols.  The symbol sequences are then clustered into causal states according to the following rule: two symbol sequences (i.e. histories of the dynamics) are contained in the same causal state if the conditional probability of any future symbol is identical for these two histories.  In other words, two symbol sequences are considered to be the same if, on average, they predict the same distribution of future dynamics. Once these causal states have been identified, the transition probabilities between causal states can be extracted from the data, and the long-run probability distribution over all causal states can be calculated. The statistical complexity is then defined as the Shannon entropy of this distribution over causal states.  Statistical complexity can be calculated analytically for abstract systems such as the logistic map, cellular automata and many basic Markov processes, and computational methods for constructing appropriate causal states in real systems, while taxing, exist and have been applied in a variety of contexts.

Predictive information (Bialek et al., 2001), while not in itself a complexity measure, can be used to separate systems into different complexity categories based on the principle of the extensivity of entropy.  Extensivity manifests itself, for example, in systems composed of increasing numbers of homogeneous independent random variables.  The Shannon entropy of such systems will grow linearly with their size.  This linear growth of entropy with system size is known as extensivity.  However, the constituent elements of a complex system are typically inhomogeneous and interdependent, so that as the number of random variables grows the entropy does not always grow linearly.  The manner in which a given system departs from extensivity can be used to characterize its complexity.

Predictive information (Bialek et al., 2001), while not in itself a complexity measure, can be used to separate systems into different complexity categories based on the principle of the extensivity of entropy.  Extensivity manifests itself, for example, in systems composed of increasing numbers of homogeneous independent random variables.  The Shannon entropy of such systems will grow linearly with their size.  This linear growth of entropy with system size is known as extensivity.  However, the constituent elements of a complex system are typically inhomogeneous and interdependent, so that as the number of random variables grows the entropy does not always grow linearly.  The manner in which a given system departs from extensivity can be used to characterize its complexity.

[[Image:complexity_figure2.jpg|thumb|400px|right|F2|Movie frames from a demonstration model of neural dynamics, consisting of 1600 spontaneously active Wison-Cowan neural mass units arranged on a sphere and coupled by excitatory connections.  Three cases are shown: sparse coupling (local connections only), uniform coupling (global connections only), and a mixture of local and global connections (forming a small-world network).  Neural complexity (Tononi et al., 1994; Sporns et al., 2000) is high only for the last case.]]

[[Image:complexity_figure2.jpg|thumb|400px|right|F2|Movie frames from a demonstration model of neural dynamics, consisting of 1600 spontaneously active Wison-Cowan neural mass units arranged on a sphere and coupled by excitatory connections.  Three cases are shown: sparse coupling (local connections only), uniform coupling (global connections only), and a mixture of local and global connections (forming a small-world network).  Neural complexity (Tononi et al., 1994; Sporns et al., 2000) is high only for the last case.]]

Neural complexity (Tononi et al., 1994), which may be applied to any empirically observed system including brains, is related to the extensivity of a system.  One of its building blocks is integration (also called multi-information), a multivariate extension of mutual information that estimates the total amount of statistical structure within an arbitrarily large system.  Integration is computed as the difference between the sum of the component’s individual entropies and the joint entropy of the system as a whole.  The distribution of integration across multiple spatial scales captures the complexity of the system.  Consider the following three cases (<figref>Complexity_figure2.jpg</figref>).  (1) A system with components that are statistically independent will exhibit globally disordered or random dynamics.  Its joint entropy will be exactly equal to the sum of the component entropies, and system integration will be zero, regardless of which spatial scale of the system is examined.  (2) Any statistical dependence between components will lead to a contraction of the system’s joint entropy relative to the individual entropies, resulting in positive integration.  If the components of a system are highly coupled and exhibit statistical dependencies as well as homogeneous dynamics (i.e. all components behave identically) then estimates of integration across multiple spatial scales of the system will, on average, follow a linear distribution.  (3) If statistical dependencies are inhomogeneous (for example involving groupings of components, modules, or hierarchical patterns) the distribution of integration will deviate from linearity.  The total amount of deviation is the system’s complexity.  The complexity of a random system is zero, while the complexity of a homogeneous coupled (regular) system is very low.  Systems with rich structure and dynamic behavior have high complexity.

Neural complexity (Tononi et al., 1994), which may be applied to any empirically observed system including brains, is related to the extensivity of a system.  One of its building blocks is integration (also called multi-information), a multivariate extension of mutual information that estimates the total amount of statistical structure within an arbitrarily large system.  Integration is computed as the difference between the sum of the component’s individual entropies and the joint entropy of the system as a whole.  The distribution of integration across multiple spatial scales captures the complexity of the system.  Consider the following three cases (<figref>Complexity_figure2.jpg</figref>).  (1) A system with components that are statistically independent will exhibit globally disordered or random dynamics.  Its joint entropy will be exactly equal to the sum of the component entropies, and system integration will be zero, regardless of which spatial scale of the system is examined.  (2) Any statistical dependence between components will lead to a contraction of the system’s joint entropy relative to the individual entropies, resulting in positive integration.  If the components of a system are highly coupled and exhibit statistical dependencies as well as homogeneous dynamics (i.e. all components behave identically) then estimates of integration across multiple spatial scales of the system will, on average, follow a linear distribution.  (3) If statistical dependencies are inhomogeneous (for example involving groupings of components, modules, or hierarchical patterns) the distribution of integration will deviate from linearity.  The total amount of deviation is the system’s complexity.  The complexity of a random system is zero, while the complexity of a homogeneous coupled (regular) system is very low.  Systems with rich structure and dynamic behavior have high complexity.

Applied to neural dynamics, complexity as defined by Tononi et al. (1994) was found to be associated with specific patterns of neural connectivity, for example those exhibiting attributes of small-world networks (Sporns et al., 2000).  Descendants of this complexity measure have addressed the matching of systems to an environment as well as their degeneracy, i.e. their capacity to produce functionally equivalent behaviors in different ways.  

 

 

Applied to neural dynamics, complexity as defined by Tononi et al. (1994) was found to be associated with specific patterns of neural connectivity, for example those exhibiting attributes of small-world networks (Sporns et al., 2000).  Descendants of this complexity measure have addressed the matching of systems to an environment as well as their degeneracy, i.e. their capacity to produce functionally equivalent behaviors in different ways.

 

==复杂网络==

==复杂网络==