香农-哈特莱定理 Shannon–Hartley theorem

任务

生产人:ZQ 下一个任务：审校

In information theory, the Shannon–Hartley theorem tells the maximum rate at which information can be transmitted over a communications channel of a specified bandwidth in the presence of noise. It is an application of the noisy-channel coding theorem to the archetypal case of a continuous-time analog communications channel subject to Gaussian noise. The theorem establishes Shannon's channel capacity for such a communication link, a bound on the maximum amount of error-free information per time unit that can be transmitted with a specified bandwidth in the presence of the noise interference, assuming that the signal power is bounded, and that the Gaussian noise process is characterized by a known power or power spectral density. The law is named after Claude Shannon and Ralph Hartley.

在信息论中，香农-哈特利定理(Shannon–Hartley theorem)给出了在存在噪声的情况下信息通过给定带宽的信道中传输的最大速率。香农-哈特利定理是噪声信道编码定理中的一个应用，是一种受高斯噪声影响的连续时间模拟通信信道的典型案例。假设信号功率是有界的，且高斯噪声过程为已知功率或功率谱密度，那么在存在噪声干扰的情况下，香农-哈特利定理为这种通信链路建立了信道容量的界限，即每个时间单元能够以指定带宽传输最多无误差信息的范围。定理以克劳德 · 香农和拉尔夫 · 哈特利命名。

定理内容

The Shannon–Hartley theorem states the channel capacity {\displaystyle C}C，meaning the theoretical tightest upper bound on the information rate of data that can be communicated at an arbitrarily low error rate using an average received signal power S through an analog communication channel subject to additive white Gaussian noise (AWGN) of power N.

香农-哈特利定理给出了信道容量[math]\displaystyle{ C }[/math]的计算方法，表示理论上的信道传输速率的上限可以用信号的平均接受功率S以任意错误率通过模拟通信信道传输，并且会受到加性高斯白噪声（AWGN）的影响:

[math]\displaystyle{ C = B\log _{2}{(1+\frac{S}{N})} }[/math]

where

{\displaystyle C}C is the channel capacity in bits per second, a theoretical upper bound on the net bit rate (information rate, sometimes denoted {\displaystyle I}I) excluding error-correction codes; {\displaystyle B}B is the bandwidth of the channel in hertz (passband bandwidth in case of a bandpass signal); {\displaystyle S}S is the average received signal power over the bandwidth (in case of a carrier-modulated passband transmission, often denoted C), measured in watts (or volts squared); {\displaystyle N}N is the average power of the noise and interference over the bandwidth, measured in watts (or volts squared); and {\displaystyle S/N}S/N is the signal-to-noise ratio (SNR) or the carrier-to-noise ratio (CNR) of the communication signal to the noise and interference at the receiver (expressed as a linear power ratio, not as logarithmic decibels).

其中:

• [math]\displaystyle{ C }[/math]为信道容量，单位为比特每秒或者奈特每秒等。为理论上的最大比特率（即信息速率，也可用[math]\displaystyle{ I }[/math]表示），不包括纠错码。

• [math]\displaystyle{ B }[/math]是信道的带宽，单位为赫兹（在带通信号的情况下为通带带宽）；

• [math]\displaystyle{ S }[/math]是带宽上的平均接收信号功率（在载波调制通带传输的情况下，通常表示为[math]\displaystyle{ C }[/math]），以瓦特（或伏特的平方）为单位；

• [math]\displaystyle{ N }[/math]是在带宽上的噪声干扰的平均功率，以瓦特（或伏特的平方）为单位；

• [math]\displaystyle{ \frac{S}{N} }[/math]为信噪比（SNR）。

历史发展

During the late 1920s, Harry Nyquist and Ralph Hartley developed a handful of fundamental ideas related to the transmission of information, particularly in the context of the telegraph as a communications system. At the time, these concepts were powerful breakthroughs individually, but they were not part of a comprehensive theory. In the 1940s, Claude Shannon developed the concept of channel capacity, based in part on the ideas of Nyquist and Hartley, and then formulated a complete theory of information and its transmission.

在1920年代后期，哈里·奈奎斯特（Harry Nyquist）和拉尔夫·哈特利（Ralph Hartley）提出了信息传输的基本思想，特别是在电报作为通信系统的背景下。当时，这些概念的提出都是非常大的突破，但这些都不是没有综合统一起来的理论。1940年代，克劳德·香农（Claude Shannon）基于奈奎斯特（Nyquist）和哈特利（Hartley）的思想提出了信道容量的概念，然后制定了完整的信息及其传播的理论。

奈奎斯特速率 Nyquist rate

In 1927, Nyquist determined that the number of independent pulses that could be put through a telegraph channel per unit time is limited to twice the bandwidth of the channel. In symbolic notation,

1927年，奈奎斯特发现单位时间可通过电报信道发送的独立脉冲数最大只能为该信道带宽的两倍。公式表示如下：

[math]\displaystyle{ f_{p} \lt 2B }[/math]

where {\displaystyle f_{p}} f_{p} is the pulse frequency (in pulses per second) and {\displaystyle B} B is the bandwidth (in hertz). The quantity {\displaystyle 2B} 2B later came to be called the Nyquist rate, and transmitting at the limiting pulse rate of {\displaystyle 2B} 2B pulses per second as signalling at the Nyquist rate. Nyquist published his results in 1928 as part of his paper "Certain topics in Telegraph Transmission Theory".

其中[math]\displaystyle{ f_{p} }[/math]为脉冲频率(单位为脉冲/s)，[math]\displaystyle{ B }[/math]为带宽（单位为赫兹）。公式中，2B后来被称为奈奎斯特速率，表示传输的极限速率为2B脉冲每秒。奈奎斯特在1928年发表该研究成果在论文《Certain topics in Telegraph Transmission Theory》中。

哈特利定律 Hartley's law

During 1928, Hartley formulated a way to quantify information and its line rate (also known as data signalling rate R bits per second).[1] This method, later known as Hartley's law, became an important precursor for Shannon's more sophisticated notion of channel capacity.

1928年，哈特利提出了一种量化信息及其线速（也被称之为数据信令速率，R比特每秒）的方法。这种方法后来被称之为哈特利定律，为香农提出更加复杂的信道容量的概念奠定了基础。

Hartley argued that the maximum number of distinguishable pulse levels that can be transmitted and received reliably over a communications channel is limited by the dynamic range of the signal amplitude and the precision with which the receiver can distinguish amplitude levels. Specifically, if the amplitude of the transmitted signal is restricted to the range of [−A ... +A] volts, and the precision of the receiver is ±ΔV volts, then the maximum number of distinct pulses M is given by

哈特利认为，在保证可靠性的条件下，信道中能够传输和接收的可分辨的最大脉冲会受到两个因素的影响和限制，一个为信号振幅的动态范围，另一个为接收机能够分辨的振幅电平的精度。具体来说，如果发射信号的振幅大小范围为[-a,+a]伏，接收机的精度为±ΔV伏，那么不同的脉冲的最大数m满足以下公式：

[math]\displaystyle{ M = 1 + \frac{A}{\bigtriangleup V} }[/math]

By taking information per pulse in bit/pulse to be the base-2-logarithm of the number of distinct messages M that could be sent, Hartley[2] constructed a measure of the line rate R as:

然后可以根据一下公式计算出先速率R的值：

[math]\displaystyle{ R = f_{p}\log _{2}{M} }[/math]

where {\displaystyle f_{p}} f_{p} is the pulse rate, also known as the symbol rate, in symbols/second or baud.

公式中[math]\displaystyle{ f_{p} }[/math]为脉冲速率，也称之为符号速率，单位为符号/秒或波特。

Hartley then combined the above quantification with Nyquist's observation that the number of independent pulses that could be put through a channel of bandwidth {\displaystyle B} B hertz was {\displaystyle 2B} 2B pulses per second, to arrive at his quantitative measure for achievable line rate.

哈特利随后将上述结论与奈奎斯特的观察结合起来，即B赫兹的带宽信道中可以传输的独立脉冲为2B脉冲每秒，由此可以计算出线速率。

Hartley's law is sometimes quoted as just a proportionality between the analog bandwidth, {\displaystyle B} B, in Hertz and what today is called the digital bandwidth, {\displaystyle R} R, in bit/s.[3] Other times it is quoted in this more quantitative form, as an achievable line rate of {\displaystyle R} R bits per second:

哈特利定律有时候被用来描述两种比例关系，一是以赫兹为单位的模拟带宽B，二是以比特/s为单位的数字带宽。哈特利定律还被用来计算线速率R的取值范围：

[math]\displaystyle{ R \leq 2B\log _{2}{M} }[/math]

Hartley did not work out exactly how the number M should depend on the noise statistics of the channel, or how the communication could be made reliable even when individual symbol pulses could not be reliably distinguished to M levels; with Gaussian noise statistics, system designers had to choose a very conservative value of {\displaystyle M} M to achieve a low error rate.

哈特利并没有准确的给出M应该如何依赖信道的噪声统计，以及在无法将单个符号脉冲可靠的区分为M电平的情况下如何使通信可靠。所以在高斯噪声存在时，系统设计者需要选择非常保守的M值，从而降低错误率。

The concept of an error-free capacity awaited Claude Shannon, who built on Hartley's observations about a logarithmic measure of information and Nyquist's observations about the effect of bandwidth limitations.

在哈特利关于信息对数测量的观察和奈奎斯特的带宽限制的基础上，香农提出了无差错容量的概念。

Hartley's rate result can be viewed as the capacity of an errorless M-ary channel of {\displaystyle 2B} 2B symbols per second. Some authors refer to it as a capacity. But such an errorless channel is an idealization, and if M is chosen small enough to make the noisy channel nearly errorless, the result is necessarily less than the Shannon capacity of the noisy channel of bandwidth {\displaystyle B} B, which is the Hartley–Shannon result that followed later.

哈特利定律可以看作是无误差M信道的容量为2B符号每秒。有些研究者将其称之为容量。但是这种无误差的信道是理想条件下的，如果M足够小以至于使有噪声的信道几乎没有误差，那么计算结果必然会小于带宽的有噪信道带宽B，即为之后哈特利-香农定律的结论。

噪声的信道编码定理和容量

Claude Shannon's development of information theory during World War II provided the next big step in understanding how much information could be reliably communicated through noisy channels. Building on Hartley's foundation, Shannon's noisy channel coding theorem (1948) describes the maximum possible efficiency of error-correcting methods versus levels of noise interference and data corruption.[5][6] The proof of the theorem shows that a randomly constructed error-correcting code is essentially as good as the best possible code; the theorem is proved through the statistics of such random codes.

克劳德香农（Claude Shannon）在第二次世界大战中对信息论的研究为在有噪信道中进行可靠信息传输的突破提供了基础。在哈特利的基础上，对于噪声干扰和数据损坏水平，香农的噪声信道编码定理描述了纠错算法的最大效率。通过对随机编码的统计结果，定理表明随机构造的前向错误纠正本质上是最好的编码方式。

Shannon's theorem shows how to compute a channel capacity from a statistical description of a channel, and establishes that given a noisy channel with capacity C and information transmitted at a line rate {\displaystyle R} R, then if

香农定理给出根据信道的统计描述来计算信道容量的方法，并且证明了当在一个给定容量为C的有噪信道中，传输线速率

从统计描述信道的统计描述计算信道容量的方法，并且证明了在一个有容量 c 的噪声信道中，信息以一个线速率传输为R时有：

[math]\displaystyle{ R \lt C }[/math]

there exists a coding technique which allows the probability of error at the receiver to be made arbitrarily small. This means that theoretically, it is possible to transmit information nearly without error up to nearly a limit of {\displaystyle C} C bits per second.

此时的可以理解为存在一种编码方式可以使得接收端的出错率任意小。这意味着从理论上来讲，几乎可以毫无差错的传输信息，最高可以大道C位每秒的速率上限。

上面的不等式反过来同样重要：

[math]\displaystyle{ C \lt R }[/math]

the probability of error at the receiver increases without bound as the rate is increased. So no useful information can be transmitted beyond the channel capacity. The theorem does not address the rare situation in which rate and capacity are equal.

反过来的含义为：随着速率的增加，接收端的错误率会一直增加。所以超过信道容量后就不能传输。但是该定理没有解决速率和容量相等的情况（R=C）。

The Shannon–Hartley theorem establishes what that channel capacity is for a finite-bandwidth continuous-time channel subject to Gaussian noise. It connects Hartley's result with Shannon's channel capacity theorem in a form that is equivalent to specifying the M in Hartley's line rate formula in terms of a signal-to-noise ratio, but achieving reliability through error-correction coding rather than through reliably distinguishable pulse levels.

香农-哈特利定理定义了受高斯噪声影响的有限带宽连续时间信道的传输容量。它将哈特利的结果与香农的信道容量定理联系起来，其形式等效于将香农信道容量公式中的信噪比替换成哈特利的线速公式中的M，但是是通过纠错编码而不是可区分的脉冲电平来实现可靠性。

If there were such a thing as a noise-free analog channel, one could transmit unlimited amounts of error-free data over it per unit of time (Note: An infinite-bandwidth analog channel can't transmit unlimited amounts of error-free data, without infinite signal power). Real channels, however, are subject to limitations imposed by both finite bandwidth and nonzero noise.

如果存在无噪声的模拟信道，那么每单位时间就可以传输无限量的无错数据(注意: 无限带宽的模拟信道在没有无限信号功率的情况下不能传输无限量的无错数据)。 然而，实际信道会受到有限带宽和非零噪声的限制。

Bandwidth and noise affect the rate at which information can be transmitted over an analog channel. Bandwidth limitations alone do not impose a cap on the maximum information rate because it is still possible for the signal to take on an indefinitely large number of different voltage levels on each symbol pulse, with each slightly different level being assigned a different meaning or bit sequence. Taking into account both noise and bandwidth limitations, however, there is a limit to the amount of information that can be transferred by a signal of a bounded power, even when sophisticated multi-level encoding techniques are used.

带宽和噪声会影响信息在模拟信道上的传输速率。带宽限制本身并不限制最大信息传输速率，因为信号仍然可能在每个符号脉冲上承受无限多的不同电平，每个稍微不同的被赋予不同的意义或位序列。 但是，考虑到噪声和带宽限制，即使采用复杂的多级编码技术，每个略有不同的电平被赋予不同的含义或位序列。然而，考虑到噪声和带宽限制，即使用复杂的多级编码技术，也会限制有限功率的信号传输的信息量。

In the channel considered by the Shannon–Hartley theorem, noise and signal are combined by addition. That is, the receiver measures a signal that is equal to the sum of the signal encoding the desired information and a continuous random variable that represents the noise. This addition creates uncertainty as to the original signal's value. If the receiver has some information about the random process that generates the noise, one can in principle recover the information in the original signal by considering all possible states of the noise process. In the case of the Shannon–Hartley theorem, the noise is assumed to be generated by a Gaussian process with a known variance. Since the variance of a Gaussian process is equivalent to its power, it is conventional to call this variance the noise power.

在香农-哈特利定理考虑的信道中，噪声和信号会叠加。也就是说，接收端测量的信号等于编码信息和随机噪声信号之和。这种叠加产生了原始信号值的不确定性。如果接收端有一些有关产生噪声的随机过程的信息，则理论上可以通过噪声的所有可能状态来恢复原始信号信息。香农-哈特利定理中，假定的噪声为已知方差的高斯过程。由于高斯过程的方差等于其幂，因此通常将此方差称为噪声功率。

Such a channel is called the Additive White Gaussian Noise channel, because Gaussian noise is added to the signal; "white" means equal amounts of noise at all frequencies within the channel bandwidth. Such noise can arise both from random sources of energy and also from coding and measurement error at the sender and receiver respectively. Since sums of independent Gaussian random variables are themselves Gaussian random variables, this conveniently simplifies analysis, if one assumes that such error sources are also Gaussian and independent.

这样的信道称之为加性高斯白噪声信道，名字源于高斯噪声被叠加到信号中。“白色”表示在信道带宽内的所有频率上相等数量的噪声。这种噪声既可能来自随机能源，也可能分别来自发射端和接收端的编码和测量误差。由于独立的高斯随机变量之和本身就是高斯随机变量，因此，如果人们假设这样的误差源也是高斯且独立的，则可以进行方便的简化分析。

定理含义

香农的信道容量定律和哈特莱定律比较

Comparing the channel capacity to the information rate from Hartley's law, we can find the effective number of distinguishable levels M 将信道容量和哈特莱定律中的信息速率比较，我们能发现可区分的电平的有效数目M：

[math]\displaystyle{ 2B2B\log _{2}{M} = B2B\log _{2}{(1+ \frac{S}{N})} }[/math]

[math]\displaystyle{ M = \sqrt{1 + \frac{S}{N}} }[/math]

The square root effectively converts the power ratio back to a voltage ratio, so the number of levels is approximately proportional to the ratio of signal RMS amplitude to noise standard deviation.

平方根将功率比转换回电压比，所以电平数与信号RMS幅度和噪声标准偏差之比成正比（近似）。

频率相关(有色噪声)的情况

In the simple version above, the signal and noise are fully uncorrelated, in which case {\displaystyle S+N} S+N is the total power of the received signal and noise together. A generalization of the above equation for the case where the additive noise is not white (or that the {\displaystyle S/N} S/N is not constant with frequency over the bandwidth) is obtained by treating the channel as many narrow, independent Gaussian channels in parallel:

在前面的内容中，信号和噪声是完全不相关的，所以[math]\displaystyle{ S + N }[/math]是接收信号和噪声的总功率。对于加性噪声不是白色的情况（或S/N在带宽上的频率上不是恒定的）,通过将信道并行地看作许多窄的、独立的高斯信道，可以得到上述方程:

[math]\displaystyle{ C = \int_{0}^{B}log_{2}{(1 + \frac{S(f)}{N(f)})}df }[/math]

其中

• [math]\displaystyle{ C }[/math]为信道容量，单位为比特每秒

• [math]\displaystyle{ B }[/math]为信道的带宽，单位为赫兹

• [math]\displaystyle{ S(f) }[/math]为信号的功率谱

• [math]\displaystyle{ N(f) }[/math]为噪声功率谱

• [math]\displaystyle{ f }[/math]为频率，单位为赫兹

Note: the theorem only applies to Gaussian stationary process noise. This formula's way of introducing frequency-dependent noise cannot describe all continuous-time noise processes. For example, consider a noise process consisting of adding a random wave whose amplitude is 1 or −1 at any point in time, and a channel that adds such a wave to the source signal. Such a wave's frequency components are highly dependent. Though such a noise may have a high power, it is fairly easy to transmit a continuous signal with much less power than one would need if the underlying noise was a sum of independent noises in each frequency band.

该公式仅适用于高斯平稳过程。

近似计算

For large or small and constant signal-to-noise ratios, the capacity formula can be approximated:

对于恒定的信噪比，信道容量公式可以近似计算。

有限带宽

When the SNR is large (S/N >> 1), the logarithm is approximated by

当SNR较大时(S/N>1)，对数近似为

[math]\displaystyle{ log_{2}{(1 + \frac{S}{N})} \approx log_2{\frac{S}{N}} = \frac{ln10}{ln2}\cdot log_{10}{\frac{S}{N}} \approx 3.32\cdot log_{10}{\frac{S}{N}} }[/math]

此情况下，信道容量在功率上取对数，而在带宽上是近似线性的（N还会随着带宽的增加而增加，从而产生对数效应，所以是近似线性）。称为有限带宽机制。

[math]\displaystyle{ C \approx 3.32\cdot B \cdot SNR(in dB) }[/math]

其中

[math]\displaystyle{ SNR(in dB) = 10 log_{10}{\frac{S}{N}} }[/math]

有限功率

类似地，当SNR较小时（S/N << 1），则可以使用对数的近似值：

[math]\displaystyle{ log_{2}{(1 + \frac{S}{N})} = \frac{1}{ln2} \cdot ln(1 + \frac{S}{N}) \approx \frac{1}{ln2} \cdot \frac{S}{N} \approx 1.44 \cdot \frac{S}{N} }[/math]

此时信道容量是线性变化的。称为有限功率机制

[math]\displaystyle{ C \approx 1.44\cdot B \cdot \frac{S}{N} }[/math]

In this low-SNR approximation, capacity is independent of bandwidth if the noise is white, of spectral density {\displaystyle N_{0}} N_{0} watts per hertz, in which case the total noise power is {\displaystyle N=B\cdot N_{0}} {\displaystyle N=B\cdot N_{0}}.

这种低SNR近似值时，如果噪声为白色，则容量与带宽无关，且频谱密度较高。谱密度为[math]\displaystyle{ N_{0} }[/math]的时候，总的噪声功率为[math]\displaystyle{ N = B \cdot N_{0} }[/math]。

[math]\displaystyle{ C \approx 1.44 \cdot \frac{S}{N} }[/math]