7,129
个编辑
更改
→信息论
在信息论中，熵是接收之前丢失的信息量的量度，有时也称为香农熵。<ref>{{cite book|first=Roger |last=Balian |chapter=Entropy, a Protean concept|editor-last=Dalibard|editor-first=Jean|title=Poincaré Seminar 2003: Bose-Einstein condensation – entropy|year=2004|publisher=Birkhäuser|location=Basel|isbn=978-3-7643-7116-6|pages=119–144}}</ref> 香农熵是信息理论和[https://en.wikipedia.org/wiki/Maximum_entropy_thermodynamics 最大熵热力学]中广泛使用的一般概念。它最初是由[https://en.wikipedia.org/wiki/Claude_Shannon 克劳德·香农 Claude Shannon]于1948年设计的，用于研究传输消息中的信息量。但是，信息熵的定义相当笼统，用离散的概率<math>p_i</math>表示，有
在信息论中，熵是接收之前丢失的信息量的量度，有时也称为香农熵。<ref>{{cite book|first=Roger |last=Balian |chapter=Entropy, a Protean concept|editor-last=Dalibard|editor-first=Jean|title=Poincaré Seminar 2003: Bose-Einstein condensation – entropy|year=2004|publisher=Birkhäuser|location=Basel|isbn=978-3-7643-7116-6|pages=119–144}}</ref> 香农熵是信息理论和[https://en.wikipedia.org/wiki/Maximum_entropy_thermodynamics 最大熵热力学]中广泛使用的一般概念。它最初是由[https://en.wikipedia.org/wiki/Claude_Shannon 克劳德·香农 Claude Shannon]于1948年设计的，用于研究传输消息中的信息量。但是，信息熵的定义相当笼统，用离散的概率<math>p_i</math>表示，有
<math>H(X) = -\sum_{i=1}^n p(x_i) \log p(x_i).</math>
::<math>H(X) = -\sum_{i=1}^n p(x_i) \log p(x_i).</math>
在发送消息的情况下，这些概率是实际发送特定消息的概率，而消息系统的熵是对消息中平均信息量的度量。对于相等概率的情况（即，每条消息均具有同等概率），香农熵（以位为单位）只是确定消息内容所需的是/否问题的数量。<ref name="Perplexed" />
在发送消息的情况下，这些概率是实际发送特定消息的概率，而消息系统的熵是对消息中平均信息量的度量。对于相等概率的情况（即，每条消息均具有同等概率），香农熵（以位为单位）只是确定消息内容所需的是/否问题的数量。<ref name="Perplexed" />
信息熵和热力学熵之间的联系问题是一个有争议的话题。大多数作者认为两者之间有联系，<ref>{{Cite book|last=Brillouin|first=Leon|title=Science and Information Theory|year= 1956|isbn=978-0-486-43918-1}}</ref><ref name="Georgescu-Roegen 1971">{{Cite book|last=Georgescu-Roegen|first=Nicholas|title=The Entropy Law and the Economic Process|publisher=Harvard University Press|year=1971|isbn=978-0-674-25781-8 |url = https://archive.org/details/entropylawe00nich}}</ref><ref>{{Cite book|last=Chen|first=Jing|title=The Physical Foundation of Economics – an Analytical Thermodynamic Theory|publisher=World Scientific|year=2005|isbn=978-981-256-323-1}}</ref><ref>{{cite journal | last1 = Kalinin | first1 = M.I. | last2 = Kononogov | first2 = S.A. | year = 2005 | title = Boltzmann's constant | url = | journal = Measurement Techniques | volume = 48 | issue = 7| pages = 632–636 | doi=10.1007/s11018-005-0195-9}}</ref><ref>{{cite book|last1=Ben-Naim|first1=Arieh|title=Entropy demystified the second law reduced to plain common sense|date= 2008|publisher=World Scientific|location=Singapore|isbn=9789812832269|edition=Expanded}}</ref>中有一些人认为它们之间没有任何关系。<ref>{{cite book|first1=Joseph J.|last1=Vallino|first2=Christopher K. |last2=Algar|first3=Nuria Fernández|last3=González|first4=Julie A.|last4=Huber|editor-first1=Roderick C.|editor-last1=Dewar|editor-first2=Charles H. |editor-last2=Lineweaver|editor-first3=Robert K.|editor-last3=Niven|editor-first4=Klaus|editor-last4=Regenauer-Lieb|date= 2013|chapter=Use of Receding Horizon Optimal Control to Solve MaxEP-Based (max entropy production) Biogeochemistry Problems |department=Living Systems as Catalysts|chapter-url=https://books.google.ca/books?id=xF65BQAAQBAJ&pg=PA340|title=Beyond the Second Law: Entropy Production & Non-equilibrium Systems|page=340|isbn=978-3642401534|publisher=Springer |accessdate=2019-08-31 |quote=Inked page forms pattern w/ information → page entropy < diff page w/ randomized letters. Reduced entropy trivial compared to inked paper entropy. If the paper is burned, it hardly matters in a thermodynamic context if the text contains the meaning of life or only jibberish.}}</ref>两个熵的表达式相似。如果w ^是微观的是可以产生一个给定的宏观态的数目，并且每个微态具有相同的[https://en.wikipedia.org/wiki/A_priori_and_a_posteriori 先验概率]，则该概率为p = 1 / W。香农熵（单位nats）是：
信息熵和热力学熵之间的联系问题是一个有争议的话题。大多数作者认为两者之间有联系，<ref>{{Cite book|last=Brillouin|first=Leon|title=Science and Information Theory|year= 1956|isbn=978-0-486-43918-1}}</ref><ref name="Georgescu-Roegen 1971">{{Cite book|last=Georgescu-Roegen|first=Nicholas|title=The Entropy Law and the Economic Process|publisher=Harvard University Press|year=1971|isbn=978-0-674-25781-8 |url = https://archive.org/details/entropylawe00nich}}</ref><ref>{{Cite book|last=Chen|first=Jing|title=The Physical Foundation of Economics – an Analytical Thermodynamic Theory|publisher=World Scientific|year=2005|isbn=978-981-256-323-1}}</ref><ref>{{cite journal | last1 = Kalinin | first1 = M.I. | last2 = Kononogov | first2 = S.A. | year = 2005 | title = Boltzmann's constant | url = | journal = Measurement Techniques | volume = 48 | issue = 7| pages = 632–636 | doi=10.1007/s11018-005-0195-9}}</ref><ref>{{cite book|last1=Ben-Naim|first1=Arieh|title=Entropy demystified the second law reduced to plain common sense|date= 2008|publisher=World Scientific|location=Singapore|isbn=9789812832269|edition=Expanded}}</ref>中有一些人认为它们之间没有任何关系。<ref>{{cite book|first1=Joseph J.|last1=Vallino|first2=Christopher K. |last2=Algar|first3=Nuria Fernández|last3=González|first4=Julie A.|last4=Huber|editor-first1=Roderick C.|editor-last1=Dewar|editor-first2=Charles H. |editor-last2=Lineweaver|editor-first3=Robert K.|editor-last3=Niven|editor-first4=Klaus|editor-last4=Regenauer-Lieb|date= 2013|chapter=Use of Receding Horizon Optimal Control to Solve MaxEP-Based (max entropy production) Biogeochemistry Problems |department=Living Systems as Catalysts|chapter-url=https://books.google.ca/books?id=xF65BQAAQBAJ&pg=PA340|title=Beyond the Second Law: Entropy Production & Non-equilibrium Systems|page=340|isbn=978-3642401534|publisher=Springer |accessdate=2019-08-31 |quote=Inked page forms pattern w/ information → page entropy < diff page w/ randomized letters. Reduced entropy trivial compared to inked paper entropy. If the paper is burned, it hardly matters in a thermodynamic context if the text contains the meaning of life or only jibberish.}}</ref>两个熵的表达式相似。如果w ^是微观的是可以产生一个给定的宏观态的数目，并且每个微态具有相同的[https://en.wikipedia.org/wiki/A_priori_and_a_posteriori 先验概率]，则该概率为p = 1 / W。香农熵（单位nats）是：
<math>H = -\sum_{i=1}^W p \log (p)= \log (W)</math>
::<math>H = -\sum_{i=1}^W p \log (p)= \log (W)</math>
如果以k / nat 为单位来衡量熵，那么熵可以通过以下公式给出<ref>{{cite web|url=http://bayes.wustl.edu/etj/node1.html|title=Edwin T. Jaynes – Bibliography|publisher=Bayes.wustl.edu|date=1998-03-02|accessdate=2009-12-06}}</ref>：
如果以k / nat 为单位来衡量熵，那么熵可以通过以下公式给出<ref>{{cite web|url=http://bayes.wustl.edu/etj/node1.html|title=Edwin T. Jaynes – Bibliography|publisher=Bayes.wustl.edu|date=1998-03-02|accessdate=2009-12-06}}</ref>：
<math>H = k \log (W)</math>
::<math>H = k \log (W)</math>
当k是玻尔兹曼常数时，这就是著名的[https://en.wikipedia.org/wiki/Boltzmann%27s_entropy_formula 玻尔兹曼熵]公式，这可以解释为每纳特的热力学熵。有许多种方法可以证明“信息熵”和“物理熵”的等价性，即“香农熵”和“玻尔兹曼熵”的等价性。然而，一些作者主张放弃信息论中H函数的熵一词，而改用Shannon的另一个术语“不确定性”。<ref>Schneider, Tom, DELILA system (Deoxyribonucleic acid Library Language), (Information Theory Analysis of binding sites), Laboratory of Mathematical Biology, National Cancer Institute, Frederick, MD</ref>
当k是玻尔兹曼常数时，这就是著名的[https://en.wikipedia.org/wiki/Boltzmann%27s_entropy_formula 玻尔兹曼熵]公式，这可以解释为每纳特的热力学熵。有许多种方法可以证明“信息熵”和“物理熵”的等价性，即“香农熵”和“玻尔兹曼熵”的等价性。然而，一些作者主张放弃信息论中H函数的熵一词，而改用Shannon的另一个术语“不确定性”。<ref>Schneider, Tom, DELILA system (Deoxyribonucleic acid Library Language), (Information Theory Analysis of binding sites), Laboratory of Mathematical Biology, National Cancer Institute, Frederick, MD</ref>