New Foundations for Information Theory: Logical Entropy and Shannon Entropy

This monograph provides a new foundation for information theory based on information-as-distinctions, measured by logical entropy, and re-quantified as Shannon entropy. It defines information sets that express the distinctions made by a partition, and logical entropy is a probability measure on these sets. Shannon entropy is derived from a non-linear dit-to-bit transform, and set concepts in this theory naturally extend to vector spaces and Hilbert spaces for quantum logical information theory.

Format: Paperback / softback
Length: 113 pages
Publication date: 31 October 2021
Publisher: Springer Nature Switzerland AG

This monograph presents a novel framework for information theory, founded on the concept of information as distinctions, directly measured by logical entropy, and re-quantified as Shannon entropy, which serves as the fundamental concept in coding and communications theory. Information is characterized by its reliance on distinctions, differences, distinguishability, and diversity. Information sets are defined to represent the distinctions made by a partition, such as the inverse image of a random variable, thereby capturing the pre-probability notion of information. Logical entropy is then defined as a probability measure on these information sets, quantifying the likelihood of obtaining a distinction or "dit" from the partition on two independent trials. The formula for logical entropy is a novel derivation of an ancient formula dating back to the early 20th century, which has been re-derived numerous times across different contexts. As a probability measure, all compound notions of joint, conditional, and mutual logical entropy are immediate.

The Shannon entropy, which is not defined in the traditional sense of measure theory, is then derived from a non-linear dit-to-bit transform that re-quantifies the distinctions of a random variable in terms of bits. This re-quantification process leads to the Shannon entropy, which represents the average number of binary distinctions or bits required to make all the distinctions of the random variable. By employing a linearization method, the set concepts within this logical information theory naturally extend to vector spaces in general, and to Hilbert spaces in particular, for quantum logical information theory, which provides the natural measure of distinctions made in quantum measurement.

Despite its relatively short length, this monograph is packed with dense content and serves as a valuable reference for researchers and graduate students alike. It provides a comprehensive introduction to the foundations of information theory, highlighting the role of distinctions and entropy in understanding and quantifying information. The work explores the relationship between logical entropy, Shannon entropy, and quantum information theory, shedding light on the applications of these concepts in various fields, including communication, data compression, and cryptography.

In conclusion, this monograph offers a groundbreaking perspective on information theory, providing a solid foundation for future research and advancements in the field. Its innovative approach to information as distinctions and the re-quantification of entropy offers a fresh perspective on understanding and manipulating information in modern technologies. By exploring the connections between logical entropy, Shannon entropy, and quantum information theory, this work contributes to the ongoing development of information theory and its applications in diverse domains.

Weight: 209g
Dimension: 235 x 155 (mm)
ISBN-13: 9783030865511
Edition number: 1st ed. 2021