In most classes on information theory (IT), the relationship between IT and physics is reduced to a remark on the origin of the term “entropy” in Boltzmann’s classical work on thermodynamics. This is possibly accompanied by the anecdote regarding von Neuman’s quip on the advantages of using this terminology. Even leaving aside recent, disputed, attempts, such as constructor theory (see here) and integrated information theory (see here), to use concepts from IT as foundations for new theories of the physical world, it seems useful to provide at least a glimpse of the role of IT in more mainstream discussions on the future of theoretical physics.
As I am admittedly not qualified to provide an original take on this topic, I will rely here on the poetic tour of modern physics by Carlo Rovelli, in which one of the last chapters is tellingly centered on the subject of “information”. Rovelli starts his discussion by describing information as a “specter” that is haunting theoretical physics, arousing at the same time enthusiasm and confusion. He goes on to say that many scientists suspect that the concept of information may be essential to make progress in theoretical physics, providing the correct language to describe reality.
At a fundamental level, information refers to a correlation between the states of two physical systems. A physical system, e.g., one’s brain, has information about another physical system, e.g., a tea cup, if the state of the tea cup is not independent of that of the neurons in the brain. This happens if a state of the tea cup, say that of being hot, is only compatible with a subset of states of the brain, namely those in which the brain has memorized the information that the tea cup is hot. Reality can be defined by the network of such correlations among physical systems. In fact, nature has evolved so as to manage these correlations in the most efficiency way, e.g., through genes, nerves, languages.
The description of information in terms of correlation between the states of physical systems is valid in both classical and quantum physics. In thermodynamics, the missing information about the microstate of a system, e.g., about the arrangement of the atoms of a tea cup, given the observation of its macrostate, e.g., its temperature, plays a key role in predicting the future behavior of the system. This missing information is referred to as entropy. In more detail, the entropy is the logarithm of the number of microstates that are compatible with a given macrostate. The entropy tends to increase in an isolated system, as information cannot materialize out of thin air and the amount of missing information can only grow larger in the absence of external interventions.
In quantum physics, as summarized by Wheeler’s “It from Bit” slogan, the entire theoretical framework can be largely built around two information-centric postulates: 1) In any system, the “relevant” information that can be extracted so as to make predictions about the future is finite; 2) Additional information can always be obtained from a system, possibly making irrelevant previously extracted information (to satisfy the first postulate).
The enthusiasm and confusion aroused by the concept of information among theoretical physicists pertain many fundamental open questions, such as: What happens to the missing information trapped in a black hole when the latter evaporates? Can time be described, as suggested by Rovelli, as “information we don’t have”? Related questions abound also in other scientific fields, such as biology and neuroscience: How is information encoded in genes? What is the neural code used by the brain to encode and process information?