Information Without Representation

Shannon’s information theory (IT) provides a rigorous framework to quantify the amount of information that a system Y has about a system X, irrespective of what the information is about.  IT metrics count the (logarithm of the) number of different states of system X that system Y can distinguish based on Y’s available information (or, conversely, the missing information on X’s state). This information can be translated into a binary file that represents the information that Y has on the state of X.

Shannon’s information measures hence concern the representation of information, while purposefully disregarding its significance to the recipient. In IT, the information system Y has regarding system X is the same whether or not Y can extract value or meaning from its ability to differentiate among some of the states of X.  In cognitive sciences, instead, the focus is on semantic information, that is, on the meaning that the information carries for the receiver.

Robert Anton Wilson proposed to measure the amount of scientific information (a subset of semantic information) by adopting as a unit of measure the amount of scientific information known during the lifetime of Jesus.

To use D. M. MacKay‘s words, semantic information is “difference that makes a difference” for the receiver. Accordingly, not all distinguishable states contribute to the amount of semantic information. Daniel Dennet in his new book gives a recursive definition of semantic information as “design worth getting“. Design refers to the use of “semantic information to improve the prospects of something by adjusting its parts in some appropriate way“. In other words, semantic information is defined by the fact that it can be leveraged to determine the form, or the design, of something for the benefit of the recipient. Semantic information hence depends on the receiver, and it need not be represented or saved to have an impact on the receiver’s design. Semantic information is valuable, and misinformation and disinformation are its perversions.


1 thought on “Information Without Representation”

  1. One way to define a significance of the information could be through a suitable definition of a prior/side information at the receiver. However, this may be difficult, since “significance” is measured by how well the system is taken into a specific, purposeful direction, while the quantity of information increases as the number of possible directions increase. I know that this is a vague explanation, but perhaps a direction worth investigating.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s