Defining Semantic Information: Beyond Shannon’s Theory

When we talk about “information,” most technical models actually measure something quite narrow: how surprising a signal is, not what it means. Claude Shannon built the foundation of modern communication on this idea, and it powers everything from your mobile network to data compression. But Shannon deliberately set aside the one thing humans care about most when they read a sentence, listen to a lecture, or scan a research paper: meaning. Semantic information theory steps in to fill that gap. It asks a different question altogether. Not “how many bits?” but “how much does this statement actually tell us about the world?” This shift has shaped how scholars in information science, linguistics, and cognitive science think about content, and it is central to understanding measures of information in informetrics.

Why Shannon’s theory leaves meaning out 🔗

In his 1948 work, Shannon was explicit about his scope. He treated communication as the problem of reproducing a message selected at one point at another point, and famously stated that the semantic aspects of communication are irrelevant to the engineering problem. What mattered to him was selection: a message is simply one choice out of a set of possible messages. The engineer’s job is to move that selection across a noisy channel reliably.

This makes Shannon’s measure purely syntactic. It deals with the form and statistical rarity of symbols, not their content. As philosophers of information have pointed out, Shannon information is neutral with respect to content, because the only relevant issue is the selection of a message among many. A string of random gibberish and a meaningful sentence of the same length and probability would carry identical amounts of Shannon information.

This is not a flaw in Shannon’s theory. It is a design choice that made the theory powerful and universal. But it created an obvious gap. The information that is truly useful to people has been described as a trinity: the form, called syntactic information, the meaning called semantic information, and the utility called pragmatic information. Shannon’s mathematical theory captures only the first. Semantic information theory is the attempt to formalize the second.

Carnap and Bar-Hillel’s information measure 🔗

The first serious attempt to measure meaning came in 1952, when Rudolf Carnap and Yehoshua Bar-Hillel published their MIT technical report, An Outline of a Theory of Semantic Information. Their starting point was a clean break from statistics. Instead of measuring the statistical rarity of a message, they treated the information carried by a sentence as synonymous with its content, normalized in a certain way, with the amount of semantic information explicated through measures based on logical probability functions.

Logical probability instead of statistical probability 🔗

This is the crucial difference from Shannon. Carnap and Bar-Hillel used logical probability, not statistical frequency. The logical probability of a sentence is measured by the likelihood that the sentence is true across all the possible situations a language can describe. A sentence that rules out many possible states of the world is highly informative. A sentence that rules out almost nothing tells us very little.

Consider a simple example. The statement “it will rain tomorrow” excludes fewer possibilities than “it will rain tomorrow between 3 and 4 PM in this exact district.” The second sentence is more specific, eliminates more possible states of the world, and therefore carries more semantic information. The core idea is that the information in a statement is proportional to the set of possible worlds it logically excludes.

The two measures: cont and inf 🔗

Carnap and Bar-Hillel defined two distinct measures. The first, called cont (content), is based directly on logical probability. The semantic content of a statement is defined as one minus its logical probability. The more improbable a statement is logically, the more content it carries. However, this content measure has a limitation: it is not additive in the way the communication engineer would want.

To solve this, they introduced a second measure, inf, defined as the negative logarithm of logical probability. Of the two measures, cont is additive for sentences whose contents are mutually exclusive, while inf is additive for sentences that are inductively independent. The inf measure turns out to be formally analogous to Shannon’s familiar information function. The same logarithmic shape appears, but the input is logical probability rather than statistical probability. This was deliberate. The authors wanted results that paralleled communication theory while resting on a foundation of meaning.

The role of context in semantic value 🔗

One of the most important features of semantic information is that the value of a message depends heavily on context. Carnap and Bar-Hillel built this directly into their framework by distinguishing absolute measures from relative ones. A statement’s information can be measured on its own, or it can be measured relative to existing evidence. The same sentence carries different amounts of information depending on what the receiver already knows.

Think about what happens when you already possess certain knowledge. If a colleague tells you a fact you already know, the message is meaningful and well-formed, yet it carries almost no new semantic information for you. The logical probability of that statement, given your existing evidence, is already close to certainty. This is why context matters so much. Meaning is not a fixed quantity stamped onto a string of words. It emerges from the relationship between the message and the background against which it is received.

The framework also distinguishes types of reasoning 🔗

Carnap and Bar-Hillel separated their measures further by the kind of reasoning involved. They defined D-functions suited to contexts where deductive reasoning alone is relevant, and I-functions suited to contexts where inductive reasoning is adequate. This distinction acknowledges that the way we draw conclusions, whether by strict logical deduction or by probabilistic inference, changes how we should measure the information a statement provides. Context is not a side concern in their theory. It is wired into the structure of the measures themselves.

The paradox that context could not fully solve 🔗

The logical-probability approach ran into a famous problem. Because information is defined as inversely related to logical probability, a self-contradictory statement, which has a logical probability of zero, would be assigned an infinite amount of information. This is the Bar-Hillel-Carnap paradox, where the approach assigns the maximum informativeness to a contradiction. Intuitively this is backwards. A statement like “it is raining and it is not raining” tells us nothing useful about the world, yet the early measure rated it as maximally informative. The paradox exposed a deep issue: a measure built purely on logical exclusion, without any role for truth, can produce results that clash with our sense of meaning.

Floridi and the demand for truth 🔗

The most influential modern response came from Luciano Floridi. He argued that the standard definition of semantic information as merely meaningful data was incomplete. In his view, meaningful and well-formed data constitute semantic information only if they also qualify as contingently truthful. In other words, well-formed means the data follow the rules of the system, meaningful means they comply with the meanings of the language, and truthful means they correctly describe the world.

This had a striking consequence. Floridi argued that false content is not a weaker form of information but something else entirely. According to his analysis, misinformation is not a type of semantic information but pseudo-information, because truth-values do not supervene on semantic information in the standard account. By building truth into the definition, Floridi’s theory of strongly semantic information aimed to dissolve the contradiction paradox: a false or self-contradicting statement simply fails to count as semantic information in the first place. His approach is, in a sense, orthogonal to the entropy-based measures inherited from physics and computer science, which assign a scalar value to a system without any direct semantic implication.

Implications for information science and beyond 🔗

For information science, the move from syntactic to semantic measures reframes what it means to be informed. Retrieval systems, indexing, and classification all rest on the assumption that documents carry content, not just strings of characters. A search engine that ranked results purely by statistical rarity of words would miss what users want, which is relevant meaning. Semantic information theory provides the conceptual vocabulary for talking about content quantitatively, which matters directly to informetrics and scientometrics where the goal is to measure the substance of scholarly communication, not merely its volume.

Linguistics 🔗

In linguistics, semantic information theory reinforces the separation between syntax and semantics that runs through the field. Just as Noam Chomsky drew a strict line between the grammatical form of a sentence and its meaning, Shannon’s theory dealt only with the engineering of symbols. A grammatically perfect sentence can be meaningless, and a meaningful idea can be expressed in many different grammatical forms. Semantic measures give linguists a way to ask how much a particular utterance narrows down the listener’s understanding of the world, which connects directly to questions of reference, truth conditions, and how speakers convey content efficiently.

Cognitive science 🔗

The connection to cognitive science was anticipated by Carnap and Bar-Hillel themselves. They suggested that semantic information is a concept more readily applicable to psychological and other investigations than its communication-theory counterpart. This makes sense. When the human mind processes a sentence, it does not count bits. It updates a model of the world, ruling out some possibilities and confirming others. The logical-exclusion view of meaning maps closely onto how cognition works: learning is the elimination of uncertainty about how things stand. Researchers studying perception, belief revision, and reasoning find the semantic framework more natural because it deals in content and truth rather than raw signal statistics.

A bridge to modern artificial intelligence 🔗

These ideas are no longer purely philosophical. Engineers building semantic communication systems and large language models have returned to Carnap and Bar-Hillel as a starting point, because quantifying semantic information is the first step toward semantic compression and handling semantic error. As machines increasingly process meaning rather than just symbols, the seventy-year-old question of how to measure content has become urgent again. The distinction Shannon drew, and the gap that Carnap, Bar-Hillel, and Floridi tried to fill, sits at the heart of how we now think about both human and machine understanding.

What do you think? If a statement must be true to count as semantic information, where does that leave the vast amount of useful but uncertain knowledge we work with every day? And when you read a research paper or a news report, are you instinctively measuring its meaning by how many possibilities it rules out?

References

1. https://arxiv.org/pdf/2207.09353
2. https://philsci-archive.pitt.edu/10911/1/What_is_Shannon_Information.pdf
3. https://www.mdpi.com/2504-3900/1/3/129
4. https://bibbase.org/network/publication/carnap-barhillel-anoutlineofatheoryofsemanticinformation-1952
5. https://arxiv.org/abs/2508.00525
6. https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1933-1592.2005.tb00531.x
7. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3845324
8. https://arxiv.org/pdf/2208.06314