The Role of Semantics in Information Theory: Beyond the Syntax of Messages

When most students first encounter information theory, they meet Claude Shannon’s brilliant 1948 model: information as the resolution of uncertainty, measured in bits, transmitted cleanly across a noisy channel. But that model has a famous blind spot. Shannon was solving an engineering problem, and he deliberately set aside the question of what a message actually means. A weather report saying “it will rain tomorrow” and a string of random gibberish of the same length can carry the exact same number of bits. Clearly, something important is missing. That missing piece is the subject of semantic information theory, the branch of study that asks not how much signal is transmitted, but how much genuine meaning, truth, and knowledge a message contains.

Introduction to semantic information theory 🔗

Semantic information theory shifts the focus from the form of a message to its substance. Where Shannon’s classical theory treats the amount of information as a measure of the statistical rarity or “surprise” of a message, semantic theory is concerned with the actual content carried by a statement. The foundational work here belongs to philosopher Rudolf Carnap and logician Yehoshua Bar-Hillel, who published An Outline of a Theory of Semantic Information in 1952 at MIT.

Their core idea was a deliberate departure from the prevailing communication theory. Instead of measuring statistical rarity, they treated the information carried by a sentence as synonymous with the content of that sentence within a given language system. In other words, the meaning of a statement is tied to what it rules out about the world. A very specific, precise claim excludes many possibilities and therefore carries a lot of content. A vague or near-tautological statement excludes almost nothing and therefore carries little.

To make this measurable, Carnap and Bar-Hillel worked within the structured framework of logic. They restricted themselves to sentences expressed in a formal language and built their measures on what they called state descriptions, which are complete specifications of how a small “world” could possibly be arranged. As the Stanford Encyclopedia of Philosophy explains, they confined their analysis to sentences in monadic predicate logic with a finite number of predicates, so that the number of possible worlds remained countable and the content of any sentence could be defined precisely. The more possible worlds a sentence eliminates, the more semantic content it carries.

Why meaning matters beyond the signal 🔗

The distinction here is more than academic. Researchers often describe useful information as a trinity of three components: the syntactic dimension (the form or structure of the message), the semantic dimension (its meaning), and the pragmatic dimension (its usefulness or utility). Shannon’s 1948 theory, despite its enormous influence, addresses only the first of these. As one IEEE study notes, the theory established by Shannon is fundamentally a statistical theory of syntactic information, which leaves the meaning and the usefulness of a message untouched.

This is why semantic information theory exists as a separate field. It tries to fill the gap that Shannon openly acknowledged but chose not to address, because his concern was the reliable transmission of symbols, not their interpretation.

Semantic-centered studies and measuring content 🔗

Once we accept that meaning matters, the next question becomes practical: how do we measure it? This is where semantic-centered studies diverge sharply from the surprise-based logic of classical theory.

From surprise to content 🔗

In Shannon’s framework, an improbable message carries more information because it reduces more uncertainty. A message that you fully expected carries almost nothing. Semantic theory keeps a related intuition but reorients it around logical content rather than probability of occurrence. Carnap and Bar-Hillel measured the amount of semantic information using logical probability functions ranging over the content of sentences. A statement with low logical probability (one that is true in very few possible worlds) is highly informative, while a logically certain truth carries no information at all.

Consider the difference between “a number was drawn” and “the number 7 was drawn from a set of 1 to 10.” The second sentence excludes far more possibilities, so in semantic terms it is richer in content. This content-based view connects information directly to knowledge and the structure of reasoning, rather than to the statistics of a transmission channel.

The Bar-Hillel-Carnap paradox 🔗

This elegant approach produced a famous and troubling result known as the Bar-Hillel-Carnap paradox. If information content increases as a statement excludes more possibilities, then a self-contradictory statement, which excludes every possible world, should carry the maximum possible information. A research analysis describes how the theory leads to the counterintuitive feature that inconsistent statements convey maximal information. This is deeply unsatisfying. A contradiction like “it is raining and it is not raining” tells us nothing useful about the world, yet the formal measure rates it as supremely informative.

This paradox has driven much of the subsequent research in the field and pushed theorists toward a key requirement: a satisfactory measure of information should somehow account for truth.

The push toward truth and knowledge 🔗

The most influential modern response to this problem comes from philosopher Luciano Floridi, who argues that genuine semantic information must be truthful. In his theory of strongly semantic information, he defines information as well-formed, meaningful, and truthful data. This factivity requirement, often called the veridicality thesis, has a striking consequence. If information must be true, then false content is not a weaker kind of information at all.

Floridi makes exactly this argument. He maintains that misinformation is not a defective type of semantic information but rather pseudo-information that is not semantic information at all. Under this view, the question of how much information a message carries cannot be separated from whether the message is true. This binds the measurement of information tightly to knowledge, because we typically only count something as knowledge if it is both believed and true. Notably, this veridical approach also dissolves the Bar-Hillel-Carnap paradox, since a contradiction is necessarily false and therefore carries no genuine information.

Contextual and user-centered information 🔗

Even truth and logical content together do not fully capture why a particular message is valuable. The same true and meaningful statement can be enormously useful to one person and completely worthless to another. This brings us to the contextual and user-centered dimension of information, which is where library and information science makes its most direct contribution.

The role of context 🔗

The value of information depends heavily on the situation in which it is received. A train timetable is rich, actionable information for a commuter standing on a platform, but the identical document is nearly meaningless to someone in another country with no intention of travelling. Classical semantic measures, anchored in logic and possible worlds, struggle to capture this because they treat content as a fixed property of a sentence. Yet in practice, the informativeness of a message is shaped by what the receiver already knows and what they are trying to do.

This is precisely why some scholars describe useful information as the trinity of form, meaning, and utility. One conference paper on semantic information argues that the truly useful information for humans must combine syntactic form, semantic meaning, and pragmatic utility so that the user can grasp the form, the meaning, and the way to actually use it. Utility is inherently relational. It only exists in the relationship between a message and a particular user with particular goals.

User perception and interpretation 🔗

Beyond context, the receiver’s own interpretation matters. The semantic dimension of information is understood not only through the truthfulness of data, but also through the perception and interpretation of that data by the consumer of information. Two readers with different background knowledge can extract very different amounts of meaning from the same article. A medical report is dense with information for a doctor and largely opaque to a layperson, even though the printed symbols are identical.

This insight is central to information retrieval and library services. When a librarian or a search engine tries to deliver “relevant” results, they are implicitly acknowledging that information value is not absolute but is calibrated to the user’s need. A concise, accurate, and relevant answer satisfies the user because it is true, meaningful, and fitted to the question they actually asked. This user-centered perspective explains why so much of modern information practice focuses on understanding the searcher’s intent rather than simply matching keywords.

Challenges in defining information content 🔗

Given all these competing concerns, it should be no surprise that the field has never settled on a single, universally accepted measure of semantic information. The challenges run deep, and they are worth understanding clearly.

No consensus on a definition 🔗

The most fundamental obstacle is the absence of agreement on what semantic information even is. As Floridi himself acknowledges, there is no consensus yet on the definition of semantic information. Different schools of thought disagree on basic questions. Must information be true, or can false statements count as information? Is meaning an objective property of a sentence, or does it depend on the interpreter? These are not minor quibbles. They lead to entirely different measurement systems.

The tension between formal rigour and real meaning 🔗

A second challenge is the trade-off between mathematical precision and faithfulness to how meaning actually works. The Carnap-Bar-Hillel approach achieved genuine rigour, but only by restricting itself to a tightly controlled formal language. Stepping outside that controlled setting reintroduces enormous complexity, because natural language allows for an effectively infinite number of possible situations to describe. Researchers note that one major shortcoming of the classical theory is that its central concern is the quantitative measurement of information rather than the qualitative ability to understand meaning and reason with it. A formula can count content, but counting is not the same as comprehension.

Disagreement even among modern solutions 🔗

Even the proposed fixes generate fresh disputes. Floridi’s truth-based theory is influential, but it has been criticised on multiple fronts. Some argue that imposing a truth requirement is unnecessary, and that notions of truth, knowledge, and meaning can be reconstructed within standard information theory without inventing a separate concept of semantic information. Others, working on the idea of truthlikeness or verisimilitude, contend that information should be measured by how close a statement comes to the truth rather than by a strict true-or-false test. This ongoing debate shows that defining and measuring information content remains an open and active research problem rather than a settled science.

Why this matters for information professionals 🔗

For students of library and information science, these challenges are not merely philosophical curiosities. Every information system, from a library catalogue to a recommendation engine, embodies some implicit theory of what makes information valuable. A system that ranks results purely by keyword frequency is leaning on a syntactic notion. A system that filters for accuracy and authority is reaching toward the truth-based view. A system that personalises results is honouring the user-centered, contextual dimension. Understanding semantic information theory helps professionals see clearly what their tools are actually optimising for, and where those tools fall short.

What do you think? If a true statement is useless to you but a slightly inaccurate one helps you make a good decision, which one carries more “information” in any meaningful sense? And as automated systems increasingly decide what information reaches us, whose definition of “meaningful content” should they be built to serve?

References

1. https://dspace.mit.edu/handle/1721.1/4821
2. https://bibbase.org/network/publication/carnap-barhillel-anoutlineofatheoryofsemanticinformation-1952
3. https://plato.stanford.edu/entries/information-semantic/
4. https://ieeexplore.ieee.org/abstract/document/7839754/
5. https://link.springer.com/article/10.1007/s11229-015-0717-1
6. https://philarchive.org/rec/FLOFDT
7. https://www.mdpi.com/2504-3900/1/3/129
8. https://www.researchgate.net/publication/235223166_Semantic_Conceptions_of_Information