Understanding Probability: Definitions, Axioms, and Theorems

Every time a researcher asks how scientific articles scatter across journals, or how often a single author publishes in a lifetime, they are really asking a question about probability. Before laws like Bradford’s or Lotka’s can describe these patterns, you need to understand what probability actually means. The word feels intuitive, yet it carries precise mathematical definitions that shape how we measure uncertainty. This post breaks down probability from its classical roots to its modern axiomatic foundation, walks through the theorems that make calculations possible, and shows how these ideas connect to real probability questions.

What probability really means 🔗

Probability is a measure of how likely an event is to happen. It is expressed as a number between 0 and 1, where 0 means an event is impossible and 1 means it is certain. A value of 0.5 means the event is as likely to happen as not. While that sounds simple, mathematicians spent centuries debating how to define probability rigorously. Two definitions dominate the discussion: the classical definition and the axiomatic definition.

Before either definition makes sense, you need a few building blocks. A random experiment is any process whose outcome cannot be predicted with certainty, such as rolling a die or tossing a coin. The sample space, usually written as S or Ω, is the set of all possible outcomes of that experiment. For a single die, the sample space is {1, 2, 3, 4, 5, 6}. An event is any subset of the sample space, such as “the result is even,” which corresponds to the set {2, 4, 6}. A random variable is a function that assigns a numerical value to each outcome, letting us work with numbers instead of descriptions.

The classical definition 🔗

The classical definition is the oldest approach and is usually applied to simple, easy-to-analyze situations like gambling games. It assumes that a random experiment produces a finite number of equally likely outcomes. Under this view, the probability of an event equals the number of outcomes favourable to the event divided by the total number of possible outcomes.

For example, when you roll a fair die, there are six equally likely outcomes. The probability of rolling a 6 is therefore 1 divided by 6. The probability of rolling an even number is 3 favourable outcomes divided by 6 total outcomes, which equals 1/2. This definition is intuitive and works well for dice, cards, and coins.

The classical definition has a clear limitation, though. It only works when outcomes are equally likely and finite in number. It cannot easily handle situations like the probability that a particular author publishes more than ten papers, where outcomes are neither equal nor neatly countable. It also runs into a subtle circularity, because “equally likely” is itself a statement about probability. These gaps pushed mathematicians toward a more general foundation.

The axiomatic definition 🔗

The axiomatic approach solved this problem in 1933, when the Russian mathematician Andrey Kolmogorov proposed something clever. Instead of trying to give one universal definition of probability, he chose to prescribe a minimum set of properties that any probability measure must satisfy to be consistent. Probability becomes any function that obeys these rules, regardless of whether you interpret it through classical, frequentist, or other lenses.

The axiomatic approach defines probability through mathematical rules called axioms. In this framework, you perform a random experiment with sample space S, and P is a function that assigns a real number to every event. There are three axioms, collectively known as Kolmogorov’s axioms.

The first axiom (non-negativity) states that for any event A, the probability P(A) is greater than or equal to 0. Probabilities can never be negative. The second axiom (normalization) states that the probability of the entire sample space is 1, written P(S) = 1. This means that whenever the experiment is performed, some outcome is certain to occur. The third axiom (additivity) states that if A and B are mutually exclusive events, meaning they cannot happen at the same time, then the probability of either one occurring is the sum of their individual probabilities: P(A ∪ B) = P(A) + P(B). This axiom extends to any countable sequence of mutually exclusive events.

From these three rules, all of modern probability theory follows. The axioms do not tell you which probability function to choose for a given problem, but they guarantee that whatever function you use stays internally consistent. This is why axiomatic probability is often called the backbone of statistical inference.

Key theorems and rules 🔗

Once the axioms are in place, several useful theorems can be derived. These theorems turn the abstract rules into practical tools for calculation.

The addition rule 🔗

The addition rule helps you find the probability that at least one of two events occurs. For mutually exclusive events, the rule is simply P(A ∪ B) = P(A) + P(B), which is the third axiom itself. But many events are not mutually exclusive and can overlap. When two events can happen together, you must subtract the probability of the overlap to avoid counting it twice. The general addition rule becomes P(A ∪ B) = P(A) + P(B) − P(A ∩ B), where P(A ∩ B) is the probability that both events happen.

Consider drawing a card from a standard deck. The probability of drawing a king is 4/52, and the probability of drawing a heart is 13/52. The king of hearts belongs to both sets, so its probability of 1/52 is subtracted once to give the correct answer for drawing a king or a heart.

Conditional probability and the multiplication rule 🔗

Conditional probability answers a focused question: how does the probability of an event change if we already know that another event has occurred? It is written P(A | B), read as “the probability of A given B.” The formula is P(A | B) = P(A ∩ B) / P(B), provided that P(B) is greater than 0. Knowing that B has happened effectively shrinks the sample space to only those outcomes where B is true.

Rearranging this definition gives the multiplication rule, which lets you compute the probability that two events happen together: P(A ∩ B) = P(A | B) · P(B) = P(B | A) · P(A). This joint probability is central to problems where events happen in sequence, such as drawing two cards without replacement.

Bayes’ theorem 🔗

Bayes’ theorem, named after the 18th-century mathematician Thomas Bayes, gives a mathematical rule for inverting conditional probabilities. It allows you to find P(B | A) when you know P(A | B). The formula is:

P(B | A) = [P(A | B) · P(B)] / P(A)

The real power of Bayes’ theorem is that it lets statisticians revise probabilities in light of new information. You start with a prior probability, observe some evidence, and update to a posterior probability. A classic application is medical screening, where you know the probability that a test is positive given a disease, but you actually want the probability of the disease given a positive test. Bayes’ theorem bridges that gap.

The theorem is itself a direct consequence of the multiplication rule. Since P(A ∩ B) is the same whether you write it as P(A | B) · P(B) or P(B | A) · P(A), setting these equal and dividing by P(A) produces the formula. This shows how the entire structure of probability rests on the foundation of the axioms.

The law of total probability 🔗

When the sample space can be divided into non-overlapping events that together cover all possibilities, called a partition, the law of total probability lets you compute the overall probability of an event by summing its conditional probabilities across each part of the partition. For a partition B₁, B₂, … Bₘ, the probability of any event A equals the sum of P(A | Bᵢ) · P(Bᵢ) over all i. This law is often used together with Bayes’ theorem, since it supplies the denominator P(A) when that value is not directly known.

Applying these ideas to real calculations 🔗

The definitions and theorems are not just academic. They form a toolkit for solving practical problems, and the skill lies in choosing the right tool.

Choosing between joint and conditional probability 🔗

A joint probability is the probability that two events happen together, written P(A ∩ B). A conditional probability is the probability of one event given that another has already happened, written P(A | B). The distinction matters. If you draw two spades in a row from a deck without replacement, the second draw depends on the first, so you use conditional probability. After removing one spade, 12 spades remain among 51 cards, making the conditional probability 12/51.

A common mistake is confusing P(A | B) with P(B | A). These are generally different, and Bayes’ theorem is precisely the bridge between them. Another frequent error is adding probabilities when you should multiply, or forgetting the requirement that P(B) must be greater than 0 before you can condition on B.

Why this matters for measuring information 🔗

These principles underpin the quantitative study of literature and information. In bibliometrics and scientometrics, the measures used are typically counts of how often events of specified types occur. Once expressed as ratios of the total number of observed events, these frequencies can be treated as probabilities of occurrence. The probability distributions formed this way are the foundation of the well-known bibliometric laws.

For instance, Lotka’s law describes the frequency distribution of scientific productivity, predicting how many authors publish one paper, two papers, and so on. Bradford’s law describes how articles on a subject scatter across journals. The probabilistic model behind Bradford’s law predicts that articles are distributed randomly and unequally among journals, reflecting an underlying hierarchical structure. None of these patterns can be understood or modelled without first grasping what probability means and how its rules operate. When you calculate the chance that a randomly selected journal falls in the core of a discipline, or the likelihood that an author belongs to the most productive group, you are applying the same axioms and theorems described above.

What do you think? If the classical definition only works for equally likely outcomes, how would you explain the probability that a specific author publishes more than five papers in a year? And when you encounter a new probability problem, how do you decide whether the addition rule, the multiplication rule, or Bayes’ theorem is the right starting point?

References

1. https://arxiv.org/pdf/1302.2525
2. https://en.wikipedia.org/wiki/Probability_axioms
3. https://www.geeksforgeeks.org/maths/axiomatic-approach-to-probability/
4. https://math.mit.edu/~dav/05.dir/class3-prep.pdf
5. https://en.wikipedia.org/wiki/Bayes'_theorem
6. https://www.stat.auckland.ac.nz/~fewster/325/notes/ch2.pdf
7. https://stats.libretexts.org/Courses/Saint_Mary's_College_Notre_Dame/MATH_345__-_Probability_(Kuter)/2:_Computing_Probabilities/2.2:_Conditional_Probability_and_Bayes'_Rule
8. https://arxiv.org/pdf/1301.5380
9. https://www.academia.edu/1307666/On_the_Theoretical_Foundations_of_Bradfords_Law