Linear Regression: Predicting Dependent Variables with a Straight Line

If you have ever wondered how a researcher predicts next year’s library budget from this year’s footfall, or how an economist forecasts demand from price, the answer often lies in one of the most widely used tools in statistics: linear regression. It takes two related variables and draws a single straight line through the data that captures their relationship, and then uses that line to make predictions. This post breaks down what linear regression is, how the line is actually calculated using the least squares method, and how you can apply it to real numbers to estimate future outcomes.

The concept of linear regression 🔗

Linear regression is a statistical method that models the relationship between a dependent variable and one or more independent variables using a straight-line equation. In statistical terms, it estimates the relationship between a response variable and one or more explanatory variables. When there is only one independent variable, it is called simple linear regression; when there are two or more, it becomes multiple linear regression.

The core idea is straightforward. The dependent variable is the outcome you want to predict or explain. The independent variable is the factor you use to do the predicting. For example, if you are predicting sales from advertising spend, sales is the dependent variable and advertising expenditure is the independent variable. The dependent variable is also known as the explained, outcome, or criterion variable, while the independent variable is also called the predictor or explanatory variable.

The straight-line equation 🔗

Linear regression assumes that the relationship between the two variables can be approximated by a straight line. This line is expressed through a simple equation:

Y = a + bX

Here, Y is the predicted value of the dependent variable, X is the value of the independent variable, a is the intercept (the predicted value of Y when X equals zero), and b is the slope (how much Y changes for every one-unit increase in X). You may also see this written as Y = mx + c, or in statistical notation as Y = β₀ + β₁X, but every version describes the same straight line. The slope captures the direction and strength of the relationship: a positive slope means Y rises as X rises, while a negative slope means Y falls as X rises.

Why we use it 🔗

Regression is valuable because it does two things at once. First, it tells us whether a meaningful relationship exists between variables. Second, it lets us forecast: once we know the line, we can plug in a new value of X and read off the expected value of Y. The linear least squares method is by far the most widely used modelling approach, and it is what most people mean when they simply say “regression.” It also forms the foundation for many advanced techniques, which is why understanding it well pays off across research and analytics.

How the regression line is calculated 🔗

The obvious question is: out of the infinite number of straight lines we could draw through a cluster of points, which one is the “best”? This is where the least squares method comes in. The method finds the best-fit line by minimizing the sum of the squared residuals, where a residual is the difference between an observed value and the value predicted by the line.

Understanding residuals 🔗

A residual is simply the vertical distance from each actual data point to the regression line. If a point sits above the line, its residual is positive; if below, negative. The least squares method squares each of these distances before adding them together. Squaring serves two purposes: it removes the negative signs so positives and negatives do not cancel out, and it penalizes larger errors more heavily. The line that produces the smallest total of these squared distances is the line of best fit. A smaller residual sum indicates a closer fit to the data.

The formulas for slope and intercept 🔗

To fit the line, we need to calculate the slope (b) and the intercept (a). The slope is calculated first using the following formula, where x̄ and ȳ represent the mean of the X values and the mean of the Y values:

b = Σ[(xᵢ − x̄)(yᵢ − ȳ)] / Σ[(xᵢ − x̄)²]

In words, you multiply each deviation of X from its mean by the matching deviation of Y from its mean, add up these products, and divide by the sum of the squared deviations of X. This gives the slope of the line.

Once the slope is known, the intercept is found by subtracting the slope times the mean of X from the mean of Y:

a = ȳ − b·x̄

This equation carries a neat implication: the regression line always passes through the point of average values (x̄, ȳ). In other words, when X equals its own average, Y is predicted to equal its own average. The least square method for fitting a line can therefore be summarized in clear steps: label the X and Y values, compute their means, work out the slope using the deviations, calculate the intercept, and finally write the equation Y = a + bX.

A worked example with real data 🔗

Theory becomes clear once you push numbers through it. Suppose a college librarian wants to understand how the number of hours students study in the library relates to their exam scores, and then predict a score for a study time not yet observed. Consider the following small dataset.

The data 🔗

Five students reported their weekly study hours in the library (X) and their exam scores out of 100 (Y):

Student 1: X = 2, Y = 50
Student 2: X = 4, Y = 60
Student 3: X = 6, Y = 70
Student 4: X = 8, Y = 80
Student 5: X = 10, Y = 90

Step one: find the means 🔗

The mean of X is (2 + 4 + 6 + 8 + 10) ÷ 5 = 30 ÷ 5 = 6. The mean of Y is (50 + 60 + 70 + 80 + 90) ÷ 5 = 350 ÷ 5 = 70.

Step two: calculate the deviations and products 🔗

Next, find how far each value sits from its mean. For X, the deviations (xᵢ − x̄) are −4, −2, 0, 2, and 4. For Y, the deviations (yᵢ − ȳ) are −20, −10, 0, 10, and 20.

Now multiply each pair of deviations together to get the product (xᵢ − x̄)(yᵢ − ȳ): (−4 × −20) = 80, (−2 × −10) = 20, (0 × 0) = 0, (2 × 10) = 20, and (4 × 20) = 80. The sum of these products is 80 + 20 + 0 + 20 + 80 = 200.

Step three: calculate the squared deviations of X 🔗

Square each X deviation: (−4)² = 16, (−2)² = 4, (0)² = 0, (2)² = 4, and (4)² = 16. The sum of squared deviations is 16 + 4 + 0 + 4 + 16 = 40.

Step four: calculate the slope 🔗

Using the slope formula, b = 200 ÷ 40 = 5. This means that for every additional hour a student studies in the library, the model predicts the exam score will rise by 5 marks.

Step five: calculate the intercept 🔗

Using a = ȳ − b·x̄, we get a = 70 − (5 × 6) = 70 − 30 = 40. So the intercept is 40, the predicted score for a student who studies zero hours.

Step six: write the equation and predict 🔗

Putting it together, the regression equation is:

Y = 40 + 5X

This single equation now lets us forecast. If a new student plans to study 7 hours per week, the predicted score is Y = 40 + (5 × 7) = 40 + 35 = 75 marks. Notice that 7 hours was never in our original data, yet the line allows us to estimate an outcome for it. This ability to predict results based on an existing set of data is exactly why the method is used so widely by statisticians, economists, managers, and researchers.

Where linear regression is applied 🔗

The reach of regression goes far beyond classroom examples. It is used in market research and the analysis of customer survey results, in forecasting demand from price, and in estimating budgets from usage patterns. In library and information science specifically, you might use it to project annual circulation figures from membership growth, or to estimate staffing needs from expected visitor numbers. The interpretation always stays the same: the slope tells you how much the outcome moves per unit of input, and the equation lets you predict.

Keep the assumptions in mind 🔗

Linear regression is powerful but not unconditional. It rests on a few key assumptions: the relationship between the variables should be roughly linear, the observations should be independent of one another, the spread of residuals should stay fairly constant, and the residuals should be approximately normally distributed. The method is also sensitive to outliers, since one extreme point can pull the line noticeably. Before trusting a regression line, it is wise to plot the data on a scatter plot to check that a straight line is a sensible approximation in the first place. When the relationship is clearly curved, a non-linear model may serve better.

Used thoughtfully, linear regression turns a scatter of past observations into a clean, interpretable rule for predicting the future, all from a single straight line and a few sums.

What do you think? If you collected study-hour and exam-score data from your own classmates, do you think a straight line would fit the relationship well, or would the points scatter too widely? And which variable in your own field of study would you most want to predict using a regression line?

References

1. https://en.wikipedia.org/wiki/Linear_regression
2. https://www.scribd.com/document/547864464/Linear-Regression
3. https://www.itl.nist.gov/div898/handbook/pmd/section1/pmd141.htm
4. https://en.wikipedia.org/wiki/Least_squares
5. https://www.technologynetworks.com/informatics/articles/calculating-a-least-squares-regression-line-equation-example-explanation-310265
6. https://people.duke.edu/~rnau/mathreg.htm
7. https://www.geeksforgeeks.org/maths/least-square-method/
8. https://www.freecodecamp.org/news/the-least-squares-regression-method-explained/
9. https://www.geeksforgeeks.org/maths/linear-regression-formula/
10. https://sixsigmadsi.com/glossary/simple-linear-regression/