How Audio and Video Compression Works: A Beginner’s Guide

Every time you watch a movie on a streaming app, send a voice note, or store a wedding video on your phone, you are relying on a quiet but powerful technology working behind the scenes. Raw, uncompressed audio and video files are enormous. A single minute of uncompressed high-definition video can run into gigabytes. Without a way to shrink these files, streaming would buffer endlessly, storage would fill up in seconds, and sharing media online would be painfully slow. This is where compression steps in. For anyone studying digitisation in libraries and information centres, understanding how compression works is essential, because it sits at the heart of how we store, preserve, and deliver multimedia content.

Understanding compression: why we need it 🔗

Compression is the process of reducing the size of a digital file while keeping as much useful information as possible. It works by using mathematical algorithms to find and remove redundancy in the data. Compression makes audio and video files easier to edit, store, and share, which matters enormously when libraries digitise large collections of recordings, lectures, oral histories, and films.

To understand why this is so important, consider what audio and video actually are in digital form. Sound is captured by measuring the audio signal thousands of times per second, and each measurement becomes a number. Video is a rapid sequence of still images, each made up of millions of coloured dots called pixels. Stored without any reduction, these numbers pile up quickly. Raw, uncompressed video is enormous, which is why no practical storage or streaming system uses it directly.

Compression solves two core problems at once. First, it reduces storage requirements, allowing far more content to fit on a hard drive, disc, or server. Second, it reduces bandwidth needs, meaning files travel faster across networks and the internet. For a digital library or archive managing thousands of media files, both savings are critical for keeping systems efficient and affordable.

Lossy versus lossless compression 🔗

There are two fundamental approaches to compression, and the difference between them shapes every decision about media storage. The choice always involves a trade-off between file size and the quality of the original recording.

Lossy compression 🔗

Lossy compression reduces file size by permanently discarding some of the data. It identifies redundancy and removes information that humans are less likely to perceive. For audio, this often relies on a psychoacoustic model, which removes sound frequencies that the human ear barely notices. For images and video, it removes fine detail that the eye tends to overlook.

The advantage is dramatic size reduction. Lossy compression can achieve very high compression ratios, making it well suited to internet music streaming, downloading, and playback on portable devices. The catch is that the discarded data is gone forever. Once decoded, the file is no longer identical to the original. Push the compression too far and you get visible or audible flaws, known as artifacts: blocky patches in video, smeared motion, or watery-sounding music. Familiar lossy formats include MP3 and AAC for audio and JPEG for images. Streaming platforms like Spotify and Apple Music use AAC or MP3 to minimise data usage.

Lossless compression 🔗

Lossless compression reduces file size without throwing away any information. Instead of discarding data, it finds patterns and redundancies and represents the same information more compactly. After decoding, the file is restored exactly to its original state with no quality loss at all.

This precision comes at a cost. Lossless files are much larger than lossy ones. A 2:1 compression ratio for lossless audio is usually considered good, which is modest compared to the tenfold or greater savings lossy methods can achieve. Common lossless audio formats include FLAC and ALAC. Lossless audio retains 100% of the original recording, preserving the complete waveform, which is exactly what professionals and archivists need.

When to use each approach 🔗

The right choice depends entirely on the purpose. Lossless methods work best for archiving, medical imaging, and master copies, while lossy methods suit everyday audio, video, and images where smaller file sizes matter more. In a library setting, the rule of thumb is simple. Preservation master files, the authoritative archival copies, should be stored in lossless or uncompressed formats to protect every detail for the future. Access copies, the versions shared with users online, can be lossy to keep them small and easy to stream.

Common compression standards 🔗

Most digital video you encounter follows a standard developed by the Moving Picture Experts Group, commonly known as MPEG. MPEG is an ISO/IEC standards group that develops compression algorithms for audio and video. Understanding the main MPEG standards helps explain how digital media evolved over the decades.

MPEG-1 🔗

MPEG-1 was the first standard from the group, defining a compression format for real-time audio and video. It was designed to compress VHS-quality video and CD audio down to about 1.5 Mbit/s without excessive quality loss, making Video CDs and digital audio broadcasting practical. It introduced familiar features like fast forward and reverse playback. The well-known MP3 audio format actually comes from this family, as MPEG-1 Audio Layer III.

MPEG-2 🔗

MPEG-2 extended MPEG-1 to cover a much wider range of applications. It supported higher video resolutions for HDTV and handled both progressive and interlaced video. This made it the backbone of the early digital video revolution. MPEG-2 became the standard for DVDs and the first generation of digital television broadcasts, allowing a full-length film to fit on a single disc.

MPEG-4 🔗

MPEG-4 introduced an object-based approach and far greater flexibility. Unlike MPEG-1 and MPEG-2, MPEG-4 was designed for interactive audiovisual services and is more adaptable and scalable. Importantly, MPEG-4 is not a single thing but a standard divided into several parts. MPEG-4 Part 2 is the Advanced Simple Profile used by codecs like DivX and Xvid, while MPEG-4 Part 10 is Advanced Video Coding, better known as H.264.

H.264 (Advanced Video Coding) 🔗

H.264 is one of the most successful video standards ever created. It was developed jointly by two bodies: the ITU-T Video Coding Experts Group and the ISO MPEG group, working together as the Joint Video Team. This is why it carries two names, H.264 and MPEG-4 Part 10. H.264 is a block-oriented, motion-compensation-based standard that reduces redundancy between frames using advanced techniques.

Its efficiency is remarkable. An MPEG-2 HD stream at 15 Mbps delivers roughly the same picture quality as an H.264 stream at just 5 Mbps, a threefold bandwidth saving. This is why H.264 powers so much of what we watch, from Blu-ray discs to video on streaming services. Newer standards like H.265, also called HEVC, have since pushed efficiency even further, but H.264 remains widely used because almost every device supports it.

Popular compression tools and codecs 🔗

It helps to clear up a common confusion here. A standard is a set of rules describing how video should be compressed, while a codec is the actual software or hardware that does the encoding and decoding. The word codec itself comes from “coder-decoder.” A container, such as AVI or MP4, is simply a wrapper that holds the compressed video, audio, and subtitles together. Many people mix these up, but keeping them distinct makes everything clearer.

H.261 🔗

H.261 holds an important place in history. Designed in 1990 for transmission over ISDN lines, primarily for video conferencing and video telephony, it was the first practical digital video coding standard. Crucially, almost every video standard that followed was based on its design. It works by exploiting both spatial redundancy within a single frame and temporal redundancy between frames, using motion vectors to describe how parts of the image move.

DivX and Xvid 🔗

DivX rose to fame in the early days of internet video for shrinking films enough to share online. The original DivX codec is based on MPEG-4 Part 2, not H.264 as is sometimes claimed. Over time, the brand expanded to include H.264 and HEVC versions. Xvid is its open-source counterpart, also following the MPEG-4 standard. These codecs became popular because they balanced good quality with small file sizes and fast encoding.

DV format 🔗

DV, short for Digital Video, was the standard for consumer and prosumer camcorders for years. In the DV format, video and audio are recorded separately, which makes it possible to add or re-record audio after editing, and it stores all data digitally on tape, memory card, or hard drive. It uses relatively light compression, prioritising quality and easy editing over small file size, which is why it was favoured for capturing and editing footage.

The impact of compression on quality 🔗

Every compression decision is a balancing act between file size and media quality. The more aggressively you compress a file, the smaller it becomes, but the greater the risk of losing quality. The skill lies in finding the point where the file is small enough to be practical yet still looks and sounds good.

A key concept here is bitrate, the amount of data used per second of audio or video. A higher bitrate generally means better quality but a larger file. A lower bitrate saves space but can introduce artifacts. Well-designed lossy compression can appear visually lossless at sensible settings, until you push it too far and artifacts appear, such as blocky gradients, smeared motion, or distorted sound.

Newer standards achieve a better quality-to-size ratio than older ones. HEVC delivers the same video quality as H.264 at about half the file size, which is why each generation of codec lets us stream higher resolutions over the same connection. For libraries and archives, the practical lesson is to match the compression to the goal. Choose minimal or no compression for long-term preservation, and efficient lossy compression for the access copies that users actually stream and download. This two-tier strategy protects the original while keeping everyday delivery fast and affordable.

What do you think? If you were responsible for digitising a collection of rare audio recordings, how would you balance the need to preserve perfect quality against the storage costs of lossless files? And as compression standards keep improving, do you think today’s “good enough” formats will look as outdated in twenty years as MPEG-1 does now?

References

1. https://www.storyblocks.com/resources/blog/lossy-vs-lossless
2. https://www.iptvbasics.com/iptv-compression-standards-mpeg-2-mpeg-4-and-h-264-explained/
3. https://documents.io/blog/lossless-vs-lossy-formats
4. https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/8630861
5. https://castr.com/blog/lossy-vs-lossless-video-compression/
6. https://image-ppubs.uspto.gov/dirsearch-public/print/downloadPdf/7536305
7. https://www.avixa.org/pro-av-trends/articles/what-is-lossless-audio
8. https://celerdata.com/glossary/5-key-differences-between-lossless-and-lossy-compression
9. https://www.oreilly.com/library/view/digital-video-and/9780750689755/kindle_split_057.html
10. https://graphsearch.epfl.ch/en/publication/86790/concepts
11. https://www.divx.com/blog/the-evolution-of-video-codecs-from-mpeg-2-and-h-264-to-hevc-and-beyond/
12. https://www.linkedin.com/pulse/what-differences-between-mpeg2-mpeg4-h264-chris-chen
13. https://www.free-codecs.com/guides/mpeg-video-standards-explained-mpeg-1-mpeg-2-and-mpeg-4.htm
14. https://www.divx.com/en/software/technologies/h264/
15. https://www.w3.org/2008/WebVideo/Fragments/wiki/SotA_Codecs.html
16. https://taketones.com/blog/all-about-codecs-and-video-formats
17. https://www.divx.com/en/software/technologies/hevc/