Every time you type a few words into a search box and get back a list of relevant documents, you are using a principle that is more than 160 years old. That principle is keyword indexing, and it sits at the heart of how libraries, databases, and search engines connect people to information. Before complex thesauri and controlled vocabularies took over scholarly cataloguing, librarians realised something simple but powerful: the words already present in a document’s title could themselves act as doorways to its content. This post explains what keyword indexing is, where it came from, how it works, and why it still matters in an age of digital search.

Table of Contents

What keyword indexing means

Keyword indexing is a method of organising documents using the significant words that already appear in them, usually drawn from the title. Instead of an indexer assigning subject headings from a fixed list, the system treats meaningful words in the document itself as access points. A reader can then look up any of those words and be led straight to the document.

This approach belongs to a family called derived indexing, because the index terms are derived directly from the text rather than assigned by a human cataloguer. It is also known as natural language indexing or free text indexing, since it relies on the everyday language of the author rather than a controlled set of approved terms.

The core idea: titles carry meaning

The whole system rests on one assumption: the title of a document is a one-line summary of its content. The significant words in a title are taken to indicate the subject of the work. Words such as “the,” “of,” “in,” and “a” carry little meaning and are filtered out using a stop list. What remains are the keywords that genuinely describe the document.

For example, take a document titled “Computerisation of Libraries in India.” The meaningful keywords here are computerisation, libraries, and India. A keyword index would create a separate entry under each of these words, so a reader searching under any one of them would find the document.

The historical background

The story of keyword indexing begins not with computers but with a frustrated nineteenth-century librarian working with paper, ink, and printing presses.

Andrea Crestadoro and the birth of keyword in titles

The concept was first proposed by Andrea Crestadoro (1808-1879), an Italian-born bibliographer and inventor who settled in England. Crestadoro is credited with being the first person to propose that books could be catalogued using keywords, an idea he developed while wrestling with the slow and laborious cataloguing methods of his day. Frustrated by delays in producing the British Museum’s printed catalogue, he published a pamphlet in 1856 outlining a new method of building catalogues that gave readers access through an alphabetical index of names and subjects.

When the Public Library of Manchester adopted his approach, it hired Crestadoro to put his ideas into practice. He became Chief Librarian of the Manchester Free Library, a post he held from 1864 until his death. It was here, in the Catalogue of the Manchester Free Library published in 1864, that his system was implemented. The method was called Keyword-in-Title (KWIT), because it generated index entries from the keywords occurring in document titles.

Hans Peter Luhn and the computer revival

Crestadoro’s idea was, in a sense, ahead of its time, and it lay relatively dormant for almost a century. It was revived by Hans Peter Luhn, a computer scientist at IBM, who rediscovered and mechanised the concept in the late 1950s. In 1958, Luhn demonstrated his technique at the International Conference on Scientific Information in Washington, where he produced keyword indexes for the conference programme using new card-scanning equipment.

Luhn renamed the technique Keyword-in-Context (KWIC). The shift from “in title” to “in context” reflected a key improvement: rather than simply listing keywords, Luhn’s method displayed each keyword surrounded by the words that accompanied it in the original title, preserving its context. Because computers could rapidly sort and rearrange titles, KWIC made it practical to index large volumes of technical literature, something impossible to do by hand at scale.

How keyword indexing works

Although the mechanics differ slightly between variants, the basic process of keyword indexing follows a clear sequence.

The step-by-step process

First, the title of a document is taken as the source of index terms. Second, insignificant words such as articles and prepositions are removed using a stop list, leaving only the keywords. Third, each keyword is used in turn as the lead term for a separate index entry, with the rest of the title rotated around it so that the context remains visible. Finally, all entries are filed in a single alphabetical sequence, often with a document identification number for retrieval.

Consider the title “Classification of Books in a University Library.” A KWIC index would generate one entry for each significant word, rotating the title so that “Classification,” “Books,” “University,” and “Library” each appear as the lead term. A reader scanning the alphabetical list under any of these words would find the same document.

Why there is no vocabulary control

A defining feature of keyword indexing is the absence of vocabulary control. In a controlled-vocabulary system, an indexer maps the document to standardised subject headings drawn from a thesaurus or subject heading list, ensuring that all documents about the same topic are grouped under one approved term. Keyword indexing skips this step entirely. It uses whatever words the author chose, in whatever form they appear.

This makes the system fast, cheap, and easy to automate, since no human judgement is needed to assign terms. The trade-off is precision. Because related concepts may be described using different words, documents on the same subject can become scattered across the index, and a search may miss relevant material or return irrelevant results. The system also depends heavily on the quality of titles. If a title is fanciful or non-descriptive, the index becomes far less effective, particularly in the humanities and social sciences, where titles like “Integration: For and Against” reveal little about specific content.

The main variants

Over time, several variants emerged to address the limitations of the original method. The most common include:

KWIC (Keyword-in-Context): The classic form, where each keyword appears at the centre or a fixed position of the line, with the surrounding title kept intact to preserve context.

KWOC (Keyword-out-of-Context): A variant in which the keyword is pulled out and placed at the beginning of the entry, separate from the rest of the title. It is simpler to read but offers less contextual richness than KWIC.

KWAC (Keyword-Augmented-in-Context): An enriched version that adds significant words taken from the abstract or text of the document to the title, helping when the title alone fails to capture the subject. The CBAC service of BIOSIS, for instance, used a KWAC-style index.

Other refinements include WADEX, which treats author names as keywords alongside subject terms, and KEYTALPHA, which lists keywords side by side without preserving the sentence structure.

Importance in modern indexing systems

Keyword indexing was never just a historical curiosity. Its central idea, using the natural language of documents as access points, has proven remarkably durable and now underpins much of the digital information world.

From print indexes to digital databases

In the print era, KWIC and its variants were widely used for indexing technical reports, journals, and periodical literature, where speed and low cost mattered more than perfect precision. They were among the first indexing methods that computers could generate automatically, which made them invaluable for handling the rapidly growing volume of scientific publication in the mid-twentieth century.

As library catalogues became computerised and bibliographic databases multiplied, keyword searching became a standard feature. Online Public Access Catalogues (OPACs) and academic databases routinely allow users to search by keywords drawn from titles, abstracts, and full text, combining the speed of natural language indexing with the structure of controlled subject headings.

The most visible descendant of keyword indexing is the modern web search engine. When a search engine builds an inverted index, matching every significant word to the documents that contain it, it is applying the same fundamental logic that Crestadoro proposed and Luhn mechanised: let the words in the text serve as the keys to finding it. Full-text search, the ability to retrieve a document by any meaningful word it contains, is the natural extension of free-text keyword indexing across millions of pages rather than a single library catalogue.

Understanding keyword indexing therefore gives a clear window into how information retrieval evolved. The concepts of stop words, lead terms, and natural language indexing that students encounter in library science are the very building blocks of the search tools used every day. The vocabulary has changed and the scale has exploded, but the underlying principle has stayed remarkably constant.

Strengths and limitations at a glance

Keyword indexing offers real advantages. It is economical and quick to produce, requires no specialist indexer to assign terms, can be fully automated, and stays current with the language authors actually use. These qualities made it ideal for environments where new documents arrive faster than human indexers can process them.

Its weaknesses are equally clear. Without vocabulary control, synonyms scatter related documents and homographs pull in irrelevant ones, lowering both recall and precision. The system is only as good as the titles it works from, so poor or misleading titles produce a poor index. For deep subject analysis, controlled-vocabulary systems remain stronger. In practice, modern systems often blend the two approaches, using keyword searching for breadth and speed while retaining subject headings for accuracy.

What do you think? If a document’s title is misleading or overly creative, should an indexing system rely on the author’s words at all, or should a human always step in to assign subject terms? And as search engines grow more capable of understanding meaning rather than just matching words, how much longer will the pure keyword still matter?

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

References
  1. https://www.librarianshipstudies.com/2016/08/derived-indexing.html
  2. https://www.librarianshipstudies.com/2017/02/keyword-in-context-kwic-indexing.html
  3. https://en.wikipedia.org/wiki/Andrea_Crestadoro
  4. https://www.historyofinformation.com/detail.php?id=2062
  5. https://www.historyofinformation.com/detail.php?id=2065
  6. https://egyankosh.ac.in/bitstream/123456789/38415/5/Unit-13.pdf
  7. https://stoutewebsolutions.com/glossary/key-word-in-context/

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

Document Processing – Practice

1 Introduction, Structure and Organisation

  1. 19th Edition of DDC
  2. Notation
  3. Properties of Decimal Fractions
  4. Basic Plan and Convention of a Minimum of Three Digits
  5. Volume 1: Introduction and Tables
  6. Volume 2: Schedules
  7. Volume 3: Relative Index
  8. Transcription of a Class Number
  9. Numbers in Square Brackets, Obsolete Entries

2 Definitions, Notes and Instructions

  1. Definitions, Explanation and Scope
  2. Different Types of Notes
  3. Inclusion Notes
  4. Class Here’ Notes
  5. Class Elsewhere’ Notes
  6. Class … in…’ Notes
  7. For… see…’ Notes
  8. Class comprehensive works in…’ Notes
  9. Notes Appearing under Discontinued (Bracketed) Entries
  10. Importance of Various Notes
  11. Centred Heading/Centred Entry
  12. Number Building Notes

3 Introduction to Three Summaries and Steps in Classifying Documents

  1. Ten Main Classes
  2. Second Summary of 100 Divisions
  3. Third Summary of 1000 Sections
  4. Multi-Level Summaries
  5. Steps in Classifying Documents
  6. Subject Analysis
  7. Steps in Practical Classification

4 Relative Index and Its Use

  1. Need and Importance
  2. Nomenclature: Relative Index
  3. Scope of the Index
  4. Organisation of the Index
  5. Numbers Given Against Entries

5 Study of Tables and Schedules

  1. Tables
  2. Schedules

6 Auxiliary Tables and Devices

  1. Number-Building with Tables
  2. Use of Table 1: Standard Subdivisions
  3. Use of Table 2: Areas
  4. Table 3: Subdivisions of Individual Literatures
  5. Use of Table 4: Subdivisions of Individual Languages
  6. Use of Table 6: Languages
  7. Use of Table 5: Racial, Ethnic, National Groups
  8. Use of Table 7: Persons

7 Practical Classification

  1. Simple Synthesis
  2. Multiple Synthesis
  3. Order of Precedence
  4. Other Means for Fixing Priority of Numbers
  5. Table of Precedence for Standard Subdivisions

8 AACR-2R Preliminaries

  1. Structure of AACR-2R
  2. Levels of Description
  3. Style of Writing
  4. Types of Entries
  5. Items in the Catalogue Entry
  6. Skeleton Card
  7. Added Entries and Tracing Section
  8. Subject Headings

9 Choice and Rendering of Headings and Statement of Responsibility

  1. Personal Author
  2. Heading for Personal Author
  3. Western Names
  4. Indian Names
  5. Cataloguing Practice
  6. Single Personal Author
  7. Shared Responsibility
  8. Books under Editorial Direction
  9. Pseudonymous Authors
  10. Corporate Bodies

10 Cataloguing Multivolumes, Serial Publications and Nonprint Media

  1. Multi-volume Books
  2. Serial Publications
  3. Cataloguing of Non-Print Media

11 Marc-21 Cataloguing

  1. MARC Standards
  2. MARC 21 Structure
  3. MARC 21 Tags and Subfields

12 Structure of Sears List of Subject Headings (18th Edition)

  1. History of the Sears List
  2. Features of the 18th Edition
  3. Principles of Vocabulary Control
  4. Principles of the Sears List
  5. Structure of the Sears List
  6. Grammar of the Subject Headings
  7. Key Headings
  8. Adding Subdivisions
  9. Categories of Subject Headings Omitted
  10. Limitations

13 Keyword Indexing

  1. Keyword Indexing – Concept
  2. Structure and Format of Keyword Indexing
  3. Indexing Process
  4. Variants of Keyword Indexing
  5. Advantages and Disadvantages of Keyword Indexing

14 Chain Indexing (DDC – 19th Edition)

  1. Classified Catalogue
  2. Problems of Subject Cataloguing
  3. Advent of Chain Indexing
  4. Mechanism of Chain Indexing
  5. Step by Step Method
  6. Working with the DDC
  7. Preparation of Class Index Entries (CIEs)
  8. Chain Indexing for a Dictionary Catalogue
  9. Advantages of Chain Indexing
  10. Limitations and Problems of Chain Indexing