Every library catalogue, search engine, and database depends on one quiet but powerful process: turning the words inside documents into searchable access points. Keyword indexing is one of the oldest and most reliable ways to do this. It takes the significant words from a document’s title and arranges them so that a user can find the document no matter which word they search for first. This guide walks through the keyword indexing process step by step, from choosing the right keywords to filing finished entries in a usable sequence, with a full worked example at the end.

Table of Contents

What keyword indexing actually does

Keyword indexing is built on a simple assumption: the title of a document is a compact summary of its contents. The significant words in that title point directly to the subject. By extracting those words and rotating each one into the front position, an indexer creates multiple entry points for a single document. A reader who remembers only one word from the title can still locate it.

The technique was formalised by Hans Peter Luhn of IBM, who demonstrated his Keyword-in-Context (KWIC) system at the International Conference on Scientific Information in Washington in 1958. The idea of indexing by keywords in titles was older still, having been proposed by the librarian Andrea Crestadoro for Manchester libraries in the 1860s. Luhn’s contribution was to make the process automatic and repeatable using punched cards and early computers. The American Chemical Society soon adopted the method for its publication Chemical Titles, and KWIC became a standard tool for scientific literature.

Whether produced by a computer or worked out manually, the keyword indexing process follows three clear stages: selecting keywords, generating index entries, and arranging those entries alphabetically. Each stage has its own rules, and getting them right is what separates a useful index from a confusing one.

Step 1: Choosing keywords

The first step is to scan the title and decide which words carry meaning and which do not. Words that describe the subject are kept. Words that only serve grammar are discarded. This decision is not made word by word from scratch every time. Instead, indexers rely on a tool called a stop-list.

The role of the stop-list

A stop-list is a pre-defined list of words considered to have no value for indexing or retrieval. When a title is processed, any word that matches an entry on the stop-list is automatically rejected as a keyword. The remaining words become the access points. According to the standard KWIC method, a title is sorted and aligned so that every word except the stop words becomes searchable in the index.

Stop-lists typically include:

  • Articles: a, an, the
  • Prepositions: of, in, on, at, for, with
  • Conjunctions: and, or, but
  • Pronouns: it, this, that, they
  • Auxiliary verbs: is, are, was, has, have
  • Generic non-subject words: aspect, different, study, report, analysis, theory

That last category is worth attention. Words like report, analysis, and theory are grammatically fine, but they appear in so many titles that they fail to distinguish one document from another. A search for “report” in a large database would return thousands of irrelevant hits. Indexing literature notes that such general words are often added to stop-lists precisely because they create what are called false drops, where unrelated documents surface in a search.

Selection criteria

The guiding principle is significance. A keyword must denote a subject and serve as a genuine approach term, meaning a word a user would realistically type when looking for that document. Each major search system defines its own stop-list to suit its subject area. A medical database may treat “patient” as too common to index, while a general library catalogue might keep it. The criteria therefore balance two pressures: include enough keywords to ensure the document can be found, but exclude words so common that they drown the index in noise.

Step 2: Generating index entries

Once the keywords are chosen, the second step is to build an entry for each one. This is where the defining feature of keyword indexing appears: rotation, also called permutation.

How keyword rotation works

Rotation means producing one entry per significant keyword, with that keyword shifted into the lead position while the rest of the title trails behind to preserve context. Because the original word order is maintained, the part of the title that would fall off the front is wrapped around to the end. The result is that a single title with four keywords generates four separate entries, each beginning with a different subject word.

In the classic KWIC format, the keyword is placed at the centre of a fixed column, with the surrounding words of the title on either side. As one description of the method explains, the keyword is rotated to appear in the centre of each entry, surrounded by a portion of the title text. This lets a reader scan a single vertical line of keywords down the page while still seeing the context to the left and right.

Each generated entry contains three components:

  • Keyword: the significant subject word serving as the approach term.
  • Context: the remaining words of the title, which show how the keyword is used.
  • Identification or location code: a code, usually placed at the far right, that points to the full bibliographic record of the document.

The location code is essential. The index entry itself does not contain the document; it only tells the user where to find it. In a printed index this might be a serial number; in a database it links to the complete record.

Variations in entry format

Not every system centres the keyword. In the KWOC (Keyword-Out-of-Context) variant, the keyword is pulled out and printed at the left margin as a heading, followed by the complete title underneath. This makes the access points easier to scan in a list, although it sacrifices some of the contextual richness that the centred KWIC format provides. A further variant, KWAC (Keyword-Augmented-in-Context), enriches the title with extra terms drawn from the abstract or text when the title alone does not fully represent the document’s subject. These variants share the same underlying rotation logic; they differ only in how the keyword and context are displayed.

Step 3: Alphabetical arrangement

After all entries for all documents have been generated, the third step is to merge and file them. Every entry is placed at its correct position in a single alphabetical sequence, sorted by its leading keyword.

This is the step that makes the whole effort worthwhile. Because each document contributed several entries, each beginning with a different keyword, the same document now appears at several points in the alphabet. A user searching under any one of its significant words will land on it. The filing follows ordinary alphabetical rules, word by word, so that related keywords cluster together and retrieval becomes a matter of simple browsing.

Consistent filing rules matter here. An indexer must decide how to handle numbers, abbreviations, and hyphenated words, and then apply those decisions uniformly. Inconsistent filing scatters entries that users expect to find together, which undermines the index even when the keywords themselves were chosen well.

A worked example

The process becomes clearest when applied to a real title. Consider a document with the following title and identification code:

Title: Classification of Books in a University Library (code: 1279)

Applying step 1 (keyword selection)

Running the title against a stop-list removes the grammar words of, in, and a. The significant keywords that survive are:

Classification, Books, University, Library

Applying step 2 (entry generation by rotation)

Each of the four keywords is rotated to the front of an entry, with the rest of the title wrapped around to preserve context and the code 1279 placed at the right. The four generated entries are:

  • CLASSIFICATION of Books in a University Library 1279
  • BOOKS in a University Library / Classification of 1279
  • UNIVERSITY Library / Classification of Books in a 1279
  • LIBRARY / Classification of Books in a University 1279

This matches the worked form of the example used in standard descriptions of the KWIC indexing process, where the slash marks the point at which the title wraps around. Each entry leads with a capitalised keyword so the eye can find it quickly.

Applying step 3 (alphabetical arrangement)

Finally, the four entries are filed into the alphabetical sequence of the master index. Sorted by their leading keyword, they fall in this order:

  • BOOKS in a University Library / Classification of 1279
  • CLASSIFICATION of Books in a University Library 1279
  • LIBRARY / Classification of Books in a University 1279
  • UNIVERSITY Library / Classification of Books in a 1279

One document now has four doorways into the index. A reader who recalls only the word “University,” or only “Books,” reaches the same record. That redundancy is the entire point of keyword indexing.

Strengths and limits of the process

The keyword indexing process is fast, cheap, and easy to automate, which is why it spread quickly through scientific publishing in the 1960s. Because it draws terms straight from the title, it needs no controlled vocabulary and no trained indexer making subject judgements for every document.

That same simplicity is its weakness. The index is only as good as the title. A vague or misleading title produces vague or misleading entries, and there is no thesaurus to bring synonyms together, so a search for “automobile” will miss documents titled with “car.” Related material can scatter across the index instead of gathering under a single heading. These limitations are exactly what prompted the KWOC and KWAC variants, and later the controlled-vocabulary systems that followed. Even so, the three-step logic of selecting, rotating, and filing keywords remains the foundation beneath much of modern automated indexing.

What do you think? If you had to build a stop-list for a database in your own field of study, which common words would you exclude to cut down on false drops? And given that titles are not always accurate summaries, how far do you think a title-based index can be trusted before a system needs to pull keywords from the full text instead?

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

We are sorry that this post was not useful for you!

Let us improve this post!

Tell us how we can improve this post?

References
  1. https://www.historyofinformation.com/detail.php?id=2065
  2. https://www.emerald.com/insight/content/doi/10.1108/eb046652/full/html
  3. https://en.wikipedia.org/wiki/Key_Word_in_Context
  4. https://www.librarianshipstudies.com/2017/02/keyword-in-context-kwic-indexing.html

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

Document Processing – Practice

1 Introduction, Structure and Organisation

  1. 19th Edition of DDC
  2. Notation
  3. Properties of Decimal Fractions
  4. Basic Plan and Convention of a Minimum of Three Digits
  5. Volume 1: Introduction and Tables
  6. Volume 2: Schedules
  7. Volume 3: Relative Index
  8. Transcription of a Class Number
  9. Numbers in Square Brackets, Obsolete Entries

2 Definitions, Notes and Instructions

  1. Definitions, Explanation and Scope
  2. Different Types of Notes
  3. Inclusion Notes
  4. Class Here’ Notes
  5. Class Elsewhere’ Notes
  6. Class … in…’ Notes
  7. For… see…’ Notes
  8. Class comprehensive works in…’ Notes
  9. Notes Appearing under Discontinued (Bracketed) Entries
  10. Importance of Various Notes
  11. Centred Heading/Centred Entry
  12. Number Building Notes

3 Introduction to Three Summaries and Steps in Classifying Documents

  1. Ten Main Classes
  2. Second Summary of 100 Divisions
  3. Third Summary of 1000 Sections
  4. Multi-Level Summaries
  5. Steps in Classifying Documents
  6. Subject Analysis
  7. Steps in Practical Classification

4 Relative Index and Its Use

  1. Need and Importance
  2. Nomenclature: Relative Index
  3. Scope of the Index
  4. Organisation of the Index
  5. Numbers Given Against Entries

5 Study of Tables and Schedules

  1. Tables
  2. Schedules

6 Auxiliary Tables and Devices

  1. Number-Building with Tables
  2. Use of Table 1: Standard Subdivisions
  3. Use of Table 2: Areas
  4. Table 3: Subdivisions of Individual Literatures
  5. Use of Table 4: Subdivisions of Individual Languages
  6. Use of Table 6: Languages
  7. Use of Table 5: Racial, Ethnic, National Groups
  8. Use of Table 7: Persons

7 Practical Classification

  1. Simple Synthesis
  2. Multiple Synthesis
  3. Order of Precedence
  4. Other Means for Fixing Priority of Numbers
  5. Table of Precedence for Standard Subdivisions

8 AACR-2R Preliminaries

  1. Structure of AACR-2R
  2. Levels of Description
  3. Style of Writing
  4. Types of Entries
  5. Items in the Catalogue Entry
  6. Skeleton Card
  7. Added Entries and Tracing Section
  8. Subject Headings

9 Choice and Rendering of Headings and Statement of Responsibility

  1. Personal Author
  2. Heading for Personal Author
  3. Western Names
  4. Indian Names
  5. Cataloguing Practice
  6. Single Personal Author
  7. Shared Responsibility
  8. Books under Editorial Direction
  9. Pseudonymous Authors
  10. Corporate Bodies

10 Cataloguing Multivolumes, Serial Publications and Nonprint Media

  1. Multi-volume Books
  2. Serial Publications
  3. Cataloguing of Non-Print Media

11 Marc-21 Cataloguing

  1. MARC Standards
  2. MARC 21 Structure
  3. MARC 21 Tags and Subfields

12 Structure of Sears List of Subject Headings (18th Edition)

  1. History of the Sears List
  2. Features of the 18th Edition
  3. Principles of Vocabulary Control
  4. Principles of the Sears List
  5. Structure of the Sears List
  6. Grammar of the Subject Headings
  7. Key Headings
  8. Adding Subdivisions
  9. Categories of Subject Headings Omitted
  10. Limitations

13 Keyword Indexing

  1. Keyword Indexing – Concept
  2. Structure and Format of Keyword Indexing
  3. Indexing Process
  4. Variants of Keyword Indexing
  5. Advantages and Disadvantages of Keyword Indexing

14 Chain Indexing (DDC – 19th Edition)

  1. Classified Catalogue
  2. Problems of Subject Cataloguing
  3. Advent of Chain Indexing
  4. Mechanism of Chain Indexing
  5. Step by Step Method
  6. Working with the DDC
  7. Preparation of Class Index Entries (CIEs)
  8. Chain Indexing for a Dictionary Catalogue
  9. Advantages of Chain Indexing
  10. Limitations and Problems of Chain Indexing