Knowledge Organisation III: Indexing Systems and Techniques, Assigned and Derived, Pre-Coordinate and Post-Coordinate; Vocabulary Control; Abstracting, its Types and Guidelines; and Information Retrieval Systems, their Features, Components, Models and Evaluation

The third chapter of Unit V covers items 8 to 10, the part of knowledge organisation that serves searching rather than shelving. It sets out indexing systems as the syllabus classifies them, assigned indexing in its pre-coordinate and post-coordinate forms and derived, title-based indexing, with worked chain indexing; vocabulary control through subject heading lists and thesauri; abstracting, its types and the standards that guide it; and information retrieval systems, their features, components and models, ending with evaluation by recall and precision, worked with numbers.

1. Indexing systems: assigned, pre-coordinate and post-coordinate

Indexing represents the subject content of documents by index terms so that they can be retrieved. In assigned indexing, a trained indexer chooses terms, usually from a controlled vocabulary, that need not appear in the document. In derived indexing, the terms are taken from the document itself, from its title, abstract or text. Assigned indexing is further divided by when the concepts are combined. In pre-coordinate indexing the indexer combines the concepts into a string at the time of indexing, in a set citation order, so the index shows compound subjects as ready-made headings (the classic printed index). In post-coordinate indexing each concept is indexed separately and the searcher combines them at the time of searching, which is how every computer database works with Boolean operators.

SystemOriginator and dateKey features
Kaiser's systematic indexingJ. Kaiser, 1911Pre-coordinate; terms are concretes, processes and countries, cited in that order
Chain indexingS. R. Ranganathan, 1938Pre-coordinate; derives subject index entries from the chain of a class number, each link giving one entry, last link first
Relational indexingJ. E. L. Farradane, 1950sPre-coordinate; terms joined by operators expressing relations between concepts
PRECIS (Preserved Context Index System)Derek Austin, British National Bibliography, 1974Pre-coordinate; role operators give the string's order; "shunting" rotates terms so each entry keeps its context; replaced at the BNB by COMPASS in 1991
POPSI (Postulate-based Permuted Subject Indexing)G. Bhattacharyya, DRTC, 1970sPre-coordinate; based on Ranganathan's postulates, with elementary categories Discipline, Entity, Action and Property, and permutation of terms for entries
UnitermMortimer Taube, 1953Post-coordinate; single-word terms each on a card listing document numbers; the searcher compares cards for common numbers
Optical coincidence (peek-a-boo) cardsW. E. Batten, 1940sPost-coordinate; one card per term with holes punched at document-number positions; stacking cards and holding them to the light shows documents with all the terms

Chain indexing worked with DDC: the number 330.954 has the chain 300 Social sciences, 330 Economics, 330.9 Economic conditions, 330.954 India. Chain procedure writes one entry for each link that users might look up, beginning with the last and adding the higher links as qualifiers: "India, Economic conditions, Economics 330.954"; "Economic conditions, Economics 330.9"; "Economics 330"; "Social sciences 300". A user who looks up India, economic conditions or economics is led to the right class number. The method is economical, but only the last link of each entry is fully specific, and it depends entirely on the classification scheme's chain.

2. Derived, title-based indexing

Derived indexing takes significant words from the document, originally from titles, which is cheap enough for a computer to do. H. P. Luhn of IBM introduced the keyword-in-context (KWIC) index around 1958-60: each significant title word, after stop words such as "a", "of" and "the" are removed, becomes an entry point, printed aligned in a central column with the rest of the title wrapped around it, so the context is visible. Keyword-out-of-context (KWOC) prints the keyword as a heading in the margin with the full title beneath; keyword-and-context (KWAC) keeps the title intact and appends the displaced words. Title-based indexes are fast and current (Chemical Titles, 1960, was an early KWIC index) but depend on how informative titles are, cannot bring synonyms together and scatter related concepts; enrichment with added keywords partly remedies this. Citation indexing, proposed by Eugene Garfield in 1955, is another derived method: the "terms" are the references a document cites.

⚠️ KWIC is derived, not assigned
KWIC, KWOC and KWAC use words that actually occur in titles, with no controlled vocabulary, so they are derived indexing. Chain indexing, PRECIS and POPSI assign terms according to rules and a classification or vocabulary, so they are assigned and pre-coordinate.

3. Vocabulary control

Natural language has synonyms (several words for one concept: cars, automobiles), homonyms (one word for several concepts: mercury the metal, the planet and the god) and variant spellings, and it does not show hierarchies. Vocabulary control fixes one preferred term for each concept, refers users from non-preferred terms, distinguishes homonyms with qualifiers, and records broader, narrower and related concepts, so that indexer and searcher use the same term. The tools are subject heading lists, such as the Library of Congress Subject Headings (first published 1909-14) and the Sears List of Subject Headings (1923) for smaller libraries, and thesauri, which list terms and their relationships for a particular field: MeSH (Medical Subject Headings, National Library of Medicine, 1960), the ERIC Thesaurus, the UNESCO Thesaurus, AGROVOC (FAO) and the INSPEC Thesaurus. Thesaurus construction is standardised in ISO 25964 (2011-13), which replaced ISO 2788 and ISO 5964.

RelationshipAbbreviationsExample
EquivalenceUSE / UF (used for)Automobiles USE Cars; Cars UF Automobiles
HierarchicalBT (broader term) / NT (narrower term)Libraries NT Academic libraries; Academic libraries BT Libraries
AssociativeRT (related term)Libraries RT Librarians
ScopeSN (scope note)Defines how a term is used in the vocabulary

4. Abstracting: types and guidelines

An abstract is an abbreviated, accurate representation of the contents of a document, without added interpretation or criticism, according to the ANSI/NISO definition. It helps readers decide whether to read the full document, saves time, overcomes language barriers when written in another language, and supports retrieval when abstracts are indexed. Abstracts may be written by the author (author abstracts), by subject specialists or professional abstractors for abstracting services, or increasingly generated automatically.

TypeWhat it does
Indicative (descriptive)Says what the document is about and what it covers, without giving results; suits reviews, long reports and discursive works
InformativeGives the purpose, methods, results and conclusions, with key data, so it can partly stand in for the document; suits experimental and research papers
Indicative-informativeInformative on the main points and indicative on the rest
CriticalAdds the abstractor's evaluation of the work's quality; close to a short review
StructuredUses fixed headings such as background, objectives, methods, results and conclusions; standard in medical journals
Slanted (mission-oriented)Written for a particular group, emphasising what matters to them
Statistical or tabular, and telegraphicData presented in tables; or notes in abbreviated, telegraphic style
  • Standards: ISO 214 (Documentation, abstracts for publications and documentation; first 1976) and ANSI/NISO Z39.14 (Guidelines for Abstracts, 1997).
  • Good practice: state the purpose, method, results and conclusions in that order; be concise, usually a single paragraph; use complete sentences, the active voice where possible and the past tense for what was done and found; include no information absent from the document; avoid undefined abbreviations and references to figures.

5. Information retrieval systems: features, components, models and evaluation

An information retrieval (IR) system selects, from a store of documents or records, those relevant to a user's query. Its components are a document subsystem (acquisition, indexing or representation, storage), a query subsystem (the user's need expressed as a search statement), a matching subsystem that compares the two and ranks or selects results, and an interface with feedback. Search features include Boolean operators, truncation and wildcards, phrase and proximity searching, field searching, limits, relevance ranking and relevance feedback.

ModelHow documents are matched
BooleanExact match with AND (both terms), OR (either) and NOT (exclude); a document is retrieved or not, with no ranking
Vector spaceGerard Salton (the SMART system): documents and queries are vectors of weighted terms (tf-idf); ranked by similarity, usually the cosine of the angle between vectors
ProbabilisticStephen Robertson and Karen Spärck Jones (1976): ranks by the estimated probability of relevance; BM25 is its best-known weighting
Fuzzy set and extended BooleanPartial membership and graded Boolean matching, giving ranked output
Latent semantic indexingReduces the term-document matrix to underlying concepts, so documents can match without sharing exact words

Evaluation began with Cyril Cleverdon's Cranfield experiments (Cranfield I, 1957-62; Cranfield II, 1963-66), which compared indexing languages on a test collection with relevance judgements and established recall and precision as the standard measures; F. W. Lancaster's evaluation of MEDLARS (1968) applied them to an operational system, and the Text REtrieval Conference (TREC), run by NIST since 1992, continues the tradition. Recall is the proportion of all relevant documents in the collection that the search retrieves; precision is the proportion of retrieved documents that are relevant. Worked example: a collection holds 40 documents relevant to a query; a search retrieves 50 documents, of which 30 are relevant. Precision = 30 / 50 = 60 per cent. Recall = 30 / 40 = 75 per cent. The F-measure, the harmonic mean of the two, is 2 × 0.60 × 0.75 / (0.60 + 0.75) = 0.90 / 1.35 = 0.67. Cleverdon observed that recall and precision tend to vary inversely: broadening a search raises recall and lowers precision.

🧠 Which denominator?
Precision divides by what you retrieved; recall divides by what exists. Retrieved-relevant is the numerator in both. Recall needs the total number of relevant documents in the collection, which is why it is hard to measure outside test collections.

Key takeaways

  • Assigned indexing uses terms chosen by an indexer; derived indexing takes words from the document; pre-coordinate systems combine terms at indexing, post-coordinate systems at searching.
  • Pre-coordinate systems: Kaiser (1911), chain indexing (Ranganathan, 1938), relational indexing (Farradane), PRECIS (Austin, 1974), POPSI (Bhattacharyya); post-coordinate: Uniterm (Taube, 1953) and peek-a-boo cards; chain indexing turns 330.954 into entries under India, economic conditions, economics and social sciences.
  • Derived indexing: KWIC (Luhn), KWOC, KWAC and citation indexing (Garfield, 1955); vocabulary control uses LCSH, Sears and thesauri such as MeSH (1960), with USE/UF, BT/NT, RT and SN, under ISO 25964.
  • Abstracts are indicative, informative, indicative-informative, critical, structured or slanted; ISO 214 and ANSI/NISO Z39.14 give guidelines.
  • IR models are Boolean, vector space (Salton), probabilistic (Robertson and Spärck Jones) and others; evaluation from Cranfield (Cleverdon) to TREC (1992) uses precision (relevant retrieved over retrieved) and recall (relevant retrieved over all relevant): 30 of 50 retrieved with 40 relevant gives 60 and 75 per cent.

Practice questions (10)

Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.

  1. A collection contains 80 documents relevant to a query. A search retrieves 40 documents, of which 32 are relevant. What is the recall, in per cent? Type the number.

    Numerical answer — type the value.

    Show answer

    Answer: 40

    Recall = relevant retrieved / all relevant in the collection = 32 / 80 = 0.40, that is 40 per cent. Precision for the same search is 32 / 40 = 80 per cent, showing a precise but low-recall search.
  2. A search retrieves 50 documents, of which 30 are relevant; the collection holds 40 relevant documents. What is the precision, in per cent? Type the number.

    Numerical answer — type the value.

    Show answer

    Answer: 60

    Precision = relevant retrieved / total retrieved = 30 / 50 = 0.60, that is 60 per cent. The total of 40 relevant documents is needed only for recall, which here is 30 / 40 = 75 per cent.
  3. PRECIS was developed by Derek Austin for:

    1. The Library of Congress
    2. The Documentation Research and Training Centre
    3. The US National Library of Medicine
    4. The British National Bibliography
    Show answer

    Answer: D — The British National Bibliography

    PRECIS was devised for subject indexing in the British National Bibliography and used there from the early 1970s until COMPASS replaced it in 1991. POPSI was developed at DRTC, and the National Library of Medicine uses MeSH.
  4. In a thesaurus, the relationship between "Libraries" and "Academic libraries" is shown by:

    1. SN
    2. BT and NT
    3. RT
    4. USE and UF
    Show answer

    Answer: B — BT and NT

    Academic libraries is a narrower term of Libraries, so Libraries NT Academic libraries and Academic libraries BT Libraries. USE and UF show equivalence, RT an associative relation, and SN a scope note.
  5. Which of the following are pre-coordinate indexing systems? Select all that apply.

    1. Uniterm
    2. Optical coincidence cards
    3. Chain indexing
    4. POPSI
    Show answer

    Answer: C — Chain indexing; D — POPSI

    Chain indexing and POPSI combine terms into strings at the indexing stage. Uniterm and optical coincidence (peek-a-boo) cards index each term separately and leave the searcher to combine them, so they are post-coordinate.
  6. An abstract that states the purpose, methods, results and conclusions of a research paper, with key figures, so that it can partly substitute for reading the paper, is:

    1. Informative
    2. Critical
    3. Indicative
    4. Slanted
    Show answer

    Answer: A — Informative

    An informative abstract carries the substance, including results. An indicative abstract only says what the document covers, a critical abstract adds evaluation, and a slanted abstract is written for a particular audience.
  7. Assertion (A): KWIC indexes cannot bring together documents that use different words for the same concept. Reason (R): KWIC is derived indexing that uses the words of titles without vocabulary control. Choose the correct option.

    1. Both A and R are true, and R is the correct explanation of A
    2. Both A and R are true, but R is not the correct explanation of A
    3. A is true, but R is false
    4. A is false, but R is true
    Show answer

    Answer: A — Both A and R are true, and R is the correct explanation of A

    Both are true and R explains A. Because KWIC takes words straight from titles with no controlled vocabulary, titles using "cars" and "automobiles" are filed in different places; only vocabulary control can collocate synonyms.
  8. Which information retrieval model represents documents and queries as weighted term vectors and ranks documents by the cosine of the angle between them?

    1. The Boolean model
    2. The probabilistic model
    3. Chain indexing
    4. The vector space model
    Show answer

    Answer: D — The vector space model

    Salton's vector space model, developed with the SMART system, measures similarity between term vectors. The Boolean model gives exact, unranked matches, the probabilistic model ranks by estimated probability of relevance, and chain indexing is a pre-coordinate indexing method.
  9. Which statements about the evaluation of information retrieval are correct? Select all that apply.

    1. TREC has been run by NIST since 1992
    2. The Cranfield experiments were led by Cyril Cleverdon
    3. Recall and precision tend to vary inversely
    4. Precision is relevant documents retrieved divided by all relevant documents in the collection
    Show answer

    Answer: A — TREC has been run by NIST since 1992; B — The Cranfield experiments were led by Cyril Cleverdon; C — Recall and precision tend to vary inversely

    Cleverdon led Cranfield, recall and precision usually trade off, and NIST has run TREC since 1992. The last statement defines recall, not precision; precision divides by the number of documents retrieved.
  10. Using chain indexing on the DDC number 330.954 (300 Social sciences, 330 Economics, 330.9 Economic conditions, 330.954 India), which is the first and most specific index entry?

    1. Economic conditions, India 330.9
    2. India, Economic conditions, Economics 330.954
    3. Social sciences, Economics, Economic conditions, India 330.954
    4. Economics 330
    Show answer

    Answer: B — India, Economic conditions, Economics 330.954

    Chain procedure starts with the last link, India, qualified by the higher links in reverse order, and gives it the full class number 330.954. Entries for higher links (Economics 330) follow; citing the links from the top down or attaching India to 330.9 is not the chain procedure.