Research Methodology II: Research Design; Primary and Secondary Data — Observation, Questionnaire, Interview, Library and Internet; Sampling and Sampling Techniques; Scales of Measurement; Analysis of Data and SPSS; Citation Patterns, Research Ethics, Bibliography and Report Writing

This chapter takes the second half of Unit X — from "Research Design" through "Report Writing" — the practical machinery that turns a hypothesis into evidence and evidence into a report. Each step is a set of choices with known strengths and weaknesses: which design rules out rival explanations, which method of data collection suits the question and the respondents, which sampling technique allows generalisation, which scale of measurement permits which statistics, and how the work is cited, conducted ethically and written up. The examples come from public administration research — citizen surveys, evaluations of schemes, studies of officials — and the arithmetic that sampling and analysis questions require is worked through.

1. Research design

A research design is the plan and structure of an investigation, conceived to obtain answers to the research question and to control variance — Fred Kerlinger's formulation. It specifies what data will be collected, from whom, how, when, and how they will be analysed, so that the evidence can rule out rival explanations. The main types follow the objectives of research: exploratory designs (literature surveys, experience surveys, analysis of insight-stimulating cases) are flexible; descriptive and diagnostic designs are rigid and emphasise accurate measurement and representative samples; experimental designs test causal hypotheses by manipulating an independent variable.

Designs and examples
DesignFeaturesPublic administration example
True experimentRandom assignment to treatment and control groups; manipulation of the independent variable; pre- and post-testsA randomised trial of text-message reminders on tax compliance
Quasi-experimentComparison groups without random assignment; controls through matching, time series or difference-in-differencesComparing districts that adopted an e-governance reform early with those that adopted it later
Survey (cross-sectional or longitudinal)Standardised questions to a sample, to describe and correlateA citizen satisfaction survey of public services
Case studyIntensive study of one or a few units in context, using multiple sourcesHow one state built its disaster management authority

Donald Campbell and Julian Stanley's Experimental and Quasi-Experimental Designs for Research (1963) catalogued the threats to internal validity — whether the treatment really caused the effect: history (outside events), maturation, testing effects, instrumentation, statistical regression toward the mean, selection bias and experimental mortality (attrition). External validity is the extent to which findings generalise to other people, settings and times; the Hawthorne effect — subjects changing their behaviour because they are being studied — threatens both.

2. Methods of data collection: primary and secondary sources

Primary data are collected first-hand by the researcher for the study at hand; secondary data already exist, collected by others for other purposes — the Census, National Sample Survey and National Family Health Survey reports, government annual reports, CAG and parliamentary committee reports, court judgments, archives, and published research. Secondary data are cheap and often large-scale, but may not fit the research question exactly and must be checked for reliability, suitability and adequacy.

Methods of collecting data
MethodFormsStrengthsLimitations
ObservationParticipant (the researcher joins the group) or non-participant; structured or unstructured; overt or covertRecords actual behaviour rather than reported behaviour; rich contextObserver bias; the observed may change behaviour; time-consuming; ethical issues in covert observation
QuestionnaireA written list of questions filled in by the respondent — mailed, handed out or online; closed-ended (fixed options) or open-endedCheap; covers large, dispersed samples; anonymity encourages frank answers; free of interviewer biasNeeds literate respondents; low response rates; no chance to clarify questions
Interview (and schedule)Structured (fixed questions in a fixed order), semi-structured or unstructured (in-depth); focus group discussions. A schedule is a set of questions filled in by the enumerator face to face — unlike a questionnaire, which the respondent fills inWorks with non-literate respondents; high response; allows probing and clarificationCostly; interviewer bias; respondents may give socially desirable answers
Library and internetBooks, journals, reports, databases; in India, Shodhganga (the INFLIBNET repository of Indian theses) and e-ShodhSindhu (consortium access to e-journals); JSTOR, Google Scholar, government portals and open dataLiterature review, secondary data, speed and rangeVariable quality and authenticity of online sources; the need to evaluate credibility
⚠️ Questionnaire or schedule?
The difference is who fills it in. A questionnaire is completed by the respondent (so it needs literate respondents and gets lower response rates); a schedule is completed by the enumerator or interviewer in the respondent's presence (so it suits non-literate respondents, as in most Indian household surveys, and gives higher response). A pilot study or pre-test should precede either, to catch ambiguous questions.

3. Sampling and sampling techniques

The population (universe) is the whole set of units the study is about; a sample is a subset selected to represent it; the sampling frame is the list from which the sample is drawn (an electoral roll, a list of gram panchayats); the sampling unit is the unit selected. A census studies every unit. Sampling saves time and money and can even be more accurate than a census, because resources can be concentrated on quality. Sampling error is the difference between a sample statistic and the population value that arises by chance and falls as sample size rises; non-sampling errors — frame errors, non-response, measurement and processing errors — arise in censuses too and do not fall with sample size.

Probability and non-probability sampling
TechniqueHow units are selected
Simple random (probability)Every unit has an equal and known chance, by lottery or random numbers
Systematic (probability)Every k-th unit from a list after a random start, where the sampling interval k = N ÷ n
Stratified (probability)Divide the population into homogeneous strata (by caste, gender, region), then sample randomly within each, proportionately or disproportionately
Cluster and multi-stage (probability)Randomly select naturally occurring groups (villages, schools), then study all units in them — or select in stages: districts, then blocks, then villages, then households
Convenience (non-probability)Whoever is easiest to reach
Purposive or judgement (non-probability)Units chosen by the researcher's judgement as typical or informative — for example, key informants among senior officials
Quota (non-probability)Fill set quotas for categories (so many women, so many men) by any convenient method — like stratified sampling without randomness
Snowball (non-probability)Each respondent refers others — for hard-to-reach or hidden populations such as migrant workers or bonded labourers
🧠 Working the sampling interval
To draw a systematic sample of 100 households from a list of 2,000, compute k = N ÷ n = 2,000 ÷ 100 = 20; pick a random start between 1 and 20 (say 7), then take households 7, 27, 47, 67 and so on. Only probability samples allow the sampling error to be estimated and results to be generalised statistically to the population.

4. Scales of measurement; analysis of data and SPSS

S. S. Stevens (1946) distinguished four levels of measurement, each permitting more mathematical operations than the last. The level determines which statistics can properly be used, so misclassifying a variable produces wrong analysis.

Stevens's four levels of measurement
ScalePropertyExampleAppropriate statistics
NominalCategories without orderReligion, state of residence, departmentMode, frequencies, chi-square
OrdinalOrdered categories with unequal or unknown intervalsRank in a service; satisfaction rated "low, medium, high"Median, percentiles, rank correlation (Spearman)
IntervalEqual intervals but no true zeroTemperature in Celsius; calendar yearsMean, standard deviation, Pearson correlation, t-test
RatioEqual intervals and a true zero, so ratios are meaningfulIncome, age, number of files disposed of, budgetAll of the above, plus geometric mean and coefficient of variation

Attitude scales used in social research include the Likert scale (summated ratings of agreement, usually five points, from strongly agree to strongly disagree), the Thurstone scale (equal-appearing intervals judged by experts), the Guttman scale (cumulative items, where agreeing with a stronger item implies agreeing with weaker ones), Bogardus's social distance scale, and Osgood's semantic differential (bipolar adjectives such as efficient–inefficient). Analysis of data runs from editing and coding to tabulation, then descriptive statistics (frequencies, measures of central tendency and dispersion, cross-tabulation) and inferential statistics (tests of significance — chi-square for nominal data, t-tests and ANOVA for comparing means, correlation and regression for relationships). For a chi-square test on a contingency table, the degrees of freedom are (rows − 1) × (columns − 1). SPSS — originally the Statistical Package for the Social Sciences, developed in 1968 by Norman Nie, C. Hadlai Hull and Dale Bent and acquired by IBM in 2009 — is the most widely used software for such analysis, alongside Stata, R and, for qualitative data, packages such as NVivo and ATLAS.ti.

5. Citation patterns, ethics of research, bibliography and report writing

Citation acknowledges the sources of ideas, data and words, allows readers to verify them, and places the work in its scholarly conversation. The major styles follow two patterns: author–date (in-text parenthetical citations such as "(Riggs, 1964, p. 22)" with a reference list — the APA style, now in its seventh edition, and Chicago author–date) and notes–bibliography (footnotes or endnotes with a bibliography — Chicago notes style, common in history), with MLA (author–page) prevalent in the humanities. Citation analysis measures scholarly influence: Eugene Garfield's Science Citation Index and the journal impact factor, and Jorge Hirsch's h-index (2005) — a researcher has index h if h of his or her papers have each been cited at least h times.

The ethics of research protect participants and the integrity of knowledge. The core principles, set out in the US Belmont Report (1979), are respect for persons (informed, voluntary consent, and protection of those with diminished autonomy), beneficence (maximise benefits, minimise harm) and justice (fair distribution of the burdens and benefits of research). In practice they require informed consent, confidentiality and anonymity, avoidance of deception and harm, special care with vulnerable groups, review by an ethics committee, honest reporting without fabrication or falsification of data, and the avoidance of plagiarism — presenting another's work as one's own. In India the UGC (Promotion of Academic Integrity and Prevention of Plagiarism in Higher Educational Institutions) Regulations, 2018 grade plagiarism by the similarity found: up to 10 per cent is not penalised; Level 1 (above 10 up to 40 per cent), Level 2 (above 40 up to 60 per cent) and Level 3 (above 60 per cent) attract increasing penalties.

A bibliography lists all the sources consulted, cited or not, usually alphabetically by author; a reference list includes only works actually cited in the text; an annotated bibliography adds a short description or evaluation of each source. A research report conventionally has three parts. The preliminary pages: title page, declaration and certificate, acknowledgements, table of contents, lists of tables and figures, and an abstract. The main text: introduction (problem, objectives, significance), review of literature, research design and methodology, presentation and analysis of data, discussion of findings, and conclusions with recommendations and the study's limitations. The end matter: references or bibliography, appendices (questionnaires, schedules, additional tables) and index. Good report writing is clear, objective, consistent in its citation style, and honest about limitations.

Key takeaways

  • A research design controls variance to rule out rival explanations; Campbell and Stanley (1963) listed the threats to internal validity, and the Hawthorne effect threatens both internal and external validity.
  • Primary data come from observation, questionnaires, interviews and schedules; a questionnaire is filled in by the respondent, a schedule by the enumerator; Shodhganga holds Indian theses.
  • Probability samples (simple random, systematic with k = N ÷ n, stratified, cluster, multi-stage) allow generalisation; non-probability samples (convenience, purposive, quota, snowball) do not.
  • Stevens's scales are nominal, ordinal, interval and ratio; chi-square degrees of freedom are (r − 1)(c − 1); SPSS dates from 1968.
  • APA is author–date, Chicago notes–bibliography; the Belmont principles are respect for persons, beneficence and justice; the UGC's 2018 regulations grade plagiarism above 10 per cent into three levels.

Practice questions (10)

Attempt each one before opening the answer. Every explanation names the tempting wrong option as well as the right one, because that is where marks are lost.

  1. A researcher wants a systematic sample of 100 households from a list of 2,000. What is the sampling interval? Type a number.

    Numerical answer — type the value.

    Show answer

    Answer: 20

    k = N ÷ n = 2,000 ÷ 100 = 20. After a random start between 1 and 20, every twentieth household on the list is selected.
  2. A chi-square test is applied to a contingency table with 3 rows and 4 columns. How many degrees of freedom does the test have? Type a number.

    Numerical answer — type the value.

    Show answer

    Answer: 6

    Degrees of freedom = (rows − 1) × (columns − 1) = (3 − 1) × (4 − 1) = 2 × 3 = 6.
  3. The main difference between a questionnaire and a schedule is that:

    1. A schedule cannot be used with non-literate respondents
    2. A questionnaire has only open-ended questions
    3. A schedule is used only for secondary data
    4. A questionnaire is filled in by the respondent, a schedule by the enumerator in the respondent's presence
    Show answer

    Answer: D — A questionnaire is filled in by the respondent, a schedule by the enumerator in the respondent's presence

    Who fills it in is the difference; because the enumerator records the answers, the schedule suits non-literate respondents — the reverse of the last option. Both can mix closed and open questions and both collect primary data.
  4. Which of the following are probability sampling techniques? Select all that apply.

    1. Snowball sampling
    2. Cluster sampling
    3. Stratified random sampling
    4. Quota sampling
    Show answer

    Answer: B — Cluster sampling; C — Stratified random sampling

    Stratified and cluster sampling give every unit a known chance of selection. Snowball and quota sampling are non-probability techniques; quota sampling resembles stratification but fills the quotas without randomness.
  5. Citizens' satisfaction with a service is recorded as "very dissatisfied, dissatisfied, neutral, satisfied, very satisfied". This variable is measured on:

    1. An interval scale
    2. An ordinal scale
    3. A nominal scale
    4. A ratio scale
    Show answer

    Answer: B — An ordinal scale

    The categories are ordered but the distances between them are not known to be equal, so the scale is ordinal; the median and rank correlations are appropriate, strictly speaking, rather than the mean.
  6. Assertion (A): Increasing the sample size reduces sampling error. Reason (R): Increasing the sample size also eliminates non-response and measurement errors. Choose the correct option.

    1. Both A and R are true, and R is the correct explanation of A
    2. Both A and R are true, but R is not the correct explanation of A
    3. A is true, but R is false
    4. A is false, but R is true
    Show answer

    Answer: C — A is true, but R is false

    A is true: sampling error arises from chance and shrinks as n grows. R is false: non-sampling errors such as non-response, frame and measurement errors do not fall with sample size and occur even in a complete census.
  7. Which of the following are the principles of research ethics set out in the Belmont Report (1979)? Select all that apply.

    1. Justice
    2. Beneficence
    3. Respect for persons
    4. Profitability
    Show answer

    Answer: A — Justice; B — Beneficence; C — Respect for persons

    Belmont's three principles are respect for persons, beneficence and justice, which ground informed consent, risk–benefit assessment and fair selection of subjects. Profitability is not a research-ethics principle.
  8. Under the UGC's 2018 regulations on plagiarism, similarity of above 40 per cent and up to 60 per cent falls in:

    1. Level 2
    2. Level 3
    3. The no-penalty band
    4. Level 1
    Show answer

    Answer: A — Level 2

    Up to 10 per cent carries no penalty; Level 1 is above 10 up to 40 per cent, Level 2 above 40 up to 60 per cent, and Level 3 above 60 per cent.
  9. SPSS, widely used for analysing social science data, originally stood for:

    1. Standard Program for Survey Sampling
    2. Survey Processing and Scaling Software
    3. Statistical Package for the Social Sciences
    4. Social Policy Statistical System
    Show answer

    Answer: C — Statistical Package for the Social Sciences

    SPSS was the Statistical Package for the Social Sciences, developed in 1968 by Nie, Hull and Bent; IBM acquired it in 2009.
  10. Which threat to internal validity occurs when participants drop out of a study unevenly between groups?

    1. Maturation
    2. Statistical regression
    3. Experimental mortality (attrition)
    4. History
    Show answer

    Answer: C — Experimental mortality (attrition)

    Experimental mortality — differential attrition — can make the groups unequal by the end of the study. Maturation is change within participants over time, history is outside events, and statistical regression is the drift of extreme scores toward the mean.