Skip to content
QDA2 Lab

MODULE 02

Building an Index

Some concepts are too big for one question. How several survey items become one number, and what that number can and cannot tell you.

Examples on this page use constructed or simulated teaching data, not empirical findings. About the data

  1. Research question
  2. Variables (trained here)
  3. Variable roles
  4. Measurement levels (trained here)
  5. Method choice
  6. Statistical logic
  7. SPSS workflow (trained here)
  8. Output interpretation (trained here)
  9. Defensible conclusion

Understand

Why ask more than one question?

Trust in science, xenophobia, religiosity: these are latent concepts. Nobody can answer them directly, so we ask several concrete items, each covering one aspect, and combine the answers afterwards. That combining step is data reduction.

More valid

Validity: measuring what you mean to measure. A single question such as “Do you trust science?” leaves every respondent to decide what “science” and “trust” mean. Several concrete items (relying on findings, following advice in a crisis, suspecting hidden results) pin the concept down and cover its different sides.

More reliable

Reliability: getting the same result when you measure again under the same conditions. One answer on a 0–10 scale wobbles from day to day; the average of several answers wobbles much less, because the random ups and downs of single items partly cancel out.

Two respondents, one number

Four items of a trust index, already pointing the same way (1 = low trust, 5 = high trust). Respondent A answers consistently; change respondent B’s answers and watch what the index keeps and what it throws away.

12345T1T2T3T4index

Respondent B

5
5
1
3

A: answers 4, 4, 3, 3 → index 3.50, spread across items (SD) 0.58

B: answers 5, 5, 1, 3 → index 3.50, spread across items (SD) 1.91

Same index score, different answer patterns. Once we average, the index cannot tell a consistent respondent from one who strongly agrees with some items and strongly disagrees with others. That is the price of data reduction: we give up detail to get one more valid and more reliable number.

Index: same direction, then average

Every item measures more or less of the same concept, so after aligning their direction we can add them up or average them. The result is a scale-level variable from 1 to 5.

Typology: combine categories, never average

A typology crosses categorical variables into types (for example, attends services often or rarely × prays often or rarely gives four types). Codes such as 11, 12, 21, 22 are labels of a nominal variable: their mean means nothing.
  1. 1

    Clean

    Frequencies on every item; set ‘don't know’ and refusals as missing.

  2. 2

    Align

    Reverse-score the items worded the other way, into new variables.

  3. 3

    Select

    Reliability Analysis; delete the weakest item, one at a time, while alpha rises. Keep ≥ 3.

  4. 4

    Compute

    Index = mean of the kept items. Label what a high value means.

  5. 5

    Categorise

    Optional: recode into absolute, relative or percentile categories.

Try

Build the index yourself

Steps 2 and 3, live: reverse, read the correlations, watch alpha, delete one item at a time.

Index builder: trust in science

120 synthetic respondents answered six items (1 = strongly disagree … 5 = strongly agree). We want one index where a high value means more trust. Reverse the items that are worded the other way, then drop items that do not belong, one at a time, until alpha is acceptable. Keep at least 3 items.

  • T1

    Scientists mostly act in the public interest.

    item-total r = 0.039 (below .25)α if deleted = -0.422

  • T2

    Published research findings can usually be relied on.

    item-total r = 0.042 (below .25)α if deleted = -0.417

  • T3

    Science creates more problems than it solves.

    item-total r = -0.347 (below .25)α if deleted = 0.111

  • T4

    In a health crisis I would follow the advice of scientists.

    item-total r = 0.048 (below .25)α if deleted = -0.438

  • T5

    Scientists hide results that are inconvenient for them.

    item-total r = -0.357 (below .25)α if deleted = 0.113

  • T6

    I enjoy watching documentaries about science.

    item-total r = 0.130 (below .25)α if deleted = -0.612

Cronbach’s alpha of the 6 kept items

-0.270

negative: items point in opposite directions

unacceptableweakquestionableacceptablegoodexcellent?0.5.6.7.8.91◀ below zero (-0.270)

Sum of the 6 item variances: 7.83

Variance of the respondents’ summed score: 6.39

α = 6/5 × (1 − 7.83 / 6.39) = -0.270

When items agree, high answers pile up on the same respondents, so the summed scores spread out far more than the items do one by one: the blue bar outgrows the grey one and alpha rises. Items that disagree cancel out in the sum.

T3 correlates negatively with the rest of the items. Read its wording: does agreeing with it mean more trust or less?

Correlations between items, as currently scored. Blue: they rise together; red: one rises as the other falls.
T1T2T3T4T5T6
T10.44-0.350.34-0.430.04
T20.44-0.400.50-0.43-0.06
T3-0.35-0.40-0.480.380.06
T40.340.50-0.48-0.420.13
T5-0.43-0.430.38-0.420.07
T60.04-0.060.060.130.07

Before reversing, T3 and T5 show red rows: people who trust science disagree with them. Reverse them and the whole table turns blue, except one item that stays pale next to everything else.

Alpha only checks that items move together. It cannot tell you what they measure: that is decided by reading the items.

α=kk−1(1−∑isi2ssum2)\alpha = \frac{k}{k-1}\left(1 - \frac{\sum_i s_i^2}{s_{\text{sum}}^2}\right)

k is the number of items, si² the variance of item i, ssum² the variance of the summed score. Alpha runs up to 1, and it can drop below 0 when items contradict each other.

Rules of thumb used in the course

.90 or more: excellent, but suspect redundant items · .80–.89 good · .70–.79 acceptable · .60–.69 questionable · .50–.59 weak · below .50 unacceptable. An item whose corrected item-total correlation is below about .25 does not fit the others. Delete one item at a time, and never go below three.

Try

From index to categories

Step 5 is optional: a crosstab needs a categorical variable, so the 1–5 index is sometimes cut into three groups. Where you cut decides what the groups mean.

Three ways to cut the same index

The final index (mean of T1–T5, T3 and T5 reversed) for the 120 respondents. It runs from 1 to 5 in steps of 0.2. Sample mean: 3.52. Choose how to turn it into three categories.

neutral 3mean 3.5212345
CategoryRecode rangen%
1 Low trust (mostly disagrees)1.00 – 2.491714.2
2 Mixed (around the neutral 3)2.50 – 3.503730.8
3 High trust (mostly agrees)3.51 – 5.006655.0

Absolute limits come from the meaning of the scale: the middle category is "around the neutral answer 3", whatever the sample thinks. Compared with relative limits (mean ± 0.5), 45 of 120 respondents would land in a different category.

Recode ranges must touch

If one range ends at 2.10 and the next starts at 2.20, a value such as 2.15 belongs to neither and silently becomes missing. Write the limits with enough decimals (2.10 / 2.11) and always compare the number of missing cases in the index and in its categorical version: they must be equal.

SPSS

The same steps in SPSS

The reliability analysis below is calculated from the same synthetic data as the builder, with T3 and T5 already reversed.

Before you open SPSS

  1. 1Which concept?Trust in science, high value = more trust.
  2. 2Which items?t1–t6, answered on the same 1–5 agreement scale.
  3. 3Which run the other way?t3 and t5 (agreeing means less trust): reverse them into t3_r and t5_r.
  4. 4What do I need from the output?Cronbach's alpha, and per item the corrected item-total correlation and alpha if item deleted.
Transform›Recode into Different Variables

Old → new values: 1 → 5, 2 → 4, 3 → 3, 4 → 2, 5 → 1. Recode into a different variable so that the original answers stay untouched, then check that the number of missing values did not change.

Analyze›Scale›Reliability Analysis
  1. 1

    Items

    Always the reversed versions (t3_r, t5_r), never the originals.

  2. Model: Alpha

    The default model is Cronbach's alpha.

  3. Statistics… › Scale if item deleted

    Produces the Item-Total Statistics table.

  4. Click Paste to save the syntax, or OK to run.
Reliability Statistics
Cronbach's AlphaN of Items
.6906
  1. 1.690: questionable, just short of .70. Try to improve it before computing the index.
Item-Total Statistics
Scale Mean if Item DeletedScale Variance if Item DeletedCorrected Item-Total CorrelationCronbach's Alpha if Item Deleted
t117.0812.92.500.622
t217.2912.88.536.611
t3_r17.2713.26.482.629
t417.3212.15.604.584
t5_r17.1413.30.495.625
t617.5916.97-.003.782
  1. 2t6 barely correlates with the sum of the other items: well below .25.
  2. 3Deleting t6 would raise alpha from .690 to .782, the largest gain of any item.
After deleting t6 and rerunning: alpha = .782 with five items, and no further deletion raises it. Stop there.
Transform›Compute Variable

COMPUTE trust_idx = MEAN(t1, t2, t3_r, t4, t5_r).

VARIABLE LABELS trust_idx 'Trust in science index (1-5, high = more trust)'.

RECODE trust_idx (1 thru 2.49=1) (2.50 thru 3.50=2) (3.51 thru 5=3) INTO trust_3abs.

EXECUTE.

MEAN() averages the items a respondent did answer. Writing the mean out as (t1 + t2 + t3_r + t4 + t5_r) / 5 instead gives a missing index to anyone who skipped an item. Use the one your course asks for, and know which it is. Then run Frequencies on the new variables to check the result and the missing counts.

Next, the categorical version can go into a crosstab, and the 1–5 index into a t-test or correlation.

Test yourself

Mastery check

Mastery check

0/0 correct on first attempt

Interpret alpha

Five items reach Cronbach's alpha = .84. A classmate concludes: “So the index is a valid measure of political trust.” What does the .84 actually show?

1 / 7