Implicit association test (IAT)

The implicit association test (IAT) according to Greenwald et al. (1998) is used to measure the cognitive association of two dimensions (e.g., Democrats/Republicans and good/bad). For this purpose, the participant is presented with words and/or pictures in 7 test blocks, which the participant is asked to correctly assign to a category by pressing a key. Reaction times are used to measure the strength and direction of the association.

Important: The IAT is not part of the standard scope of SoSci Survey. The test must be booked separately and for a fee as the “Implicit Methods” module.

Note: If the D-scores are missing when downloading the data in SPSS, please set a different decimal separator when downloading (Problem Solutions for Data Retrieval → Missing decimal scores in SPSS).

In the question catalog, first create a New question of the type “Implicit Association Test (IAT)” within any category.

The two dimensions are spanned by two categories each. For each category, up to 8 words (left) or images (right, file name only) can be entered – in addition to the name.

• Note: If you use images, please enter only the file name in the column Image (optional), not the HTML code to include an image.
• Note: Please upload images for the IAT “normally” and not into the “protected file repository” (recognizable by the fact that the image address starts with pro://). With individual browsers it can otherwise happen that images are not displayed, which should be shown later in the IAT.

The texts that are displayed as instructions at the beginning and between the test blocks can be adjusted if necessary. A number of text input fields are provided for this purpose in the “Explanations” tab. The respective standard text is always visible below the input field (incl. HTML tags) and can simply be copied into the input field for customization.

After defining the categories and the corresponding stimuli (items), the test – like any other question – can simply be dragged onto a page of the questionnaire during Compose Questionnaire. It makes sense not to place any other questions on this page. If the corresponding page

The measurement of the reaction times takes place with an accuracy in the range of milliseconds. To ensure that this measurement is not distorted by the data transmission on the Internet, the question loads all necessary images and content in advance.

The measurement of the reaction time starts with the presentation of a stimulus and ends with the correct keystroke of the participant. In case of an incorrect keystroke – according to the conception of the IAT – a negative feedback (red cross) is displayed and the time continues to run until the correct keystroke.

After the correct keystroke, the stimulus fades out. This is followed by a pause of 250 ms before the next stimulus is presented. This pause serves to clearly delineate the stimuli.

According to the IAT procedure, categories A to D are combined in the 7 blocks as follows (the blocks at BIAT and SC-IAT differ from this scheme):

In the evaluation according to Greenwald et al. (2003) improved (recommended value, see below), the reaction times from exercise blocks 3 and 6 are also used for the calculation.

The order of the items (stimuli) within the test blocks is varied randomly. For the longer test blocks, the items are displayed multiple times. Randomization is performed according to the following rule set:

• An equal number of items are shown from all categories active in the block.
• Each stimulus within a category is shown the same number of times (if the number of repetitions for a category is not divisible by the number of items, the frequency of display is made as equal as possible, i.e., the difference between items is no more than 1 presentation)
• Blocks with categories from both dimensions always alternate between the two dimensions.
• Direct repetitions of the same stimulus are avoided.

The IAT question stores (in addition to the raw data) an index value for the strength of association (D-score). This is calculated according to the literature cited below.

The response times of blocks 3, 4, 6 and 7 are used for the evaluation. The cleanup is based on all response times of these blocks:

• How often was the response time faster than 300ms? If this was the case for more than 10% of the responses, the test is invalid.
• Did the participant need more than 10 seconds for a single answer? Then this answer is removed from the evaluation.

The evaluation is done in the following steps.

1. After cleaning, the standard deviation of all response times in blocks 3 and 6 (together) and 4 and 7 (together) is calculated.
2. If the participant answered incorrectly, the mean response time within the block plus 600ms is used for this response. This value replaces the actual response time for the further calculations (not for the standard deviation).
3. Now the mean response time for each of the test blocks is calculated.
4. The difference of the mean response time from blocks 6 and 3, and from blocks 7 and 4 is calculated.
5. These differences are divided (for individual standardization) by the standard deviations calculated above.
6. The mean of the two quotients is the index value (strength of implicit association).

In principle, the range of values is not limited. The results can be smaller than -1 and larger than +1 if the response times in blocks 3 and 6 or blocks 4 and 7 differed very much.

The resulting value is positive if the association between categories A and C or B and D is stronger (lower response times in block 3/4) than the association A-D or B-C (higher response times in block 6/7).

The response times of blocks 4 and 7 are used for the evaluation.

1. Response times over 10 seconds are removed from the data
2. Response times <300ms are replaced by 300ms.
3. Response times >3000ms are replaced by 3000ms.
4. The mean value per block is calculated based on the logarithmized reaction times, the divisor is the number of trails minus 2.
5. The text value is the difference of the mean values.
6. If a person answers incorrectly in more than 25% of the trials (in blocks 4 and 7), the trial is considered invalid.
7. If there are less than 3 reaction times per block after cleaning, the trial is considered invalid.

The “improved” D-scores determined according to Greenwald et al. (2003) are interval-scaled (metric) variables that predominantly (not exclusively) lie in the range between -2 and +2. They are usually treated in correlations and regressions in the same way as other metric variables. Before evaluation, check for extreme outliers that may need to be removed from the analysis.

The calculation of the “improved” score is based on the training blocks and the actual test blocks. To estimate reliability, the separate D-scores for the training blocks and the test blocks are correlated. In SoSci Survey, these are the variables “partial score based on training blocks 3 and 6 (D-score)” and “partial score based on test blocks 4 and 7 (D-score)”. The estimation of reliability follows the idea of the split-half procedure as used for scale batteries.

The IAT module of SoSci Survey automatically evaluates the measured data and provides a D-score according to the common literature (see above). If the standard evaluation is not sufficient, SoSci Survey also provides the raw data of the measurement.

The raw data of the IAT include for each block the information which item was presented, whether the item was assigned to the correct category in the first attempt, and how long the participant needed for the assignment [ms]. This is not classical tabular data, because the items are presented in random order by default.

In the variables …b01i to …b07i the information is stored in a format like the competition outputs, e.g.

  ItemA 1 1183 ItemB 0 581 ItemC 1 774 ItemD 1 565 ItemE 1 565 
  

Here always an identifier for the item is given, the correct assignment (0=false, 1=correct) and the reaction time in milliseconds [ms].

For the evaluation, however, usually only the reaction times and the correct assignment are required. But in a more manageable format. The IAT module therefore additionally stores the raw data as JSON, e.g.

  [
    [
      [1183,1],[581,0],[774,1],[565,1],[565,1]
    ],[
      [1600,1],[629,1],[551,1],[677,1],[1317,1]
    ],
    ...
  ]
  

In the dataset, the rows are not so nicely indented, but the data is the same: The outermost array (`[]`) contains 7 elements corresponding to the 7 blocks. Each block is represented by another array (`[]`), which in turn contains as many arrays as trials. So in the above data e.g. 5 trials in blocks 1 and 2. Within each trial the reaction time [ms] and the correct assignment (0=false, 1=correct) is coded.

This data can be read in with any software that supports JSON, e.g. GNU R. Since reading in and processing the arrays of different lengths (i.e. no classical matrices) in R is not quite trivial, we have provided here an example R script that performs the evaluation of an SC-IAT based on the JSON data: Import and evaluation for the SC-IAT

Note: Handling the raw data is only necessary if you want to deviate from the default evaluation. Normally SoSci Survey does the evaluation automatically.

Via JavaScript some modifications to the IAT are possible. For this purpose, an HTML element with the corresponding JavaScript code is placed under the question. The identifier `iatAB01` in the following examples must be adapted to the identifier of the IAT.

To ensure a high degree of comparability, the order of the blocks in the IAT is fixed, e.g. block 3 always shows categories A and C together on the left side (blocks).

If a random rotation of e.g. the pages is desired here…

• … create a copy of the IAT question (if you select the question in the question catalog, a bar with several icons appears above the question (on the right), one of them for “duplicate question”).
• In the copy, swap the contents of the categories (e.g., the words of the positive and negative evaluation dimensions).
• Finally, follow the instructions Randomization to randomly use one of them (or more variations, if necessary) in the questionnaire.

This procedure ensures that the available variations are precisely defined. In addition, the sequence used can be precisely traced in the dataset.

The default IAT includes 7 blocks with 20, 20, 20, 40, 20, 20 and 40 trials per block. The following JavaScript code increases the number of trials.

<script type="text/javascript">
<!--
SoSciTools.attachEvent(window, "load", function() {
  SoSciTools.questionnaire.AB01.setTrials([24,24,48,48,35,48,48]);
});
// -->
</script>

Please note the square brackets inside the round brackets. These define an array. This array must have exactly 7 elements, corresponding to the number of trials.

For each presented stimulus (trial), the IAT generates a JavaScript event 'present' that contains information about the block, position in the block, and stimulus.

The following example shows how the event can be used to randomly flip presented images. The identifier of the IAT question in this case is “IA01”.

<script type="text/javascript">
 
function onTrial(evt) {
    var node = evt.detail.stimulusNode;
    // Only images, text stimuli are ignored
    if (node.tagName != "IMG") {
        return;
    }
    // Only 50% of the images are mirrored
    if (Math.random() < 0.5) {
        node.style.transform = "scaleX(-1)";
    }
}
 
window.addEventListener("load", function() {
    var iatNode = SoSciTools.questionnaire.IA01.node;
    iatNode.addEventListener("present", onTrial);
});
</script>

To test IAT questions, you can switch them to a test mode so that only a fraction of the trials are used:

<script>
window.addEventListener("load", function() {
    s2.IA01.setTesting();
});
</script>

The identifier IA01 must of course be replaced here by the identifier of the IAT question.

In addition to the classic IAT according to Greenwald et al. (1998), the “Implicit Methods” module also contains a Single Category IAT (SC-IAT according to Karpinski & Steinman, 2006) and the Brief IAT (BIAT according to Sriram & Greenwald, 2009).

In the SC-IAT, not two opposing concepts are contrasted with an evaluative dimension, but only one. The participant is then asked to categorize, for example, whether “monkey” belongs to the dimension “animal” or “positive” or not.

The implementation of the SC-IAT follows Karpinski and Steinman (2006), a study on reliability is presented by Blümke and Friese (2008).

In the question, terms (stimuli) for the evaluative dimension are already entered in the German and English versions. These terms are taken from Karpinski and Steinman (2006, p. 32) or are a literal translation of these terms into German.

The BIAT reduces the number of blocks and trials to shorten the test duration. Also, participants are presented with a slightly different set of tasks. The procedure is implemented according to Sriram and Greenwald (2009).

Blümke, M. & Friese, M. (2008). Reliability and validity of the Single-Target IAT (ST-IAT): Assessing automatic affect towards multiple attitude objects. European Journal of Social Psychology, 38, 977–997.

Greenwald, A. G., McGhee, D. E. & Schwartz, J. L. K. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74(6), 1464-1480. Available online

Greenwald, A. G., Nosek, B. A. & Banaji, M. R. (2003). Understanding and using the Implicit Association Test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85(2), 197–216. DOI: 10.1037/0022-3514.85.2.197

Karpinski, A. & Steinman, R. B. (2006). The Single Category Implicit Association Test as a Measure of Implicit Social Cognition. Journal of Personality and Social Psychology, 91(1), 16–32.

Lane, K. A., Banaji, M. R., Nosek, B. A. & Greenwald, A. G. (2007). Understanding and Using the Implicit Association Test: IV. In B. Wittenbrink & N. Schwarz. Implicit Measures of Attitudes, S. 59-102. Google Book Preview

Nosek, B. A., Greenwald, A. G., & Banaji, M. R. (2005). Understanding and Using the Implicit Association Test: II. Method Variables and Construct Validity. Personality and Social Psychology Bulletin, 31(2), 166-174. DOI: 10.1177/0146167204271418

Sriram, N. & Greenwald, A. G. (2009). The Brief Implicit Association Test. Experimental Psychology, 56(4), 283-94.