Questionnaires for Smartphones

The proportion of participants who fill out a questionnaire on a mobile phone varies greatly, depending on the study. In most studies, however, you have to assume that a relevant proportion of respondents will fill out the questionnaire on a small display. In practice, the following constellations arise.

• The questionnaire shall work everywhere
  Often, participants shall simply be able to fill out the questionnaire – regardless of whether they are currently using a laptop, a tablet or a smartphone.
• Questionnaire for mobile devices only
  Some study designs, such as diary studies or mobile experience sampling, are designed to be filled out on the mobile device only.
• Questionnaire for computers only
  Other study designs are unsuitable for smartphones, e.g. when a task requires a lot of information to be displayed on the screen at the same time or when reaction times shall be recorded by key press. In this case, you want to make sure that the questionnaire is not filled out on a smartphone.

Tip: You can also test the display for mobile devices on your computer. To do this, look for the developer tools in your browser (Firefox, Chrome) (Developer Tools in the Browser). There you will find a switch to the smartphone simulator.

Make sure that you select the “dynamic” display for your questions. This automatically adapts the display to the size of the screen.

Note: Not every questionnaire is suitable for being filled out on the small display of a smartphone. Long scale batteries (more than 10 items) are just as little fun on the small display as long question texts/instructions or long stimulus texts. Not to mention that you might happily fill out a 5-minute questionnaire on the bus, but probably not a questionnaire that takes 30 minutes. Therefore, regardless of the display, scientific questionnaires are often not suitable for the small mobile phone display.

The proportion varies between 0 and 100 percent depending on the recruitment – but on average, every second person (54%, as of Q1/2024) uses a smartphone when calling a questionnaire on www.soscisurvey.de. This does not include tablets, which are unproblematic: They offer sufficiently large screens, and all question types of SoSci Survey can be answered by touch (touch display) just as well as with the mouse. With smartphones, on the other hand, the small display can become a problem.

Basically, any common smartphone browser can display a website so small that it fits completely on the display. However, you can then neither read the texts nor hit the buttons for the answers. The term “responsive layout” covers various techniques that ensure that a website can be displayed sensibly on displays of different sizes. Essentially, unimportant elements are hidden on small displays and the remaining elements are shown in a different arrangement (e.g. one below the other instead of next to each other).

The standard layouts of SoSci Survey (with the exception of “SoSci Survey Paper”) are designed to automatically adapt to the available screen width. In the Questionnaire Layout, a maximum width is set so that the questionnaire is not stretched too much on large screens. For small displays, however, there are two major limitations:

• In the questionnaire, there are usually no unimportant elements (questions, answer options) that could be omitted.
• Not every question type can cope with arbitrarily little space.

On the small screens of smartphones, questions that use a lot of text (e.g. long item texts) or offer many options cause problems in particular. In addition, some question types such as the ranking question rely on displaying items and answer options as cards next to each other at the same time.

For the same question to be displayed sensibly on displays of different sizes, the display may have to change significantly. This is most obvious with a fully labeled scale in the display variant “dynamic” (default). Here, SoSci Survey varies between a matrix display with small selection fields on the large PC screen, large selection fields for the tablet (and other small screens) and vertically sorted selection fields on the smartphone in portrait format.

Systematic differences in response behavior are not to be expected due to the different display, because with a reliable scale, the display should not have a significant influence on response behavior. If the different display is nevertheless undesirable, you can select the “static” display in the question. However, not all answer options may then be displayed on the smartphone – therefore, we explicitly advise against this.

To ensure that the questionnaire can be filled out sensibly on smartphones, we recommend the following optimizations:

• Use short texts.
  • The welcome should not comprise more than 4 sentences plus a closing formula.
  • Explain how to fill out questions (e.g. a Likert scale) only once, when the question appears in the questionnaire for the first time.
  • Use the accordion function and the function for displaying shortened texts (Partially hide a text) so that longer texts do not impair the overview too much.
• Avoid Combined Questions – especially if you want to compress more questions into less space. If the questionnaire becomes too long without combined questions, question once more whether the questionnaire is suitable for smartphones.
• For scale questions, set the Arrangement to “dynamic” or “automatic” (default setting).
• For selection questions, use “dynamic” (default) or, even better, “cards” as Display. Cards take up more space, but are much easier to hit with the finger on a small display.
• The question type “Selection Sequence” is particularly easy to use on touch devices, because here, too, the answer options are very easy to hit. In addition, there is no extra click on “Next”.
• If you use audio or video files, embed them using HTML 5 (Media Files in Questionnaires (Audio, Video, Documents)) and make sure that all video formats are converted so that they can also be played on mobile devices (pixel size, data rate, etc.).
• Do not insert manual line breaks into items and texts, but use dollar signs ($$) to define where the text shall not break (Texts in the Questionnaire: Wiki Notation).

With the question type Device and Transmitted Variables, you can find out, among other things, whether a participant fills out the questionnaire using a smartphone:

• Create a question of the type Device and Transmitted Variables.
• In the Device tab, check Form factor.
• Drag the question onto the page where you want to filter – or directly onto the first page of the questionnaire.

This question stores whether the participant's browser identifies its device as a computer, tablet or smartphone. If, for example, the question has the ID “GG01”, the following PHP code shows smartphone users the question “FM01” and all other users the question “FS01”.

if (value('GG01_FmF') == 5) {
  question('FM01');
} else {
  question('FS01');
}

The following PHP code only displays question “FS02” if the participant does not use a smartphone. Detailed explanations of filters can be found under Filters and Conditional Questions.

if (value('GG01_FmF') != 5) {
  question('FS02');
}

Tip: The question type Device and Transmitted Variables can also capture the user's screen size. For some filters, this information can be more useful than the device format. However, the question must then be placed on the page before the filter at the latest (not on the same page as the filter).

if (value('GG01_ScW') < 1024) {
  question('FS02');
  question('FS03');
} else {
  question('FS02', 'combine=FS03');
}

It is possible to present smartphone users with a completely different questionnaire than other respondents. This can be useful, e.g., if different content shall be shown and/or if the questionnaire for smartphone users shall use a different layout.

Note: If you want to ask the respondents in the mobile version the same questions as in the PC version, this solution is not recommended.

• Under Compose Questionnaire, in the tab Listing of Questionnaires, create a second questionnaire or a copy of the questionnaire.
• Optionally, set the ID “mobile” in the new questionnaire in the Settings tab. Which ID you specify is up to you; you should just clearly recognize the mobile version. Save the changes.
• Switch to the original questionnaire and, under Settings, select the questionnaire you have just created for Version for mobile phones. Save the change.

Smartphone users are now automatically redirected to the new questionnaire when they call the main questionnaire.

Important: Changes to a question are applied in all questionnaires that use the question. If you want to modify a question for the display on the smartphone/in the second questionnaire, create a copy of this question () and replace the old question with the new one under Compose Questionnaire in the second questionnaire.

Note: If a filter is provided for the original question, an additional filter must be defined for the mobile copy of the question.

If the questionnaire shall be filled out exclusively on smartphones, the same notes on optimization as above apply.

For scales, there is a special display for narrow screens (to be selected in the first tab). In addition, the question type “Scale (with intermediate values)” can display the scale points as a dropdown – this can be clearer, especially with long answer options.

Depending on the study design, the following chapters may also be helpful:

• Multi-Wave Surveys: Diary Study
• Multi-Wave Surveys: Mobile Experience Sampling Method (MESM)

Sometimes, it is necessary to reject participants who want to access the questionnaire with a desktop computer or laptop. This is done similarly as explained above (Filter for Smartphone Users).

• You need a functional building block of the type “Device and Transmitted Variables” that determines the Form factor of the device and is placed before the filter in the questionnaire (see above).
• In addition, a text is created (Texts in the Questionnaire) which has, e.g., the ID “GG02” and politely asks the participant to fill out the questionnaire on a mobile device.

The following code example assumes that the building block “Device and Transmitted Variables” has the ID “GG01” and the text has the ID “GG02”. The following PHP code is placed as the first element on the first page of the questionnaire. It already includes “GG01” using question(), so that the building block “GG01” does not have to be placed separately in the questionnaire.

question('GG01'); // Include the functional building block GG01 to determine the device format
if ((value('GG01_FmF') == 1) or (value('GG01_FmF') == 2)) {
  text('GG02');   // Show text
  buttonHide();   // Hide buttons so that the questionnaire cannot be filled out further
}

Note: This PHP code allows filling out on a tablet. To prohibit this as well, add a check or (value('GG01_FmF') == 3) in the IF filter.

Note: The PHP code is intended to prevent accidentally filling out the questionnaire on the wrong device. By deliberate manipulation, participants may nevertheless be able to fill out the questionnaire on a device that should actually not be allowed: Some browsers (e.g. Chrome) can emulate a mobile device, other browsers can transmit a fake browser ID.

Some content is not suitable for smartphones or even for tablet computers. This may be, for example…

• Complex combined questions that query detailed information in a compact way
• Ranking questions that require a lot of space, or questions that shall measure an implicit attitude based on reaction times, e.g. an IAT (both are in principle also possible on smartphones, but may not be desirable for methodological reasons)
• Tests that require a keyboard and/or mouse as input device
• Videos that shall be watched in a quiet environment (not on a public transport bus) on a computer with speakers
• Images of websites as they look on a normal computer, whose contents would not be recognizable on a small smartphone display

With a filter, you can kindly inform participants who use a mobile device that it is not suitable for filling out the questionnaire. The procedure is equivalent to rejecting participants in smartphone studies (see above).

• Create a question/building block “Device and Transmitted Variables” (e.g. “GG01”)
• Create a friendly text (e.g. “GG02”)
• At the beginning of the questionnaire, use a filter to show the text to participants with a mobile device

question('GG01'); // Include the functional building block GG01 to determine the device format
if ((value('GG01_FmF') == 4) or (value('GG01_FmF') == 5)) {
  text('GG02');   // Show text
  buttonHide();   // Hide buttons so that the questionnaire cannot be filled out further
}

Use the Variables Overview to determine the possible values of the device format for the filter. Code 4 describes a mobile device whose format cannot be determined exactly, code 5 describes a smartphone.