Technical Test

Generally, SoSci Survey is a reliable platform. Nevertheless, errors can occur during the construction of the questionnaire, in the programming of filter questions, but also when software updates or new features are installed. It would be frustrating, ifsuch errors lead to the loss of data.

We recommend that each questionnaire is subjected to a technical function test before the survey is started. This test takes about 20 to 30 minutes, so hardly longer than any participant invests in the survey.

1. Delete the data that was saved during the design and the testing of the questionnaire. Click Collected Data → Delete All Data in the menu.
2. Now, print a list of all variables in the project. Within Variables Listing, click on Print preview. A new window will open – print this information.
3. Start the questionnaire by clicking on the green arrow in Compose Questionnaire. In case you have created multiple questionnaires, carry out the following steps with each of them.
4. Complete the questionnaire twice giving different replies. It is best to use two different web browsers (e.g. Firefox, Internet Explorer, Opera, Safari, Chrome), since the questionnaire may be displayed differently. In case you are using filters, complete the questionnaire more often to test all paths of the filters.
   • Ensure in advance that the test runs are also counted. To do this, set the appropriate check marks under Collected data → Selection criteria for valid cases.
   • Take a note of all data you enter into the questionnaire on the printed variables listing. It is best to note the number that is saved for your statement. The variables listing also contains the number being saved.
   • Make sure all questions are properly displayed – if not, often a quick change of settings directly within the question solves the problem.
   • When using filters, give different statements when completing the questionnaire and check the filter paths.
   • When using randomisations, check whether the results vary correctly.
5. Click on Collected Data → Download Data and download and open the file with the software you intend to use for data analysis. Thus, you can check whether the import works as expected.
   • You can find detailed instructions under Erhobene Daten und Dokumentation.
   • If the survey period entered in the project settings has already started, set the Collected data to → Selection criteria for valid cases another check mark at Interview context (MODE) → questionnaire development (Problem Solutions for Data Retrieval). If the survey period entered in the project settings has already started, set the Collected data to
6. Now you can easily compare the data with your notes on the printed variables listing.
   • Check whether the values noted on your list are the same as those saved in the downloaded data set. The variables in the data set are ordered in the same way as in the variables list, however, the data set (depending on your setting when downloading the data) only contains those variables that are asked for in the questionnaire.
   • Check whether all relevant variables are present in the data set. Questions that are not prompted are not displayed in the questionnaire. Maybe a question was forgotten or a filter skipped a question, although this was not intentionally. After completing the questionnaires you can easily check which variables are active in the variables listing – those not active are grey.

Once you have done this technical test, you can be sure

• that all questions appear in the questionnaire
• that all statements are being saved correctly
• that the import of data for the analysis will work as expected.

However, your participants may not properly understand all questions the way you meant them. To ensure a valid survey, a Pretest of the Questionnaire is necessary.