Define your target population and prepare your participant list

Your target population is the group of individuals on campus for whom your survey is intended. It could be undergraduate students, graduate students, faculty members, support staff, or a combination of these groups. The group selected could also be more restricted depending on your survey goals (e.g., first‑year students only, students currently registered in a particular faculty, etc.).

Given the observed decline in response rates because of survey fatigue, we strongly advise that you use a sample rather than invite all of the target population to participate, especially if the target population is students. This way, you will be helping to reduce solicitation on campus, which is in everyone’s interest. In any case, asking the entire population to respond will not improve your response rate because the rate is based on the number of people invited to participate. However, it may be appropriate to invite all members of a particular group to participate if the group is small (e.g., first‑year students enrolled in a specific faculty). 

Your sample needs to be representative of the target population if you want to be able to generalize the results. This means that the characteristics likely to influence the results—such as gender, language, status (full‑time, part‑time)—must reflect those of the target population (e.g., 60% females, 40% males). 

The date you choose to launch your Web survey is also likely to significantly affect your response rate. Needless to say, you should avoid the end of an academic session and exam periods. We ask that you consult the Institutional Research and Planning’s (RIP) Survey Schedule when choosing your dates. To avoid any unpleasant surprises, make sure you allow enough time to prepare your material, obtain the necessary translations and test the Web‑based survey and the mechanism for sending your invitations.

To avoid any unpleasant surprises, make sure you allow enough time to prepare your material, obtain the necessary translations and test the Web‑based survey and the mechanism for sending your invitations.

You should also decide how many invitation messages will be sent. Reminders sent at intervals of three to seven days are generally effective in increasing response rate. The general norm is to send no more than three participation messages in total. It is important to send reminders only to those people who have not yet completed the questionnaire. 

Your invitation‑to‑participate messages need to be short and convincing. Briefly explain the purpose of your survey, how to participate, what incentives are offered (if any), the deadline for responding and who the contact person is for any questions. As with the questionnaire, check the message’s language quality and allow time for editing and translation.  If necessary, you may contact the University’s Language Services to find out what resources are available and the cost for various services.

The invitation should also contain an URL address where participants can access your questionnaire, along with an identification number or password that they must use. If possible, make it easier for participants by using personalized URLs so they can access the questionnaire directly. A unique identification number for each participant will be inserted in the URL included in the invitation. This will avoid problems arising from the incorrect copying of the identification number and/or password.

Be careful about what you use as the subject line of your invitation. If you use expressions like “win a prize,” “help us” or “take part in a draw,” your message could be blocked by the anti‑spam filters. We suggest that you run some tests before launching the survey. If emails are not sent from an internal University of Ottawa workstation, we also strongly recommend notifying the Service Desk at 613-562-5800 extension 5000 a few days before sending the invitations. Please ensure that the ticket opened for your request is assigned to the “INF-Collaboration” team.

Some survey programs allow you to insert in your invitation message a link that potential participants can click on to remove their name from your participant list (“OptOut” function). This will prevent reminder notices from being sent to people who are not interested in participating, thus avoiding frustration.

Here are a few messaging elements that are likely to encourage participation:

• Signature: Response rates are generally higher when the people invited to participate are familiar with the person signing the invitations. It could be an individual or service.

• Length: Indicating how long the questionnaire is likely to take has a positive impact if the questionnaire is relatively short. However, both ethically and strategically, it is not recommended to deliberately underestimate the response time required.

• Confidentiality statement: Reassure participants by indicating that responses will be kept confidential (this implies that no identifying information will be disclosed with the participant’s responses). However, avoid raising concerns by using an overly detailed confidentiality 
  statement. 

• Draw: It is quite common to offer incentives in the form of a random draw as a means of thanking participants and increasing response rates. A number of studies have examined the effectiveness of this strategy. The findings are generally disappointing (Porter and Whitcomb, 2004; Warriner et al., 1996). Furthermore, the relationship between the value of incentives and the response rate is not linear; in other words, higher values do not necessarily produce higher response rates. Consequently, any prizes offered should be selected on the basis of their potential appeal to the target population and your budget.

• Benefits: You can also encourage your potential participants to respond by explaining why their involvement in the survey is important, what the benefits will be to them, or how the results will be used. 

• Personal greetings: There are a number of software programs that allow you to use personalized greeting formulas (e.g., “hello [first name]”). Given the vast volume of personalized spam messages, the effectiveness of this measure in increasing response rates is somewhat uncertain, but you have nothing to lose by using it.

Select the software or supplier

The Survey Monkey tool is available to the University of Ottawa staff members (academic and support staff). For more details, contact Service Desk by phone at extension 5000.

You may also decide to use a different software or an external supplier, while assuming the related costs. If you decide to use an external supplier, make sure that you have the supplier sign a confidentiality agreement before forwarding any files containing personal information, such as names and electronic addresses. IRP may provide you with a confidentiality agreement template to be updated by modifying the text in yellow using the information specific to your survey initiative. After updating the agreement, please send it to the Office of the Vice President Governance at  [email protected]  to have it signed by the Vice-President and the President. Only the Vice-President, Governance and the President are authorized to sign legal contracts in the name of the University.  

Tips on creating your Web survey

Here are some tips on creating your Web‑based survey that will help you make it easier for participants to respond and thus increase your chances of receiving a greater number of completed questionnaires:

• Restrict access to your survey to people on your participant list. The goal is to obtain a response rate from a representative sample and not to get the largest possible number of responses. 

• It is also recommended that participants not be required to answer one question before moving on to the next one (Umbach, 2004). This could mean that you will have some missing information but forcing participants to answer will not solve the problem because they may be just as likely to close the questionnaire. From an ethical perspective, participation should be voluntary, which means participants have the right not to answer all questions or to leave the survey when they wish. 

• Use a simple, easy‑to‑read format similar to a paper questionnaire, without too many colours or images.

• Provide clear, detailed instructions that make navigation easy (e.g., how to move to the next page, how to submit the questionnaire, etc.) and explain how to answer the questions (e.g., “Choose only the most important reason” vs. “Choose all applicable reasons”). 

• Wherever possible, participants should be able to see the next question or to access navigation buttons without having to scroll down the page.

• Avoid drop‑down menus and use multiple-choice questions instead; this way all respondents see all the options available without first having to click on an arrow. 

• To keep respondents motivated, add a progress bar to your questionnaire. This could have a negative impact however if the questions requiring the greatest response time (open‑ended questions, matrices with many statements, etc.) are placed at the beginning of the survey.

• Run some tests to see how your questionnaire looks on different operating systems, screens or browsers. Trial runs are also useful to ensure that any skip patterns in the survey are working.