Once your PhD thesis proposal is approved, the next step involves actual data collection. Depending on the study, a student may choose to collect secondary data, primary data or both.
A questionnaire is used to collect primary data. It is a set of questions that a researcher asks his/her research respondents so as to understand the problem under investigation.
A questionnaire can be structured, unstructured or semi-structured. The differences among these three are explained below:
Has closed-ended questions.
Example:
Q1. Has covid-19 affected your business?
- Yes 2. No
Has open-ended questions.
Example:
Q1. What is your opinion about the covid-19 vaccine?
_____________________________________________________________
Has a mix of both closed-ended and open-ended questions.
Example:
Q1. Has covid-19 affected your business? 1. Yes 2. No
Please explain _____________________________________________________
This guide discusses the best practices for designing and administering a questionnaire.
- Designing a questionnaire
- Introduction
- Questions
- Conclusion
- Administering a questionnaire
- Facilitated questionnaires
- Self-administered questionnaires
- Training research assistants
- Pre-testing a questionnaire
- Protocols for administering questionnaire
- Recommended resources
Designing a questionnaire
The design of a questionnaire is a key determining factor of how it will be received by the potential respondents.
A questionnaire has three distinct parts: the introduction, the questions, and the conclusion.
Introduction
In the introduction section of a questionnaire, the researcher outlines the following: who is carrying out the study, the purpose of the study, what information will be collected from the respondent, the process used to identify the respondent, use (and permission for use) of audio recording, the average amount of time required to fill the questionnaire, how the data collected will be used, the risks or inconveniences of participating in the study, the benefits of participating in the study, and any compensation plans for participants.
Additionally, the introduction section also includes ethical issues related to participating in the study such as protection of personal privacy of the respondents, upholding the confidentiality of the data collected, voluntary nature of participation in the study, and informed consent of the respondents. The researcher should also include contact details in case the respondents would want to reach out to them in future.
Below is a sample of introduction to a questionnaire.
Questions
The design and structure of the questions in a questionnaire matter greatly. The following factors should be considered when designing the questions:
Flow of questions: the questions should flow in a logical manner. It is therefore important to organise the questions by sub-topics.
In the example of effects of covid-19 on micro and small enterprises, the student can organise the questionnaire by the following sub-topics: revenue, customers, employment etc.
Language used: the language should be simple and clear. Do not use jargons that a layman cannot understand. Not all respondents have high level of education, and even if they did, they may not be experts in your field of study.
Types of questions: there are different types of questions that can be used. Make sure to mix them up to make the questionnaire more appealing.
The following table lists the different types of questions you can include in a questionnaire:
These are questions on the socio-economic background of the respondent.
They are best used to disaggregate the data or to check for differences between different socio-economic groups.
They are best placed at the beginning of the questionnaire, after the introduction.
Examples include:
Q1. What is your age? ___________
Q2. What is your highest level of education? 1. No education 2. Primary 3. Secondary 4. Tertiary
Q3. What is your marital status? 1. Single 2. Married 3. Divorced 4. Separated
These are questions whose responses fall into two groups only.
Examples:
Q1. Do you have children? 1. Yes 2. No
Q2. Are you employed? 1. Yes 2. No
Q3. What is your gender*? 1. Male 2. Female
- It is important to be careful about some questions such as gender because some people do not identify with either of the options provided.
Unlike the two-option response questions, these questions provide many response options.
The response options should be as exhaustive as possible.
Examples:
Q1. For your last birth, who assisted you to deliver? 1. Traditional birth attendant 2. Midwife 3. Nurse 4. Doctor 5. Other (specify) _____________
Q2. What is your main source of water for your household? 1. Piped water 2. Borehole water 3. Rain water 4. Surface water (river, lake) 5. Other (specify) _______________
Always include “other” and/or “don’t know” responses for the multiple-choice questions.
A rating scale question asks respondents to rate a product or service on a scale. The scale can be 0-10, 0-100 etc.
Example:
Q1. On a scale of 0-10 (where 0 is extremely dissatisfied and 10 is extremely satisfied), what is your level of satisfaction with the services provided by the Water Services Board?
Likert scale questions are used to gauge the opinions and feelings of respondents on a particular issue.
Examples:
Q1. I am satisfied with the measures the county government has put in place to cushion micro and small enterprises from negative effects of covid-19 pandemic.
1. Strongly disagree 2. Disagree 3. Neutral 4. Agree 5. Strongly agree
Matrix questions are used when several questions require similar response type. For instance if you have multiple rating scale or likert scale questions with the same response types, you can put the questions in a table.
The questions can be on the first column and the responses on the first row. The respondent then selects the response that applies to each of the questions in the table.
Include “skip logic” where applicable: skip logic instructs respondents to skip certain questions based on their responses to previous questions.
For example: if question 2 is “Do you have children?” and questions 3 and 4 are questions about the children (e.g where they were delivered, if they have been vaccinated etc), then those who responded “No” to question 2 should be asked to skip to question 5 because question 3 and 4 do not apply to them.
Recall questions
Some questionnaires include questions which respondents to remember an event that happened in the past. If the recall period is long (longer than one year), the respondent is highly likely to give false responses. In such situations, the respondents can use aids to help them remember the actual details of the event.
For example: a question about the types of vaccinations a child below the age of 5 has received would be easier on a mother whose child is less than one year. If the child is say 4 years old, it may be difficult for the mother to remember all the vaccinations the child received and when they were administered. In such a situation, the mother may refer to her child’s vaccination schedule handbook which lists all the vaccines the child got and when they got them.
Length of the questionnaire
There needs to be a balance when it comes to the length of the questionnaire.
A very short questionnaire may increase response rate but may fail to achieve all the study’s objectives as it may leave out some key questions.
On the other hand, a very long questionnaire may lead to high response rate due to fatigue of the respondent (and the interviewer) even though it may include all the necessary questions needed to achieve the study’s objectives.
Conclusion
After you are done asking the respondent questions, it is important to give them the opportunity to ask any questions they may have.
Additionally, thank them for the time they spent answering your questions/filling in the questionnaire.
Restate how the data will be used and how they would be able to get the results of the study should they be interested.
Administering a questionnaire
There are two methods of administering questionnaires: facilitated questionnaires and self-administered questionnaires.
Facilitated questionnaires
In facilitated questionnaires, the researcher (or a trained research assistant) administers the questionnaire directly to the respondent. This can be done either through a face-to-face interview or telephone interview.
If a researcher plans to use research assistants to help with the questionnaire administration, he must train them to ensure they are on the same level of understanding.
Facilitated questionnaires have advantages and disadvantages:
- They result in higher response rate compared to self-administered questionnaires.
- The researcher is present to clarify the questions if need be.
- The respondents do not need to have high literacy levels.
- They are costly (especially the face-to-face method of administration). The costs may include: travel costs to meet the respondents, personnel costs if using research assistants, accommodation costs if the interview is taking place far from the interviewers’ area of residence).
- They are more time-consuming compared to self-administered questionnaires.
- They are prone to interviewer bias. The interviewer can influence the responses given through questioning the respondent on the response given, prompting responses from the respondent, interpreting the questions for the respondent etc. To minimise bias, the interviewer (and research assistants) need to be trained on effective interviewing skills.
- They do not offer privacy to the respondents and may be uncomfortable for sensitive questions.
Self-administered questionnaires
In self-administered questionnaires, the respondent fills in the questionnaire without the presence of the researcher.
There are different ways of delivering self-administered questionnaires: through post office, email address, mobile phone, or web-based.
Questionnaires sent through the post office should include a stamped envelop that the respondent can use to mail back to the researcher. This ensures that the respondent does not incur costs for mailing and is one way of increasing the response rate.
Self-administered questionnaires should be designed taking into consideration the layout of the questionnaire (font type used, font size, order of questions, simple instructions and skip logics). The layout of the questionnaire should make it easy for the respondents to fill in the questionnaire.
Self-administered questionnaires also have their advantages and disadvantages:
They are less costly.
They can be distributed to a wider reach in a short time.
They are not prone to researcher bias because the respondent fills the questionnaire without any assistance of the researcher.
They are the best option for sensitive questions because of the privacy they offer.
They require the respondents to have high literacy levels.
They lead to low response rates.
Web-based and mobile phone-based questionnaires require respondents to have high internet connectivity.
Training research assistants
In some cases, the researcher may not be able to carry out the data collection all by himself. This may be the case if the sample size is large and the time is limited.
The researcher can enlist the services of research assistants to help with the data collection. The research assistants should be trained on the questionnaire before the data collection exercise starts.
When training the research assistants:
Set a date and time when all the research assistants will be available and train them at the same time.
Explain to the research assistants the purpose of the study, the importance of the data being collected and how it will be used, who the respondents are and how they were selected etc.
Be clear to the research assistants on what is expected of them, for example, how much they will be paid, how many questionnaires they are expected to administer, the start and end dates of the data collection exercise.
During the training, go through all the questions, one-by-one. Discuss each question on its own making sure that all the research assistants have the same understanding about the question. Discuss if the question is simple and easy to understand, and is not ambiguous.
For closed-ended questions, go through the responses making sure they make sense and are exhaustive enough.
Go through the instructions for each question, making sure that the instructions are easy to understand.
Go through the skip logics used in the questionnaire, making sure that they are simple and easy to follow.
Allow for a Q&A session where you respond to all the questions the research assistants may have.
Sign a contract with the research assistants before they start the data collection exercise.
Pre-testing a questionnaire
After designing the questionnaire and training the research assistants, the next important step is to pre-test the questionnaire.
Before the questionnaire is administered to the target population, it is good practice to pre-test it.
Pre-testing vs. pilot-testing
Most people use pre-testing and pilot-testing interchangeably but the two are different. Whereas pre-testing entails testing the usefulness and adequacy of a data collection tool (like questionnaire), pilot testing goes beyond that. Pilot testing entails testing the feasibility of the entire study and involves conducting a mini-study before the entire study is launched. It therefore checks for the viability of the research design, the data collection tool, the logistics of conducting the study, the adequacy of the sampling technique, sample frame and sample size etc. Pre-testing is therefore nested within a pilot study. Pilot testing is best done when the target population is large.
Pretesting the questionnaire entails administering the same questionnaire to a small sample from the target population before it is administered to the larger sample.
Pre-testing should be done with individuals from the same target population, not just any population. For instance, if the target population is pregnant women, the pretest should be done with a few pregnant women, not just any woman. However, respondents who take part in the pre-test should not be included in the main study.
Additionally, if using research assistants for the study, the (trained) research assistants should be involved in the pre-test. Do not use someone else who will not be part of the main study to pre-test the questionnaire. Pre-testing is a good way for the trained research assistants to gain experience with the tool and the process and bring up any issues they may encounter so that they can be corrected before the questionnaire is administered.
The purpose of pre-testing is to:
- Identify any ambiguous or unclear questions in the questionnaire.
- Ensure that the targeted respondents have a good understanding of the questions that are in the tool.
- Check if the wording and ordering of the questions is correct.
- Check if there are unnecessary questions that need to be removed or important missing questions that need to be added.
- Check if the layout, structure and length of the questionnaire is OK.
After the pre-test is done, all the issues raised should be addressed and the questionnaire corrected before it is launched.
Protocols for administering questionnaire
Depending on the nature of study, the student may need to observe some protocols before the questionnaire can be administered.
Some common protocols include:
- Obtaining ethical clearance from the relevant authority.
- If data will be collected from organisations, the student should visit the organisations first and create rapport with the management before the study is launched. During the visits, the student should explain the purpose of the study and the need for collecting data from them. An introduction letter is key for this protocol.
- Sending an introduction letter in advance to the identified sample.
- Seeking audience with community leaders and informing them about the study, its purpose, how the sample was identified and what the data will be used for. This is very important when the target population involves members of local communities. Establishing rapport with the local leaders not only helps with their buy-in but the leaders can be instrumental in mobilising the community members to ensure they are co-operative during the study.
In conclusion, designing and administering questionnaires are vital skills for PhD students to gain especially if their future career will be in research. This article provided some key guidelines on designing effective questionnaires and administering them so as to achieve the study’s objectives.
Recommended resources
https://journals.sagepub.com/doi/pdf/10.5005/jp-journals-10021-1104