Stage 1: Design and Planning (cont.)
Formulating the questions
Once topics or modules are identified, specific questions can be drafted. Formulating succinct and simple questions that address survey objectives can be time consuming and a challenge.
Several useful survey or questionnaire banks exist that provide access to questionnaires implemented by national and international institutions. These use standardised questions and responses with good working scales and categories that are valid and reliable . These are useful if an objective of the survey is to provide comparable data. A pilot test is still recommended even if standard questions are used.
Useful Survey Question Banks
- International Household Survey Network. http://ihsn.org
- Survey Resources Network coordinated by UK Data Archive at University of Essex http://surveynet.ac.uk/sqb/surveys/introduction.asp
- An interagency question bank (Q-bank) involving Center for Disease Control (CDC) http://wwwn.cdc.gov/qbank/Home.aspx
Two major question formats exits – open and closed.
Q1. How do you feel about the quality of care you received from the family planning worker at her last visit? ................................................................................................ Q2. How would you rate the quality of care you received from the family planning worker at her last visit?
|
There are advantages and disadvantages to both. Open ended questions may elicit more detail but are harder to analyse and may take longer to administer. Clearly, response categories for closed questions need to be fully comprehensive and require careful consideration (Polgar et al. 1995).
Most questions are designed for the interviewer to read verbatim. However, some questions may require additional questions or probes to obtain a complete answer. These probes should be neutral and interviewers trained in probing techniques. Other questions may need adapting to fit the respondent’s specific situation.
In the DHS, questions that need adapting include words in parentheses. How the question is phrased will depend on the respondent’s specific circumstances. In the question below the interviewer must choose the appropriate term depending on whether the respondent is married or not.
How old was your (husband/partner) on his last birthday? | Age in completed years |
|
In the question below the capitalised word in parentheses is an instructions to the interviewer, and should not be read out. The interviewer must substitute the instruction for the respondent’s child’s name.
Was [NAME] weighed at birth? |
Yes.........................1 No...........................2 Don’t know...............3 |
In general, survey questions should:
- Use simple, non-judgemental, everyday language
- Avoid bias
Biased questions elicit a response that does not accurately reflect the respondent’s position. A question can be biased in 3 ways:
- If it implies the respondent should be engaged in a particular behaviour.
- If the response categories are unequal or loaded in a particular direction.
- If words with strong positive or negative connotations are used.
For example, "What side effects did you experience?" assumes side effects were experienced. First ask "Did you experience any side-effects?"
- Contain only one idea or question.
- Be short and succinct.
- Follow logical, natural sequence with clear skip and filter instructions for the interviewer.
- Contain a defined and specific scope, such as the time period or activity relevant
to the question
For example studies show that recall of behaviour, such as condom use, is easier if it is associated with a specific event, for example last sexual event, rather than a specific reference period, such as last 4 weeks. Questions associated with specific events may also be less prone to social desirability bias. Moreover, evidence suggests that use of condoms at last sex correlates strongly with measures of consistent use (Cleland et al. 2004).
For example, the meaning of terms may vary by age, sex and locality. In the National Survey of Sexual Attitudes and Lifestyles (NATSAL) conducted in the UK, the term "sexual partner" was carefully explained to all participants because in some cases it was considered too "casual" to refer to a married partner, while among single respondents it was considered appropriate to describe a steady relationship but not a casual encounter (Fenton et al. 2001).
- Contain response options that are simple, clear, consistent, comprehensive and precoded.
- For categorical responses, be mutually exclusive and exhaustive
- For numeric responses, guide the respondent to provide the response in a consistent format and units
- Not require the respondent to recall several different elements at one time before attempting a response
"How much did your last visit to the clinic cost?" requires the respondent to recall the 3 bits of information - last visit, the travel cost and the clinic costs. A series of more simple questions would be preferable.
- For example: "When did you last visit the clinic?
- How did you travel to the clinic?
- How much did you pay for the transport?
- Did you have to pay any money to the clinic?"
Breaking the questions down helps the respondent recall relevant details and that should be more accurate than a single, inclusive question.
Attitudinal questions are challenging (what is your attitude to condoms?). It is often better to pose a series of statements such as:
"If I loved someone I would rather not use a condom than risk losing this person", "It is not ‘cool’ to use condoms", or "It is very embarrassing to obtain condoms" can be combined into a score for a multi-item scale. The individual statements of such a scale could be answered by "Yes" or "No", or by "Likert scale" response options ranging from strongly agree to strongly disagree. By combining the response to all statements, a score is computed and an attitude towards condoms estimated.
Likert scale is a widely used method to rank responses and typically takes the following format.
- Strongly disagree
- Disagree
- Neither agree nor disagree
- Agree
- Strongly agree
Source: adapted from Collumbien et al. 2012.
- Have clear examples of probes and instructions for the interviewer
Translating survey documents
Once the questionnaire is finalised it may require translation into several languages. The aim is to minimise the need for on-the-spot translation by interviewers, and to interview respondents in their own language, so as to minimise error. Translation of questionnaires and survey documents is an important and time consuming undertaking. For instance, the three questionnaires (household; women’s; and men’s questionnaires) that make up the Kenyan DHS were all translated from English into 11 different languages, including Kiswahili.
The process typically involves a core questionnaire (usually in English) being translated into a local language by a skilled individual. If a similar questionnaire has been fielded in the past (as with the DHS), these translations are also referred to. The translated questionnaire is then translated back into the original language by an independent translator, without reference to the original. The back-translation and original questionnaires should be as independent as possible. The two documents are compared and any discrepancies resolved. It is then necessary to test the translation in the field before finalisation and printing.
It may also be necessary to translate other survey documents, such as interviewer manuals etc., from English into local languages.