Tips for Developing a Survey
Developing and administering a survey is a good way to gather information about your organization and the success of your programs. Here are a few tips to help you make your surveys as effective as possible.
Review previously administered surveys. You may decide that you need to start from scratch, but if you can use some questions from a previously administered survey, you'll be able to compare your results with results from past years. Also, other surveys will give you ideas about things you might want to include or exclude from your survey.
Keep your survey short. Try not to exceed one double-sided page. If your survey is longer than that, many people will not respond.
Start with some demographic items. These are items that tell you something about your respondents. You will probably want to know about gender and ethnicity. If your survey is for students, you'll want to know their grade level. You can ask just about anything, but be aware that some people will not respond if they feel their response will identify them. Demographic items are critical if you want to disaggregate your results. For example, do you want to know how the responses of males differed from those of females? If your survey is for students, do you want to know if there are differences in responses for students at different grade levels?
Make sure the instructions are very clear. Explain exactly what you want respondents to do. If they should mark an "x" in the box, say so. If they should use a pencil or certain color of pen, state that clearly. If you are using a scale (1-4, etc.) for responses, make it clear what each response means (1 = strongly disagree, 2 = disagree, etc.). Keep the language simple and comprehensible for the widest possible population.
Phrase your survey items positively. Avoid negative items because the responses get confusing. For example, if your item states, "I do not services offered in the evening," respondents who like services offered in the evening have to select "disagree" to indicate their preference. This will undoubtedly confuse some respondents and your results will be skewed.
Start with a list of the things you really want to know. Ask yourself (and your staff) what information you really want to get from the survey. Resist the temptation to ask many interesting, but useless questions. Focus on gathering information that you need to demonstrate your program's effectiveness or to improve your program.
Structure your response scale carefully. Many surveys contain a response scale that asks respondents to select a response from 1 to 4 (or 1 to 5, etc.). This is known as a Lickert Scale and it can be very helpful in allowing you to gather some quantitative data. However, you need to structure the scale carefully. Make sure the lowest number indicates the most negative response (1 = strongly disagree). Start your scale with 1, rather than zero. Consider offering an even number of responses, rather than an odd number. If you offer a range of 1-5, many of your respondents will select the middle-of-the-road response just because they don’t want to think about it. On the other hand, if you offer a range of 1-4, respondents are forced to choose and this will make your results more meaningful. Also, think about adding an "I don’t know" or "not applicable" option. If you do, make sure it is clearly marked and separated from the other options. Don’t list it as number 5 because those who do not read the directions (and there will be more of these than you think) will assume the highest number is the most affirmative choice and your results will be .
Provide an open-ended section for written responses. Some of your most useful information will come from open-ended responses. Think about what you really want to know. Phrase your questions carefully. Always give respondents a chance to offer other comments. When analyzing written comments, be aware that open-ended responses appear to carry more weight than they should. For example, ten people may comment on the same topic and this may lead you to think that you must do something about that topic, but when you analyze it carefully, you'll realize that you sent out 600 surveys and you received 250 back. Of those 250, only 10 people made that comment. While you should never ignore any comment, you should think carefully before basing program decisions on the opinion of slightly less than 2% of your total population and 4% of those who returned surveys.
Conduct a pilot before formally administering the survey. Have some members of your staff and/or a few trusted friends review your survey before you administer it. This will help you sort out vague and poorly worded questions.
Developing an effective survey is not as difficult as you might think, but it does require careful thought and planning. If you have questions about developing a survey, you can email them to CRR (veronica@grantgoddess.com). You'll receive a prompt response.
