Dear Editor,
Prior to my retirement I was Chairman of the Business Administration and Commerce program at Red Deer College for many years. In addition, I taught a senior-level Marketing Research course on research design and questionnaire design. This course transferred to third/fourth year Bachelor of Commerce/Management programs at all Alberta Universities. I did this for 33 years.
I also taught this course at U of C. If, during these 33 years, a student handed in a questionnaire like the one the town (Survey Monkey) is presenting online, that student would have received an "F" as a grade. The survey is flawed in a myriad of ways. The following is far from an exhaustive list of things which would bring about a failing grade.
The basic question is analogous to asking how you would prefer to be executed; hanging or electric chair. What about all the other options including not being executed at all?
The two most important things in quantitative research design are reliability and validity. Reliability means that if you replicate the survey you will receive the same response every time, given a small + or - error factor based on sample size. You can not get a high degree of reliability with a self-selecting sample like the one being used. Administer this questionnaire 20 times and you will probably get 20 very different sets of responses.
Validity means asking the questions in a manner which will elicit accurate, unbiased data. Providing exhaustive lists of options and no biased or leading questions are three examples of how to get valid data. Unfortunately the questionnaire fails in all three of these areas.
There is no guarantee of anonymity for the respondent. You have to identify your property which identifies you. This inhibits critical responses.
Anyone on the planet who logs onto the Sylvan Lake web site can respond to the questionnaire. The questionnaire should only be responded to by those affected (cottage owners).
You should never ask a question like “Should the roadway be narrower in order to save trees?”. How many people are going to say "Screw the trees, let's have a wide street”? This is a classic leading question.
People rarely take the time to provide written responses. They generally choose from the options presented. The personal biases of the researcher also influence how written responses are grouped and quantified.
The diagrams are confusing at best and are hard to see given that you need to scroll across to see everything.
After viewing the “survey” and based on my years of experience, I can only arrive at one of the following two conclusions or some combination of both:
The town has already decided what it wants to do regarding modernization and has purposefully designed a survey to gather data in support of that decision. From a research perspective this is extremely unethical and from my perspective, immoral.
Both the town, and Survey Monkey do not have a clue how to gather reliable and valid data. Basing any decisions on responses received from this terrible survey would be a travesty.
The town and Survey Monkey, should be ashamed of what they are calling a survey.
Michael O’Hanlon,
Sylvan Lake