Date: Mon, 1 Mar 2010 10:47:31 -0500
From: P & S Cabe
Subject: Chatham Tax Survey
Steve Candelori deconstructs an earlier post with a series of pretty cynical barbs toward that earlier writer. I won’t attempt to respond to Candelori’s comments in any detail, but I think there are some general comments worth making about the tax survey.
Here is the last question cut-and-pasted from this Chatham Tax Survey. I’ve inserted letters to separate its various parts, and grant that I may have overlooked one or more. (Those of you who notice should ignore any grammatical mistakes in the question.)
“A proposal to request that commissioners to reduce spending:
Entreat the commissioners (a) to reduce spending (b) by twenty-two & one half percent (22.5 %) (c) over three years, 2010-2013, (d) seven & one half percent (7.5 %) per year, and (d) submit to timely reporting on spending (e) and/or to monitoring (f) by a responsible outside entity. Do you support such an initiative?”
The only two responses are yes and no. If you choose not to answer it, the survey comes back with an error message to the effect that an answer is required.
A recent poster admonished, “It’s a question on a survey…just answer it.” Look at that question for a minute. How exactly is a reasonably thoughtful person to answer it? You might agree with ALL the parts of the question; easy answer “yes.” But what if you agree with some, but not all, of the parts? How do you answer then?
For instance, I might think it is a good idea to reduce spending (possible “yes” response), but not by a highly specific 22.5% (possible “no” response). Where did that number come from anyway? Or I might think even the specific percentage is a good idea, but not over three years…or maybe over a different three years, say, 2012-2015. Or maybe staged in a different progression, rather than 7.5% a year. Is reporting the same as or different from monitoring? What is this “outside entity?” Etc., etc., etc.
In the end, the proportion of yes and no answers to this hugely complicated question must really be pretty meaningless, because we have no idea to which of its many parts any given respondent is answering yes or no. Candelori says “Surveys like this help keep those that care informed.” One might seriously doubt that conclusion. Rather, surveys like this tend to obscure issues and muddy the decisional waters for all of us.
I’ve seen lots of surveys and developed some myself. They are blessed difficult to do well. Tons of the ones I’ve encountered are simply poorly thought out, poorly designed, poorly constructed, poorly administered, and poorly analyzed. The worst by far, in my opinion, are the ones (like this one, I’m afraid) that lead the respondent on and on with seemingly innocuous questions (is the register of deeds doing a good job?), then fire a terminal zinger that asks for a categorical response (typically, yes/ no or agree/disagree) to a multi-prong question such as the last one in this survey. Rather typically, too, there is no space provided whatsoever to explain a response or to offer a comment for a given question.
Every time you are asked to complete a survey, you should ask yourself two questions: (a) who is doing the survey, (b) how are the results to be used (that is, what is the purpose of collecting this information). Taken together, the answers to those questions go a long way toward helping to decide if you want to expend your time, energy, and thought on the survey. If you can’t see answers to those questions, or find them out easily, don’t waste your time with it. Maybe someone can help me with answers to those two questions for this tax survey, but I couldn’t discover them for myself.
Unfortunately, many surveys are not intended for benign purposes, but rather to promote someone’s or some group’s special agenda (which may be disguised with an appealing title and which you may or may not agree with). Further, such individuals or groups appear to have little compunction about using your responses for those objectives. In the absence of clear indications about what that agenda might be, it’s probably better simply to keep your thoughts to yourself.
Or simply lie. Who’ll know the difference? Well done surveys go to great lengths to identify the characteristics of the sample of respondents they query and to promote the validity of the responses they get. There appears to be none of that care evident in this tax survey. In principle, anyone from anywhere for any purpose could respond to the survey, apparently as many times as they want to, Chatham resident or not, taxpayer or not, voter or not. And no one would be the wiser. How could any reasonable person trust the results? If someone throws those results up as a reflection of “public opinion,” either that person doesn’t care about the quality of the information or doesn’t know how bad it is.