Blame the user
Thursday, July 24, 2008 at 09:08AM When three people call you an ass, put on a bridle. Spanish proverb
I will be running close to the edge of my "Complain globally, praise locally" blogging guidelines in this post . But I am pretty angry.
Our state's Library Services Department wanted to collect data on school library programs using an online survey tool. Great!
We need a good set of data. We don't know for certain how many libraries, librarians, resources or computers we have in our fair state's schools - and whether those numbers are increasing or decreasing. It was embarrassing during legislative testimony to be asked for school library data and to not have such numbers available. The lack did not help our case.
So the intent itself was outstanding.
But the execution was terrible. Irrelevant questions, confusing questions, unreadable formatting, unreasonable tech requirements, malfunctioning website, and just an incredibly daunting length were all "features" of this survey. But school librarians in 42% of schools bravely made the attempt - including our district. Many of us tried working the department to make the survey more useful and meaningful - work with seemed to have been simply ignored.
But this is what put me over: a scolding letter from the department saying...
Please note that of the 383 respondents, only 80 reports were correctly answered. Every library has a dictionary because of the importance of understating the meaning of a word. It’s equally important to understand the intent of the question to obtain comparable data.
So let me understand this... Of the 42% of surveys completed, only 21% of those were completed "correctly?" That is a rate of less than 9% of possible survey returns that the state deems as "correct."
Uh, might the problem be with the survey and not with the 91% of us who either didn't complete the survey or got it wrong?
Creating a good survey is a task best left to professionals, not well-meaning amateurs. The validity of the data requires it.
There is a larger issue here as well: When any of us don't get the response we were anticipating (amount of use of a new resource, attendance at an in-service, number of readers or responses to our blog, etc.), it's very easy to "blame the user." Maybe we should be looking at what we are offering instead.
Good intentions do not make up for incompetence.
Original image source: www.bryantnielson.com/
Reader Comments (10)
What an understanding state library department you have. Guess they don't understand the importance of positive communication. I mean, after all, library professionals are the information and communication experts, right?
What a patronizing comment: 'Every library has a dictionary because of the importance of understating the meaning of a word." That is NOT how to treat professionals, or any intended audience, for that matter.
And unless you're the Amazing Kreskin, how are you to know "intent of the question"? Clarity of writing should make that unmistakable.
I'd give the state Library Services Dept. a failing grade and provide some remedial work. Ignorance and rudeness are an unfortunate combination.
A couple of years ago the TEA required that every teacher fill out a survey re. their technolgy use. We all had to watch a talking head video, read the questions on line and then were given paper and pencil and told to bubble away.
Beyond dumb.
Hi Rob, Diane and Guusje,
Normally I'd single out an individual with these less than stellar interpersonal skills, but the department is so small, the individual really IS the department. From the other responses, it does not sound like an uncommon problem.
All the best,
Doug
Very well said, Obi-Wan. The intent is beyond applaudable, but the execution needs help.
M
For years I filled out a long library survey for Colorado school libraries. It was well written, AND it was longitudinal, although as the library field and technology changed, it did add some items to reflect the realities of this century. We were fortunate to have Keith Curry Lance in our state, and there's a message in that. To do a great survey you need number crunching statisticians as well as experts in the field being surveyed. You have written in the past about the importance of well-written surveys, and it's probably still in your back pocket waiting for time and energy to consider survey writing skills more in depth. As wonderful as the latest online survey software may be, it's still garbage in, garbage out. I'm sorry to say that it sounds as if you got both.
WOW that was rather condescending of them! Our state department of education (sde) does an annual library program survey too, and in the month leading up to it, we have teleconferences that are live (that are archived), and LMS's can tune in and call in with questions. Our SCASL listserv is also a great resource to get answers for anyone confused, and our sde liaison, Martha Alewine, is a member who frequently answers questions publicly posed there. She also provides a link on her site with FAQ, which are extremely helpful too.
Saying all this, we still have a large number of LMS's who just do not fill out the survey, making the data generated not really representative of the state of school library programs in SC. While it angers me and many others that people do not fill it out, there was very little one could do to get them to fill out the voluntary survey, save doing it for them. After all, it is a voluntary thing, and not really a requirement.
In comes the monkey wrench last year. In recent years SCASL lobbied hotly with "Strive for $25" trying to get our legislature to by law, give schools $$ for library programs, and we used the slogan "Strive for $25" on the premise that the average cost of a new book is about $25, and every student in the building should be able to check out at least one new book each year→ $25 per student per year.
I won't say our campaign was a total success, as the legislature tried to appease us by giving a $1 million dollar line in the state budget, and it was to be a one time funding. We STILL have our work cut out for us, and the campaign is far from over. I share this story because what our state department did was take all the schools that submitted their surveys, and divided up the $1 million between them. The schools that did not fill out that survey were not included in this special funding. That was a huge surprise for many who failed to complete the voluntary survey--but they couldn't really say a thing because the reminders for filling out the survey were continuous from May until mid September.
I would be willing to bet THIS YEAR we will have more participants completing the survey than ever before and get a much better representation of data. But I digress. My point here is that brow beating and berating (on the sde's part) will not help encourage participation. Once many begin participating, then and only then will sde leaders know what questions participants have in regards to the instrument, and then they can address the needs. KUDOS to SC's SDE, and specifically Martha Alewine, and the methods of implementation and encouragement for participation for our survey. (P.S. I changed districts this year, and I can PROUDLY boast that my former school and my current school both benefited from our annual survey with special funding from the sde as a reward for submitting data. W00T!!)
(P.S.S. I I were really a greedy person, I’d just stay quiet and hope fewer would fill it out, hoping the in the least same allocation of money is in the state budget, which in turn would mean MORE money for me. But I’m not that evil or greedy.)
@ Mary,
Thanks for the reminder that the success of the Colorado survey was in large part due to Lance's expertise both in libraries and in statistics. (And he is a very nice man as well.)
And yes, the how to create a good survey is on the back burner. All these projects for my retirement!
All the best,
Doug
@ Cathy,
What a great idea to give funds to only those schools who complete the survey! We currently allocate telecommunications funds only to those districts who are part of a consortium. A bit of a carrot to collaborate.
I think there must be a formula that relates length of survey to being in an inverse ratio to completions. Perhaps 2% decrease in return for every question asked??? Had I been running the survey I would have limited my questions to 10 at the most, then build in subsequent years.
The world would be such a better place when they just let me run it!
All the best and thanks for the ideas,
Doug
We completed that survey, too. All the LMS's in my district got together to interpret the questions together so we could at least be on the same page. The discussion of what questions meant took over two hours and several follow up e-mails. I couldn't see how all LMS's in the state could be expected to interpret the questions the same way. Like you said, good intentions, poor execution, insulting summary response.
Hi Linda,
We did the same thing in our district. With similar results.
Thanks for adding to the discussion!
Doug