A trick question?

The stumper, it seemed, during yesterday's interviews for our new high school library media specialist was:

How will you demonstrate that the library media program is having a positive impact on student achievement in the school?

How did that evil question get in there with "Tell us a little about yourself" and "Describe a successful lesson you've taught"? - those questions most of us could answer with one frontal lobe tied behind our backs.

Given the increased emphasis on accountability and data-driven practices, it's question all of us, librarians and technologists alike, need to be ready to answer - even if we are not looking for a new job. (Of the four candidates we interviewed yesterday, three were looking for a job because their previous positions had been cut.)

 While I would never be quick enough to have said this without knowing the question was coming, I believe the best response to the question would be another question: "How does your school measure student achievement?"

If the answer was simply, "Our school measures student achievement by standardized or state test scores," I would have had to have replied, "There is an empirical way of determining whether the library program has an impact on such scores, but I don't think you'd really want to run the study. Are you willing to have a significant portion of your students (and teachers) go without library services and resources as part of a control group? Are you willing to wait 3-4 years for reliable longitudinal data? Are willing to measure only those students who are here their entire educational career? Are you willing to change nothing else in the school to eliminate all other factors that might influence test scores? Will the groups we analyze be large enough to be considered statistically significant?" No school I know of has the will to run such as study. Nor do many of us have the statistical or research expertise to conduct it any who.

If the answer to the question "How does your school measure student achievement?" was more complex - though successful completion of rigorous course work, through  authentically assessed mastery of problem-solving skills, through reports of post-secondary success by graduates, through successful participation in extra-curricular activities, though high graduation rates, through alumni or employer satisfaction surveys, etc.., then my response could not be so flip.

How can we hold ourselves accountable/answerable in an education environment in which a empirical means to demonstrate our value is impractical if not impossible?

Joyce Valenza's exemplary "End of Year" report that she shares in a recent Neverending Search blog entry is one example of demonstrating a program's (and the program director's) impact on student and school success. Why do I think Joyce's position will probably never be cut so long as she is in it?

OK, Joyce is probably among the half dozen smartest and hardest working people in all library-land.  Nobody should be expected to create a report as good as Joyce's. But we can all steal really good stuff from the report. I especially like the idea of exit interviews with seniors. (OK, making videos is annoyingly over-achieving, but regular interviews or a survey is do-able.) Replicate a mini-OHIO study  (TODD, KUHLTHAU, AND OELMA, 2004). Track the circ and use statistics and make the numbers meaningful. (Is there a spike in checkouts after a particular initiative?) David Loertcher has been on the bandwagon for use to compile lists of units we work with teachers on and the skills being taught in them.

And I would still advocate creating and then reporting on the successful completion of annual goals that are owned by not just the librarian, but the staff and administration too. That's how my department remains "answerable."

 How will you demonstrate that the library media (or technology) program is having a positive impact on student achievement in the school?

Think about it. I bet you'll be asked in the near future. How will you answer? 

Comments

[Jacquie Henry]

Doug,

Take a look at this blog post:
http://hollywolf.typepad.com/electronic_pencilhigh_sch/2006/06/this_spring_alo.html

Holly is a librarian in a high school near mine. She is an amazingly talented librarian, and has recently joined the blogging world. Her posts will be well worth reading. She has done exactly what you suggest - replicated a mini-Ohio Study.

[Artichoke]

Interesting - I have come up against this question before - in New Zealand and in the context of ICT PD for teachers and student learning outcomes.

How will you demonstrate that the library media (or technology) program is having a positive impact on student achievement in the school? is unanswerable given the complexity of factors impacting on student achievement. The question/s we should be asking and answering is

What are the conditions of value in teaching and learning that have a positive impact on student achievemnt in the school?

and then

How can the library media (or technology) program enhance these conditions of value?

[Betsy]

In Texas, where some features now in federal education were first tried out, this question has become more and more important. I have used Keith Currey Lance's studies quite a bit. We have also tried to remind folks that, yes indeed, student achievement goes beyond state testing (try to tell that to a teacher under the gun to raise test scores so the school can achieve a certain rating).

[Lisa]

I teach at one of those schools in Texas. (I am also working on my library science degree) We just had a change this last year in our school librarian. The new librarian was absolutely wonderful. She really turned the kids on to reading. I saw a definite shift in our reading environment and a desire (latent in the past, if any) bubble up in the kids to read for pleasure. Yes, our school is going up a level in our state's accredidation. I am glad to be a part of that but I know our librarian should also get a ribbon.

[Doug Johnson]

Hi Lisa,

Thanks for your very nice comment. I believe most librarians work very diligently to create good readers without much recognition for their accomplishments. I hope many of them see themselves when reading this.

All the very best,

Doug

[Janice]

I am working on my National Boards. This is the question I have to ask myself with each entry: How does what you do impact student learning? The impact can be direct or indirect. I thought Entry 4 (Documented Accomplishments) would be the easiest entry--NOT! I am really having to dig deep in my thinking process to answer the question. I must demonstrate that my program is having an impact on student learning by reflecting upon the National Standards. NBPTS candidates must be able to prove that what they do impacts student learning "through your work with families and the community, with colleagues and other professionals, and as a learner." I have also purchased Joyce Valenza's Power Tools so that I could have forms to document impact on student learning. Another book that I have purchased that has data driven forms to prove impact on student learning is by Frances Bradburn -- Output Measures in the School Library Program.

Even if I do not pass the NBs the first time, the experience has already caused me to ask "How will that impact student learning?" If it impacts student learning, then it should impact student achievement.

Although some folks might think your question is a trick question, it is right on target with the National Boards for Professional Teaching Standards. I would venture a guess that if you asked a NBC media specialist, the question would be answered without hesitation.

[Toni]

Interestingly enough, the question which seemed to "stump" your candidates was the very first question I asked myself when I was blessed with my new assignment for next year (K-5 Media). Having just completed a stint as our district's Reading First Literacy Coordinator, I have to admit, I'm maybe a little bit too "data driven." But, I am also goal-oriented, and wanted to create a vision of success in the media center. I appreciate the various links offered here that provide some direction, and measurement tools. However, my 13-year-old son summed up a response that is kid-friendly, with, "Mom, if you want to know if you've done your job, just watch and see if kids come to the library because they WANT to, not because they HAVE to. I only went to the library one time this year, and that was because I HAD to for our research project."

Good idea.