Data or gut?

The Top 10 Reasons Why the Top 10 Reasons Don't Matter (Heart of Innovation)

1. Reason is highly over-rated.
2. If you need more data to prove your point, you'll never have enough data to prove your point.
3. Analysis paralysis.
4. You're going to follow your gut, anyway.
5. By the time you put your business case together, the market has passed you by.
6. "Not everything that counts can be counted; not everything that can be counted counts." - Albert Einstein
7. The scientific method came to Rene Descartes in a dream!
8. Most reasons are collected to prove to others what you have already decided to do.
9. "The reasonable man adapts himself to the world. The unreasonable man persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man." - G.B. Shaw
10. I am, therefore I think.

We're wired to be superstitious (so are dogs, parrots and most other creatures trying to survive), and if your favorite false causation makes you feel like you have a bit more control over things, enjoy it. But just as we'd rather not have a veterinarian that brings a rabbit foot into the operating room, when in doubt, it pays to understand what's actually happening and what's merely a crutch. Seth Godin

Data about our children are piling up like snowdrifts on a January morning in Minnesota. We spend immense amounts of (instructional) time, teacher time, technology support time, equipment usage time, in-service time, and administrative time on creating, reporting and organizing these flakes. And time costs money, of course. Of course time also has a value that cannot be measured by dollars - it cannot be replaced or reused once it is gone and time that could have been spent on actual teaching and learning is the gone as well.

Were these data used in ways that actually help schools become more effective in providing a quality, personalized education, I would be totally on board. But does this happen? Do any of us in education have the skills or will to make hard judgements based on hard numbers? Or do we pay lip service to data use, use data to support our pre-made assumptions, or use only the data that support the programs and projects our guts tell us work?

I will start believing schools take data-driven decision-making seriously when all administrators and teachers are required to demonstrate proficiency in statistical analysis. Numbers alone don't mean a damn thing. In fact, I would say that numbers used badly, can be harmful.

From my vague recollection of my (much enjoyed) grad school statistics class in the previous century, terms like "standard deviation from the norm," "valid sample size," and "correlation vs causation" are pretty important when looking at numbers. In using test results, for example, one first needs to know if, say, a five point difference between two groups is statistically significant before trying to figure out either the cause or the cure for that gap.

Until schools are willing to invest in training their staff on statistical analysis, let's just save our dollars and use them to buy rabbit feet and other luck charms.

And surrender the test data to those who simply wish to use it to bash and defund public education.