Subjectivity and multiple assessors

Rubrics are great formative assessment tools for projects and performances. When well-designed, they are great ways to self-assess and learn exactly how to improve in order to move to the next "level" of the rubric. 

But a genuine deficiency of too many rubrics, especially those attempting to measure creativity, is the ambiguous nature of the terms. Take this example by Susan Brookhart from her article Assesing Creativity:

Descriptors like:

• startling
• important
• same or similar
• wide variety
• original
• surprising
• interesting
• helpful
• excellent
• good
• some
• many
• few

What separates excellent and good?

What is startling to a first grader may be tired old stuff to his teacher - and perhaps vice versa.

One rubric level for creativity I’ve seen simply asks, “Is the “wow factor” present?” Can something wow one person and cause yawns in the next?

One way of overcoming the inherently subjective nature of scoring the degree to which a student product or idea is effective or original is to use multiple assessors. 

Projects that are more likely to produce creativite efforts have end products that are shared with people who care and respond. A good assessment tool includes blanks for the student, a peer, and the instructor to each rate the project. If the assignment is team-based, each member should rate each element that is being assessed. Comparing and discussing multiple results may reduce inaccuracy or unfairness in an inherently subjective evaluation.

Figure skating in the Olympics uses multiple judges for just that reason

Add parents and guardians as potential assessors. While no one wants parents doing the child’s work for them, I believe we can define a role for them. When my less-than-enthusiastic-about school son came home from a science class with a mousetrap and the assignment that it need to power a self-made car, he descended into his basement lair. After a very short time, I heard the familiar soundtrack of a videogame coming up the stairs. When I asked him to show me his product, the duct-taped set of Kinex plastic parts lept six inches in the air then crashed to the floor. I suggested to him the assignment was to build a car that  moves horizontally, not a helicopter - and that he could play videogames after his project could move at least five feet across the floor. “Quality control” is the correct role for parents and they can help provide that if they too have the assignment and tool by which it will be judged. Do you have links to these materials on your school website for easy parental access?

Older students, local experts, public blog comments, and classes in other schools might also prove useful for any given assessment effort. Just use your imagination.

Forgive me a short rant. In our one-right-answer, test-driven educational environment, we've somehow lost our faith in subjective evaluation. Yes, many projects that we as adult must complete have some form of metric. Did the project come in on time and under budget? Did the project have its intended effect? Did we sell more widgets as a result of the project? By meeting metrics like this we can objectively show success or failure. But truly successful projects have less measurable outcomes as well. Did the team who completed the project work well together? Were there unintended consequences of the project? Has the company’s reputation and public image improved as a result of the project? Subjective evaluations still have a place in organizational work - and especially in schools. One big difference between a teacher and computer program is that the teacher has a heart as well as a brain and uses both when needed.

And as Einstein also famously wrote in his blog, “Not everything that can be counted counts, and not everything that counts can be counted.”