Evaluating teachers
Monday, January 21, 2008 at 07:04PM As reported by a number of horrified bloggers, the New York schools are experimenting with evaluating teachers based on their students' test scores. Bad, bad idea. Right? No control over the input; therefore not control over the output:
... teachers are being measured on how many students in their classes meet basic progress goals, how much student performance grows each year, and how that improvement compares with the performance of similar students with other teachers.
But put yer parent hat on for minute, and consider this scenario...
Your child's school district gives a "value-added" test like the NWEA MAPS test. These computerized tests are given in the fall and again in the spring and are designed to measure individual student growth on specific skills. RIT scores show just how many months/years of progress each individual makes between tests. Whether a student has a 1st grade reading level or a 10th grade reading level on entering the 5th grade classroom, the test will tell whether that student makes one year of skill growth - or not. At least that's the theory.
Now, my little boy Skunkie Jr is going into 4th grade and there are three possible 4th grade teachers that he might get. Might not I, as a parent, want to look at the track record of each teacher the Skunkster might get next year - as demonstrated by the percent of students that made or exceeded a year of growth in each of those teachers classes, over say, the past three years. The records indicate that an average of 75% of Mr. Chip's kids make a year of growth; 90% of Ms. Brodie's kids made a year's growth; but only 50% of Mr. Holland's get that year of progress. Might that not be a good thing to know - as a parent or an administrator or a staff development coordiantor or as a taxpayer?
It certainly wouldn't be the only factor I would consider in choosing a teacher (or school) for my child, but I'd rather like knowing it. I do want little Blue to be able to read and do his sums.
Norm-referenced standardized tests can't be used to measure the quality of teachers, of course. But that doesn't sound like what NYC schools are doing. I can't see how it would be a bad thing if my son making a year's worth of progress on a set of standards was seen as very, very important to his teacher.
I suspect there are plenty of really good teachers who would welcome some form of objective evaluation criteria for the work they do. Even without merit-based pay.
I'm not that horrified. Should I be? Why?
Reader Comments (17)
As always, I have more question than answers:
Should the "best" teachers have the highest achieving students in their class or the ones who are most in need of improvement?
What if you requested the teacher with the highest success rate and they "rejected" your child? How would you and your student feel?
Teacher "style" and students' learning styles vary greatly. What if your child and his/her teacher were not a good match?
If a teacher started out with a high-performing class, how could they possibly achieve AYP?
Wouldn't this system discourage young teachers who might need a few years of seasoning to reach their full potential?
Lots of questions, for which I have no good answers. I'd appreciate hearing your responses.
diane
I'm not horrified, but I am a little concerned. This method is a vast improvement over the way we are currently judging schools and teachers based on testing, but it is still far from perfect. In your scenario it is possible that Mr. Holland's class had a significant number of students with little or no support at home (or various other factors that could affect test scores). Simply looking at the growth over a year will tell you something about him as a teacher, but he can only control so much.
Again, I agree that this is a better system than our current one. But I'm not convinced that this is information that should be available to the general public. It's too complex an issue to present simply through test scores.
i'm not sure such info ought to be in the hands of parents. maybe. i'm really not sure. my gut says it sets up a competitiveness and publicity nightmare that pits colleagues against one another. but i absolutely don't think it's bad for the teacher to have pressure along these lines and i don't even think it's bad (don't tell my union) if teachers are somehow compensated (with extra pay) using these sorts of results. and to those who say there are things teachers can't control, like home environment---well, sure, but the system you described is measuring things that teachers can control---individual academic growth over the course of a year.
Thanks for offering another perspective on this issue.
I wonder how a school would accommodate all 4th-grade parents requesting (demanding?) Ms. Brodie...
The concern for younger teachers is a legitimate one, and another reason I wish education had an "internship" system.
And I would reply to Amy S. regarding individual growth: if I take two students with the same teacher, X with excellent support at home and Y with a serious lack of support, their progress will (almost always) be vastly different. Now, instead of averaging a full-year's growth for all students, that teacher is averaging perhaps only 75%. So the at-home factors would still have bearing.
Still, assessing individual student progress year-to-year is an improvement. The next questions: what are we assessing? how? and how do we account for outside factors?
Hi Diane,
People who think tend to have more questions than answers, I believe. Yours are great questions. Most of your questions seem to concern values and the answers can't be definitive. Here is how I personally would answer them:
> Should the "best" teachers have the highest achieving students in their
> class or the ones who are most in need of improvement?
I would hope all teachers have a mix. Such testing might ensure all teachers are fairly treated by having a classroom that has both the brightest and the most needy.
> What if you requested the teacher with the highest success rate and they
> "rejected" your child? How would you and your student feel?
Bad? Disappointed? Wondering why the school was not making sure all teachers were equally effective?
> Teacher "style" and students' learning styles vary greatly. What if your
> child and his/her teacher were not a good match?
As I said in my earlier post, test scores would be only one way I would evaluate what teacher I wanted my child to have. Again, testing might make matching learning and teaching styles important enough for administrators to actually pay attention to when placing kids.
> If a teacher started out with a high-performing class, how could they
> possibly achieve AYP?
I am not sure what you mean by this? It is not high-performing students who are keeping schools from meeting their AYP goals. Using value-added testing, high performing students could also be shown to make improvements or not.
> Wouldn't this system discourage young teachers who might need a few years
> of seasoning to reach their full potential?
We've long needed an "internship" category for new teachers. Such a test might encourage the adoption of an internship category. Such testing would also be a morale booster if teachers see their effectiveness growing!
Just some ideas. Thanks again for writing in. This really is the 3rd rail of education!
Doug
Hi Jenny,
To make sure that no one year's class would "describe" a teacher's effectiveness is why I would look at averaging results over a 3 year time period. Were struggling students equally divided among all sections, every teacher (not just Mr Holland) would have the same challenges.
It is complex!
All the best and thanks for writing,
Doug
Hi Amy,
The competition factor is something to consider, I'm sure. My sense is that most good teachers want to see their colleagues succeed. Maybe I am naive. If all teachers at a grade level were shown to be doing very well, it would not be a negative on any one teacher IMHO.
Thanks for bring up the issue.
Doug
Hi Eric,
Great comments/concerns!
Don't we to a degree already have the issue of parents requesting popular teachers? Look at it the other way, what if all parent requested their children NOT have a teacher with a poor track record? Seems like some serious remediation might then in store - which would benefit all that teacher's future students.
I agree about the internship program. One thing we have noticed is that our teachers like the NWEA tests because they do show measurable growth which is a morale booster. Ever teach a year and wonder if you really had any impact at all? I have and it is discouraging.
Again, your X and Y students would need to be equally distributed among teachers for this to work.
Thanks again for the comments!
Doug
On one hand I think the accounting of student progress is a good idea since some teachers will be able to show the results of their efforts with students. I also think this could help administrators get struggling teachers the necessary assistance for helping students increase capacity and for improving the teacher's skills.
However, I also believe, being a high school teacher, that having an honors class would mean that my students may already be maxing out on the scale and may not show much growth while my "regular" English course students would reveal more growth. Did I do a better job in the one over the other?
I hesitate to openly publish the results, though I also feel most parents can probably figure out which teachers are more effective than others.
Hi there Dr. Pezz,
From what I have seen and heard of NWEA MAPS test, even the brightest, older kids are very challenged since the test modifies itself to the user. The more right answers, the harder the questions become (and vice versa).
I will admit this idea is not as practical for students who have mastered the basics.
Thanks for the reflection,
Doug
This post is an essential look into the reality of implementing value-added assessment of teachers:
http://edwize.org/the-real-true-facts-behind-the-doe-teacher-value-added-project
Yes, it could be implemented better, but the bottom line is you're talking about spending A LOT OF MONEY to improve the quality of the data, increase the amount of time giving higher quality tests, etc. It is EXPENSIVE to do it right, and it isn't clear the testing industry or schools have the capacity to do it right. No, I take that back, they've PROVEN they don't have the capacity to do it right, so really, it is just a fantasy.
Hi Tom,
Considering edwize comes from the teachers' union, the response is not exactly unpredictable. And given the stupidity with how standardized testing has been used, one can understand the sentiments.
A couple things about the NWEA MAPS testing that we are in our third year of using:
-NWEA is non-profit
-NWEA compares a students performance to his/her own performance rather than to a peer group
-NWEA results are used in out district primarily to help identify specific areas of need for individuals and facilitate differentiated instruction (they have test results as they walk out the lab in the fall for immediate use)
We have never considered (that I know of) using NWEA scores to evaluate teachers and having and aggregated student performance measures made public is not done - yet
There is no such thing as a perfect test, I'm afraid, but as long as our communities and politicians demand them, I think we should go with the test that can be used for formative purposes, not just normative.
Yes testing is expensive. But don't be too quick to judge all testing products and companies by Pearson!
All the best and thanks as always for a provocative perspective,
Doug
My kids take the NWEA tests. They're GT, so I don't worry too much about their performance from year to year, figuring that it will all balance out eventually. However, one year my son progressed three grade levels in math from fall to spring. He had an amazing teacher who not only taught well, but assigned lots of homework. His previous teacher had no idea how to work with a GT student, even though she had years of experience teaching. She frequently used him to tutor her slower students, including ESL students. She stuck to her guns on their reading program, so even though he was reading at 12th grade level in 4th grade, she made him read silly little fourth grade books. That wasn't a huge problem because I'm a librarian and I augmented his reading at home. He's in 8th grade now, so I asked him, do you remember Mrs. C. He said, no, I had Mrs. Lewis for 4th grade. Nope, I said, that was 3rd grade. Well then, I had Mr. Lundy... Nope, that was 5th grade. He doesn't even remember her. She didn't *hurt* him, but she didn't help either. My point? The evaluation takes place whether you acknowledge it or not.
Hi Jude,
Thanks for the great example. One's view of testing changes, I think, when it is one's own kids (or grandkids in my case) who are impacted by the effectiveness of teachers. While these tests are imperfect, they are what we have.
My 6-year-old last year decided he didn't like reading! As it turned out, he didn't like reading "little bunny books" as he called them. His parents get him science, history and Captain Underpants books, and he reads all the time now. Teachers made such a difference!
All the very best and thanks for commenting,
Doug
With all due respect, the logic in your scenario is flawed.
The assumption is that Skunkie's learning depends on Ms. Brodie being his teacher since she had the greatest number of students that made or exceeded a years growth academically. His learning would more likely depend on having Ms. Brodie if he were her only student or he was a fish. In acutality, Skunkie's learning could depend on Ms. Brodie, or it could depend upon his classmates, or it could depend on your families well being, or it could depend on the allignment of the planets.
A childs ability to learn, or a teacher's ability to teach is not a simple experimental design as judging teachers based on standardized test would imply. Learning and teaching is an incredibly complex design the cannot be judged so quantatatively.
Hi Rick,
I would certainly agree that a wide variety of factors are a part of why kids learn and that skill-based tests don't measure everything. This is why any look at judging a teacher's effectiveness would need to be an aggregate of several years and that students be randomly assigned to classes. It should also be just one of several teacher performance indicators.
You've expressed the feelings of many teachers, I'm sure. Thanks for the comment.
All the very best,
Doug
Ok people, stop the nonsense! I teach in a gifted classroom... I have students that go from 240 to 246 on the READING NWEA. They are fourth graders... however, if they only make 6 points I do not make my goal. Keep in mind, these are fourth graders at an 8th grade level. I have to get them all to a 9th grade level???? That is insane. The higher you are, the harder it is to make those kind of gains. Also, where is the curriculum/activites to make sure your already high students make a years growth according to the NWEA??? This is just insane.