By Beth Hawkins | Published Fri, Apr 1 2011 9:34 am
In coming days, Minnesota schoolchildren — many of them fresh from spring break and thus possessed of sunny spring dispositions ripe for the crushing — will be asked to eat a nutritious breakfast, sharpen their No. 2 pencils and submit to a series of standardized tests that virtually everyone agrees are not just useless, but an expensive waste of classroom time that might otherwise be used for learning.
That’s right — it’s time for the Minnesota Comprehensive Assessments, or MCAs, as everyone over the age of, oh, 7, calls them. If you know an educator who has to administer one and then wait to see whether he or she is deemed a failure based on the outcome, this would be a good time to volunteer to buy them a drink.
Or better yet, buy them a copy of Todd Farley’s hilarious and outrageous new memoir “Making the Grades: My Misadventures in the Standardized Testing Industry.” Farley spent 15 years working his way up from scoring tests to writing them, and the book is a tour de force of the ineptitude, corruption and squishy statistical analyses he claims to have been privy to.
Me, I often wonder about the credibility of a writer who stays for 15 years in an industry he purports to have concluded PDQ was a total sham. But the publication of his book last fall has drawn other embittered test scorers out of the woodwork, including Dan DiMaggio, whose trenchant blog entry on the topic was brought to my attention by MinnPost reader and former teacher Ray Schoch.DiMaggio’s review of the book is a terrific read, so much so that I think my role can be limited to offering you a few excerpts in an effort to persuade you to click on the link above and read it.(And yes, I realize Truthout is a “lefty” blog. Lots of other people penned reviews of “Making the Grade” that made similar points, just not nearly as well.)
In both Farley and DiMaggio’s experiences, many of the tests plaguing K-12 today are scored by temps who may or may not understand the material before them. Which is a very big deal given that relatively few tests that are scored by live humans require written, open-ended or critical responses.
The scorers are supervised by other temps who pretty quickly figure out that their chief unwritten mandate is to make sure that the company selling the tests and the scoring can tout their statistical reliability. Oh, the wacky tricks desperate seasonal workers can come up with to make sure their numbers look good!
Synopsizes DiMaggio:
“To back this up, [Farley] rolls out an intriguing cast of characters he met over 14 years of scoring projects. There's Keith, a chiseled ultimate fighter who believes that he, rather than the students, is being tested (psychologically), and who is amazed to discover he is actually ‘deciding if kids are smart or not.’ There's Maureen, who frequently flags essays that mention students smoking or swearing because, she says, ‘their souls are in imminent danger’ …
“On one project in Phoenix, Farley describes watching what he calls ‘the backbone of the standardized testing industry’ as they file through the door: ‘a woman wearing a surgical mask, a la Michael Jackson on a shopping spree; a fellow in a full safari outfit, khaki pants and khaki vest, and even a pith helmet atop his head; an elderly African American man in shiny, pointy-toed shoes and a dapper zoot suit, a black fedora perched jauntily atop his silver curls; a fellow, in the 100+ degree Arizona heat, wearing a wool blazer and an Irish driving cap pulled low over his eyes; and a guy carrying under his arm, into his first day of work, a George Foreman grill, the family-sized one.’ ”
Adds the Washington Post, in its own capsule review:
“Ethics weren't the only problem. The guidelines used to assess the responses were often vague, and answers were awarded points based on absurd criteria. Farley's account is often downright funny, particularly in one case where students had to list their favorite food and describe its taste. The scorers hotly debated whether grass is a food, if pizza can be considered salty, and whether a food can be both sweet and bitter.”
So, how does one square such absurd criteria with that omnipresent demand for "reliability rates" and "validity statistics"? Reports DiMaggio:
“For example, Farley tells the story of ‘The Guy Who Gave All 2's.’ This scorer gave a 2 to every paper he saw for an entire day — over a thousand of them — and still managed to have completely acceptable statistics.”
The richest irony here, of course, is that if The Guy Who Gave All 2's were a teacher, it would be more or less acceptable right now in most states to have him drawn and quartered in the public square, with his union stewards forced to look on.
But he’s not. He’s the guy whose 2s ultimately will be used to evaluate teachers and schools who may or may not do a fine job teaching during those precious weeks of the year when they are not prepping for tests.
Beth Hawkins writes about education — from preschool through college and beyond — in Learning Curve. Hawkins has covered a variety of subjects for MinnPost since its launch in 2007, but has a particular passion for reporting on education. In addition to her conviction that education is one of today's most pressing issues on a policy level, she's motivated by her experience advocating for her two sons in great schools and not-so-great ones. Hawkins is the recipient of numerous national and regional awards, including several first-place Society of Professional Journalists Page One awards for investigative reporting and feature writing. Her work has appeared in More, Mother Jones, Minnesota Monthly and many other publications. She can be reached at bhawkins@minnpost.com.
Copyright © 2011 MinnPost.com
