Capitol Chronicle
Would You Like to Swing on a Star? The Educational Test Mavens Test Everybody but Themselves
By Michael King, Fri., Nov. 9, 2001
The JFTK report, a centerpiece in that phalanx, is essentially an upgraded analysis of the data produced by the Texas Education Agency through its Texas Assessment of Academic Skills (TAAS) test. JFTK has reworked the TAAS numbers for nearly all Texas schools, purporting to deliver "the most accurate and comprehensive ranking ... ever produced" -- or so trumpets the current issue of Texas Monthly. (I've no doubt, by the way, that in 20 years the Chronicle has published some cheesy cover stories -- but a borrowed and prettied-up list based on some highly theoretical tap-dancing with an already dubious set of state agency numbers? In its own movie-review breathlessness, this Monthly flatulence, "How Does Your Kid's School Stack Up?" earns no better than one star.)
JFTK claims that their massaging of the data produces much more accuracy than straightforward state TAAS rankings for two reasons: (1) It ranks schools by the number of students showing "proficiency" (an 85% TAAS score) rather than "passing" (a 70% score); and (2) It presumably controls for socioeconomic differences. That is, schools with similar numbers of poor or disadvantaged (non-English-speaking) students are ranked against each other rather than against more privileged schools. S.C. Gwynne writes in the Monthly, "If schools were ranked only with their true counterparts and it turned out that some were achieving good results and others were not, the underperforming schools would have no excuse for failure." Sounds reasonable enough -- although even without looking at the data, one wonders whether a relative "success" in these highly qualified terms can mean very much.
But can a careful reader -- say, a parent -- gather sufficient information from these tabulated lists to enable her to "forget about those TAAS rankings you see in the paper"? Can she take a look at these stars, passing rates, percentages, and "opportunity gaps" and feel confident that her child's school -- with its three, four, or even five Monthly-approved stars -- is above-average in its educational performance? Or are the report cards issued by Just for the Kids rather less than meets the eye?
Swingin' on a Star
Frankly, in this instance more might not be less -- but it doesn't seem to be all that much more, either. Consider only the rankings for Austin and there is little to suggest that these JFTK rankings are dramatically different from the standard TAAS results. Anderson and LBJ High score well, Johnston and Crockett stumble. Embattled Kealing Middle School gets five stars, while Mendez and Covington each get one. Will one star from Texas Monthly and an "opportunity gap" of -31 help a parent in the Pearce neighborhood decide if she should send her child to the neighborhood public school, or else jump through various and sundry AISD hoops in the hope that, say, Kealing will find a home for her budding scholar in the already oversubscribed magnet program? And do those five stars for Kealing mean that it's a "better" school for little Johnny than, say, O. Henry, which only garnered three? Certainly the often polarized Kealing community needs all the good news it can get -- but high average scores on the TAAS test may say a great deal or little about whether a school is truly educating its children, or will do so with yours.
AISD spokesman Andy Welch says the district found "no big surprise" in the JFTK numbers for Austin, and that they essentially echo what AISD's own analysts say about the schools' progress. "We're better off than we were in 1999 [when Superintendent Pat Forgione was hired]," said Welch, "but we're still not where we need to be." Brad Duggan, executive director of Just for the Kids, says these rankings are much better indicators of future student success than are standard TAAS results, and that regular universal testing now produces "longitudinal data" allowing administrators and teachers to identify and help students lagging behind. "We're not testing for testing's sake," said Duggan. "We're producing a common measure of performance that will allow problem areas to be identified, and a teamwork approach to solve those problems."
This Is Not a Test
Maybe he's right. But for me the most discouraging passage in Gwynne's story is his conclusion: the "success" profile of Hambrick Middle School in the "high-performing Aldine district" (north of Houston). "The best schools," Gwynne reports, "seem to be the ones that give the most tests." I'm delighted that the learning progress of Hambrick's mostly minority students is closely monitored by their teachers and that roughly 85% of Hambrick's eighth-graders are "proficient" on the TAAS. But that thinking adults who have themselves matriculated from any educational system at all should consider it a sign of "progress" to conclude, "The best schools seem to be the ones that give the most tests," seems both a self-fulfilling prophecy and a failure of public imagination on a nationwide scale.
Associate professor Angela Valenzuela of UT's School of Education and the Center for Mexican-American Studies is skeptical of the state's increasing emphasis on single-number educational standards, like the TAAS. "I'd like to ask these legislators and these real estate agents [who sell property by pointing to school rankings]," she said, "What would that particular number be, that would precisely capture the particular quality of your performance? But we allow for children things that we would never allow for ourselves."
Of course, for a real estate agent, that "particular number" would undoubtedly be the ubiquitous "bottom line." The Legislature's interim education committees have begun once again their struggle to find the equitable funding that would raise the Texas educational system out of the 19th century. I wonder how many of the more than 100 corporate underwriters donating $1,000 or more to Just for the Kids -- from Advanced Micro Devices through Enron to Wells Fargo -- will be showing up soon in Austin, to say that they've seen the light, they've answered the call, they've taken the test -- and now they understand that without a fair and progressive system of taxation, Texas cannot hope to provide a constitutionally mandated, fair and effective educational system to all its children.
When that happens, we'll declare a national holiday -- and give 'em all five stars.
Got something to say on the subject? Send a letter to the editor.
