December 18, 2000
How Not To Do Education Testing
"Too busy with tests to teach?" asked a recent front-page headline in the Winnipeg Free Press. Grade 3 school teachers, the story explained, are spending so much time evaluating their students that they don't have enough left to cover the basic curriculum. This is a classic case of getting what you wished for . . . and regretting it.
Standardized testing, which was abolished in Manitoba high schools in 1968, worked. It provided an accurate picture of how well or poorly students were learning course material and basic skills. But high schools stopped conducting province-wide exams, and all schools were allowed to carry out their own testing. The prime mover of these "reforms", the Manitoba Teachers' Society, complained the old approach encouraged a mind-numbing conformity among students and compelled its members to "teach to the test."
Beginning in 1996, Minister of Education Linda McIntosh restored a limited regime of annual standards tests in Grades 3, 6, 9 and 12. The MTS launched a blistering campaign against the policy, arguing teachers should perform internal assessments on the progress of each student.
The New Democratic Party took up the suggestion and, when Drew Caldwell became Minister last year, he cancelled standardized testing and replaced it with a Comprehensive Assessment Program which involved individual diagnostic assessment tests. Including pupils from nursery school through Grade 3, its goal was to ensure that all of them could read, write and count after five years in the system.
What went wrong?
You can't find out by magic if students have acquired basic skills. Right off the bat, the MTS warned that teachers would have to spend about 90 minutes per child to conduct the assessments. Evaluating the average class would consume about 40 hours a year, time current schedules simply don't allow. As it turned out, a minimum of two hours per child is more realistic, plus two more to fill out the paperwork, although some schools are reporting even longer interruptions. One principal in Rosser was apparently so incensed by the burden that she sent the Minister an invoice for overtime.
The glitch points to what has become the consistent failure of many public policies: lack of transparency. An accurate, complete understanding of costs and benefits should precede a policy innovation, not follow it. During the brief re-instatement of standards testing, the Province covered the costs, which amounted to only a small percentage of our annual public school budget. The program offered participating teachers a chance to augment their incomes because they were paid separately for marking. The tests themselves absorbed only the amount of class time spent writing them. Now, however, the costs have been imposed without reimbursement on both teachers and divisions, the latter having to hire a substitute to head each class while the regular teacher is assessing.
The burden on teachers seems even more onerous because most of them don't appreciate the nature or benefits of testing and evaluation. The information provided on the subject by our university degree programs is remote from the teachers and scratchy, at best. This lapse in their training exists despite ample support in the academic literature for the usefulness of standardized tests.
Their ignorance breeds an unjustified fear in teachers. Since we spend a billion dollars a year teaching children, it follows that we would want to measure the effectiveness of our expenditure. We need to know if the cash is actually teaching young people to read, write, count and maybe even think.
What little data could be gathered from the short experiment with standards testing indicated that the public system is falling down badly on that job. This may explain the haste of the present government, which is heavily beholden to the public monopoly, in ditching performance measurement. The public record contains only brief, but troubling, glimpses of the test results. The total picture, which reflects a disastrous failure to teach basic skills, travelled under deep cover to individual divisions. For ten years, the University of Manitoba conducted its own assessments of enrolling students -- a program that was stopped when the Province put in standards tests. The results showed that public school grades reported by entrants were far above their actual skill levels.
What lessons can we learn from all this? First, and most important, without standardized testing there's no way to find out if kids are learning. Second, all public spending must undergo rigorous and constant performance measurement. Our focus on the political glow of inputs has led to another low-performance masterpiece, a 40-year decline in basic skills despite a tripling of real-dollar expenditures and crippling property tax levels.
Third, we need to recognize that the paralytic state of public schools has little to do with teachers' competence and a lot to do with the structural problems inherent in monopolies. Across the industrialized world, a renewal of basic education (the three r's)is gaining momentum, fuelled by an ambitious expansion of school choice, curriculum diversification and -- you guessed it -- standards testing. Most people agree that public funds should underwrite the cost of basic education, but that doesn't mean the government has to specify the nuts and bolts of the delivery system. The search for better practices benefits from the innovation spurred by competition and by the energy released through freeing schools to experiment.
The proper role for politicians and bureaucrats in this process is that of "supercop." Test every child every year, and report the results. Don't impose more burdens on teachers, but separate the program from the providers it is partly designed to measure. Use the results to recognize superior pedagogy and point failing schools in the right direction. Tests need not be the enemy of teachers; they can be their best friend.
The MTS now says that the program of internal assessments should have been piloted in a small number of schools. But that would have left most teachers in the province without the means to measure the success of their efforts. They may rightly wonder if their leadership, which steered them into this impasse, knows what it's talking about.
The Frontier Centre for Public Policy
is an independent public policy think tank whose mission is "to broaden the debate on our future through public policy research and education and to explore positive changes within our public institutions that support economic growth and opportunity."