A propos of our discussions on assessment, evaluation, and research, this article appears in the Dept of Education website:

November 14, 2006 — “NCLB Achieves Its Top Goal — Accountability” appeared in the Wisconsin State Journal
When he signed the bipartisan No Child Left Behind Act into law, President Bush said the laws “first principle is accountability.”

Five years later, its a good time to hold the law itself accountable. Is it doing what it promised? Is it working for Wisconsins and Americas kids?

The answer, on both counts, is yes.

No Child Left Behind was the nations collective statement that every child can learn and must be taught.

I encourage you to read the whole thing because I think it really outlines some of the more problematic issues.

The main problem is spelled out in this excerpt — “The first principle is accountability.” My heartburn here is that I believe the “first principle” should be learning. While I appreciate the point that one needs to have some kind of yardstick — a scoring mechanism, as it were — the implementation of accountability in the law is badly flawed.

First, holding schools — and only schools — accountable to student learning is missing a critical reality. Learning — and, by extention, Education — is a complex system that extends outside of the school. A whole range of factors have an influence on learning. Most of them are out of the control of the teacher. Parently socio-economic standing has long been known to be a factor. Recent studies indicate that it’s less about how much the family earns as it is about where poor families tend to live — in areas that cannot afford to provide schools with good facilities, current books, and up to date equipment. Parental and student involvement are also critical factors. Studies have long shown that simple things like parents reading habits have large implications on a child’s performance in school. None of these factors are considered in the accountability plans.

Second, the methods of measuring accountability are badly flawed. The basic method is to test the students’ knowledge against a state standard curriculum. These tests are alleged to be criterion-referenced and not norm-referenced, so that they are intended to measure mastery. The problem is that - by the government’s on research criteria - the tests being used have not been validated using the evidence-based research required by the law. In Colorado, for example, the Colorado State Assessment Program (CSAP) tests have been shown to be more tests of reading skill than tests of subject mastery. The method — of establishing Acceptable Yearly Progress based on grade level — is also misguided. Comparing the skill mastery of this year’s third graders against last year’s and trying to draw any conclusions about the effectiveness of any school is silly. Doing this kind of comparison on a manufacturing line — this batch against the last batch for quality control purposes — has some validity because the input parameters of the production process can assure a relatively stable and homogeneous baseline. The same cannot be said for children where the “raw materials” in the process can have very wide ranges of skills, abilities, and backgrounds.

The underlying problem, of course, is that actually dealing with making sure that no child is left behind is a complex and difficult problem which cannot be solved by simplistic means. We measure achievement with a standardized test because it’s relatively easy even though we know that the tests are not valid assessments of a child’s learning. We use a flawed methodology because it’s a standard practice in one context even though we know that the educational context shares few (if any) of the original. Then we draw conclusions based on the findings.

The net effect is like that old joke. A man walking down the street at night finds a man frantically searching around the base of a streetlight. “What ya looking for?” asks the first man.

“My contact lens!” answers the second.

After about twenty minutes of fruitless searching the first man asks, “Are you sure you lost it here?”

“Oh, no. I lost it down the block, but the light’s better here.”

By choosing to measure things that are convient using methods that are familiar, regardless of the applicability in our context, we’re doomed to never find the contact lens because we’re looking in the wrong place.

This entry was posted on Thursday, November 16th, 2006 at 9:31 am and is filed under Education. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.