One of the problems we face it how to asses whether our students learned what we intended them to learn. In a classroom setting, we give tests and quizzes. We give homework that has to be passed in. We assign projects and grade them. But we also look to see who’s keeping up with classroom discussion and who’s actually making cogent commentary. We tend to think that, for the most part, the student in our class did the work if we can see them doing it. We tend to evaluate homework with an eye toward, “Does this sound like Johnny?”
We tend to think of this as valid assessment strategy. Is it?
One of the problems is that the creation of tests and quizzes tends to fall into one of two camps. First, you use the publisher provided tests associated with a book or curriculum. Second, you create your own. The problem is that the instrumentation in either case has not necessarily been tested for validity or reliability. When it comes to the state-wide assessment tests this becomes an even greater problem. If the instrumentation is not valid, or not reliable, then you cannot draw conclusions about the outcomes you measure with it.
To illustrate this problem, suppose you need to find out how much each of your students weighs. You’re given an instrument by the school and told to use it. On test day, you dutifully line the students up, use the instrument according to directions, record the results and discover that all your students scored within a few tenths of 98.6 on the scale. The problem, of course, is that the school gave you a thermometer to measure weight. From our perspective we know a thermometer doesn’t measure weight so the use of this instrument, while reliably measuring something, isn’t measuring what we intend it to. Knowing what we know about weight, temperature, and the instrumentation needed to measure them, we can say pretty emphatically that using a thermometer to measure weight is silly.
But how do you know if your classroom tests are measuring what you intend? We don’t have the experiential knowledge or background information that would allow us to say, “This is silly!” How do we know when it comes to assessing our students learning?
Reliability is the other issue here. Any instrument needs to measure what we think and it needs to do it with some predictable measure of reliability. Reliability is one of those slippery constructs because it’s difficult to really appreciate all the things that could go wrong with reliability. Take the idea of measuring weight. All scales are not as reliable as others. My bathroom scale varies by as much as 20 pounds based on where I stand on tthe platform. It is measuring what I think it is, but the results are not consistent. I can’t say with any degree of confidence what my real weight (mass, really) might be. You have the same problem with your classroom assessments. Assuming validity (which you are, but shouldn’t be) then how do you know it’s working the same way for all students? Does the ESL student have any special problems? How about the SPED student? Does it work the same way with boys and girls? How about with morning or afternoon sessions? Do you know?
October 24th, 2007 at 8:09 am
[...] This echoes a question from the comments on Lexie’s blog: phaedrus » Blog Archive » On Assessment One of the problems we face it how to asses whether our students learned what we intended them to learn. In a classroom setting, we give tests and quizzes. We give homework that has to be passed in. We assign projects and grade them. But we also look to see who’s keeping up with classroom discussion and who’s actually making cogent commentary. We tend to think that, for the most part, the student in our class did the work if we can see them doing it. We tend to evaluate homework with an eye toward, “Does this sound like Johnny?” [...]
October 27th, 2007 at 4:04 pm
In my opinion I do not think that quizzes and test are a valid way to measure what a student knows. I for one have test anxiety, so almost every test I take, I fail. I know the material and have studied multiple times, but I can not score well on test. Having students complete projects is another way to measure what a student knows, but how do you know they completly understood what they were suppose to do. If they don’t understand then they can not complete the project to the fullest. What if they missed a part of the lesson or didn’t understand all the lessons associated with the project? Then they can’t complete the assignment to its fullest.
I can tell you first hand that assessments that I give do not measure what we intend for them to know. My students struggle all together and giving them a test to assess what they know does not work. They can not retain the information given and they are totally lost.
If a test is valid how do we know it is valid for all students? We don’t. Many test that my students are forced to take throughout the year by the state are not measuring what the students really know. We give a GRADE test three times a year and my students (SPED) are not allowed to have accomodations on this test. Many of my students can comprehend what is read to them, but they can not read. I think this is a bunch of crap that we are taking time away from teaching to test students without accomodations. My students look as if they are making no progress from this test.