.
Testing students is very widespread by those of us in (or formerly in) teaching. We use the results to measure how well our students have done their job and learned what we were teaching them. However, as a body we have vehemently opposed using the results on any standardized tests as an indication of how well WE were doing our job.
It appears that that has changed.
The NEA has finally come around to the view that how much your students learn might be a legitimate measure of how well you are teaching.
The next step is to get them to agree that those who do it better should get paid more and those who don't do it well should be removed from the classroom. (or do it without their agreement)
.
USA PLUTOCRACY CONSTITUTION (CARTOON)
2 hours ago
I weep for that future utopia that has optimized the educational process through the metric of standardized tests. It is self evident that the great advances in human society almost never arise from "standard" or orthodox responses to the challenges of the time.
ReplyDeleteObviously it is a leap on my part to think standardized testing and "out of the box" thinking are not correlated, but my intuition argues that they are not.
So I guess I join those who would doubt that standardized testing is necessarily a good thing.
I don't know who Michael is, but I agree with his comment. As a teacher with 40+ years experience, I am virtually certain that the best learning is NOT demonstrated on standardized tests (most of what we study and "learn" for such tests is very soon forgotten) but occurs through reading, writing and thinking, which is best evaluated in oral and written speech that involves contact between a student and a teacher. This, however, is very inefficient and so is being replaced by the lesser or short term "learning" that can be easily evaluated quantitatively and digitally. And I weep for that.
ReplyDeleteI agree with both Michael and KW. I work in an industry where a professional certification is almost universally required. To get the certification an applicant must pass a standardized test. Being able to pass the test and get certified provides only a loose correlation in being able to actually perform in the business world.
ReplyDeleteIn my profession (and for the most part the private sector) when an employee cannot perform on the job we simply invite them not to report to work tomorrow. I’m not sure that is a common practice with teachers.
One additional thought. If credentials and degrees, mostly based on standardized test, are essential to qualifying for a teaching position, it would seem slightly askew to say they have no value in the evaluation process once they have been hired.
My experiences in teaching began in the Fall of 1965 in the Northeast Houston Independent School System at a junior high school. I did not have a college degree, but the school suffered about a 45% annual turnover in faculty at the time so any warm body that could keep the "students" of 9th grade general math in the the classroom most of the day would be a welcome asset. I received $3100 for the full 9 month academic year.
ReplyDeleteMy "warehousing" skills were probably uppermost on the administrator's mind when I was hired and they would not think of firing me until they had a replacement in hand. That I was one German class short of a degree in Mathematics sealed the deal.
A lot of learning took place in my classroom that year, mostly by the "teacher." Some people don't have the wit and temperament to be in that job for very long, but with new teachers you aren't going to know for quite a while. One lesson they must learn quickly is that a teacher must measure out their "relationship" with their students over several months. One probably cannot put too high a value on acting skills.
Paddling by a bull-necked Assistant Principal was a frightening thing to witness and astonishing in the brevity of its effect.
The striking thing for me is the clear recollection that many of the teachers I met in Houston and in the Dallas Independent School District were marginally accomplished in their field in the period from 1965 to 1968. And this was at the height of the Post-Sputnik/Race-to -the-moon era.
The National Assessment of Educational Progress was begun around 1970 with a battery of standardized student achievement tests. Despite all the changes in technology and teaching methodology over the course of the 40 years from then until now, student scores in mathematics have varied little from the approximate 300 mean that was measured in the earliest tests.
What has changed dramatically is that in 1960 about 40% of the average high school graduating class went to college. The percentage in the early 2000's was approaching 75 or 80%, if I recall correctly. Clearly one would expect that if you only had students from the top 40% their performance should be superior to the average performance of the top 80% and the perception would be that performance capabilities of incoming college freshmen had diminished on average over the period from the 60's to the present.
However, it would not be true that the performance of the educators had diminished, because they seemingly produce essentially the same performance over the 40 years the NAEP achievement tests have been given. Teachers do not, after all, teach "intelligence," so it should not be surprising that attainment levels remain fairly constant for average slices of the general population.
Michael, your last point that the NEAP test scores indicate that educators have delivered the same performance over the last 40 years suggests a couple of questions.
ReplyDelete1. 40 years is approximately the life span of the US DOE. After 40 years is a result of “the same” acceptable or is that a negative indictment of the DOE’s performance or effectiveness?
2. Does the community of educators consider “the same” an accurate assessment and do they consider “the same” acceptable?
Some things seem harsh in writing that would not even be noticed in verbal conversation and it is not my intention to be that.
Under contract I facilitate approximately 4-6 adult classes per year for an international certifying body. I can provide feedback on the course material, but I have no direct control of that material and my contract says I will deliver the material as written, classes are designed as knowledge transfer tools and are monitored by the certifying body to insure the instructor is not teaching to the exam, students are polled post class/exam and asked about the instructor including whether the instructor furnished any exam answers or contacted them in any way, I proctor the exams but I do not grade the exams, the certifying body will tell me my aggregate pass rate, but not who passed/failed so other that what I can observe in the class room I get no feedback related to a particular student.
We cannot teach intelligent either, however, every new instructor must have an acceptable pass rate and to remain in the instructor core, over time, your pass rate must improve over past performance (student pass %). Over the 11 years I have been working with this group I have seen literally 100s of instructor come and go and as harsh as this system may be I can also report that the 12 - 15 instructors (worldwide) that have managed to retain their status as instructors are by no means average.
This system was put in place in an attempt to have the best possible instructors in front of the group and makes no pretence of being fair. As you might imagine my support level concerning arguments for retaining/releasing educators based on seniority, tenure, last hired, union membership, or that student scores do not (in a meaningful way) reflect the educator’s competence is somewhat reserved.
Very interesting comments.
ReplyDeleteI pretty much agree with what was in the responses, but some of the responses did not deal with the central question of the post which is this: Should how much students learn from a teacher be part of the evaluation of that teacher? Should how well a teacher does the job that they are hired to do be part of their professional evaluation?
I think that the NEA’s position (widely followed) has been absolutely not. They have preferred using seniority and some variation of “credentialization” as the only factors in the pay scale.
I believe that the fact that they are now agreeable to using student performance as part of faculty evaluation is a welcome change.
So I wonder whether Michael and KW agree.
I would not presume to speak for the community of educators in judging the significance of the NAEP scores.
ReplyDeleteThat the scores remain relatively constant I assume to be meaningful in the sense that if the measurement tools were continually being adjusted to reflect a current political viewpoint then I don't know what their relevance would be in judging the current state of the educational product.
While public schools are fairly rigidly controlled by their respective administrations, as a college professor at a state university I work with students from a large variety of sources and there is little direct control or assessment of what I do with students aside from student evaluation forms and success/failure rates.
Funding being directly dependent on enrollment probably tells you all you really need to know about what practical parameters are used to judge faculty.
Your description of the quality control procedures involved in your classes bears little resemblance to what actually happens in my circumstance.
I teach 10 classes a year at an essentially open admissions state university. Courses range from remedial math up through what passes for graduate mathematics here. Because remediation has become a larger portion of our service to the university community, in recent years I have been teaching more basic and intermediate algebra classes.
Unlike your situation, I design and implement the materials that I teach. I do this because remedial math has been an especially difficult subject to teach with satisfactory success rates. Like that first class that I taught so many years ago, I learn a great deal in these classes, but I also know that some of my students are actually learning more.
To return to your two questions, if the purpose of the DOE is to assess and help improve education effectiveness, then the first mandate must be to assess accurately.
Until you do that there is little likelihood that you can go on to the other. As KW noted there is a strong tendency to find some measure that is produces distinguishable results easily and to go with that rather than worry over hard to distinguish parameters, ... grading for spelling and punctuation is easy, but grading for creative writing skills is difficult.
Do the NAEP tests assess only rudimentary skills or do they assess more subjective higher order skills? Honestly, I can't say, but until a measurement can do both, then I would very much doubt that assessment has been done as well as it needs to be done to rely on entirely. If DOE did nothing more than that it would be a worthy enterprise.
The job of actually improving how certain subjects are learned is not really the job of the DOE. It is the job of those of us that actually do the "teaching." Once we have found a better way, an entity like DOE or a book publisher can help disseminate that information. However, many teachers are a walking contradiction: they expect their students to learn a new thing everyday, but they think they only needed to learn new things when they were students. Thus widely implemented change will come slowly.
While I don't speak for anyone but myself, I don't think NAEP tests are the final standard for mathematics nor do I think that it was intended to be. I think it was just meant to assess basic skills. That is important in and of itself.
Mathematics has been taught certainly for thousands, perhaps even tens or hundreds of thousands, of years ... ever since that first person instructed a child to go gather as many fruit as the fingers on their hand. Have we improved on that?
I hope so, but I actually had a student last spring who apologized because, she said, she had taught herself since her school was so bad, ... she did her arithmetic on her fingers.
YA,
ReplyDeleteIt kind of reminds me of that old saw "appropriate use of technology" that was tossed around in reference to the latest calculators. Everyone agreed that the calculators were handy gadgets and made some tedious tasks much easier, but they also hedged because they knew that some calculators were dangerously powerful.
As a part of assessment the tests might have a role, but administrators too often fall in love with easy to obtain scales. Assessing qualitative skills such as true problem solving is much more difficult to assess reliably since most people have good days and bad days and others simply do not test well. Besides timed tests may just be a poor vehicle for assessing some skills.
I have complained for years that if the administration really wanted to assess their faculty, then they should insist that their representative, the head of the department in our system, to frequently visit faculty classes.
The last time I had a supervisor in one of my classes was in 1965. I appreciate the vote of confidence, but it seems a strange way to run a system.
Beyond that I have thought for several years that the real correction for assessing both students and teachers lies in the direction of a different sort of grading scheme. Since we now can easily profile every class along any set of parameters we deem important, we should set the basic performance mean and distribution for a class and only allow the teacher to define the rank of the students in such a way that the basic mean and distribution has not changed. Giving high grades to improve evaluations would end.
Statistical evidence that later performance of the students was improved or declined would indicate that the teacher deserved commendation or correction. Later standardized testing could be a factor also if such test could also address the hard to test more subjective qualitative skills, but I am not familiar with tests that do that.
Well, some more comments on the comments: no, it is not he job of the DOE to evaluate teaching, but yes, teachers should be evaluated to some extent and in some responsible ways on what students learn. The problem is finding the right way to do that. Supervisors visiting classes is OK, so long as there is more than one visit per year. However, how does one fairly judge the skill of a creative writing teacher, for example? Such a teacher can only take what students write and critique it, perhaps with the aid of other class members, as was the case in a CW class I took. The teacher cannot make the student improve, but can only tell him/her if he/she has improved. Students at a CW class at Yale may be more likely to improve than students in such a class at Murray State. So is that the teacher's "fault"?? It seems to me that much of what a good teacher does--especially in the Humanities classes--is irremedially subjective.
ReplyDeleteOf course, that suggests another form of evaluation, doesn't it--a subjective one. I have been a party to situations in which someone whom "everyone knew was a bad teacher" was denied tenure, but have also seen situations in which such a person who already had tenure was not even warned or seen his/her salary cut?
And I have seen good teachers denied tenure for reasons that had little to do with their teaching.
All this suggests to me that in the matter of evaluation, we do not need bureaucratic systems fostered by politicians or by the NEA, but rather the cultivation of people who are willing to make professional judgments about the quality of the work done by their colleagues and who are supported by those above them in the hierarchy.
Sounds utopian, doesn't it. Reminds me of my high school speech teacher who made us memorize Quintilian's definition of a good speaker. "A good speaker is a good man skilled in the art of speaking."
A good teacher (or evaluator of teachers)is a good person skilled in the art of teaching--or evaluation.
I re-read the article on which this blog was based and it covers most of the thoughts expressed in the comments on this blog. However, there is one other means of educator evaluation, common in my situation, but not mentioned in the article and that is to ask the students. It has been my experience that student evaluations are, for the most, part sincere and based on a realistic perception.
ReplyDelete