The newly proposed “All Children Can Achieve Act” includes evaluating “teacher effectiveness” based on increases in students’ test scores over a four-year period. Here’s a press release from Senator Norm Coleman, R-Minnesota:
<http://coleman.senate.gov/index.cfm?FuseAction=PressReleases.Detail&PressRelease_id=1365&Month=7&Year=2007>
Here’s a key statement from this release:
“One of the most important factors in school and student achievement is teachers. The quality of teachers should be determined by their effect on students’ learning, not just their qualifications. All students should have effective teachers. Thus, these data systems must link student achievement data to teachers, allowing states to measure teacher effectiveness.”
The bill does allude to increasingly “flexibility” in terms of relying solely on test scores as a measure of student achievement: “States should be held accountable for student achievement. However, students do not progress at the same pace or start in the same place. Thus, states are allowed the flexibility to measure student academic growth, rather than looking at absolute test scores. States are also encouraged to look at merit pay including getting the best teachers to teach in the poorest schools.”
However, the bill also refers to “comprehensive data systems”: “To ensure parents that all students are achieving, states must create comprehensive data systems that track students’ academic progress and other factors that affect their success.” My own reading of this is that existing reading/math test scores will remain as the primary measure of “achievement” and “data systems” in Minnesota, which has, unlike Nebraska and other states, demonstrated little interest in exploring alternative achievement measures for meeting NCLB mandates.
As teachers know, determining if a teacher is “effective” based on test scores ignores all of the other factors that influence those scores. It will also further perpetuate teaching for the test that already is having adverse effects on Minnesota schools.
Given the push for “teacher accountability,” there is a good possibility that NCLB revisions will include this provision, something that the NEA opposes. However, the public often assumes that teacher opposition to such provisions is only a defensive attempt to “protect themselves.”
Dick Allington, University of Tennessee, notes some of the problems with a growth model for determining teacher effectiveness:
1. Who is the teacher? Some RF kids see three different teachers during the school day (and get three different reading lessons from three different reading programs). So will every teacher be credited with proportional responsibility based on proportional minutes of instruction offered? This is only getting worse with the advent of RTI models.
2. What's the test? As Anne and I noted in our Kappan article last year, TN DOE allows the state test to be read aloud to pupils with disabilities This makes the teachers of these children look like miracle workers, at least in the first year. The modified tests are used in calculating state school grades and AYP. KY has a similar policy and probably other states. But FL has revoked the licenses of teachers and principals for doing the same thing. TX allows teachers to read the proper nouns aloud on state tests. So how can you calculate teacher effects if the read aloud to kids accommodation is allowed?
3. What happens when a kid is retained? How is progress going to be calculated? Against grade placement standards or against the grade standards the kid should be in? Remember that by mandating the flunking of some 35,000 kids, FL state and NAEP scores hit all time highs year before last. Now that those flunked kids are back in the 4th grade pool the state scores dropped dramatically.
4. What score will be assigned to a kid on a state test? Here I am thinking about the use of confidence intervals. Some states use straight score attainment, some use 95% confidence interval for determining who passes state test and some use a 99% confidence interval. In either case, using confidence intervals, which I recommend, means kids pass in those states that would fail to meet AYP in states that don't use confidence intervals.
5. Mobility is also a problem. The former accountability director for Philly schools published an analysis of Philly schools showing that some schools had enormous mobility in and out. Do you only count the students enrolled for the full year or any kid there on testing day? Such decisions will make a big difference. If only kids there all year, some teachers will be graded based on 5-6 kids. Maybe only kids from the most stable families. If every kid there on testing day counts then teachers will be graded based on 12-14 kids they only taught for part of the year, perhaps only a few weeks.
And all of this assumes state test are valid and reliable measures of reading development.
6. Most growth models ignore the well established phenomenon of summer reading setback that Anne and I have written fairly extensively about. The setback most often occurs and is largest for children from poor families. Unless the method requires twice yearly testing, fall and spring, the models ignore the fact that poor kids lose 3-4 months reading achievement every summer. Best analyses indicate that about 80% of the rich/poor achievement gap can be assigned to summer setback not to poor teaching. But if use of Fall-Spring tests in required, when teachers produce a year's growth poor kids continue to fall behind because of summer setback. Such a model would produce effective teachers when poor kids were two or more years behind by 5th grade.
Recent Comments