a strong voice for kids 2nd Congressional District
a strong voice for kids 2nd Congressional District
Colorado is rushing headlong into the third generation of education reform, and yet it hasn’t completed the second—or even the first! Despite a slow and thoughtful start, reform efforts in the past few years have lurched forward without a steady or coherent plan. It’s a little like a child growing up too fast. Not surprisingly, some people have begun to refer to it as educational “deform”!
The first step taken in Colorado for education reform, House Bill 93-1313, was well thought out and allowed for careful reflection and slow implementation. The state adopted model content standards in 12 content areas, to be implemented in two “tiers,” the first of which included reading, writing, math, science, geography, and history. Each school district then adopted its own content standards that had to “meet or exceed” the state model standards. The Standards and Assessments Development and Implementation (SADI) Council, a group of educational experts, wrote the model standards. Over the course of two years, the SADI Council issued several drafts of the standards and sought input from people around the state before submitting the final drafts to the Colorado State Board of Education (CSBE) for approval. Districts then had another year to implement the standards. This slow, deliberative process ensured that educators and the public had sufficient “buy-in” to the standards. Unfortunately, it was the only stage of reform that did.
The implementation of assessments was not done coherently or deliberatively or in accordance with how the Act was written in 1993. The state was originally supposed to do assessments in all content areas by means of the CSAP (Colorado Student Assessment Program) – to be done randomly (i.e., in random districts each year) in grades 4, 8, and 11, in order to “check” that districts had implemented the standards. However, when it came time for the Legislature to fund the CSAPs, the decision was made to give the tests to all students in all districts in grades 4, 8, and 10. Because of the high cost, the tests were to be given in only the first four subjects of the first tier; as a result, the districts were given the responsibility for testing the other 8 content areas, as well as developing their own tests for the main 4 areas in years in which the CSAP was not given in their district. The CSAP implementation was to be phased in over several years, which indeed it was, but by the second year that state funding was needed to pay for the tests, the Legislature moved the math assessment 5th and 8th grades, and later the science to only 8th grade. For a while, every year there was a change in which CSAPs were to be given and to which students. Furthermore, the SADI Council was abandoned, leaving the writing of the CSAPs to the vendor chosen by the . No longer would there be model assessments for districts to base their own on; eventually the interpretation of HB 1313 changed regarding the requirement that districts evaluate in the other content areas changed, so that now tests are not necessarily required.
Additions made in the late 1990s added numerous other dimensions to the confusing state of the first generation of reform. First was the Third Grade Literacy Act, which, among other things, added a CSAP in reading in 3rd grade. Then came the Educational Accreditation Act, which required the CSBE to base the accreditation of schools and districts upon specific criteria, the most prominent of which was the CSAP. Superintendents and other education professionals around the state questioned the validity of measuring schools’ and districts’ effectiveness based on a few “snapshots” taken by CSAPs given intermittently to different cohorts of students. And the rollout of the CSAPs hadn’t even reached the middle school or high school levels at this time!
Nevertheless, in the 2000 session of the General Assembly, the second generation of education reform was enacted, Senate Bill 186. Part of this legislation addressed the concern about the “snapshot” aspect of CSAP by requiring that CSAPs be given yearly (but only in reading, writing, and math, with the lonely science CSAP just in 8th grade). The key aspect of SB 186, the part that created a whole new generation of reform, was of course the School Accountability Reports (SARs), requiring schools to be ranked and graded on their CSAP scores. SB 186 was neither enacted nor implemented through a slow, deliberative process. The bill of more than 100 pages was rushed through the Legislature; for example, dozens of vital amendments were given only 20 minutes of consideration in the House. And before the first computations for the school ratings on the SARs were even begun, the Legislature made changes to SB 186 in its regular session of 2001, changing the “grades” to “academic performance ratings,” adding categories of students who were exempted from the ratings, and including the results of the CSAPs given in Spanish.
When the process for producing the SARs was finally clear, the Colorado Department of Education (CDE) had just a few weeks to enter all the last-minute changes into a never-before-used program and do the complex calculations for the school ratings. This unrealistically short timeline was necessary because the CSAP results were not available until July, and the information for the SARs had to be sent to the printer by August so that the SARs could get to the school districts by September. During those few summer weeks, CDE’s new computer went through the intricate process of doing the following:
Dividing schools into the categories of elementary, middle, and/or high school was not an easy task, given the variety of grade levels in schools around the state. Many “elementary” schools got ratings for both elementary and middle, as did “middle” schools, and many “high” schools got ratings for both middle and high schools. Criticisms of the determinations for the divisions were voiced, and it is likely that the Legislature will make changes during the 2002 session.
Also, it appears that when the first list of ratings was produced, Commissioner Moloney decided to take 29 schools out and have the ratings re-calculated. He never gave exact reasons for doing this, simply claiming that they were “alternative” schools, although many other schools with the same characteristics as the 29 were not taken out. One of the most dramatic examples of an alternative school that was not exempted is the Gilpin Montessori School, where the school’s rating was based on the CSAP scores of only one student (this school, by the way, received a rating of “excellent”).
The fact that the process for determining the final weighted score upon which schools’ ratings are based is so complicated means that schools cannot easily determine if their rating is accurate. The “formula” is touted as clear-cut, but many schools have alleged that their score is wrong. CDE does not provide a “road map” through the calculation for each individual school, so there is no way for anyone to check to see if their score was figured properly.
Another complication is that the method used to convert CSAP results into “weighted scores” for ranking and grading the schools is generally not well understood. Many people, including those in the press, refer to the schools’ “academic performance ratings” (SAR language) as the schools’ “CSAP scores.” Assumptions are made that a school’s rating of “excellent” means that all of its students are “proficient” or “advanced,” when in fact all the ratings show is how schools compare with the others. The ratings are norm-based measurements of a standards-based system. It isn’t surprising that people are confused.
And still the complete rollout of CSAPs in all grades and subjects had not occurred when the SARs were produced in 2001! Nevertheless, SB 186 set 2001 as the “baseline” year.
To make matters worse, SB 186 requires the SARs in 2002 to include an improvement grade, based on a comparison of the school’s weighted score in 2002 compared with 2001. In 2002, there will be 8 new CSAPs given (writing in grades 3, 5, 6, 8, and 9, and math in grades 5, 7, and 9), and the scaling for all the tests in each content area will be redone to make them consistent. Thus, the total weighted score for each school in 2002 will be based on more tests than in 2001, ensuring that the comparison won’t be based on identical information. It also will be based on one grade’s worth of a different cohort of students. With these aspects, the improvement grades will certainly be an inaccurate picture of the extent to which improvement or decline has taken place in the schools.
With an eye to the problem of comparing different cohorts of students, SB 01-129 contained a component that launched the state into the first part of the third generation of education reform: a longitudinal analysis and reporting system (LARS) that allows tracking of individual students from year to year. The thinking behind this is based on William Sanders’ “value-added” research, which allows one to see the “value” that has been added to students from year to year in knowledge gained through their education. The members of the State Board of Education favor the longitudinal approach to evaluating schools’ success, as evidenced by the Rules for Accreditation of Schools and Districts written in 2001, which will base their accreditation on showing that students make “reasonable progress over time.” SB 129 gives CDE the funding to establish a student tracking system, and it requires the to make rules establishing the procedure for how this will be done.
While the CSBE considers and various education groups debate how the longitudinal analysis should be done, a state legislator is in the process of introducing a bill to push LARS forward even further. His proposal is to have the SARs show a grade for each school’s “academic growth” based on a longitudinal analysis and to provide a report for each student showing his or her individual academic growth. Although everyone seems to agree that the determination of longitudinal growth should be based on CSAP scores, there are still big problems. For one thing, the experts believe that the CSAP tests are not good enough for diagnosis of students at the highest and lowest ends of achievement. Another concern is whether to base evaluation of longitudinal growth on the large four performance categories of CSAP (advanced, proficient, partially proficient, and unsatisfactory) or to look at growth in finer increments within the categories. In either case, some testing experts maintain that the CSAPs do not have enough questions in each particular standard/content area to warrant using only CSAP for a valid evaluation of longitudinal growth. The Sanders project uses other assessments, but incorporating them into the SAR scores would be impossible. Thus, it appears that the LARS will have to be based on only partial information.
Comparisons of different cohorts, incomplete data, rushed and error-ridden reporting, changes in the process every year – given these problems, it is clear that as long as we continue to rush through education reform without having worked out the “bugs,” we are destined to persist in the awkward, messy state we are currently in. That doesn’t bode well for the future of education reform in Colorado. I am constantly reminded of a guiding principle in the computer world: GIGO (garbage in, garbage out). What concerns me most is the extent to which we have impacted the lives of our school children in this unfortunate process.
Copyright © 2001-2007 Evie Hudak - All Rights Reserved