EdAction
Maple River Education Coalition PAC
105 Peavey Rd, St 116 
Chaska, MN  55318 
952-361-4931
http://www.EdAction.org
E-mail

June 9,  2001

Recalibrating the Tests

The first article is from Texas regarding their assessment system, called TAAS (Texas Assessment of Academic Skills). The results have been "recalibrated," meaning that students have to give fewer correct answers to get the same percentage score. The reason given is that some questions are weighted as being "more difficult."

In the second article, the same kind of "recalibration" is being reported in North Carolina. Clearly, such manipulations undermine the entire credibility of basic skills test.

The same manipulations are being used in Minnesota's Basic Skills Tests. The MN Department of Children, Families and Learning calls it "scaled scoring." Kentucky is showing evidence of manipulation of scoring results, as well.

All of this represents the failure of a centrally planned federal education system. Outcomes are micromanaged by an education bureaucracy to produce the required results, one way or another. Inevitably, definitions of success and "performance indicators" will be massaged to demonstrate progress.

Historically, that kind of corruption is always the result of central planning, whether in education, workforce planning and management, or production of goods. It is the opposite of freedom. There was a specific reason that our founders included the 10th amendment to the U.S. Constitution. It was to protect future generations against just this type of takeover, leading to corruption. A constitution, however, is of little value unless it is taken seriously by its citizens and defended against encroaching powers.

The Texas Essential Knowledge and Skills (TEKS) is the equivalent of Minnesota's Profile of Learning "state standards." Keep in mind as you read this, that the federal testing of HR1 uses Texas testing as its model. In truth, the Texas model is not substantially different from Minnesota's model: the Profile of Learning, the Basic Skills Tests and the Minnesota Comprehensive Assessments (MCA's).

Bar for Passing TAAS Lowered Cutoff Scores for Math Test Debated

06/09/2001
By Terrence Stutz / The Dallas Morning News

AUSTIN - Texas students - widely praised for their recent performance on the state's basic skills exam - were graded with a lower passing standard in 2001 than in previous years, test documents show.

The 1.9 million students tested in math, reading and other subjects this spring were required to correctly answer significantly fewer math questions to pass the high-stakes Texas Assessment of Academic Skills.

In most of the grades tested, students had to get only about half the math questions right. Two years ago, they had to get about 70 percent of the questions correct.

Preliminary TAAS scores

Critics said the lower passing standard calls into question the much-touted improvements on the TAAS this year. Texas Education Agency officials said they lowered the cutoff score to pass because the test was harder than in previous years.

This year's TAAS included more questions reflecting new curriculum standards - the Texas Essential Knowledge and Skills - that took effect in fall 1998.

State Education Commissioner Jim Nelson said the math cutoff scores were adjusted this year so that test results could be accurately compared with results for past years. He also insisted the higher passing rate on the exam this year is real.

"The TEKS are much broader, and as a result, the questions were more difficult and students had more trouble with them," the commissioner said.

"It was a policy decision [to adjust the cutoff scores]. We needed to keep the [testing] system level."

For example, sixth graders had to get 28 of 56 math answers right, compared with last year's requirement of 34 of 52 questions. In 1998 and 1999, they had to get 38 correct responses out of 56 questions - 10 more than this year.

Passing standards changed only slightly on the reading section of the TAAS even though new curriculum standards for reading also were reflected in the exam along with the new math standards.

In sixth-grade reading on the TAAS, students have needed 27 of 40 questions right to pass for three consecutive years, including this year's exam.

Critic's response

Walt Haney, a Boston College professor and testing expert and critic of Texas' testing program, said the changes this year have made the TAAS results suspect.

"It is very, very unusual to see that kind of change in the passing scores from one year to another based on recalibrations," Dr. Haney said.

"In most testing programs, when there is a change from one year to the next you may see a passing level bounce by up to 5 percent of the test items, but this change [on the math portion of the TAAS] is three times that much. You just don't see changes of this magnitude."

Dr. Haney said even "if you buy their argument that they have changed the math test dramatically," that would mean that this year's scores are not truly comparable to last year's because one of the fundamental premises of equating annual tests is that they must have equivalent content.

He also pointed to the downward trend in the passing standard in recent years.

The Texas Education Agency reported last month that 82 percent of students passed the TAAS this year, the eighth consecutive year in which the passing percentage rose.

Last year, 80 percent passed. In the two years before that - 1998 and 1999 - the passing percentages were 73 percent and 78 percent, respectively.

At a recent hearing before a legislative committee, Mr. Nelson said the gains on the TAAS are valid despite the changes in the cutoff scores to pass math.

"If a kid in 1997 gave a certain effort, he got a certain score. And if he gave the same effort last year, he got the same score," the commissioner said.

While many people think that passing the TAAS requires the student to answer at least 70 percent of all test items correctly, that is not the case.

Complex system

For several years, the education agency has used the Texas Learning Index - a complex system to evaluate the level of difficulty of each test - to ensure that annual TAAS results are comparable from year to year.

Because the overall difficulty of each TAAS test is only approximately equal to versions in other years, the number of items a student must answer correctly to pass can vary slightly from year to year. Those decisions are based on field testing of questions and other analysis making up the index.

For example, one year's math test might be found to contain a few easier questions than the test given a year before, so more of the items must be answered correctly on the newer exam to pass. That helps make sure the level of difficulty is the same on both tests.

From 1998 through 2000, the adjustments made under the index were minor in a majority of cases.

But this year, the number of correct answers that were necessary to pass in math dropped sharply across all grade levels.

The TAAS is administered to all students in grades three to eight and grade 10. Reading and math skills are tested at all grade levels, writing skills are tested in grades four, eight and 10, and science and social studies are tested in eighth grade.

"Math had a sizable shift this year," said Debbie Graves Ratcliffe, a spokeswoman for the Texas Education Agency.

She attributed the lower passing cutoff scores in math to more difficult questions that require higher order thinking skills than previous versions of the TAAS.

"We are starting to see more algebra and some geometry on the TAAS," she said.

Dr. Haney, on the staff of the Center for the Study of Testing, Evaluation and Education Policy at Boston College, sees nothing wrong with making the exam more difficult. But, he said, problems arise when the state tries to adjust the scoring and then says there have been improvements in student achievement.

"The lower minimum scores to pass [math] could be a factor in the increased percentage of students who passed the TAAS this year," he said, noting that passing rates typically would be expected to drop with the introduction of tougher test items.

The new TAAS

The Texas Assessment of Academic Skills is scheduled to be replaced with a new version - the TAAS II - in spring 2003. The new exam is expected to be more difficult, and school superintendents have been warned that passing rates likely will drop across all grade levels. That same year, third-graders will be required to pass the reading section of TAAS II to be promoted to fourth grade. That requirement is the first phase of former Gov. George W. Bush's plan to stamp out social promotion - the practice of automatically passing students regardless of achievement - in Texas schools. The 1999 Legislature approved the new promotion standards.

CER NEWSWIRE
Vol. 3, No. 23
June 5, 2001

* TESTING: The argument for a national benchmark against state tests was never made better than by the North Carolina fifth grade end-of-grade math test this year. This year's edition of the exam, which was not statistically validated, was revised because so many students didn't do well last year. The result was an exam so easy that kids could pass it by answering only 28 percent of the problems correctly, compared with 45 percent last year. Random guessing of the four possible answers would generate a 25 percent correct score.

EdAction - 105 Peavey Rd, Ste 116, Chaska, MN  55318 
952-361-4931 - edaction@lakes.com - (c) EdAction - All rights reserved.