During their monthly meeting this week, Kansas State Board of Education members listened to a presentation regarding “The State Report Card.” This was when they heard that public schools in the state were doing a pretty good job. Scores in reading and math are going up. Science scores need some work. More schools are making Adequate Yearly Progress. Most teachers are highly qualified, although there is a shortage of highly qualified special education teachers. Attendance is good.
Board member Walt Chappell, however, questioned the accuracy of this report, which showed steady improvement over the past 11 years. After reminding the board that they had adopted new curriculum standards during this time period, he asked if it was accurate to compare the before-and-after test results. He wondered whether that was like comparing apples to oranges.
Chappell, who is from Wichita and represents District 8, has been a controversial member of the Kansas State Board of Education since his election in 2008. He often plays devil’s advocate, asking questions other board members do not think are important enough to spend time on during their day-long marathon meetings. Several times during this week’s meeting, the board’s chairman — David T. Willis — firmly directed Chappell to end a line of questioning no one else was interested in pursuing.
Among other questions, Chappell wondered why Kansas’ own assessments showed student improvement in reading and math, while the National Assessment of Educational Progress (NAEP) — which is administered by the U.S. Department of Education — shows Kansas students’ performance in these subjects has remained relatively unchanged since at least 2007.
According to Kansas’ education department, comparing the NAEP and Kansas assessment results is problematic. For example, state tests and the NAEP have different performance and content standards. In other words, NAEP could be testing Kansas students over information they have not yet had the opportunity to learn.
Another question Chappell asked was why the state’s scores on the ACT — a national college entrance examination — have not increased along with student performance on Kansas’ own assessments? Both test student knowledge of English, math, science and reading.
However, educators knowledgeable about testing maintain that results of different standardized test results are not necessarily comparable. That is one reason why many object to the idea of using assessments designed to test student skills and knowledge in order to judge the effectiveness of schools and teachers. This is not what these tests were designed to do. Assessments that are superficially similar may actually be measuring different skills and knowledge. For example, as the Kansas State Department of Education points out on their website, their tests and the NAEP measure different content. And the ACT is a test of the knowledge of college-bound students, not all students, which is the purpose of the state exams. Therefore, Chappell’s questions about the NAEP and ACT may have been asking state officials to make false comparisons.
However, he did ask that question about comparing apples and oranges.
The state’s curriculum did change during the 11 years covered by this year’s State Report Card. It is possible that comparisons of student academic performance before and after this change are not valid. Yet state officials maintain their statistics are accurate measures of improvement in reading and math over that entire time period. In another part of their report, though, Kansas officials refused to compare this year’s data with previous numbers. The way states calculate graduation rates changed this year at the direction of the U.S. Department of Education. Using the new calculation method, Kansas’ graduation rate isn’t as high as it used to be, and the state maintains that comparisons of graduation rates before and after the change are not valid.
As the Topeka Public Schools news blog explains it, “Since the way graduation from high school is calculated has changed, trying to compare this year’s graduation rate to last year’s would be like comparing apples to oranges.”
One might say that Chappell’s questions were “fruitless,” because the rest of the Kansas Board of Education did not share his concerns about the validity of the State Report Card. And — indeed — there are valid reasons for dismissing his concerns.
Perhaps the best lesson to take away from Chappell’s differences with the board during this week’s meeting was the importance of understanding the various ways that statistics and standardized testing can be used and understood.
In reporting this data on the State Report Card each fall, the Kansas Board of Education is fulfilling its mission, which is to “prepare Kansas students for lifelong success through rigorous academic instruction, 21st century career training, and character development according to each student’s gifts and talents.” Early next year, the education department will present an Accountability Report based in part on this data to the State Legislature, which will be making decisions about school funding and other education issues.
The Kansas State Board of Education met Tuesday, Oct. 11, in Topeka.
If you would like to read more about the Kansas State Report Card, here is a link to the Kansas City Star‘s article about this week’s meeting: http://www.kansascity.com/2011/10/11/3201812/kansas-students-show-greater-proficiency.html
And here is the Kansas State Department of Education’s own report: http://www.ksde.org/Default.aspx?tabid=36&ctl=Details&mid=1030&ItemID=569
Read our Twitter feed for more education news from local, regional and national sources: http://twitter.com/#!/JW_Martinez