APR Scores: What They Are and What They Miss
Missouri’s APR Scores
By: Kristi L. Donaldson, Ph.D. & Evan Rhinesmith, Ph.D.
The Missouri Department of Elementary and Secondary Education evaluates and accredits its public schools using Annual Performance Report scores (APRs). APRs are used to determine a district’s accreditation status and charter schools’ renewal and closure decisions. APR scores are calculated using the Missouri School Improvement Plan (MSIP) performance standards, which are currently in their fifth iteration. As Missouri prepares for the sixth iteration of MSIP and the planned release of 2019 APRs in November, we are providing a look at Missouri’s current way of calculating the APR. Specifically, we show there is a heavy emphasis on achievement and status measures.
APR Components and Accreditation
Under MSIP 5, 2018 APR scores used the following standards:
Academic Achievement, measured by English Language Arts (ELA), math, and social studies Missouri Assessment Program scores and End-of-Course assessments.
Subgroup Achievement, measured in the same ways as academic achievement, but for specific groups (i.e., race-ethnicity, free and reduced-price lunch status, English language learners, and students with disabilities).
College and Career Readiness (K-12 districts), measured by assessments like the ACT/SAT; Advanced Placement, International Baccalaureate, and Dual Enrollment courses; and the 180-day high school graduate follow-up; OR High School Readiness (K-8 districts), measured by the percentage of students who earn a proficient score on one or more End-of-Course exams prior to high school.
Attendance Rate, measured by the percentage of students who regularly attend school. To earn full points for attendance, 90 percent of students must attend 90 percent of the time.
Graduation Rate (K-12 districts only), measured by the percentage of students who graduate high school.
Districts earn points in each category (Table 1). APRs are the percentage of points earned of total points possible. For the 2018 APR, K-8 districts could earn up to 60 points, and K-12 districts could earn up to 120 points. The full APR includes achievement on science tests, but these were excluded as the current science assessments are in the pilot phase (Section 161.855.4, RSMo).
Districts’ APR percentage then determines their accreditation status for the year. The Department of Elementary and Secondary Education makes an accreditation recommendation to the State Board of Education using the current year’s accreditation status and two prior years of APR performance (alongside other factors like financial status).
Source: MSIP 5 Comprehensive Guide, pg. 58
Charter schools receive APRs just like traditional districts, though their accreditation does not depend on their APRs. Instead, charter school sponsors use APR scores to determine renewal and closure decisions. Much like K-8 districts held to only four of the five standards, charter schools with varying grade configurations are also only held accountable to specific standards. For instance, a charter school serving only grades K-2 receives an APR score based solely on the attendance standard and is held responsible for other aspects of their contract from their sponsor.
APR Issues
The largest percentage of total APR points comes from the first standard, Academic Achievement (Figure 1). ELA and math drive most of this (Table 1). Fifty-three percent of K-8 districts’ and about 27 percent of K-12 districts’ total APR points come from performance in these two subject areas. K-12 districts can also earn a quarter of their APR points from college and career readiness, and another quarter from graduation rates.
Points in each standard are calculated from a combination of Status, Progress, and Growth measures.
Status: the overall percentage of the school or district that met the standard, averaged over three years.
Progress: a measure of the district’s or school’s continuous improvement based on a rolling average of three years of data.*
Growth: the change in individual students’ achievement scores between two points in time. Each student’s growth score is used to create an overall district or school effect estimate.
All three measures of Status, Progress, and Growth are used to calculate scores for Academic and Subgroup Achievement (standards one and two). The remaining standards use only Status and Progress measures (Table 2).
While APR is meant to reward a combination of Status, Progress, and Growth, currently, districts can earn all of their APR points based on Status alone. As long as districts meet the goal for the standard, even if their students perform worse than the previous year, they will not be penalized in the current APR. As the Table below shows, districts with high achievement Status across all five standards can earn 100 percent of APR points.
MSIP sets goals for each standard. Target, On Track, Approaching, and Floor are Status designations that indicate how close the district is to the set goal. If a district meets or exceeds the goal, they earn a “Target” designation and earn all Status points in the particular standard. On Track and Approaching are intermediary categories, indicating districts are on track to achieving the goal in the future, or close to it. The last category, Floor, reflects districts that have a long way to go before they reach the standard. For example, the Target for Attendance, the fourth standard, is 90 percent or more students attending school 90 percent of the time (sometimes referred to as the 90-90 rule). Target districts are 90 percent or higher, On Track districts are between 85-89.9 percent, Approaching districts are between 80-84.9 percent, and Floor districts are below 79.9 percent.
Districts with low Status (“Floor”) and high Growth or Progress are only able to earn 75 percent of APR points. While 75 percent is above the provisionally accredited level, there is little room for error. Low-Status districts would need to be in the highest Progress or Growth categories in all standards to reach the 75 percent mark.
Conclusion
We’ve provided a 30,000-foot overview of the components of APR and what goes into each of these measures used to judge Missouri’s school district and charter school performance. However, there are many nuances, details, and—for lack of a better term—flaws in the APR system. The most obvious issue is how much Progress and Growth are undervalued compared to Status in current APR calculations. An accountability system that more strongly prioritizes growth would provide a closer look at how much schools and districts are actually helping students learn throughout the year. The PRiME Center will be closely tracking this with the MSIP 6 standards currently under review and how districts performed this past year when the 2019 APRS are released later this year.
————————————————————————-
*In the case of Status and Progress measures, if three years of data are not available, fewer than three years are used for reporting purposes. If three consecutive years of data are not available, the three most recent are used for accountability purposes.