IBO Fiscal Brief.eps

 

February 2016

New York State Student Achievement Test Results
:
New York City Public Schools No Longer Lag Rest of the State

PDF version available here.

Summary


In early August, the State Education Department released the results of the 2015 English Language Arts and mathematics tests administered in grades 3 through 8 in spring of 2015. In the city, critics have emphasized the low performance of the city’s traditional public schools—with only 30.4 percent of tested students showing proficiency in English Language Arts and 35.2 percent doing so in math.

IBO has analyzed the test results for every public school in New York State, both traditional public schools and charter schools, and we see a more nuanced picture. The 2015 test results not only show continued improvement in student proficiency rates in city schools, but also the continuation of a slow and steady shift in the relative performance of city schools compared with statewide results—a shift that has been underway across dramatic changes to the state’s testing program and across city mayoral administrations. Among our findings:

  • The performance of city students now is essentially the same as those in the rest of the state in English Language Arts and less than 3 percentage points behind the state average in math.

  • This is dramatically different than the relative standing of the city and state in 2006, when the city lagged the state by 11 points in English and 9 points in math. (The state testing program was overhauled beginning in 2006; comparisons cannot be readily made to prior years.)

  • New York City-based charter schools outperform schools throughout the state when test results are adjusted to control for demographic differences.

  • Nearly 330 schools throughout the state—both traditional public schools and charter schools—remain at the lowest levels of achievement, with fewer than 10 percent of students reaching proficiency in English Language Arts.

This report is structured around five observations that are critical to understanding these test results.

The Tests Used Since 2013 Are Designed to Be More Challenging Than Previous State Tests

The State Education Department has consistently emphasized that the tests introduced in 2013 are designed to hold students to a much higher level of proficiency than the previous testing program. A recent report in Education Next cited New York State among the top 10 states to set its proficiency bar at a level that is equivalent to or tougher than the standard set by the National Assessment of Educational Progress (NAEP)—long considered the gold standard of assessment of student achievement across the country. The higher standards imposed in 2013 need to be considered when parsing both state and city proficiency results.

It is useful to compare the results of the state test to those of the New York State results on the 2015 NAEP—administered in grades 4 and 8. In English Language Arts (ELA), proficiency rates on the state grade 4 ELA were 3 points lower than on the NAEP Reading; in grade 8, the state test results were 2 points higher than NAEP. In math, the state test results were 8 points higher in grade 4. The results of the state’s grade 8 math scores are not comparable to NAEP because the state test offers an exemption to high achievers who take a Regents exam instead.

Controlling for Differences in Student Characteristics, NYC Schools—Traditional Public and Charter—Are Doing Better Than Those in the Rest of the State

Based on the 2015 New York State test results for grades 3 through 8 for each traditional public and charter school in the state, New York City appeared to lag behind schools in the rest of the state in terms of raw proficiency rates for ELA and math. In ELA, 31.2 percent of students in the city’s traditional public and charter schools tested proficient, whereas 32.5 percent of students in the rest of the state tested proficient. In math the gap was greater: 37.1 percent of students in New York City schools tested proficient compared with 40.5 percent of students in the rest of the state. Comparing these proficiency rates, test results for the city’s students had proficiency rates that were 1.3 percentage points to 3.4 percentage points below those for students in the rest of the state.

These raw differences, however, do not account for the fact that test results for schools are strongly related to the demographic characteristics of their students. By controlling for these differences statistically, we can estimate school performance in light of the demographic differences across schools or groups of schools. This allowed us to compare the relative performance of New York City’s schools, including the Department of Education’s traditional public schools and charter schools, to those in the rest of the state. The results showed that New York City schools are actually doing better than schools in the rest of the state.

IBO analyzed the ELA test results for all 3,553 public schools (district and charter) across New York State, where we had complete demographic data; for our analysis of math test results the sample was slightly smaller, consisting of 3,546 public schools. The demographic variables we controlled for were: the share of students who are economically disadvantaged; the share of students with disabilities; ethnicity, measured as the share of Asian, black, and Hispanic students relative to whites; and gender, measured as the share of females.

Effects of School Demographics on Proficiency Rates

 

Percentage Points

English

Language Arts

Math

Positive Effects

 

 

Percent Asian

26.6

32.5

Percent Female

20.7

7.5

Being a New York City School

(Traditional or Charter)

14.1

15.2

Negative Effects

 

 

Percent Economically Disadvantaged

-44.7

-45.2

Percent Students

With Disabilities

-43.9

-77.8

Percent Black

-6.8

-13.8

Percent Hispanic

-5.9

-8.5

NOTES: Controls for differences in student characteristics. All reported results are statistically significant at the 5 percent level except for the effect for percent female on math proficiency rates, which was statistically significant at the 10 percent level.

New York City Independent Budget Office

With these controls in place, the model was able to account for 68.1 percent of the variation in school ELA proficiency rates and 62.4 percent of the variation in math proficiency rates. All seven variables had a statistically significant effect on ELA and math proficiency rates, although the percent female was only statistically significant at the 10 percent level for math proficiency rates (see table below).

The impact of being a school in New York City was both considerable and statistically significant. After controlling for demographics, we found a 14.1 percentage point advantage in ELA and a 15.2 percentage point advantage in math for the average New York City public school, including charter schools, over the average school in the rest of the state. While the direction of the effects for the demographic variables was similar for ELA and math (positive versus negative), the effects were larger in magnitude in math for all demographic groups except for females (which was only marginally significant).

Charter Schools in New York City Do Better Than Traditional Public Schools

Once again controlling for student characteristics, IBO also identified the New York City advantage based on whether students attended a charter school or a traditional public school. Again, performance was gauged against schools in the rest of New York State (all schools outside of the city). After controlling for demographics, traditional public schools in New York City have a 13.1 percentage point advantage over the rest of the state on ELA and a 12.5 percentage point advantage on math proficiency. The gaps are larger for charter schools in the city: 18.8 percentage points on ELA and 30.1 percentage points on math.

As is the case with traditional public schools, there is a wide range of performance across charter schools. Among the 20 charter schools in new York City with ELA proficiency rates above 50 percent, 12 are part of the Success Academy network, four are part of the ICAHN network, three other schools are each part of different networks (Family Life Academy Charters, Classical Charters, and Uncommon Schools), and one is an independent charter (The Neighborhood Charter School of Harlem). The 20 New York City charter schools with math proficiency rates above 70 percent are mostly the same schools as those with high ELA rates. Among the other highest-achieving charter schools in math are Children’s Aid College Prep and Leadership Prep Ocean Hill.

 

Schools at the Very Lowest Achievement Levels Are Found in Every Part of the State and Include Traditional Public and Charter Schools

A considerable number of schools remain at the lowest levels of achievement. Across the state, 329 schools have ELA proficiency rates below 10 percent. Of these, 161 are traditional public schools in New York City. District-run schools in the rest of the state include 154 such schools. Finally, 14 charter schools are among these low-achievement schools, including 7 located in New York City.

NAEP Trends Also Indicate That New York City Has Narrowed the Gap With New York State, Especially for Eighth Graders

Despite a dip in fourth grade NAEP proficiency rates for New York City schools in 2015, the gap between New York City and New York State has narrowed over time, most significantly for eighth graders. Note that for NAEP results, we are not able to control for demographic differences, so these results are as reported. The gap in eighth grade reading proficiency rates has been halved over 12 years—from 13 percentage points to 6 percentage points. The gap in eighth grade math proficiency rates has shrunk even more—from 12 percentage points to 4 percentage points. The gaps for fourth graders shrank by much less—2 percentage points in reading and 3 percentage points in math.

Importantly, New York State is found to be at or around the national average on the NAEP. Thus, while the New York City proficiency rates are low when compared with where the state has decided students need to be, it is also true that on the ELA test, city students largely match the results for New York State, and the NAEP tells us that New York State is performing one point above the national average in grade 4 ELA and at the national average in grade 8. For math, the city performs slightly below the state, and the state’s NAEP results are 4 points below the national average in grade 4 and 1 point below in grade 8.

Conclusion

Comparing the proficiency rates for students in New York City with those of students in all of New York State over time shows that the city has considerably narrowed the gap with the state, although the city still trails. This trend is evident when looking at both the state tests and on the NAEP. When we control for demographic differences across districts in New York State, however, we find that the city is actually doing better than the rest of the state on the state tests.

Disaggregating the effects for New York City traditional public schools from charter schools, we find that charter schools have a greater advantage than district-run schools, relative to the rest of the state, particularly in math proficiency. Among both charter schools and traditional public schools, it is important to note that performance on the state tests varies widely within those groups. This variability, in terms of both performance and the demographics of students they serve, are important considerations in efforts to replicate and scale up high-achieving schools in both the Department of Education and charter sectors.

This report prepared by Sarita Subramanian
and Joydeep Roy

PDF version available here.

Receive notification of free reports by e-mail

Facebook

Twitter

RSS