How New Hampshire’s PACE is Redefining Measurement and Accountability

One of the biggest changes ESSA brings to education policy is a requirement that states start including a broader range of measures for student learning and school success in their accountability systems (you can see an approach Education First proposed in a recent design competition here). But even as every state redoes its accountability system to add more measures, the law also provides the opportunity for up to seven states (or groups of states) to experiment with using locally developed or performance assessments in place of state tests.

New Hampshire’s Performance Assessment of Competency Education (PACE)—a two-year pilot with eight districts that the U.S. Department of Education approved as part of its ESEA waivers—serves as an inspiration for what’s possible. In PACE districts, New Hampshire expects educators to judge and validate student learning using robust performance assessments they’ve developed, rather than relying only on an annual state test.

Many state policymakers from around the country are already looking to the Granite State for lessons about what to import into their own proposals for accountability innovations. With its unique mix of using Smarter Balanced to measure student performance for just three grade levels and district-developed performance assessments for other grades, some celebrate PACE as an opportunity to de-emphasize state testing while others worry about the impact of fewer state tests (and the objective snapshot of student learning they provide) on populations of low-performing students.

But the take-aways from PACE aren’t so obvious for those watching from afar. While the New Hampshire Department of Education has published all sorts of helpful details, information and analysis about PACE (see here, here and here, for example), I recently asked Paul Leather, the deputy state commissioner and a key architect of PACE, to reflect on what he’s learned so far and what advice he offers other states as they consider whether and what sort of accountability pilot of their own to pursue. His advice is below, followed by a few more reflections from me.

*  *  *  *  *

Briefly describe how PACE came about.

School district leaders who were driving our competency-based education reforms in the state told us the current accountability system was not helpful and actually impeded them in what they were trying to do. They wanted to see if there could be more flexibility. At that time, we had yet to develop a system of performance assessments to support competency education; the state was moving to a new large-scale assessment with Smarter Balanced but there were indications that it wasn’t going to give us as much performance assessment as we had hoped. Two years prior, Secretary Duncan indicated interest in giving New Hampshire the flexibility to try out a different approach, but he pushed us to make sure we had a system in place to ensure success. Over the last two years, we went back to our stakeholders here and built the system, which became PACE.

What has been the biggest obstacle you’ve faced with the PACE approach?

PACE requires transformative work from educators in how they deliver instruction, how they measure student learning and how they hold themselves accountable. It takes time and positive energy. A recent paper from the National Center on Education and the Economy looks at high-performing systems in British Columbia, Hong Kong, Shanghai and Singapore and finds they all have a learning and innovation cycle embedded in schools. PACE requires districts and educators to really commit to a similar kind of learning cycle. The lift is high. Districts that do not already have strong professional learning communities with strong innovation cycles struggle the most with the expectations of PACE.

Describe the sorts of performance assessment” students take during the years when they aren’t taking Smarter Balanced.

In grades 3 (English/language arts), 4 (math) and 8 (both subjects), the PACE districts use Smarter Balanced assessments, and in grade 11 they administer the SAT. For all other tested grades and subjects, there are 17 common, complex performance assessments the PACE districts have created together and use to measure student performance.

Don’t think about the performance assessments as in lieu of the state summative assessment; these common assessments are not the state assessment. These are common assessments that are used for calibration purposes to demonstrate reliable administration across schools and districts. We are trying to prove that local educators can be held accountable for their own assessments, using a system of checks and balances. This is such a different idea than what we normally think about when it comes to school accountability. We want to prove we can raise the level of rigor of local assessments so they become comparable to a more traditional state summative assessment.

The results from our first year of PACE showed a similar differentiation of student performance, just like the state test does. Essentially, the same percentage of students are demonstrating competence as we see in non-PACE districts on Smarter Balanced. In other words, the degree of rigor for the local assessments is being borne out. The common assessments use the same achievement level descriptors as Smarter Balanced.

Also, these performance assessments are not what you would typically see on a PARCC or Smarter Balanced test. They are based on real-world problems and ask students to explain, apply and model. Many go across discipline areas, and some include a mix of group activities and individual activities, so we can see skills like collaboration (although these success skills are not measured as a part of the state accountability process). Most take several weeks for students to complete and are embedded in the regular curriculum. They are not tests that are given statewide on a certain day.

What have been the biggest keys to success so far?

The ownership and engagement of the participating districts. They believe: This is our assessment and accountability system. They want to prove it can work. We’ve also learned that networking districts to work together on creating the assessments and strengthening teacher capacity and assessment literacy becomes greater than the parts. It’s the districts working together that makes PACE work. By having guardrails by which districts enter the pilot, we are able to control for effective implementation.

PACE began by engaging a set of high-performing school districts that started with lots of capacity and strong leadership. Scaling this effort statewide (and engaging lower-performing schools especially) is a major concern for NHDOE. Share some of your thinking about what it will take to make PACE successful statewide.

We created PACE’s three-tier system to provide different levels of support to New Hampshire districts. Tier 3 districts start by focusing on implementing competency-based education in a deeper way (such as changing grading and units of instruction and making greater use of project-based learning and performance tasks). Tier 2 districts receive support and training from other districts and the Center for Assessment to design performance assessments that meet high standards. Tier 1 districts participate in PACE’s new accountability structure. There are eight Tier 1 districts this school year, including the four original districts who led and designed PACE and four others who joined this school year (the second year of the pilot).

We think this tiered approach can help lower-performing districts succeed with PACE. For example, Manchester, the state’s largest district, has historically had many low performing schools—and it is now in Tier 2. What we’re finding is what the NCEE report found in other countries: Working in a network with higher-performing districts and educators pushes everyone to do better. And a focus on student work and looking at student work has a profound impact.

Several other states are eyeing the idea of designing an innovation pilot under ESSA and perhaps replicating some of what NH is doing. What advice do you have for them?

Start by creating an explicit theory of action about student learning, assessment and school improvement. Don’t cut corners. It took three years for districts in New Hampshire to be ready to use the PACE framework. Too many people don’t want to wait that long.

I used to think you could do a lighter version of PACE that didn’t require as much investment and work. I now believe that without the deep and rich engagement of teachers in transforming their work, the PACE districts could not have done what they have done. We have to invest significantly in educators. Our pilot requires effective educator judgments about how well students are doing. Those who want to replicate this work need to invest in professional learning and hold participants to high standards. Build the system deliberately and engage highly skilled technical partners.

*  *  *  *  *

What, then, is a policymaker interested in exploring accountability alternatives to do? While the U.S. Department of Education still needs to publish the regulations that will guide pilot proposals—so states should be careful about getting too far ahead in their thinking—I see three considerations:

  1. States need to articulate a more compelling reason and commitment to a new accountability approach than “we don’t like state tests or NCLB accountability.” New Hampshire’s approach is well-grounded in a strong state vision for instructional change and a long-standing commitment to competency-based education. States proposing a winning pilot will need their own compelling rationale for doing something different, backed up by a clear belief system. What goals would a different approach to accountability help meet? How does the new approach reinforce and match existing reform priorities? And how would a new approach still keep a focus on leaving no child left behind?
  2. Similarly, rethinking a new approach to school accountability is not for the faint of heart. As Paul points out, the work to support the PACE approach has been harder not easier than the status quo in New Hampshire, as the state now has even higher expectations for educators and a more substantial commitment to providing support and help. New Hampshire took three years to figure out the details of PACE and to ensure educators were ready to tackle their new responsibilities. Winning pilot states will need to show how they’ve thought carefully and invested deeply in making their approach successful.
  3. Attend to equity. This is an issue the U.S. Department of Education will certainly worry most about as it reviews pilot applications. Whether all districts can rise to the expectations of PACE is an issue New Hampshire leaders have explicitly struggled with. The very first, pioneering PACE districts were all high-capacity, high-performing systems with visionary leaders and more homogeneous student bodies. However, New Hampshire has been deliberate about building supports and networks to boost the capacity of other districts to succeed with the PACE framework, and they’ve deliberately recruited more challenged districts like Manchester. The burden will be on states who want to test a pilot to show how their approach can help all schools, especially those who are struggling the most, better attend to student learning and increase achievement.

You might also be interested in:

Helpful Resources

The High Quality Assessment Project

Over three years, Education First has supported grantmaking and technical assistance to a network of grantee local and state advocacy organizations as they engaged in the critical tasks of educating state boards, legislators governors and the media; identifying and supporting teacher advocates; and engaging parents as new tests are adopted and administered and as score reports are released.

Read More >

Our Vision for a Next-Generation School Accountability System

When Mike Petrilli of the Thomas B. Fordham Institute declared an “accountability design competition” on January 5, 2016, with a submission deadline of January 26...
Read more >

Read More >

Putting it All Together: Using Multiple Measures to Improve Teaching

Student achievement data can be used to help states and districts improve instruction, design professional development opportunities, and assess the quality of teacher preparation programs,...
Read more >

Read More >

Fewer and Better Local Assessments

It’s time for a more strategic approach to assessment. Schools, school districts and charter management organizations (CMOs) need to take a fresh look at what...
Read more >

Read More >