What Can “No Child Left Behind” Teach Higher Education?

Introduction

In 1983, the release of the National Commission on Excellence in Education’s report, “A Nation at Risk” sparked national outrage about the state of our country’s education system. And rightfully so—its assessment was abysmal: 23 million Americans were identified as functionally illiterate; most 17-year-olds failed to possess “higher order intellectual skills”; and international comparisons put Americans dead last on 7 out of 19 academic tests.¹ These findings are widely recognized for galvanizing the country to think critically about measuring student success and holding schools accountable for their students’ outcomes for the  first time.

Most notably, the report spurred Congress to apply a newfound focus on transparency and accountability to the K-12 system (which spans from kindergarten through 12^th grade) that culminated in the 2001 rewrite of the Elementary and Secondary Education Act. More commonly known as “No Child Left Behind” (NCLB), this legislation made sweeping changes to the federal government’s oversight of K-12 schools, including requiring annual testing of students in grades 3 through 8, measuring achievement gaps for students of color, low-income students, English language learners (ELLs), and students with disabilities, and setting the ambitious benchmark that 100% of American students would be proficient in math and reading by the year 2014. And this effort was deeply bipartisan, with both parties maintaining a strong focus on common ground and the need to take a stand against failure throughout the process.

This approach was by no means perfect, and its flaws required Congress to work together once again in a bipartisan manner to make some significant course corrections in a later reauthorization of the bill in 2015. However, there’s no denying that its focus on student outcomes helped improve scores after decades of stagnation and brought measurable academic gains to the millions of vulnerable student populations who had been previously ignored by districts, states, and the federal government.²  

Despite the significant gains in accountability made in our K-12 system since NCLB, there has been almost no comparable federal effort to apply similar scrutiny to our higher education system in the three and a half decades following the release of “A Nation at Risk.”³ This is true even though the federal government spends nearly six times more to fund postsecondary education than it does on our country’s K-12 schools, and despite the fact that a vast number of colleges and higher education programs produce inexcusably poor results each year, including stagnant graduation rates over the last two decades.⁴ These problems aren’t just limited to the for-profit college sector, which has received the lion’s share of any higher education-related accountability focus to date. Today, half of students who start college of any kind fail to earn a degree. And at the average four-year private non-profit or public institution, one-third of students don’t earn more than a high-school graduate and one in five students don’t earn enough to repay their loans.⁵

That is why federal higher education policy currently finds itself at a crossroads, similar to where our K-12 system was nearly two decades ago. There is a rising discussion across both parties of the need for institutions to have some “skin-in-the-game” and to shift the focus towards “outcomes” over inputs. This is especially true as increasing college tuition and rising student debt are forcing more students and taxpayers to ask the question each year, “What am I getting from the higher education system in return for what I pay?” And while states have already begun these conversations in earnest through the development of performance-based funding systems that provide financial incentives to institutions based on key performance results, gaping holes remain in the way we measure and prioritize student outcomes at the federal level.

Luckily, Congress has an opportunity in the next reauthorization of the Higher Education Act (HEA) to change this trajectory and strengthen federal oversight for our country’s institutions of higher education. Doing so will ensure that more students actually graduate from college with the skills and degrees they need to get good jobs that will allow them to pay back their student debt. But as anyone who followed the lifespan of NCLB knows, implementing a large-scale accountability system can come with adverse consequences, especially if those systems are inflexible, set unachievable benchmarks, fail to put in place proper tools for measuring success, or don’t address the realities of an uneven playing field or challenges of capacity to improve from the start. This paper aims to highlight key lessons learned from the NCLB era in an effort to help Congress avoid the same pitfalls as it looks to strengthen accountability in higher education.

Lesson 1: It Matters What You Measure.

Prior to NCLB’s passage in 2001, the only accountability requirement for states to improve student performance in K-12 schools was to “set targets for continuous and substantial improvement.”⁶ With such vague instructions from the federal government, it was no surprise then to see that states set for themselves unclear and unambitious goals. For example, 17 states simply had no statewide accountability systems at all, and some states, like Mississippi, had nonsensical goals, such as the mathematically impossible feat of “decreasing the percentage of students scoring in the lowest quarter on state assessments.”⁷ NCLB set out to ensure that states had in place more academically rigorous and standardized performance goals by requiring them to measure both the number of students who tested “proficient” in math and reading on statewide assessments each year and student performance on at least one other academic indicator, such as graduation rates.

Where NCLB Went Wrong

While noble in its effort—and a huge improvement over the Wild West approach to accountability found in states before it was enacted—NCLB’s hyper-focus on math and reading proficiency came at the detriment of other subjects, and the high-stakes nature of testing quickly led to the criticism that NCLB took the fun out of learning by making school all about “teaching to the test.” And to some extent, this criticism was true. Studies found that the introduction of a federal requirement for annual testing led some states and districts to layer on unnecessary and duplicative tests, ultimately increasing the amount of time students spent taking assessments in a given year.⁸ In addition, because states had the flexibility under NCLB to set their own bars for proficiency on their annual tests and graduation rates, a majority of states actually chose to make their standards less rigorous than before NCLB to inflate their students’ overall performance and avoid additional scrutiny.⁹ Oftentimes, states and districts made the decision to teach to the test, cut other courses, and implement drill and kill pedagogy because they simply didn’t know what else to do. They lacked the capacity to raise scores through higher-quality curriculum, better teaching, and/or more student supports.

But as anyone who followed the lifespan of NCLB knows, implementing a large-scale accountability system can come with adverse consequences, especially if those systems are inflexible, set unachievable benchmarks, fail to put in place proper tools for measuring success, or don’t address the realities of an uneven playing field or challenges of capacity to improve from the start.

Lessons Learned for Higher Ed

Similarly, in higher education there is a lot of debate about what even constitutes success in college. We can measure a college’s value by looking at statistics like graduation rates, students’ future earnings, and their ability to pay back their loans, in addition to more difficult-to-measure metrics like self-sufficiency, belonging, and personal growth. To mitigate an over-reliance on one factor, it is important for any accountability system in higher education to take into account multiple measures that can provide a more robust and well-rounded picture of a college’s performance and the value add it is providing to its students. This is particularly true for one of the most commonly discussed proxies of college success on the table today—college completion. Improving graduation rates is critically important, considering that only half of students that start college today actually finish with a degree or credential of any kind.¹⁰ But similar to how measuring math and reading proficiency alone under NCLB created an incentive for schools to overemphasize those subjects and diminish their success by lowering the bar for proficiency, a system that measures only completion rates could have the unintended consequence of turning colleges into diploma mills that hand out meaningless degrees.

That is why much of the current accountability discussion in higher education focuses on also looking at factors that ostensibly do a better job than completion alone at indicating students’ preparation for today’s workforce. This includes pairing completion rates with factors like the percentage of students who get jobs, the percentage of students able to earn above a high school graduate ($25,000/year), and the percentage of students who can successfully pay down their student debt. But even focusing on just these “hard skills” may fail to fully capture the value an institution brings to a student’s life. Take, for example, Historically Black Colleges and Universities (HBCUs). Not only do HBCUs graduate a higher proportion of low-income students and students of color than predominantly white institutions, but they also do a significantly better job at making their students feel supported during and after graduation than their non-minority-serving peers.¹¹ In crafting policy that will hold institutions accountable for the outcomes of their students, Congress should explore ways to give colleges credit for some of the intangible outcomes they may provide in addition to the more academic- and economic-related factors currently on the table. And there should be conversations about the use of multiple metrics, as well as which metrics to use across sectors, with the understanding that a certificate program or community college may have different outcomes than a traditional four-year institution (conversations that many states are already having as part of their performance-based funding systems).¹²

Lesson 2: Growth Can Be As Important As The End Goal.

The alarming findings revealed in “A Nation at Risk” forced Congress to dramatically correct course and make clear that the United States was serious about improving its academic achievement in our K-12 schools. That is why NCLB swung for the fences by putting in place the highly ambitious goal that 100% of U.S. students would be on-or-above grade level in math and reading within a decade and a half of the law’s passage (that and the fact that all involved in crafting the legislation expected it to be revised long before 2014). Despite these good intentions, it became clear as 2014 quickly approached—and no update to the legislation to adjust these goals was in sight—that every single state would fall short of meeting this benchmark, and ultimately be subject to sanctions, as outlined in the law. As a result, the Department of Education began issuing waivers to states in 2012 to both give states flexibility to update their accountability goals and to encourage states to adopt the administration’s K-12 priorities, like teacher evaluations.¹³ However, some states like Washington failed to have their waivers approved in time, resulting in the publicity nightmare of the State Superintendent having to send every parent in the state a letter telling them their child was attending a “failing” school.¹⁴   

Where NCLB Went Wrong

There is no doubt that setting a 100% bar for proficiency was an untenable goal for states to ever meet, let alone within a 14-year timeframe. However, NCLB’s biggest flaw around goal setting was not its unrealistic focus on having every single student in a state reach proficiency but rather its reliance on measuring school performance by a static bar of proficiency alone, as opposed to also measuring growth (otherwise known as improvement). Even though NCLB did require schools to measure their “Adequate Yearly Progress” (AYP) towards reaching 100% proficiency each year, these interim benchmarks still did nothing to reward a school’s growth over time. Having this kind of binary system where a school either did or did not meet a predetermined annual goal did nothing to acknowledge the genuine improvement a school may have made during the year, even if it still fell short of the benchmark it was required to meet. For example, if a school had increased its number of proficient students in math and reading from 50% to 59% in one year, but still fell short of a 60% AYP goal, NCLB labeled that school a failure despite the clear progress it was able to make.

This type of proficiency-centric system was particularly punishing for schools that served a disproportionate number of high-needs students, as they may have had much more ground to gain each year to hit their proficiency targets than well-resourced schools that serve primarily wealthier students. This created a demoralizing game of “catch-up” for many schools, placing additional stresses on the districts and schools that needed support the most. Measuring proficiency alone also had the unintended consequence of forcing schools to focus primarily on students right above or below the threshold—ironically leaving behind students who may have already exceeded proficiency standards or students so far below grade level that any growth would likely still fall short of the static “proficiency” bar.¹⁵

Lessons Learned for Higher Ed

The limited accountability measures we have in higher education today at the federal level are already subject to the same pitfalls that plagued NCLB’s singular focus on proficiency. For example, setting one-size-fits-all outcomes (such as saying we want at least 80% of students to graduate) runs the risk of putting in place a bar that may not be achievable for all institutions, especially if they fail to take into account the challenges different institutions face, including receiving fewer federal and state resources or serving a higher share of at-risk students. In addition, having an “on-or-off” switch alone does nothing to capture the large number of colleges or programs that may fall right above a threshold but still have middling outcomes. Take, for example, the fact that over 1,700 programs still passed the Gainful Employment rule (a rule designed to measure a program’s debt-to-earnings ratio), even though a majority of their graduates were earning below the federal poverty line.¹⁶

Instead, any accountability system in higher education should move away from only setting cliffs that serve as inadequate proxies for quality and only measuring proficiency, which may miss a broad swath of institutions that are ripe for improvement. Rather than thinking of accountability options as either proficiency or growth, higher education should embrace models that both set a floor and incentivize growth. For example, this could include setting reasonable federal bottom lines for completion rates while still measuring and rewarding the progress institutions make to improve student outcomes year after year. We saw movement towards this approach with the 2015 rewrite of NCLB, as now states are required to identify and intervene in the bottom 5% of institutions as determined by a set of multiple measures, while still providing states flexibility to measure schools using student growth over time on those indicators. And to make sure accountability systems fairly recognize institutions that take in harder-to-serve populations, higher education accountability should include risk-adjusted measures (in some ways similar to the concept of value-added methodology in the K-12 space) that set reasonable growth targets for institutions based on the demographic of students that an institution serves. This would not excuse institutions simply because they serve low-income students, first-generation students, or students of color but recognize the resources and capacity required to serve these students well and take into account how institutions with similar demographics are getting wildly different results. Taking this kind of risk-adjusted approach would allow the federal government to set appropriate  guardrails as well as growth targets that can recognize and reward institutions for the true value they provide to their students each year.   

Lesson 3: Sticks Need Carrots, Too.

One of the most powerful tools Congress has to force an industry to alter its behavior is to write into law various “carrots” and “sticks” that will incentivize a desired change. This can include popular “carrots” like pay-for-performance bonuses or loosened regulations, or popular “sticks” like reduced investments or increased federal oversight. But finding a balance between such rewards and punishments can often be tricky—and if done incorrectly, it can result in unintended consequences, “gaming” of the system, or unfair burdens that ultimately cripple an industry altogether.

Where NCLB Went Wrong

Despite Congress’ good intentions, NCLB fell victim to this type of imbalance. Given the abysmal state of our nation’s schools and an absence of federal oversight up to that point, Congress used a heavy hand when crafting NCLB to instigate tough sanctions for schools that failed to meet its ambitious goals. For example, NCLB used what it called a “cascade of sanction,” marching schools toward closure if they didn’t turn around on unrealistically short timelines. Later, under the Obama administration’s Race to the Top initiative, the federal government began requiring states to implement one of four predetermined turnaround models for schools that missed consecutive years of their annual proficiency benchmarks, even if they showed growth but still fell short of meeting their progress goals. This included sanctions like forcing the school to hire a new principal and replace half of the staff; converting the school into a charter school; implementing a mandatory set of new strategies, like increasing the school day or lengthening professional development; or the most harsh penalty of them all—closing the school down altogether.¹⁷ By prescribing such a limited menu of punishments to failing schools, NCLB gave little flexibility to states to determine how best to handle low-performing schools and gave schools no real support to improve. Instead, this “my way or the highway” approach left some schools feeling demoralized and created a backlash within many communities, as no parent or teacher wanted to see a school in their own neighborhood close down—no matter how bad its performance seemed to be.¹⁸

Rather than thinking of accountability options as either proficiency or growth, higher education should embrace models that both set a floor and incentivize growth.

Lessons Learned for Higher Ed

As early conversations around how to implement accountability systems at the federal level in higher education heat up, it is already clear that policymakers and advocates are struggling to figure out what an appropriate balance of carrots and sticks could be to motivate and help colleges improve their results. And we’ve already seen the severe backlash that can occur from students, parents, and policymakers when a college is threatened with closure—even if that college is found to be financially insolvent or has abysmal student outcomes. Take, for example, the City College of San Francisco, whose accreditor in 2012 recommended its closure due to poor financial management and lax academic oversight.¹⁹ Rather than shutting its doors or putting in place a plan of intensive supports and necessary changes to right the ship, community stakeholders and policymakers instead engaged in years of legal battles to help keep the college open without any real improvements, as it remains today.²⁰ This hesitancy to impose the toughest sanctions on a school—cutting off its access to Title IV financial aid dollars—also explains why the Cohort Default Rate (one of the only accountability measures in higher education today) is only used to sanction a tiny handful of institutions each year.²¹ This is not to say that we should not have sanctions at all; however, penalties that do exist could include graduated sanctions for schools that do show progress year after year. Another way to avoid this all-or-nothing approach could be to focus not just on institution-level penalties, but also on program-level penalties that avoid shutting down an entire school and target penalties to the programs within an institution that make students worse off.

In addition, the federal government does almost nothing today to recognize and reward the colleges that are consistently doing their part to improve the lives of their students by taking in their fair share of low-income students and helping them graduate. As a result, colleges feel little pressure to increase completion rates, take in greater numbers of low-income students, or close equity gaps for the $130 billion federal investment they receive each year. That is why any new accountability system in higher education should also include “carrots” that reward institutions for meeting or exceeding designated performance metrics, similar to how performance-based funding systems operate within a handful of states.²² This is especially true since, unlike in K-12, institutions could choose to stop enrolling harder-to-serve populations altogether if the punishments of accountability appeared greater than the rewards. This could include incentives like financial bonuses for exceptional student outcomes or improvement on those measures, additional points on federal grant proposals, a new designation of “blue ribbon colleges,” or reduced regulatory burdens—especially for schools that have proven success with harder-to-serve populations.

Lesson 4: Accountability Needs Transparency.

It is difficult for any accountability system to achieve its goals if the metrics for success are incomplete, unclear, or not well targeted to measure what matters most from the start. But in addition to figuring out exactly what to measure, there must also be clear parameters about how to collect and disseminate information in a way that will fairly determine whether a school is on track to meet or exceed an accountability system’s prescribed objectives. That is why transparency is a critical success factor for any entity looking to improve student outcomes. However, there is often uncertainty around how to achieve transparency and accountability at the same time.

Where NCLB Went Wrong

This inherent tension played out clearly throughout the NCLB era, especially surrounding the issue of teacher quality. With new research emerging at that time highlighting teacher effectiveness as one of the biggest factors for improving student success, NCLB and later the Obama Administration’s “Race to the Top” competitive grant program, both heavily emphasized the importance of having a high-quality teacher in front of every classroom.²³ As a result, there was an effort to usher in rigorous teacher evaluation systems as a way to better identify the teachers who were having a more positive or negative impact on their students’ learning. However, the rapid emergence of teacher evaluation systems hit a major roadblock after it became clear that standardized tests—which states often required to make up “a significant or the most significant factor in teacher evaluations”—were imperfect measures. This was due to the inconsistencies in testing quality across districts and states, and a lack of capacity and resources to present testing data in a clear and consistent way, including for the large number of teachers in non-tested subjects.²⁴

The federal government does almost nothing today to recognize and reward the colleges that are consistently doing their part to improve the lives of their students by taking in their fair share of low-income students and helping them graduate.

The simultaneous introduction of new tests as part of the Common Core State Standards, as well as the failure to come to consensus on how test scores would be used prior to putting in place teacher evaluation systems—all in a high stakes environment—resulted in a severe backlash that threatened to derail teacher evaluations altogether. Some teachers were held accountable for the test scores of subjects they didn’t teach, while others strongly resisted the notion of using test scores as proxies for student learning altogether. As a result, what started as a well-intentioned effort to improve transparency and reward teachers for helping their students succeed instead became sullied by years of infighting over whether we should evaluate teachers at all (we should). This distraction came at the detriment of answering more important questions, such as “how can we best use student outcome data to support teachers?” or “how can we provide all students with the effective teachers they need?”²⁵

Lessons Learned for Higher Ed

To avoid a similar fate in higher education, policymakers debating the intricacies of any new accountability system must first ask themselves if the tools they have to measure institutional and student success can fairly and accurately capture an institution’s impact on students’ lives. Without increasing transparency around student outcomes data in higher education and making clear what metrics are being used and why, it will be impossible to fully implement an accountability system with the buy-in and support needed to make it work. We’ve already seen how having incomplete data in higher education—such as graduation rates that only measure first-time, full-time students or earnings data that only takes into account students who have received federal aid—has given many institutions an excuse to shun responsibility for poor student outcomes. (It is a common refrain used by institutions and those that represent them that the data is imperfect, and therefore we cannot hold them accountable for outcomes).²⁶ One of the clearest steps Congress could take to mitigate this concern would be to overturn the federal ban on reporting student-level data in the next HEA, so that at minimum, the data we do have paints a more fair and accurate picture of an institution’s outcomes.

Lesson 5: Averages Aren’t Sufficient.

With limited bandwidth, staff capacity, resources, and a strong culture of local control, it was rational for states to take the path of least resistance when it came to proving student success. That is why prior to NCLB, 48 states chose not to measure the performance of subgroups of students like English Language Learners, students of color, or students with disabilities, and instead looked only at the overall performance at individual schools. But measuring only the averages made it nearly impossible for any stakeholder to assess how well schools were serving our nation’s most vulnerable student populations. Too often, the data erased students and hid clear equity gaps.    

Where NCLB Went Right

While the above sections all highlight the miscalculations Congress made when setting up stronger accountability in NCLB, it is important to recognize that lawmakers did get critical components of the law right: including the requirement that states had to measure and report the performance of all students as well as the performance of individual subgroups of high-needs students. Shining a light on these student populations for the first time uncovered just how deep and pervasive equity gaps were in our K-12 system, even in the highest-performing districts in the nation. This also created pressure to put in place more targeted support for these underserved students—an effort that resulted in major progress towards closing these gaps. For example, when looking at National Assessment of Educational Progress (NAEP) scores from 1999-2012, the gaps in reading scores between white and African American students closed by 12 points, and the gap in math scores between white and Hispanic students closed by 8 points over this time.²⁷ High school dropout rates for African American and Hispanic students also dropped dramatically during this time.²⁸ While we still have a long way to go toward improving equity in our K-12 system, NCLB’s foresight to not just look at average student performance provided a much-needed and long overdue boost in helping states and districts start to make progress.

Any accountability system must be willing and able to look below the surface to encourage institutions to institute support systems to help all students succeed.

Lessons Learned for Higher Ed

There is no doubt that this same principle can and should be applied to any accountability system implemented in higher education. From the limited disaggregated data that we do have available, we already know that there are wide disparities in outcomes for different student populations across higher education. For example, Census data shows that people of color are half as likely to get a postsecondary degree as their peers, and that this gap has grown wider over the last 30 years.²⁹ New research from Judith Scott Clayton out of Harvard University finds that even when African American students do graduate, they are still more likely to default on their loans than white students who never earned a degree.³⁰ Similarly, a new study by the American Association of University Women finds that women hold almost two-thirds of all student loan debt and take longer to pay back their loans due to the gender pay gap.³¹

Given such inequities already clearly exist, we can no longer allow colleges to hide behind their averages. This is particularly true given what is at stake for students investing significant amounts of time and money to attend colleges in the hopes that it will better their lives. Students should be able to clearly see how likely it will be for a student like them to succeed at any given institution before they start. Our own research found that on average, Pell students graduate at a rate 18-percentage points lower than their non-Pell peers.³² Any accountability system must be willing and able to look below the surface to encourage institutions to institute support systems to help all students succeed. Holding colleges accountable for their performance overall, as well as with subgroups of students, will also bring much-needed attention to the institutions that are providing greater access to and success for vulnerable and high-need student populations. 

Conclusion

Thirty-five years after the National Commission on Excellence in Education wrote “A Nation at Risk,” there is finally an opportunity for higher education to have the quality overhaul it deserves. This includes recognizing the importance of setting high expectations, recognizing that schools are able to serve students better if they are motivated and equipped to do so, and having differentiated sanctions that don’t employ an all-or-nothing approach to accessing Title IV dollars. Similar to what we witnessed in the K-12 space, implementing new accountability mechanisms in higher education will likely require some course correction along the way. Our hope is that this post-mortem of NCLB will prevent policymakers from making some of the same mistakes on higher education, speed up our country’s ability to effectively recognize and reward the institutions that are serving students well, and dissuade or prevent students from attending the institutions that are not. Earning a degree or credential of some kind beyond high school is now a prerequisite for the lion’s share of good jobs in today’s economy. Our commitment to equipping students with the skills they need to succeed must not end once a student leaves our K-12 system, and neither should our responsibility to hold schools accountable for their students’ outcomes.