Growth & Justice Blog

High-stakes tests don’t raise achievement

April marks the start of the state-wide education testing season. While 4th-8th graders complete the Minnesota Comprehensive Assessments (MCA-II’s), the class of 2010 faces the strictest high school tests yet — the Graduation Required Assessment for Graduation (GRAD), a series of reading, writing and math tests. 

The GRAD 11th grade math test, however, has created a problem. Roughly half of Minnesota’s students are predicted to fail the exam, potentially denying them a diploma if they cannot successfully pass the test by the end of their senior year. Such a widespread consequence has been deemed unacceptable, at least politically, sending politicians and policymakers back to the testing drawing board.  

The good news, from the students’ perspective, is that the GRAD math test will likely never be a graduation requirement in its current form. The GRAD also has sparked more debate over how testing could affect student learning for better and for worse, but without really illuminating a fundamental issue related to the state’s low test scores. 

Different types of tests are designed to accomplish very different ends. In the nomenclature of the testing industry, a test, regardless of its content, has either a summative or formative purpose. A formative test is any assessment — quiz, test, assignment, etc. — that can inform a teacher’s future instruction. It’s an evaluation designed for improving student achievement.     

The GRAD, however, is a summative test — designed not to inform teachers, but to quantify what a large groups of students has learned over the course of the entire school year. An individual’s score becomes one of thousands of data points used to rank and compare schools, districts or even states as a whole for accountability purposes. 

Here is the crux of the issue: we assume that a summative test like the GRAD — when paired with severe consequences, like denying a diploma to a student — can fix a formative problem.

Recent research is proving the futility of this approach. A review of 30 years worth of state testing data [Download PDF], conducted by the University of Minnesota, found no instance of high-stakes tests raising student achievement. Worse, not only do students face harsh consequences if they fail, passing the test leads to no added benefits: no scholarships, no wage increases, nothing beyond what a regular pre-test diploma would have afforded.      

Unfortunately, there is an inverse relationship between the utility of high-stakes summative testing and its political appeal. The tests seek to replace the complex contradictions and paradoxes of educating students with a black-and-white definition of success and failure, while shifting the onus of success solely onto the students themselves.

The easy refrain becomes: if lazy, underperforming teenagers can’t perform in school, how can we expect them to perform later in life? The tough medicine of the GRAD —  and withholding diplomas from underperforming students, even in the face of high failure rates —  is justified as a necessary mass vaccination that must be done for the common good.         

This line of thinking is logically sound but metaphysically dishonest, because the data gathered by summative tests measures the quality of institutions, not individuals. Dismal performance on the GRAD is not the actual problem — it is a symptom of a systemic disease. And while summative tests are powerful diagnostic tools, they can only measure — not fix — the shortcomings of Minnesota’s educational system.

Solutions need to be instead based around the classroom instruction a child receives from the first day of school until graduation, with particular emphasis on the elementary grades.  Researchers from the Public Policy Institute of California [Download PDF] found they could predict — with a high degree of accuracy — which 4th grade students would eventually pass or fail the California high school exit-exam using a range of emotional, social and academic indicators. If a student’s ultimate success is more a product of early-childhood trajectory than adolescent motivation, effective education policies need to become preventative rather than punitive.

Making that turn, however, will not be possible until we acknowledge the causal relationship between individuals and the institutions they inhabit. Every student in Minnesota should be subject to high standards, consequences, and, yes, frequent testing. But not without access to quality teachers, equitable school funding, and saner testing policies that are designed improve student achievement, rather than just measure it. 

Simply put: the rhetoric in favor of the GRAD is obscuring the inconvenient fact that we have yet to consistently provide the institutional elements necessary for state-wide student success. To do better, we need policy based on evidence rather than assertion, designed for sustained results over the long term. 

Adolescents make decisions based on emotion and impulse; it would be both ironic and unfortunate if our testing policies continue to follow their example.

– Kevin McNellis is an intern at Growth & Justice

Minnesota compared: Colorado and the allure of limits

As the Minnesota Legislature struggles to close a gaping revenue shortfall, I'm in Colorado watching how another set of lawmakers attacks the same problem.

The two states are similar in size, ethnic makeup and per capita personal income. They are weathering the recession better than many others, but are dealing with the harsh fact that a contracting economy shrinks revenue at the same time it increases demand for government services.

So it’s not surprising some of their budget-balancing efforts look similar, too.

They’ve both searched out one-time infusions beyond federal stimulus dollars. (Colorado is testing whether it can draw upon reserves in the state-founded workers comp insurance company, and Minnesota has proposed borrowing backed by tobacco settlement revenues.) Lawmakers in both places consider rescinding hundreds of millions of dollars in tax credits and exemptions — called tax expenditures — that divert money that would otherwise flow into the tax system.

And, Democratic Gov. Bill Ritter, like Minnesota Gov. Pawlenty, has been raising fees, which may look like taxes but — like all these other measures — don’t constitutionally quack like them.

In fact, the greatest similarity may be this urgent pantomime of raising enough revenue without appearing to raise taxes. The main difference? Colorado is constrained by constitutional amendment; Minnesota, by a tax-averse governor.

Strict revenue limits and no-tax pledges were enabled by a growing economy that supplied sufficient public dollars without the states having to face severe consequences… until later.

Colorado went furthest in imposing what the state’s Bell Policy Center calls “the most restrictive tax and spending limitation in the country” — TABOR, the Taxpayer's Bill of Rights. Designed to hold down taxes and shrink government, TABOR requires voters to approve tax increases that exceed a certain formula of growth. About 90 percent of revenues are dedicated, leaving little room to deal with changing conditions. Fed up with how the state was shortchanging education and transportation, Coloradoans have twice voted to loosen TABOR’s revenue straightjacket.

An inflexible, rule-based approach appeals to those who want the pre-emptive accountability of a bathtub-sized limited government. But it has little to offer the majority who have a broader definition of accountable government. They want the overall tax burden to be distributed fairly and expect spending programs and tax breaks to deliver their promised benefits.

On paper, Minnesota’s more flexible accountability framework looks pretty good.

The Legislature enacted a law in 2008 to revive the Minnesota Milestones established by Gov. Arne Carlson in the 1990s that mark progress against long-term goals for the state, and now there’s an Accountability Minnesota website that reports on the milestones by category and agency. Many legislators from both parties also have initiated goal-setting and accountability initiatives in the Legislature.

But reporting on progress against long-term goals and high-level priorities is only part of holding government accountable for results. The real test of these principles comes at budget time when dollars are scarce and choices must be made.

The scrutiny forced by scarcity can be healthy. For example, both states are finally taking a serious look at whether tax expenditures actually achieve their intent, and how they tend to benefit businesses and higher-income taxpayers over those whose benefits must go through the appropriations process.

According to a recent Public Strategies Group report, “Minnesota’s Bottom Line,” [Download pdf] regular legislative review is already required to evaluate the cost, effectiveness and fairness of this $11 billion in “hidden spending.” Yet cost is the only aspect publicly reported by the Department of Revenue.

And that’s a general shortcoming. Measuring cost is relatively easy, since expenses are routinely captured in the system. Determining effectiveness is more difficult and costly, particularly when the greatest payoffs come from education, health care and transportation systems that deliver value over the long term.

Minnesota needs to know what works as well as what things cost. It’s the most fiscally responsible way to govern and the only way to ensure government can deliver what we expect. 

The apparent simplicity of rules that limit revenue and formulas that disperse money has some appeal in a complicated and changing world. But it’s not the way to manage for results. Maddening as it may be to muddle through this year and push some problems into the next, Minnesota is on a better path  — provided we continue to emphasize fairness and effectiveness, guided by facts and a long-term vision for Minnesota.

— Charlie Quimby 

Tax Appreciation Day!

It's a beautiful day and just the right time to make a public statement that taxes are necessary and not at all evil. 

If you have time at lunch today (April 15), and if you are one of those informed and practical progressives who knows that your taxes are providing vital investments for our community, then I urge you to get on down to the state capitol and join a marvelously inventive demonstration under the banner of Tax Appreciation Day.   

Apparently a number of progressive groups are involved in planning this antidote to the anti-tax, anti-government venom that is always on display on tax deadline day on the front steps, orchestrated by talk-radio types and various hard-line conservative groups.  

The event begins at noon in the Capitol Rotunda. Sources say the event will include some entertaining street theater. One can only hope that Tax Appreciation Day becomes an annual tradition at the capital, as the force for progress and smart public investments builds.

— Dane Smith

Unenforced regulation benefits no one

Spending some quality time in another state inevitably invites comparisons back home. Nearly every day, I read a story in the Colorado press that evokes a comparison with Minnesota.

The story that prompted this post describes a suit against the Colorado insurance commission and department of public health:

State officials have allowed nursing homes to operate without insurance in violation of state law, leaving vulnerable residents without recourse when they are abused and neglected, lawyers charged in a lawsuit filed Friday.

Colorado state law requires nursing homes to have liability insurance. When the industry complained that the required $3-million policies were getting too expensive, the department of health adopted a policy allowing the homes to self-insure if they submitted affidavits to the insurance commission stating they had set aside up to $1 million to handle claims.

The lawsuit claims at least one company with a poor record with regulators submitted false information to the state, which did nothing to verify it. The suit asks the courts to require the state to make sure the homes are properly insured.

Stories like this raise broader questions about whether regulation is working the way it should, as well as arguments that government should not be regulating so many aspects of business operations.

The anti-regulation screed sounds something like this: "This is just one more example of a nanny state failure. Regulation adds to business costs and doesn't actually protect consumers who assume the government is taking care of a problem. Citizens would be safer if they took responsibility for their own affairs, instead of trusting government to regulate private businesses."

When small government advocates look for places to cut "non-essential" government services, this sort of regulation might well be on their list.

It certainly sounds like an agency fell down on the job — at least to the extent it trusted a national company to file truthful reports. We also know Colorado is one of the lowest-spending states on a per-dollar-of-personal-income basis, and in 2006 it ranked last among states on spending for public welfare. Presumably, officials in state agencies have to prioritize where they will spend their limited dollars. Monitoring compliance with insurance regulations may seem like a lower priority than making sure all patients are being taken care of appropriately.

If the company named in the suit was indeed skirting the liability coverage regulations, what other corners were being cut? And how do we know those shortcomings will come to light?

Starving government starts with starving inspections and audits that show taxpayers their dollars are being well-spent. Then, when problems surface, that's used as an example of how regulation isn't working and should be abolished.

But what's the alternative? If an underfunded agency can't verify a company was properly insured, how will individual consumers make that determination? How efficient is it for companies to handle individual queries that could be handled on a statewide basis? And what are the costs to society if "the marketplace" doesn't self-regulate?

These questions are not just about this specific issue of nursing home liability insurance. They apply broadly to the roles we want government to play on our behalf and what happens when we set the expectation without funding the follow through.

Finally, I wondered, does Minnesota require similar liability insurance coverage or reserves to handle claims? Speaking as a member of the general public, I can't easily tell whether state agencies or the nursing home companies have addressed this. And speaking as a taxpayer, I don't know how much those assurances would be worth to me.

[Note: Lee Egerstrom of Minnesota 2020 comes at insurance regulation from another angle, how closely the state reviews insurance products offered to consumers.]

— Charlie Quimby