The Legibility Series · Essay Two

Follow the Money

What one state’s checkbook reveals about a decade of spending, and the question no one was able to ask.

Download the full essay (PDF) ← Essay One 5 pages · two-column whitepaper

Key Points

A note on method

This essay, like the one before it, is written from the builder's chair rather than the policy desk. It is a companion to Reward Follows Legibility, which argued that the education market fails to reward what works because the evidence of what works is invisible at the moment and the place money actually moves. That essay made the case in principle. This one does something smaller and more concrete. It takes a single state, asks a single question any taxpayer might ask, and lets the money answer.

The question is this. Who are we actually paying, how long have we been paying them, and did it work.

I. The two answers that were always in the building

For most of the last two decades, a school leader in Maryland could have asked that question and gotten only the first part of an answer, and only from the people being paid. The companies that sell curriculum, reading and math catch-up programs, and benchmark tests arrive with a presentation and a familiar name. What they do not arrive with is an independent account of how their product performed for districts like yours, or a record of how much your own state has already spent on it, or for how long. That account was never assembled, because no one whose job it was to sign the order had it in front of them.

It can be assembled now, because the state writes its spending down. Maryland, like a small number of other states, publishes its districts' payments to outside companies.1 Read sixteen years of it at once and the first two answers come into focus with uncomfortable clarity.

Across those years, the two dozen districts of Maryland sent more than a quarter of a billion dollars to the small group of companies that supply their core academic materials. The curriculum. The programs meant to help struggling readers and math students catch up. The tests that measure whether any of it landed. Not buildings, not buses, not salaries. Learning. And that figure is a careful one. It counts only the spending that can be tied to teaching and learning, and sets aside the far larger checks every district writes to run a payroll. A single one of those academic companies has collected more than ninety million dollars on its own. Another has turned up in the checkbooks of twenty-three of the state's twenty-four districts, and has been paid somewhere in Maryland every year for sixteen years running.

These are not purchases in any ordinary sense. A purchase is a decision you make once and can unmake. What the checkbook shows instead is permanence. A small number of providers, woven so deeply into the budget of nearly every district that the line items renew each year without anyone pausing to ask what they bought. They have stopped being vendors and quietly become institutions.

There is a smaller lesson hiding inside the larger one, and it is worth a sentence, because it is the part most people get wrong about a checkbook. The single largest checks a district writes do not go to anyone who teaches a child. They go to the tax authority and to the health insurer. A ledger does not interpret itself. The work of reading one is the work of telling the cost of running a payroll apart from the cost of a decision about how children learn. Only the second kind of spending is the subject of this essay, and once you separate it cleanly, the quarter of a billion dollars above is what remains.

II. The answer no one collected

So we have the first two answers. Who, and how long. A quarter of a billion dollars, a handful of companies, sixteen years of renewals.

Now lay that spending next to the thing it was supposed to buy.

Set the money beside the results, measured not by any company's own dashboard but by an independent yardstick that no district and no seller controls, on a scale that lets you compare a child in one county to a child in another, and to children across the country.2 Over the same sixteen years that the spending climbed, the results did not. Taken together, the students in these districts lost ground. Not against some impossible ideal, but against where children their age used to stand. Fewer than one in ten of the state's districts improved at all.

Then comes the fact that should end the argument, and is almost never reached, because the spending and the results are so rarely placed on the same page. If these products were the thing moving the needle, you would expect the districts that spent the most on them to have the most to show for it. They did not. For six of the seven largest providers, the districts that spent the most on them slipped the furthest, not the least.3

I want to be careful about what that does and does not prove, because the credibility of the whole argument depends on the care. It does not prove that any one company sells a bad product. Districts buy many things at once, and the districts that spend the most on outside help are often the ones already struggling the hardest, which is part of why they are spending. A single state across a single stretch of years cannot, by itself, isolate the cause of anything.

But it proves something, and the something is not small. After sixteen years and a quarter of a billion dollars, the results are not there to be found, and the harder a district leaned on these tools, the less it had to show for it. Whatever else is true, the money and the outcomes are not moving together. They are barely acquainted.

III. The excuse that has expired

There is a familiar answer to all of this, and it deserves to be stated at its strongest before it is set aside. The answer is that the products are fine, and the problem is implementation. The teachers were not trained well enough. The program was not used the way it was designed to be used. The rollout was rushed, the coaching was thin, the thing was bought and then left on a shelf. Give it more time, more training, more support, and the results will come.

For a new product in its first year or two, that answer is not only fair, it is usually correct. Most things fail in a classroom for want of implementation long before they fail on their merits.

But that defense has a clock on it, and here the clock has run out. We are not talking about a pilot in its second year. We are talking about products bought, renewed, and bought again, across a dozen districts, for a decade and a half. Sixteen years is long enough to train every teacher in the state, and then to train the ones who replaced them. A tool that has been the established way of doing business since before today's high-school seniors started kindergarten is not a tool waiting on better implementation. It has become the implementation. At that point, the sentence "we never really used it well" stops being a defense of the spending and becomes the gravest possible charge against it. It says that a quarter of a billion dollars bought something no one ever managed to make work, across sixteen years and twenty-four districts, and no one in a position to stop it ever noticed, because no one was ever set up to look.

That is the real finding, and it is worse than a bad product. A bad product is a mistake you can correct. What the checkbook reveals is the absence of the correction itself. The scandal is not that the results are missing. It is that for sixteen years the question was never asked, because the two halves of it, the money and the outcome, lived in separate worlds and were never brought into the same room.

There is a second objection, fairer than the first, and it deserves the same hearing. These tools were never bought for every child. A reading program meant for the students furthest behind, a math program aimed at a single grade, could be helping exactly the children it was bought to help without ever lifting a district's overall number. That is true, and it matters, and I will not wave it away. But look at where it actually leaves us. If, after sixteen years in nearly every district, those targeted gains were real and large, they would be very hard to hide inside a decline this wide. And if they are real but small, or real in corners no one has ever thought to check, then the honest reply is not reassurance. It is the admission that we spent sixteen years and a quarter of a billion dollars and never built the means to see whether the very children we were aiming at actually moved. The objection does not rescue the spending. It names the blindness more precisely, and the blindness is the whole problem.

Implementation explains a bad year. It does not explain a bad decade. Spending without a result is just a receipt. A renewal without a result is just a habit. And a quarter of a billion dollars without a single answer is not an investment in children. It is a story a system told itself about one.

IV. What a checkbook is for

A checkbook is not a verdict. It cannot tell you what works, and anyone who claims to rank these companies by their results from spending records alone is selling you the same kind of story the brochures sell. Spending is not proof. One state is not the country. The honest use of a ledger is narrower than that, and more important than that.

What a checkbook can do is make the spending legible, and legibility is the thing that has been missing the entire time. You cannot ask whether a decade of spending earned its keep until you can see the decade of spending whole. The moment you can, the question that was unaskable for sixteen years becomes not only askable but obvious, along with the discomfort of having gone so long without it.

This is the argument of my earlier essay, Reward Follows Legibility, run in reverse. That essay said the market cannot reward what works because what works is invisible at the moment money moves. Maryland's checkbook shows the other face of the same coin. When you finally make a decade of spending visible, the first thing you see is that no one could have told you whether it worked, because the means to know was never built. Reward follows legibility. So does accountability. So does the kind of giving that insists on a result before it gives again. None of them can begin until the money and the outcomes are written on the same page, in a hand that neither the buyer nor the seller controls.

That is within reach now, and it does not require a new agency or a new authority appointed to crown winners. It requires only that we put what was already spent next to what was actually achieved, in public, on a common scale, and let everyone see the same thing at the same time. The company that would rather compete on results than on logos. The superintendent who signs the orders and has never once been handed the state's own track record on what she is signing. The donor who wants every dollar to move a number, and who deserves to know whether the dollars already in the system moved one.

Because here is where sixteen years of a single state's checkbook leaves you. A state can spend more than a quarter of a billion dollars, with the same few companies, for sixteen years, and be unable to tell you, from its own records, whether any of it moved a single child. The first thing this field needs is not a better product or a bigger grant. It is the ability to see the question.

Follow the money. It leads you straight to it.

Follow the money yourself

This essay was written from inside PILLAR's Vertical Intelligence. Ask it the same question for any state, any category, any company, in plain English on independent government data.

Explore Vertical Intelligence
Eli Jameson

About the Author

Eli Jameson is the founder of PILLAR, which builds revenue and evidence infrastructure for the EdTech and GovTech markets. This is the second essay in a series that began with Reward Follows Legibility. It draws on a state's own published spending records, set against independent measures of student outcomes, and names no company because no company is the point.

Notes

  1. Source for the spending figures: Maryland's own published record of school-district payments to outside companies (the state's vendor-payment disclosures), which list payments at or above a statutory threshold. Each payee was matched to the company behind it, and only payments for academic materials, curriculum, intervention and practice programs, and assessment, were counted toward the figures cited here; payroll, benefits, construction, transportation, and tax remittances were set aside. Figures are read across sixteen fiscal years and the state's twenty-four school districts, and are kept deliberately conservative.
  2. Source for the outcomes: an independent, nationally-comparable measure of district-level student achievement built from public assessment results that no district or company can edit, expressed on a common scale so that districts can be compared to one another and to the country. The comparison window runs across the years for which that measure has a consistent start and end point. "Fewer than one in ten districts improved" refers to the share of the state's districts whose measured standing rose rather than fell over that window.
  3. A necessary caveat, stated plainly: this is a descriptive comparison, not a causal claim about any company. Districts adopt many products at the same time, a payment is not the same as classroom use, and the districts that spend the most on outside help differ in systematic ways from those that do not. The point of the essay is not that any product caused a decline; it is that, across sixteen years and a quarter of a billion dollars, the spending and the outcomes show no relationship a buyer could have used, and the means to look was never built.