The NAEP is a nationwide standardized test of reading and math proficiency. The 2024 scores are out, and they’re not good:

Most headlines have said something like New NAEP Scores Dash Hope Of Post-COVID Learning Recovery, which seems like a fair assessment.

I feel bad about this, because during lockdowns I argued that kids’ educational outcomes don’t suffer long-term from missing a year or two of school. Re-reading the post, I still think my arguments make sense.

So how did I get it so wrong?

When I consider this question, I ask myself: do I expect complete recovery in two years? In 2026, we will see a class of fourth graders who hadn’t even started school when the lockdowns ended. They will have attended kindergarten through 4th grade entirely in person, with no opportunity for “learning loss”.

If there’s a sudden switch to them doing just as well as the 2015 kids, then it was all lockdown-induced learning loss and I suck. But if not, then what?

Maybe the downward trend isn’t related to COVID? On the graph above, the national (not California) trend started in the 2017 - 2019 period, ie before COVID. And the states that tried hardest to keep their schools open did little better than anyone else:

And the states that closed schools the longest did, if anything, a little better than average:

On the other hand, some of the other subtests show a more obvious COVID effect:

A second possibility is that it is COVID, but it has more to do with schools than students. For example, schools lowered their standards during COVID (because achieving the old higher standards was impossible under the circumstances), then never raised them again.

This would explain the timing and (if it materializes) the effect on not-yet-in-school classes. But did anything like this really happen?

These two articles warn of a “parent-expert disconnect”; experts monitoring standardized test scores are panicking, but parents monitoring their own child’s learning think everything is okay. They indeed blame declining standards:

Unfortunately families’ main sources of information about their children — report cards and other school-based sources — are not painting the same picture [as standardized tests]. Schools eased grading expectations during the pandemic and likely haven’t returned to pre-Covid standards. The representative education surveys we conduct indicate that school-based sources are telling 75-80% of families (depending on subject) that their children are doing fine, earning mostly B’s and above. As a result, less than 20% of parents/caregivers, in California and nationwide say they’re worried about their child’s academic performance.

There’s also an “absence crisis”. During COVID, there were so many reasons to be absent (sickness, fear of sickness, the school itself was barely functional, etc) that administrators stopped enforcing tough-on-absence policies. By the time they changed their mind, a new culture of absenteeism had taken hold, and has since proven hard to uproot. AEI reports that rates of “chronic absenteeism” (missing 10% or more of the school year) rose from 15% before the pandemic to 29% afterwards. And this could also cause a parent-expert disconnect: the parents of chronically absent children probably aren’t paying too much attention (or at least aren’t blaming the schools themselves) so everyone who expresses an opinion on the schools say they’re fine.

If the problem was lower standards or more absences, we might expect it to concentrate among the worst performers - students more likely to be on the edge of meeting standards, or to play hooky. I looked to see if this was true, but data are too weird to draw any conclusions. Choose your poison - do you want this graph, where the high-performers have inexplicably great years from 2017 - 2019, but then return to average?

Or this graph from Indiana, where low-performers are already in free-fall by 2019, and COVID barely even shifts the trend:

Or this graph, which makes total sense and matches my prediction, but comes from another test entirely?:

With some trepidation, I stand by the conclusion of my 2021 article, which was addressed to parents worrying about the individual decision to send their child (eg with some immune deficiency) to school after it had reopened. In this case, when they could look forward to returning to a well-functioning school and friends who were slightly ahead of them, I thought (and still think) they would eventually catch up.

I predict that what we’re seeing here is not each individual child’s learning loss multiplied across all children, but a systemic effect where something about the pandemic made schools worse - in a way that would set back even some hypothetical child who stayed in school throughout the pandemic and suffered no learning loss.

We’ll know more when we get the 2026 test results, and see scores for kids who hadn’t even started school during COVID.