Tangle - Is crime getting better or worse?

I’m Isaac Saul, and this is Tangle: an independent, nonpartisan, subscriber-supported politics newsletter that summarizes the best arguments from across the political spectrum on the news of the day — then “my take.”

Are you new here? Get free emails to your inbox daily. Would you rather listen? You can find our podcast here.

Introduction. 

Is crime increasing or decreasing in America? This seemingly simple question is not as easy to answer as you might think.

This week, the Federal Bureau of Investigation (FBI) released its annual Crime in the Nation report, which shared data on over 14 million criminal offenses reported to the agency in 2023. News outlets like The Associated Press, The New York Times, NPR, and others ran headlines on a “steep” decline in crime since 2020, but others like Fox News and The Daily Caller raised concerns about the FBI’s data or highlighted areas where the crime rate increased. The opinion pages, too, showed a stark contrast in how the report was received. 

“Contrary to media myth, U.S. urban crime rates are up,” Jeffrey H. Anderson wrote in The Wall Street Journal. “Police and victim data finally agree that crime is falling,” Bloomberg’s Justin Fox said. 

Earlier this month, the Bureau of Justice Statistics released the National Crime Victimization Survey (NCVS), an annual report on self-reported instances of crime. The survey found that the overall rate of violent victimizations dropped slightly from 2022 to 2023 but remained higher than the rate in 2020, exemplified by an increase in the rate of violent crime (excluding simple assault) from 5.6 per 1,000 in 2020 to 8.7 per 1,000 in 2023. Unlike the FBI report, though, few news outlets wrote about the NCVS data.

What’s going on here?

In last week’s Friday edition, we shared a side-by-side breakdown of former President Donald Trump and Vice President Kamala Harris’s record and positions on eight of the biggest issues in the 2024 election. In the section on crime, we noted that voters have diverging views not only on the best policies to address crime but on the prevalence of crime itself — in 2023, 77% of Americans thought there was more crime in the U.S. than the year prior. What explains the gap between perception and the data? Or could former President Trump and others be correct that the FBI’s data is fatally flawed? 

Today’s edition focuses on three core questions about crime in America: 

1. How reliable is the crime rate data?
2. What is driving the gap between Americans’ perception of crime and the crime data?
3. What policies are most effective at addressing crime?

To answer these three questions, we spoke with three experts who span the political spectrum in their views but have all dedicated their careers in part to studying crime and advocating for evidence-based solutions to the issue. Ames Grawert is senior counsel in the Brennan Center’s Justice Program, leading quantitative and policy research focused on crime and criminal justice reform. John Lott is president of the Crime Prevention Center and served as the Senior Advisor for Research and Statistics in the U.S. Department of Justice during the Trump administration. Robert VerBruggen is a policy researcher at the Manhattan Institute, where he covers issues like crime, education, and family policy. 

Before we dive in, here are a few pieces of background information on the sources crime analysts use:

• The FBI’s annual Crime in the Nation report contains data collected from the agency’s Uniform Crime Reporting program, which compiles reports on crime from cities, universities, counties, states, tribes, and federal law enforcement agencies across the U.S. In the 2023 report, the FBI says approximately 16,000 sources covering areas with 94.3% of the U.S. population submitted data.
• In 2015, the FBI announced it would transition from collecting data using two systems — an aggregate crime-reporting system called the Summary Reporting System (SRS) and an incident-based reporting system called the National Incident-Based Reporting System (NIBRS) — to only using the incident-based NIBRS system. 2021 was the first and only year the FBI accepted only NIBRS data. In 2022 and 2023 it once again accepted both NIBRS and SRS data. 
• The National Crime Victimization Survey (NCVS) is an annual survey sponsored by the Bureau of Justice Statistics, and it collects data on self-reported instances of crime (excluding homicide) from a random sample of roughly 230,000 U.S. residents aged 12 or older. 
• The Centers for Disease Control and Prevention (CDC) also publishes annual data on homicides based on death certificates for U.S. residents. 
• Changes in the rates of crime vary between type of crime and location in the FBI and NCVS reports. You can explore the full FBI data here and the full NCVS report here. 

We spoke with Grawert, Lott, and VerBruggen individually this week, and the interviews below have been lightly edited for clarity.

Is crime data reliable?

Tangle: Ames, you’ve written several pieces in the last year looking at the crime rate in the U.S. and found that crime has decreased meaningfully since the pandemic. What sources of data are you using to reach that conclusion?

Ames Grawert: The gold standard of crime data in the United States is the FBI's annual Uniform Crime Reporting publications. The FBI publishes two series of tables — one is called Offenses Known to Law Enforcement, and the other is called Crime in the United States. These are essentially the final say on crime, at least according to police data in the United States. The problem is they don't come out very regularly, typically about nine months after the end of the year that they cover. For example, just three days ago we got the final data on 2023. So it's gold standard data, but it takes time to produce, takes time to get right, and we have to wait for it. 

There are other sources out there that help to fill the gap between the lag time created by this wait for FBI data. One is the Council on Criminal Justice, a nonprofit, nonpartisan organization run by a team of eminent criminologists. They collect police data from across the country and do their best to standardize it and publish a look at crime trends in this group of major cities they study. Major city crime trends don't always match the national trend, but I’m not aware of a year that the sign and trend of the city crime trends were significantly different from the national trends.

Others do similar work. Jeff Asher is an analyst based in New Orleans who runs a site called the Real-Time Crime Index, which is a really, really exciting site. He's essentially built a digital pipeline from hundreds of police departments to his website, and they publish — in relatively short order — up-to-date crime data from police departments, which gives a much more timely picture of national crime trends. 

A third source that's brand new is quarterly data from the FBI. With the launch of this new data system that they've been rolling out for decades, they’ve also started publishing quarterly data. It still has a lag of a couple quarters and it's also preliminary, but it publishes more frequently. It did diverge from what the final data showed, but it did get the sign and the magnitude right at showing large declines in murder and modest declines in violent crime.

Tangle: John, you’ve argued that the crime data from sources like the FBI is not painting a full picture of the true crime rate in the U.S. Why do you say that’s the case?

John Lott: Well, the FBI deals with reported crime. We've known for many decades that not all crimes are reported to police. About 40% of violent crimes are reported to police; about 30% of property crimes are reported to police. And that share over the last few years in particular has fallen. 51 years ago, precisely because of those concerns, the Department of Justice set up something called the National Crime Victimization Survey inside the Bureau of Justice Statistics. It is a truly massive survey. A lot of large surveys you hear about are 8,000 or 10,000 people; this survey is 240,000 people each year, and they've been doing it for 51 years. 

That allows you to break out both reported and unreported crime to go and get a measure of total crime. There are other measures outside of the Department of Justice — like murders tracked by the Centers for Disease Control — but one of the big problems that you have with the FBI data over the last few years (from 2021 on) is that a huge percentage of police departments have stopped reporting or are only partially reporting crime data to the FBI [Editor’s note: When the FBI changed its reporting system in 2021, law enforcement agencies covering approximately 215 million U.S. inhabitants reported data for that year’s report. In 2023, when the FBI again accepted both the NIBRS and SRS data, agencies covering approximately 314 million inhabitants submitted data.]

In 2020, police departments covering 97% of the population of the United States were reporting crime data to the FBI. In 2022, 31% of police departments weren't reporting any crime data, and that included three of the five largest cities in the United States: New York, Los Angeles, and Phoenix. Then you had another 24% of police departments that were only partially reporting crime data to the FBI [Editor’s note: Lott is referring to the FBI’s quarterly crime report. In 2022’s Q4 report, only 44% of agencies had reported a full 12 months of crime data, 24% submitted partial data, and 32% did not participate at all, including cities like New York City, Los Angeles, and Phoenix. However, overall participation was higher for the full (non-quarterly) report — approximately 87% of agencies submitted data. New York City did not submit complete data in 2021 or 2022 but did in 2023; Los Angeles did not submit any data in 2021 but did in 2022 and 2023; Phoenix did not submit any data in 2021 but did in 2022 and 2023 (you can search for crime data by city here)].

We know crime varies a lot over the course of the year — you have a lot more violent crime in the summer than you have in the winter, for example — and these police departments that were only partially reporting data overwhelmingly tended to report the data that they did report during the relatively slow crime rate parts of the year. The FBI — let's say you got three months of the winter — the FBI would take those three months and extrapolate them over the rest of the year, assuming that the rest of the year had the same crime rate that you had during those three months that they reported. You can see pretty clearly why that could create an underestimate of crime. 

There are other problems that have occurred. The FBI has been found in a couple of cases to take the crime data that they did get from the cities and report a lower number than those cities had given them. In fact, one thing that's gotten almost no news coverage is that the FBI said in the 2022 data on reported crimes that it had fallen by 2.1%. Well, they've gone back and re-looked at the numbers that they had, and now they've revised it from a drop of 2.1% to an increase of 4.5% — a net swing of 6.6%. 

The government overestimated the number of people employed over the course of last year by 818,000. Well, it turns out that they did something similar with regard to reported crime numbers. For the last year, we've had news articles saying crime is down, and people are mistaken — even though they should say reported crime is down — but even that turned out to be wrong when they revised these numbers up. If you look at the National Crime Victimization Survey data, reported crimes have moved in dramatically different directions from the FBI's measure of reported crimes. For example, in 2022, while the FBI initially reported a 2.1% drop and revised it to a 4.5% increase, the NCVS data showed a 29% increase in reported violent crime numbers. That's a huge increase. In the numbers that just came out for 2023, the FBI claims a 3% drop in violent crime, but the NCVS data showed a 4.5% increase in reported violent crime. 

The FBI data claims that violent crime — reported violent crime — has fallen by 5% during the Biden administration. The NCVS data says that total violent crime has increased by 55% [Editor’s note: Total violent crime as measured by the FBI data is up between 2020 and 2023, rising from 16.4 per 1,000 to 22.5 per 1,000, or a 37% increase. However, the total violent crime rate was 21.0 per 1,000 in 2019 — the last pre-pandemic year — resulting in a 7% increase between 2019 and 2023]. That's the largest three-year percentage increase on record in the 51 years that the National Crime Victimization data has been collected — by far the largest. The largest previous [three-year] increase was [from 2004 to] 2006, which was 27%. You have a 42% increase in rape, a 63% increase in robbery, and a 55% increase in aggravated assault. 

David Muir, when he fact-checked Trump on the crime rate, basically said, “Well, the FBI's measure of reported violent crime has fallen.” But Trump was not only right about the claim about violent crime — because Trump didn't say reported crime, he talked about total violent crime changing. He was also correct in his response to Muir about the fact that a lot of major cities with high crime rates aren't reporting their data. 

Tangle: Robert, how reliable is the data that we're getting from sources like the FBI? Are there reasons to doubt that this data is capturing the whole picture?

Robert VerBruggen: There are a few different reliable sources that you can use to look at crime trends. One is called the National Crime Victimization Survey. Another is to use the FBI data, which is drawn from reports by police departments. This only includes crimes that are actually reported to the police, and many of the agencies don't report their data to the FBI — though the FBI does make estimates to account for the fact that the data isn't reported. 

My personal favorite is the CDC’s homicide data based on death certificates. They have a broader definition: Homicide is any situation in which one person kills another person. The FBI keeps tabs on murders and non-negligent homicide — basically, a killing that is found to be legally wrong. And they show very, very similar trends. Both show that if you divide American history up into a few different periods, we had a really big surge of crime between the 1960s and the early 1990s, followed by a really big drop in crime in the 1990s that carried over into the 2000s. 

Tangle: Ames, do you think there are flaws in the FBI’s new system of collecting crime data?