Astral Codex Ten - Nobody Knows How Well Homework Works
Yesterday I wrote about bottlenecks to learning. I wanted to discuss the effectiveness of homework. If it works well, that would suggest students are bottlenecked on examples and repetition. If it works poorly, it would have to be something else. Unfortunately, all the research on this (showcased in eg Cooper 2006) is terrible. Most studies cited by both sides use “time spent doing homework” as the independent variable, then correlate it with test scores or grades. If students who do more time on homework get better test scores, they conclude homework works; otherwise, that it doesn’t. One minor complaint about this methodology is that we don’t really know if anyone is reporting time spent on homework accurately. Cooper cites some studies showing that student-reported time-spent-on-homework correlates with test scores at a respectable r = 0.25. But in the same sample, parent-reported time-spent-on-homework correlates at close to zero. Cooper speculates that the students’ estimates are better than the parents’, and I think this makes sense - it’s easier to reduce a correlation by adding noise than to increase it - but in the end we don’t know. According to a Washington Post article, students in two very similar datasets reported very different amounts of time spent on homework - maybe because of the way they asked the question? I don’t know, self-report from young kids seems fraught. But this is the least of our problems. This methodology assumes that time spent on homework is a safe proxy for amount of homework. It isn’t. Students may spend less time on homework because they are smart, find it easy, and can finish it very quickly. Or they might spend more time on homework because they love learning and care about the subject matter a lot. Or they might spend more time because they’re second-generation Asian immigrants with taskmaster parents who insist on it being perfect. Or they might spend less time because they’re in some kind of horrible living environment not conducive to sitting at a desk quietly. All of these confound the time-grades correlation. Most studies don’t bother to adjust for these factors. The ones that do choose a few of them haphazardly, make wild guesses about what model to use, and then come up with basically random results. Both homework proponents (Harris Cooper) and opponents (Alfie Kohn) briefly nod to this problem, then take these studies seriously anyway. If you do that, you find that probably homework isn’t helpful in elementary school, but might be helpful during high school (though some people disagree with either half of that statement). But why would you take these seriously? Are there any real randomized studies? Cooper finds six for his review article (page 17), none of which are published or peer-reviewed. Only one is randomized by students, and it contradicts itself about how random it actually was; the other five are cluster-randomized by classroom (which means they have very low effective sample size). Several of are bungled in confusing ways. Still, these pretty consistently show a positive effect of homework with high effect size. The one that might have been randomized by students (and so might possibly be okay) had an effect size of 0.39. Some of the cluster randomized ones that weren’t bungled too badly had effect sizes in the 0.9 range; the cluster randomization makes it hard to call this significant, but unofficially it seems impressive. Since Cooper wrote his 2006 review, I was able to find one actually good, individually randomized study of homework, Nawaz and Welbourne. They took 368 students taking algebra classes using a digital platform, and randomly assigned them either 0%, 50%, 100%, or 150% of the ordinary homework load (corresponding to 0, 15, 30, or 45 minutes/night). Results: P < 0.0001. Looks solid. Probably 9th grade algebra homework is useful. But everyone already expected high school homework to be more useful than elementary school, and math homework to be more useful than other subjects. So it’s unclear if eg 4th grade reading homework would follow the same pattern. Still, this is the one firm fact about homework which we have managed to produce in several million child-years of assigning it. For everything else, just go with your priors, I guess. You’re a free subscriber to Astral Codex Ten. For the full experience, become a paid subscriber. |
Key phrases
Older messages
Study: Ritalin Works, But School Isn't Worth Paying Attention To
Wednesday, July 6, 2022
...
Open Thread 231
Monday, July 4, 2022
...
Your Book Review: The Internationalists
Friday, July 1, 2022
Finalist #8 in the Book Review Contest
Links For June
Friday, July 1, 2022
...
Highlights From The Comments On San Fransicko
Wednesday, June 29, 2022
...
You Might Also Like
Welcome to The Flyover
Sunday, May 19, 2024
Thanks for joining The Flyover! ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Most Trustworthy Crypto Exchanges | Irrational Bitcoin 13F Exuberance
Saturday, May 18, 2024
The asset manager's new short-term credit fund is hosted on the Ethereum blockchain. ADVERTISEMENT Forbes START INVESTING • Newsletters • MyForbes Mitchell Martin Senior Editor, Forbes Money &
Expedia Group dismisses CTO and SVP, citing ‘violation of company policy’
Saturday, May 18, 2024
Microsoft's gaming gamble | The rise of 'Technocapitalism' ADVERTISEMENT GeekWire SPONSOR MESSAGE: Register now for AWS re:Inforce: Hear from AWS experts on all things cloud security in the
Scottie Scheffler, Pelosi Attacker, and a Pigeon Parachute
Saturday, May 18, 2024
Facts, without motives.
How to Profit Like a Real Estate Tycoon, Without the Hassle of Being a Landlord
Saturday, May 18, 2024
Save 50% On Forbes Real Estate Investor View in browser LIMITED-TIME OFFER—SAVE 50% Have you ever dreamed of profiting from real estate like a tycoon but without the headaches of property management?
Weekend Briefing No. 536
Saturday, May 18, 2024
Meaning > Purpose at Work -- The Land That Doesn't Need Ozempic -- The Forgotten War on Beepers ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
The best mascara
Saturday, May 18, 2024
Va va voom ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Originalism Means Never Having to Say You’re Sorry
Saturday, May 18, 2024
Columns and commentary on news, politics, business, and technology from the Intelligencer team. Intelligencer Weekend Reader Required Reading for Political Compulsives 1. Running From Inside Is it
YOU LOVE TO SEE IT: The Consumer Watchdog Catches A Break
Saturday, May 18, 2024
Plus, sick leave is sticking around, Medicaid tackles climate change, criminal justice reform is helping communities, and another state aims to make Big Oil pay. YOU LOVE TO SEE IT: The Consumer
The problem with No Mow May
Saturday, May 18, 2024
+ how college presidents pick commencement speakers