As I scrolled through my twitter feed this morning, a tweet from NPR jumped out at me:
All I could think was, here we go again.
If you’ve followed education policy for any length of time this routine looks familiar. Researchers or policymakers visit some other country that performs better than we do on international assessments and then come back with the secret sauce that makes them do so well. The recommendations are so anodyne that that anyone with a passing knowledge of the education system will probably agree with them. The policy flavor of the month (national education standards, universal pre-K, etc.) is usually highlighted. Rinse and repeat.
For a while we were told to emulate Shanghai—until we learned that Shanghai systematically excludes disadvantaged students from its testing pool to juice its scores. Then it was Finland, at least until Finland’s scores dropped precipitously on the very test that ushered in its rise to prominence.
Here is the problem with this approach, and why it never actually yields the information we’re looking for: it violates the basic precepts of research design. If you want to know if a certain policy affects an outcome, you develop a hypothesis and test it. You follow children that are subject to the policy and children that aren’t—doing your best to make sure all other aspects of the two groups’ educational environments are identical—and you see what happens. If the children who were subject to the policy do better than those who weren’t, then you have reason to believe that the policy caused the improvement.
“Do what the best schools do” research does the exact opposite of that. It sees a result it likes and then tries to work backwards to the cause without isolating other variables that might also explain the outcome. That is not how science works. Are there places, for example, that do the very things that Finland or Shanghai or any of these other countries do that don’t meet with success? Are there countries meeting with success that don’t do these things? As I’ve written before, the Netherlands is a very high-performing country that is marked by an incredible degree of school choice. So why does a universal voucher system always seem to be missing from the list of recommendations? In short, we have no idea whether the policies these folks advocate are really behind these countries’ successes.
I don’t want to say there’s nothing we can learn from other states or other countries about how to improve education here at home. However, if we are going to make claims that one policy or another causes a particular outcome, we need to back those claims up with research done the proper way.
