Saturday, December 16, 2006

I Can Give You Facts and Figures, I Could Give You Plans and Forecasts

If you want to see me angry, remind me about the Bush administration's insistence that sex education funds be given exclusively to abstinence-only programs. This isn't because I think teenagers shouldn't be abstinent (it didn't hurt me none). It's not even that I think abstinence-only education is the underhanded tool of fundamental Christians trying to shoehorn religious teachings into public education (though it totally is).

No, the reason I get so steamed about abstinence-only sex ed is because it doesn't even work! More comprehensive sex ed programs, such as those touting abstinence and condoms, are more successful at preventing teen pregnancy, STDs, and sexual activity. So, based on the research, if fundies really wanted to curb the clap and unwanted babies (and, by extension, abortions), they'd promote comprehensive sex ed over the laughable "not 'til you're married" policy.

Beware--I invoked the holy name of Research when debating an emotionally-charged topic. For some reason research isn't a popular tool in policy making; in some contexts, it's practically a dirty word. There's something about a nerd with a pie chart telling you that your 12-year-old daughter should watch her teacher roll a condom over a banana that, well, rubs parents the wrong way. Who the hell do you think you are, with your fancy applied statistics degree and flashy bar graphs. Do you even have a daughter of your own?

Sure, we pretend to advocate the use of research in civic decision-making. Representatives like to walk onto Capitol Hill with a stack of charts and data under their arm. Constituencies like the authenticity that research affords their lawmakers. When a bearded Berkeley professor goes on the local news to act as an expert about the connection between carbon emissions and climate change, audiences hold that researcher in a certain regard. For a moment.

When it comes down to it, though, beliefs will always trump facts.

There are plenty of reasons for this. The first and most subtle is that Americans don't like know-it-alls. Data are exotic and mysterious, and come to think of it most of us have always hated math. Sarah Vowell writes about this phenomenon when assessing why voters undervalued Al Gore in the 2000 presidential election. Al Gore embodies what the populace hates about research-oriented thinkers: he uses big words, cites numbers and trends and published reports, and then acts like he knows more than I do because, most likely, he knows more than I do. He's a big nerd, and he makes us feel dumb. Researchers, experts, and Al Gore spend their whole careers rubbing it our face that they know better than we do how we should spend our lives. The audacity!

That brings me to the next reason Americans dismiss poindexters and their highfalutin' data: we don't like people telling us what to do. A lot of research out there is exploratory--it seeks to discover unknown facts. It's when research gets prescriptive, however, that we get ornery. And when research hops in bed with policy, the most prescriptive element of our society, it feels like mom and dad are making us eat our veggies.

Along those lines, the most obvious reason we poo-poo research in policy is that we're confident that we know how to take care of ourselves, no matter what the bespectacled number-crunchers say. It's the same mentality that prompts some of us to never wear our seat belts; seat belts are for bad drivers, not me. Or sometimes the research results in question counter everything we've experienced in our lives. How can pork be bad for you if my granny ate a rack of ribs every day until the day she died at the age of 98? Or the research results might simply be counter-intuitive. You mean red light cameras actually increase car accidents? No way! Very simply, research can prescribe behaviors that completely violate preexisting beliefs, morals, or folk knowledge, and it's going to take more than a few papers published in Harvard Educational Review to change our minds about how schools should be run. If my parents always told me that milk builds strong bones, I'm not going to value research that says it doesn't. If I think pre-teens shouldn't be exposed to condoms, I don't care if the Mighty World of Research says they should. If I think God created the Earth in six days, fossil records mean nothing to me.

There is one last, and semi-logical, reason to question the role of research in your daily life. Frankly, research results are confusing, conflicting, and ever-changing. Is milk bad for me or good for me? What about beef? Vitamins? Some research says yes, and some says no! From a layperson's perspective, the Research Community can't make up its mind. "Research" is viewed as an amorphous lump of scientists in lab coats, a unified coalition of smarty-pants who convene every year to agree on the Truth. So it's a little confusing when one member of the research community says milk makes you fat while another equally believable source says it'll help you lose weight. To decipher all the conflicting information would require a degree of research methods and statistics literacy that very few people actually have.

The truth is that most the research that gets published and makes headlines is internally valid, and usually fairly externally valid. The source of confusion is in how these data are presented to the public: the USA Today headline says something earth-shattering like "Watching TV Makes your Kids Stupid," when in reality the research study simply correlated television viewing with lower grades. How is Joanne Meatball supposed to detect the nuanced differences between those two statements? If you carefully examine every study that makes the front page, you'll find that most of them are sound, and most of the studies that reveal ostensibly opposed truths are actually measuring completely different constructs. But if I can't understand what the data imply, why should I trust it?

The question is whether these are actually good reasons to discourage the marriage of research and law.

The advantages of research-based policy are clear. We would expect to see laws actually yield their intended results at a greater rate than currently enjoyed, assuming the policies are enforced properly. Fewer pregnant teenagers, fewer women with osteoporosis, fewer inmates on death row, fewer gun deaths, fewer accidents at lit intersections. It's a utopia waiting to happen.

The disadvantages are not easily overlooked, though. What if the policy that research prescribes violates the sanctity of civil liberties? So what if tracking sex offenders with GPS devices is effective at reducing repeat crimes? It's also invasive and cruel. (Come to think of it, though, I doubt if it'll prove effective.) Relying heavily on research to inform policy is precariously undemocratic. When lawmaking is put in the hands of unelected experts, the public's right to enforce their beliefs, however unfounded, is limited. For better or worse, it's my right as an American to demand that morals, not science, dictate policy. And given how inscrutable most research is, who would mediate between the academic world and the policy-making world? Would their motives be free of bias? Finally, as new data are collected, the "facts" can change with generations. Can we afford to bend law around a seemingly fickle collection of truths?

Despite these drawbacks, research should play a more prominent role in lawmaking. I think resistance to fact-based policy is rooted primarily in unfounded skepticism and ignorance, and perhaps a little in hardheadedness. Americans display a distrust of science not observed in comparable nations, and while it hasn't been stifling our short-term economic health, our environmental and personal health is suffering as a result.

Plus my income for the last three years has relied on publicly funded education research, so maybe I'm just looking out for Numero Uno.


Thinker said...

In my opinion, one of the major reasons so many people have the attitudes about research and researchers that you've so ably presented is the abysmal science education so many of us endured in secondary school. Unfortunately science classes are taught like so many others - a series of lists to memorize and regurgitate. People come away with the idea that we live in a binary world - things are either right or wrong, or true or false.

I've thought for awhile that we need to create and teach about 'truth-o-meters'. Instead of asking whether an assertion (or a report) is true or false, we should ask students to search out and evaluate the evidence supporting and opposing it. Once they've collected as much as they can, they can decide where it belongs on the truth-o-meter - 100 being true without doubt, 0 being absolutely false, and 50 being unable to tell. After years of such study, perhaps a future generation of Americans will look on your post and scratch their heads, wondering how any group of people could develop in such ways.

Paul said...

Propositions are either true or false. That law even has a name: the law of the excluded middle. The solution to the problem Rebecca highlights isn't to introduce some sort of relativism or skepticism, it's to clarify our propositions, or to help laymen better understand the nature of the disagreement. The fact that people disagree is not evidence that objective truth doesn't exist.

Some of the worst lessons I ever received in school involved being taught that the difference between a "fact" and "opinion" is that an opinion is just something a person thinks, but a fact can be proven, or something to that effect. That's the kind of thinking that gets us "teach the controversy" arguments in favor of creationism in school. It was silly and pernicious, but I think similar sorts of lessons are still widely taught.

Thinker said...

I agree with you Paul, objective truth does exist. However, as humans we're forced to seek it out with senses and sensory extensions (microscopes, telescopes, microphones, spectrometers etc.) that limit our ability to know it with certainty. We're rather like the Blind Men and the Elephant. We can collect a lot of data, but then we have to decide what it means. It is in that decision making process that dispute arises among thoughtful people. Does this mean that all positions are equal. Absolutely not! If I give you an equation (A + B = 6), then ask you to tell me what A and B equal; there are several correct answers ( 1 and 5, 5 and 1, 4 and 2, 2 and 4, etc.), but many more wrong ones (4 and 3, 3 and 4, etc.).

In my earlier comment, I was simply suggesting that we need to teach ways to separate possibly correct assertions from demonstrably incorrect ones; not that all must be taken equally.