Now that McCain is flip-flopping on affirmative action, let me go ahead and make my in-principle defense of those sorts of racial preferences.
I think there's an instinctive resistance to the idea of "racial preferences" since, in this country, such preferences have tended to be horrifically unjust. Realistically, though, pointing to a long legacy of institutionalized and informal racism to show that affirmative action is, ipso facto, wrong is a "Hitler was a vegetarian" kind of an argument, a lazy non-sequitur.
The right way to think about the issue, I think, is to imagine yourself as the dean of admissions to a college or university. You have made most of your admissions decisions for the coming school year, but you have one more spot to fill, one more student you can accept. Complicating things, however, is the fact that the two best candidates left in your applicant pool are, on paper, identical in every way. That is, they have the same SAT scores, the same high school GPA, the same extra-curricular involvement, etc. Perhaps they even have the same economic background. They appear to be totally the same, except that one is white, and one is black.
Who do you admit?
It seems to me there's a very, very plausible argument to the effect that, even if your only goal is recruiting the strongest possible student body, you should prefer the black student. The reasoning is pretty simple: by stipulation, the two students have accomplished precisely the same things, but one has done it in spite of what were probably non-negligible obstacles due to his race, the subtle and not-so-subtle manifestations of racism in America.
If high school for these students was a race, they both reached the finish line at the same time, but odds are good the white student had a head start. It is therefore likely that the black student ran faster. It is therefore reasonable to prefer to admit the black applicant. Note that this is true even if you do not care about "diversity" in your student body.
Now, you might object that this argument is too probabilistic: maybe the black student faced more difficulties than the white student, but maybe not. And this is true; race, here, is serving as a proxy for factors we cannot evaluate directly. But so what? The same can be said for an SAT score or a GPA. The goal for an admissions committee is to acquire as much relevant information about the applicants as is practical. Information about race is both relevant and easy to obtain.
The upshot of all of this, I think, is that if it makes sense for a university to prefer a black student to an otherwise identical white student - and my argument is that it does make sense - then I think the case in favor of affirmative action in general is very strong. Of course, particular affirmative action policies might, say, give too much weight to race, but I do not find it plausible that race should have no weight at all.
P.S. - If information about race is, in fact, relevant, then it shouldn't surprise us that when affirmative action policies are challenged in court, selective organizations like universities, businesses, and the military rise to their defense.
P.P.S. - Note that analogous arguments can be mustered to consider other possible factors in admission. My sense is that, in addition to race, factors like SAT scores, GPA, and economic status hold up pretty well to the "hypothetical committee" test, but that things like volunteerism, extra-curricular participation, and athleticism hold up more poorly.