I don't often post in comment threads, because 1. I really hate arguing, and 2. Other people usually make my points better than I can. This is equally true of this thread, but I think the issue is important enough, and I have come so far in my own understanding of it in the last few years, that maybe I can say something of value.
My favorite description of privilege is John Scalzi's article describing being a straight white male as playing a video game on easy mode:
The answer to Chris Lozos's question (what can a horrid white man do?) is, first, to stop taking it personally. When someone says your race/class/sex benefits from certain factors of society, they are not saying you are a bad human being or that you've automatically had a perfect life. But it's a pretty safe bet that if you were black or gay or female, it would have been harder. (I can't argue with you about the draft sucking for men! But I'd take it over childbirth and the treatment of mothers in America any day. Different discussion!)
At the end of the day, "privilege" is statistics, and human beings suck at statistics. We have to turn everything into a personal anecdote in order to process things. This is why the responding to "white men generally have it easier" with "but I had it hard" doesn't help.
So what we white men can do is:
listen to the people who are affected by the problem
resist the natural urge to be defensive
figure out how to explain the issues to your fellow white dudes, because a side effect of all this is that they will probably hear the message louder and clearer from you than someone more "other"
I'm still figuring all this out myself. In fact, thinking back on just this thread, I can see my own biases at work. The posts that really stand out in my memory are the ones from Tom and Ray (ha! Car Talk!) rather than, say, Nina and Elizabeth who have explained things equally well. Feels weird saying that out loud, but there it is.