Well I'm still not 100% convinced by the idea. I've basically moved on from utter derision, to speculating that there might be something to think about, but I'm not convinced that to lay the situation out in these terms is the best philosophical basis for discussion.

One difficulty is that the whole idea has aspects of the 'emperor's new clothes' about it. That is to say only the members of a self-selected elite can apparently diagnose this privilege, and you have to acknowledge it to get into their club. It's not saying 'when you do x you hurt me*', it's saying that merely by you being you, you are hurting me.
* and ultimately yourself, and society, under the whole 'the patriarchy oppresses men too' concept.

The other thing I suppose is the middle ground argument above. I cannot say that I accept your idea and I will act upon it in a way I see fit - it appears that if I accept your argument I must accept your course of action.

I suppose I also want someone to explain it to me in my own language - that is to use a clichéd idea, physical scientists don't get the language of the social scientists.

Ultimately I just want to scream out 'stop being oppressed!' to people, but I don't imagine for a second that is useful.