It bothers me more when the word is used casually instead of as an attack. If you call my son a retard, I can take you on directly, because he is most certainly not a retard, nor will I accept you labelling him (or anyone) in such a derogatory manner. It's the tag-ons that I find most challenging. The "omg that's so retarded!" comments. When I see this type of language, it knocks the wind out of me. I typically comment back and say "not a good word - please use language that actually describes what you are trying to articulate like "omg that's just not right" instead of insulting a group of individuals who deserve to be embraced not maligned".
My friend Ellen over at Love That Max wrote a brilliant post about this subject yesterday, as did Tanis (who I have not yet met, but hope to) at Red Neck Mommy. There is also a terrific site called The Social Challenge that highlights all tweets using the word "retard" and has a program set up where you can respond to these tweets with an informative response without using your personal twitter handle. I encourage you to check all of these sites out. If you are currently a casual "r" word user they will provide you with additional insight as to why you should delete this word from your vocabulary.
No person deserves to be referred to as "retard". It is incredibly hurtful, not only to the person it is directed at, but the people who love that person. I challenge you to challenge yourself and others. Let's create a community based on equality.