I never thought I fit in with LGBT because I'm pansexual but never felt "gay." Come from a generation of playing "smear the q-s." And teachers telling us that "there are two kinds of men, steers and q-s."
Due to this, I definitely see the your point. I was called this a lot as a kid, and it definitely hurt, but it was before I was out even to myself, and haven't been called that as an adult, so I don't have that perspective.
However, it wasn't until it started to be reclaimed that I came to terms with who I am and the people I am attracted to. Q just feels right for where I am on sexuality and gender if that makes sense. It is the only slot I feel like I belong.