I don't think it's fucked up; I for instance am also not attracted to gay guys, and it's not just because they wouldn't be interested in me, because I'm attracted to plenty of people who wouldn't be interested in me. Think also of the many many many men who talk about being attracted to lesbians but are not actually attracted to any lesbians. This is part of why I think talking about two genders is misleading, even though I do it too.