Is it normal to be afraid of men?

I'm uncertain when it comes to my feelings of the opposite or same sex and when it comes to entering a relationship with either. I was abused as a child. Both sexually, physically, emotionally, and physical torture. I feel that this -- might be why I feel I cannot be around men, or be in a relationship with them, but I can with a woman?

I consider myself Pansexual. Though I realize it's easier for me to have a crush on another woman, and be able to pursue it if I wish; but if it involves a man, it stifles me. I know I'm attracted to men, but how can I ever be with them if I'm always uncertain, afraid, and that stifles me from every actually being with one sexually? There is just something about it -- I don't know. Being around men makes me constantly think about what happened to me as a child. I'm worried about what they think, what they would do, if I'm safe, and all these overwhelming feelings I find it hard to be myself and just -- normal.

I've been involved with one man in my life and during the times he'd want to touch me, and I'd allow it I couldn't really enjoy it. My mind was somewhere else and I was thinking about what happened and it wouldn't allow me to enjoy it. It came to the point that I just wasn't attracted him (Not my type, though) and I refused to let him touch me.

Though it's easy for me to be involved with someone on an emotional level, I find my attraction to woman is usually just that -- on an emotional level. I have no real desire, unless I find I may really -- really -- love the woman, to please her sexually or be involved with one sexually. I have no qualms when it comes to thinking about sex with a man; it's something I'd "prefer" and desire, yet though the desire is there, the preference is there, I can't.. seem to go about it in actuality. I never fantasize about woman. I can't look at lesbian porn and enjoy it, and I'm frankly not very fond of the female genitalia. I'm just not turned on by them physically, and it's exceptionally hard to get me that way unless they've rubbed me the right way, so to speak -- emotionally. I'm confused by this. I'm having trouble understanding.

I've come to believe that though I'm accepting of both sexes, if I'm really attracted to woman, or if I really am to men but cannot seem to get past this odd stifling fear -- is it normal?

How can I even solve this? I want to live my life not being afraid of men or being with one. I want to be able to pursue a happy, healthy relationship with one and not be afraid of real intimacy.