Is it normal my fiance feels indifferent towards his family?

My fiance is an incredibly sweet man. I call him "Butters" because he has a crop of blond hair on his head and matches in personality to the character (He literally used to walk into rooms saying, "Hey fellas!" until I pointed it out and he got self conscious lol, he hates the Butters comparison).

What I find strange is he is very apathetic towards his family members. He is the oldest boy and was the oldest sibling in the household growing up because his sisters were practically adults when they were born (they have an old dad).

He has two younger brothers. The older of the two is 22 and dealing with a lot of issues. He has epilepsy and bipolar disorder. He got kicked out of his father's house for threatening his father with a fireplace poker. He lives with his mother and her boyfriend in Florida. Their mother's boyfriend and his brother frequently get into conflict. This brother reaches out a lot to my fiance. I can tell my fiance's brother, who I have known since he was 12, really loves his brother. However, my fiance treats him with annoyance and that really confuses me given my fiance's demeanor otherwise.

A few days ago he finally shared with me, "If I didn't see my family ever again, I wouldn't really care." I had a feeling he felt this way but I didn't want him to.

I feel like I am part of the reason. We got together in high school and for some reason his family HATED me. His mom went to one of our teachers at one point to try and break us up and the teacher responded, "Well, he's 18 now and she's 17. Get him arrested for sex with a minor." The brother mentioned above was incredibly antagonistic towards me as well, and still is. I don't think he's aware I'm the only reason his brother responds to him, when I force him to so his brother doesn't feel so alone. I don't want his brother to feel like no one loves him. The only real back and forth I had with my fiance's father was years ago when I pleaded for help during a mental breakdown of my fiance (early college years, no money, big amount of stress culminating in a breakdown). His father said I am the reason his son was in that state and I will never be any help to him. The youngest brother, who is an adult now (18) is likely joining the military to leave the household. That's all I know.

I had a big family and my parents fostered other kids so I have siblings that aren't even related to me. I think my family also may have issues with codependency on each other, so we stick together even if we shouldn't. So apathy towards family is a foreign concept to me. I also feel guilty if I am the reason he doesn't want to associate with his family.