Is it normal to stress the importance of affluence in careers?

So we've all heard the saying that "money doesn't mean everything", but it grates me every single time somebody says it. I grew up with two parents under the age of 20 with no college degrees who bought an apartment with love to pay bills. It was terrible. We lived in one of the worst cities in America in a rat-infested house. Hell, I used to get bullied in school because the water was turned off so often that I "showered" at a nearby gas station. Speaking of school, I didn't even learn basic multiplication or division until I was fifteen years old.

Now I'm a university student majoring in Medicine. I wouldn't call it my passion directly, but it provides a promise that I'll never have to go back to poverty. It just annoys me that people always judge when I'm frank about my career decision. They say that you should follow your "passion" - but my passion is providing a life for my children and spouse where they don't have to worry about dinner on the table. I could certainly major in Creative Writing and do what I love but, hell, I'd end up just where I started but in debt, too. I see a career as a means to the financial comfort of my passions (travel, theater, politics) - rather than an end itself.

Anyway, does anyone else feel this way?