Is it normal to really hate the contacts feature of email services?

Okay, maybe it's a little convenient to be able to put someone's email address in the "To:" field just by clicking their name, for the few people I actually email very often. But what I hate is that everyone I've ever sent an email to since the beginning of time is automatically added to the list. On my Gmail account, I have upwards of 80 people listed as contacts for me, including tech support and information service emails, and only a handful of them are people I've emailed more than once or twice. I'd delete the ones I don't want but in order to do that you have to go to their contacts profile then select to delete them. It's one by one, there's no way to mass delete people you don't talk to. That's just more effort than it's worth. Is anyone else bothered by this?