I used to assume Gym Leaders were just the strongest trainer in their city/town, although with Blue, it seems like you don't have to live there to be the Gym Leader. It's possible places like Pallet Town are too small to build a gym in.
I've always wondered why Blue is a Gym Leader when he's at Champion level. IMO he should be an Elite Four at least, and the Champion himself if Lance didn't surpass him. I don't really expect a gym challenger to be able to beat a former Champion unless like the player they're on their second region. Cheren makes sense considering he started over with a new team.