Gym leaders I struggled with were Aya and Florinia.
The easiest for me were Charlotte (I didn't know she was the hardest until I checked the forums ._.; ), Serra, Terra, Kiki, Samson, Luna (am I the only one who uses Gardevoir w/ Moonblast?) and Radomus.
With everyone else it took a couple of tries sometimes.