No matter how you look at it, Swagger IS completely luck based, there is absolutely NO skill involved with that move. I don't care how much you want to delude yourself in thinking otherwise. It does not take skill to predict a switch with Swagger, because that is gonna happen since you're spamming the move.(not to mention that prediction isn't a skill, it's more based on experience with the game). It is a 50/50 chance for the opponent to hit themselves, that's how confusion works.
Moving on. Luck has no counter, none. You cannot say you can counter a 50/50 chance. You can prevent the move with things like Magic Bounce and Own Tempo, but that's not countering luck, that's countering Swagger. We're getting that confused. It is a luck abusing move, not luck itself. If it lands you can't do anything but hope you are the lucky one.(With the exception of Own Tempo).
Regarding your edit: 10% miss chance is not a lot, so it missing would be unlucky for the user, and lucky for the opponent. Which means it would still take luck to avoid confusion. The 50/50 only applies when Swagger hits and confuses the target. So yes, it's always 50/50.
Any physical OU ground type(and some that are not ground types) can OHKO Klefki without +2, and common knowledge would tell you to bring in something that can hit back with SE damage. What you're forgetting is that thing also has Substitute and can just spam that until you hit yourself.
Anyway. Despite all that, Swagger by itself wouldn't be bad. A lot of people here use Klefki as their example, and while I tried to avoid that, it is the the main reason. There are many factors in why Prankster+Swagger wasn't a thing in 5th. Weather is the biggest, namely Sand. If Sableye(which was the only good Prankster then) was used, it had Will-O-Wisp to support the team. The reason was for things like Excadrill, Landorus, Tyranitar, etc. Will-O-Wisp, as you can imagine, would completely contradict the point of Swagger. So it was not used. Now that weather has been nerfed and we have another good Prankster Pokemon(with great typing I might add) you can see its rise in popularity.
As I mentioned above, Swagger by itself wouldn't be bad. So, I believe that if we do have a ban, it should be a complex ban, banning Prankster+Swagger. Klefki will still have use, as will Sableye, and even Whimsicott. Anyone that still wants to use Swagger still can, and it will be easier to deal with.
Voting has started. What we should focus on is how we'll go about banning if the votes lean towards a ban.
As for Baton Pass, I've only seen it from Slant and DD. It was seen last gen too. I don't have much to say on that one, and voting has started anyway, so it hardly matters now.