Not a huge deal, but it's happened to me and others where you inadvertantly drop a team game. If everyone is trying to set up with a particular partner, it really mucks things up.
A very simple solution would be to add a "are you sure" prompt when doing this.