at anytime during a game. at the moment people are only removed from games for missing 3 turns in a row, i recently played a game which was held up constantly by someone missing a turn in every 3 or so rounds. it's unfair for the other players, not only is the game held up but you're never quite sure who you should attack because of the history of missed turns and the possibility of them being removed which can end up benefiting the person who persists on missing turns.
i understand that in games which last for 200+ turns it would be unfair for someone to be removed for missing 5 throughout the game, in long games like this the number of turns missed could be reset to zero after every 100 rounds or so.
on a site were you can give your password to someone site to play your games if you're unable to there really is no excuse for missing so many turns throughout a game.