Timing is important in using the signal...for the most part, you don't see officials using the signal right as the action is finishing...there's usually a delay to accommodate for the fact that there might be a whistle from someone's secondary. Of course, if your partner(s) come in any later than that, there's gonna be an issue anyhow with the opposing coach.
Personally, I don't have a problem with the signal when used correctly. It's just another means of giving additional information that can help with coaches/players.