The CFL (don't watch NFL so I don't know what it's like down south) has over complicated the issue and it has made video replay detract from the game rather than make it better by assisting the officials get the call right.
First, the officiating situation in the league is a mess. Officials on the field blowing calls, command centre blowing video replay calls and on field officials out right misintepreting and incorrectly applying rules. How can we even think about video replay when we can't even get the initial call correct?
Second, why are they so many situations that can be challenged? In the NHL there is only one thing that can be reviewed, goal/no goal. Why does football have to go beyond the obvious? For every situation you can come up with for football that is reviewed I can give you an eqivilent in hockey that is not. "Did they score or not" should be the only reviewable play. And only on that portion of the play that is directly responsible for the score. Did the ball break the plane? Did the player have possesion and control of the ball when it broke the plane? Did the ball go through the uprights? Did he step out of bounds at the 30 on his way to the end zone would not fit this criteria.
Third, why is the referee staring up into the stands during video replay? He was on the field, he is suppose to know the rules and how to apply them, should he not be looking at the play with the command centre so he can communicate his interpretation of what happened? I believe in the NFL the referee on the field goes under the hood to make the call why do we take our official out of the call making process?
Last, impose a time limit. Two minutes, three minutes the actual length is less important than having a limit the fans and players can see. Knowing there is a limit and seeing it countdown will actually give the impression that the process is faster than there was no limit.
Personally I say get rid of video review until we can find officials capable of calling the game at a level equal to that of the play of the players. If they must keep it limit its use to situations of score/no score.