Of course colour has an effect - not only on the impressions of an observer but on the attitude of the team wearing the colours. Studies show that colour impressions vary across cultures and between age-groups. Black for instance makes a huge impression on younger people because it's 'bad' and apparently something that they are convinced had never occured to their parents. In western culture blue is seen as more positve that red. Eastern cultures are the opposite.
There are some universals that are applicable to football - all white makes the players look bigger to the opponent. Black makes the wearer feel tougher and intimidating.
There have been huge leaps in team performance based upon uniform changes. Tampa went from last to first in one season, Atlanta and Philidelphia both became contenders.
So what does this say for CFL refs? not much really. You have to take a lot of factors into account and you'd have to run an analysis of the past 20+ years, game by game, to see trends.
Good topic. It adds one more factor to our post-game analysis.