A Swedish minister's phobia of bananas has reportedly led to government officials asking for rooms to be free of the fruit.