The precious metal has struggled to live up to its safe haven credentials