Sandy signed from Butterburn Youth Club in August 1961, after having trials with United, Dundee and West Ham United. Originally the third choice keeper behind Alec Brown and Lando Ugolini, he made his first appearance as a substitute in a friendly against a British Army side in October 1961, and made his debut in major competition against Partick Thistle at the end of that season. Ugolini and Brown were then released but United signed Donald Mackay who began season 1962-63 as first choice.
Following injury to Mackay early in the season, Sandy took over in goal and retained the number one jersey for the next two years, playing in the Club's first ever semi-final appearances in the Scottish Cup in 1963 and the League Cup in 1964. Originally signed on a part-time basis whilst working as a gardener, he lost his job in July 1963 when he chose to travel with United on the tour of Southern Africa. Now full-time with United, he was named in the Scotland Under-23 side to face France in May 1964. United were fortunate to have two good goalkeepers throughout the sixties, with Davie and Mackay sharing goalkeeping duties at Tannadice throughout most of the decade. By season 1966-67 Sandy was again first choice and was in the line-up that recorded famous home and away wins over Barcelona as United stepped on to the European stage for the first time. He also played in both legs against Juventus in the next round, and in the summer was part of a sixteen man squad that toured the United States as United represented Dallas Tornado in the United Soccer Association.
By January 1968, Sandy was deputising once more, and after handing in a transfer request he was sold to Luton Town in October 1968 for a fee of around £8,000. After two seasons at Kenilworth Road he signed for First Division side Southampton, but found it hard to displace regular keeper Eric Martin. With Don Mackay released, new United manager Jim McLean brought Sandy back to Tannadice in May 1972 for a reported fee of £5,000. Now battling for the keepers jersey with the established Hamish McAlpine, it was Sandy who was the preferred choice towards the end of the 1973-74 season, and reached the pinnacle of his United career when he was in the side that faced Celtic in the Club's first appearance in the Scottish Cup Final in May 1974.
Shortly afterwards, he emigrated to New Zealand where he signed for North Shore United. Sandy also played at international level for New Zealand, winning eight caps and he was later appointed Director of Coaching for the New Zealand FA.
In February 2016, Sandy returned to Dundee to be inducted into the Dundee United Hall of Fame.