Taylor was born in Crawfordsville, Arkansas. He had one release, "Somewhere to Lay My Head", on Chicago's Chance Records in the 1950s, as part of the doo-wop group Five Echoes. His singing was strikingly close to that of Sam Cooke, and he was hired to take Cooke's place in Cooke's gospel group, the Soul Stirrers, in 1957.
A few years later, after Cooke had established his independent SAR Records, Taylor signed on and recorded "Rome Wasn't Built In A Day" in 1962. However, SAR Records quickly became defunct after Cooke's death in 1964.
In 1966, Taylor moved to Stax Records in Memphis, who dubbed him "The Philosopher of Soul". He was, for a time, their best-selling artist, outselling such stars as Otis Redding and Sam & Dave. His hits included "I Had a Dream" and "I've Got to Love Somebody's Baby" and especially "Who's Making Love?", which reached No. 5 on the Billboard Top 40 and No. 1 on the R&B charts in 1968.
As Stax folded in the early 1970s, Taylor switched to Columbia Records, where he made his best-known hit, "Disco Lady", in 1975. "Disco Lady" was the first certified platinum record.