Phillips was born Esther Mae Jones in Galveston, Texas. Her parents divorced when she was an adolescent, and she divided her time between her father, in Houston, and her mother, in the Watts section of Los Angeles. She was brought up singing in church and was reluctant to enter a talent contest at a local blues club, but her sister insisted. A mature singer at the age of 14, she won the amateur talent contest in 1949 at the Barrelhouse Club, owned by Johnny Otis. Otis was so impressed that he recorded her for Modern Records and added her to his traveling revue, the California Rhythm and Blues Caravan, billed as Little Esther. She later took the surname Phillips, reportedly inspired by a sign at a gas station.