Phillips served as the defensive assistant coach for the San Diego Chargers under Sid Gillman. Gillman later became head coach of the Houston Oilers, and he brought Phillips with him as his defensive coordinator.
Phillips took over as the head coach of the Oilers on January 25, 1975. Under Phillips, the Oilers reached the AFC Championship Game in two consecutive seasons, losing to the Super Bowl champion Pittsburgh Steelers 34–5 in 1978 and 27–13 in 1979.