The Steered Response Power-PHAse Transform (SRP-PHAT) localization algorithm has high accuracy but poor real-time performance. In response to this problem, this study introduces a localization algorithm based on Time Difference Of Arrival (TDOA) to improve real-time performance and then proposes a combination algorithm based on TDOA and Search Space Clustering (SSC), search space shrinking clustering, to optimize SRP-PHAT. First, the TDOA localization algorithm is used to estimate the range of the sound source in the direction angle and radial distance after outlier correction. Then, the search area is shrunk according to the estimated sound source range. Finally, fine-grained (5 cm) space search calculations in the shrinking area are performed by the SRP-PHAT-SSC algorithm to obtain the three-dimensional (3D) coordinates of the estimated sound source. A five-element microphone array and the virtual source method are employed to simulate the indoor sound field, and the 3D localization of the sound source is simulated by Matlab. The experimental results show that compared with the Full Grid Search (FGS) algorithm and the SSC algorithm based on SRP-PHAT, the improved algorithm has great real-time performance and robustness in 3D localization.