Hesitance Fuzzy Linguistic Term Sets (HFLTSs) allow decision makers to evaluate a property in several possible linguistic terms. Recently, HFLTSs based fuzzy clustering analysis draws increasing attention. Considering that the current fuzzy clustering algorithm based on HFLTSs still costs large computation, this study proposes a novel orthogonal fuzzy clustering algorithm. Firstly, calculate the distance measures between samples to construct distance measure matrix, and then calculate the matrix's equivalent matrix. Secondly, cut the equivalent matrix according to its confidence level to obtain the corresponding cutting matrix. Finally, obtain the clustering result based on the orthogonal relationship between the column vectors of the cutting matrix. This algorithm has simple steps and low computational complexity. It is also suitable for large-scale fuzzy clustering problems. At last, the feasibility and efficiency of this algorithm are proved by a practical application with k-means clustering algorithm.