As one of the most widely used clustering techniques, the fuzzy k-means (FKM) assigns every data point to each cluster with a certain degree of membership. However, conventional FKM approach relies on the square data fitting term, which is sensitive to the outliers with ignoring the prior information. In this paper, we develop a novel and robust fuzzy k-means clustering algorithm, namely, joint learning of fuzzy k-means and nonnegative spectral clustering with side information. The proposed method combines fuzzy k-means and nonnegative spectral clustering into a unified model, which can further exploit the prior knowledge of data pairs such that both the quality of affinity graph and the clustering performance can be improved. In addition, for the purpose of enhancing the robustness, the adaptive loss function is adopted in the objective function, since it smoothly interpolates between l(1)-norm and l(2)-norm. Finally, experimental results on benchmark datasets verify the effectiveness and the superiority of our clustering method.