Stipa purpurea is among constructive endemic species in the alpine steppe on the Qinghai–Xizang Plateau. To reveal the fungal community structure and diversity in the rhizosphere and roots of this important grass and to analyze the potential influence of different habitats on the structure of fungal communities, we explored the root endophyte and the directly associated rhizosphere communities of S. purpurea by using internal transcribed spacer rRNA cloning and sequencing methods. We found that the roots of S. purpurea are associated with a diverse consortium of Basidiomycota (59.8%) and Ascomycota (38.5%). Most fungi obtained from rhizosphere soil in S. purpurea have been identified as Ascomycetes, while the high proportion detected in roots were basidiomycetous endophytes. The species richness, diversity, and evenness of fungal assemblages were higher in roots than in the rhizosphere soil. Fungi inhabiting the rhizosphere and roots of S. purpurea are significantly different, and the rhizosphere and endophyte communities are largely independent with little overlap in the dominant phyla or operational taxonomic units. Taken together, these results suggested that a wide variety of fungal communities are associated with the roots and rhizosphere soil of S. purpurea and that the fungal assemblages are strongly influenced by different habitats.