Biodiversity interactions between aboveground and belowground biota have been a central focus of ecology in recent years, particularly in the context of environmental change. However, few studies have systematically assessed the biodiversity relationships of plants and soil fauna from rare to abundant taxa along environmental gradients, especially in the rhizosphere. This study aimed at exploring the alpha and beta diversity associations between plants and soil fauna along an elevational gradient while systematically considering rare to abundant plants and animals, and different plant life forms. We sampled soil fauna (litter fauna and rhizosphere fauna of trees, shrubs and herbs) and investigated plant diversity in 119 plots along an elevational gradient (from 1,020 to 1,770 asl) on Dongling Mountain, China. We calculated alpha diversity from both Renyi's generalized entropy function and Jost's effective species numbers assigning different weights to abundant and rare taxa. Beta diversity was obtained based on a presence-absence metric (beta-sim) and an abundance-based metric (beta-morisita). Abundant litter fauna showed decreasing alpha diversity patterns with increasing elevation, whereas rare litter fauna showed hump-shaped patterns. Abundant litter fauna diversity was positively correlated with abundant tree diversity, but rare litter fauna were negatively correlated with abundant shrub diversity. The diversity of rare fauna associated with the herb rhizosphere showed hump-shaped elevational patterns, whereas no patterns were found for abundant fauna associated with the herb rhizosphere. Rare fauna diversity associated with the herb rhizosphere was negatively correlated with rare herb diversity. No beta diversity correlations between rhizosphere fauna and plants were detected, whereas the beta-sim diversity of litter fauna was positively correlated with that of trees and herbs. Our study suggests that litter fauna are more closely coupled with plant diversity than rhizosphere fauna. We conclude that the diversity associations of plants and soil fauna are dependent on plant life form and are largely driven by rare taxa. Furthering this understanding will enhance our ability to predict the responses of belowground communities to environmental change based on information of plant communities.