Cities are prone to ecological problems, yet the impacts of rapid global urbanization on the feedback between above- and belowground subsystems remain largely unknown. We sampled the roots of 8 common herbaceous plants within the Fifth Ring (urban areas) and in Jiufeng National Forest Park (rural areas) in Beijing (China) to assess the impacts of urbanization on the network of plant-arbuscular mycorrhizal (AM) fungal associations. Using Illumina MiSeq sequencing, 81 AM fungal OTUs were identified in 78 herb root samples. The Shannon, Simpson, and Pielou indices of root AM fungi in urban areas were significantly higher than those in rural areas. In this study, a significantly nested mycorrhizal association network was observed in rural areas (NODF = 64.68), whereas a non-nested pattern was observed in urban areas (NODF = 55.50). The competition index C-score (0.0769) of AM fungi in urban areas was slightly lower than that in rural areas (0.1431), and the species specialization (d') of 8 host plants and fungal dissimilarity among 8 host plants in urban areas were significantly lower than those in rural areas. Convergent associations among hosts may be an important factor influencing this non-nested pattern of the plant-AM fungi network in urban areas. Generalists, rather than specialists, were enhanced during the establishment of mycorrhizal associations in urban areas. Our results suggest that reduced selectivity of host plants, and generalist promotion and specialist reduction of AM fungi during urbanization may contribute to the non-nested network of plant-AM fungal associations.