Abstract It has long been suggested that nearby pulsars within ～1 kpc are the leading candidate of the 10–500 GeV cosmic-ray positron excess measured by PAMELA and other experiments. The recent measurement of the surface brightness profile of TeV nebulae surrounding Geminga and PSR?B0656+14 by the High-Altitude Water Cherenkov Observatory (HAWC) suggests inefficient diffusion of particles from the sources, giving rise to a debate on the pulsar interpretation of the cosmic-ray positron excess. Here we argue that GeV observations provide more direct constraints on the positron density in the TeV nebulae in the energy range of 10–500 GeV and hence on the origin of the observed positron excess. Motivated by this, we search for GeV emission from the TeV nebulae with the Fermi Large Area Telescope (LAT). No spatially extended GeV emission is detected from these two TeV nebulae in the framework of two-zone diffusion spatial templates, suggesting a relatively low density of GeV electrons/positrons in the TeV nebulae. A joint modeling of the data from HAWC and Fermi-LAT disfavors Geminga and PSR?B0656+14 as the dominant sources of the positron excess at ～50–500 GeV for the usual Kolmogorov-type diffusion, while for an energy-independent diffusion, a dominant part of the positron excess contributed by them cannot be ruled out by the current data.