Cross-modal selective attention enhances the processing of sensory inputs that are most relevant to the task at hand. Such differential processing could be mediated by a swift network reconfiguration on the macroscopic level, but this remains a poorly understood process. To tackle this issue, we used a behavioral paradigm to introduce a shift of selective attention between the visual and auditory domains, and recorded scalp electroencephalographic signals from eight healthy participants. The changes in effective connectivity caused by the cross-modal attentional shift were delineated by analyzing spectral Granger Causality (GC), a metric of frequency-specific effective connectivity. Using data-driven methods of pattern-classification and feature-analysis, we found that a change in the alpha band (12 Hz-15 Hz) of GC is a stable feature across different individuals that can be used to decode the attentional shift. Specifically, auditory attention induces more pronounced information flow in the alpha band, especially from the parietal-occipital areas to the temporal-parietal areas, compared to the case of visual attention, reflecting a reconfiguration of interaction in the macroscopic brain network accompanying different processing. Our results support the role of alpha oscillation in organizing the information flow across spatially-separated brain areas and, thereby, mediating cross-modal selective attention.