Merged satellite ocean color data were used to examine trends in the timing and magnitude of phytoplankton blooms. Special emphasis was placed on the peak shift of spring and autumn/winter blooms in the North Pacific and North Atlantic Oceans with bimodal seasonal cycles. Ensemble empirical mode decomposition and Holo-Hilbert spectral analysis were combined to extract seasonal signals and investigate their modulation at multiple time-scales. In the temperate North Atlantic Ocean, earlier and decreasing spring blooms were detected with delayed and increased autumn blooms. The temperate North Pacific Ocean presented delayed and increased spring and autumn blooms, with the delay in fall blooms was larger than spring ones. The separation between two bloom peaks was increasing in both temperate North Atlantic and Pacific regions. The intrinsic variation in bloom timing and magnitude in these selected regions could be clearly extracted by ensemble empirical mode decomposition. Further Holo-Hilbert spectral results showed that changes in the annual cycle and bloom characteristics were modulated by interannual variability. These results suggest the critical role of interannual variability in the modulation of phytoplankton seasonality.