Learning an efficient and safe driving strategy in a traffic-heavy intersection scenario and generalizing it to different intersections remains a challenging task for autonomous driving. This is because there are significant differences in the structure of roads at different intersections, and autonomous vehicles need to learn to generalize the strategies they have learned in the training environments. This requires the ego vehicle to capture not only the interactions between agents but also the relationships between agents and the map effectively. To address this challenge, we present a technique that integrates the information of high-definition (HD) maps and traffic participants into vector representations, called Lane Graph Vectorization (LGV). In order to construct a driving policy for intersection navigation, we incorporate LGV into the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm with Prioritized Experience Replay (PER). To train and validate the proposed algorithm, we construct a gym environment for intersection navigation within the high-fidelity CARLA simulator, integrating dense interactive traffic flow and various generalization test intersections with different maps. Experimental results demonstrate the effectiveness of LGV for intersection navigation tasks and outperform the state-of-the-art in our proposed scenarios.