Space target detection technology based on visible light image sequence is of great significance to improve the defense capability of space debris. The path of an empty debris target is usually a straight line on the image. Firstly, this paper established the motion model of the space target according to its motion characteristics, and proposed a target initial screening method based on image sequence projection frame. In the initial screening target, a directed graph is established according to the constraints of motion, and the trajectory parameters are obtained simultaneously. Then calculating the similarity between the trajectory parameters, from which the track correlation matrix could be obtained. According to the result of correlation, the initial target is reduced to form a directed graph, and the selected node is the target point. Extensive experimental results show that the proposed method can effectively detect space moving objects, and the performance of the proposed algorithm is better than other algorithms. ? 2018 Copyright SPIE.