Online video text detection is important in many applications, such as real-time translator and wearable camera system for visually-impaired. Existing methods for video text detection perform unsatisfactorily mainly because of the inferior text detection result and insufficient utilization of spatial and temporal information. Besides, the majority of them work in offline mode. In this paper, we propose an online video text detection method which works nearly in real time. We detect texts in each frame using a EAST based text detector, and formulate the online text tracking problem as decision making in Markov Decision Processes (MDPs). The similarity function in tracking stage can be learned by reinforcement learning. Besides, text detection and tracking are naturally unified by state transactions in the MDP. Extensive experiments on three benchmark datasets, ICDAR 2015, Minetto, and Youtube Video Text, verify the effectiveness of our method.