With the expansion of robotics application fields, robot always faces unpredictable manipulation tasks in hazardous and unstructured environments. Compared to tradition programming and robot learning methods, teleoperation approaches shows great potential in assisting the operator to perform and accomplish complex and uncertain tasks. In order to achieve the goal of manipulating complicated tasks with high accuracy, stability, and convenience, in this paper, we design a robot teleoperation system based on virtual reality device. Our designed system converts the motion information of the VR hand controller into the motion information of the end-effector of a robot arm, realizing the goal of teleoperating robot arm to manipulate complicated tasks, and concurrently viewing the robot working scenarios in real time with headset. Experiments show that our designed system can efficiently perform complicated manipulation tasks with high accuracy and stability, which shows great potential in accomplishing complex tasks and expanding our teleoperation system in more application areas.