As an important part of the human-computer interaction system, facial expression recognition has been a hot research field. The convolutional neural network cannot recognize expression sequence although it can train the classification features automatically for the reason that the direction of feature training need to be specified manually. In order to solve this problem, this paper improves the network structure, and proposes a multi convolutional network fusion method that can be used to identify the expression sequences containing multiple frames. First, we construct a number of convolutional neural networks, so that each network processes one frame, and then merge the results in the merge layer, and finally pass the softmax classifier to give the identity result. On the CK+facial expression database, experiments are carried out on the 3rd, 4th and 5th frames of expression sequences, and a high recognition rate is obtained for all experiments. © 2018, The Editorial Board of Journal of Xidian University. All right reserved.