Recognition of individual’s emotions using the data of speech and natural gait has been proved effective by previous studies. This study establishes a supervised regression model based on individual’s speech and natural gait data to predict the scores of depression and anxiety. The results show that the performance of the predictive model based on gait features is better than that of speech features. The basic processes are as follows: First, we recruited 88 participants and collected their speech data and natural gait data in laboratory; Second, we required each participant to finish the anxiety and depression scales (PHQ-9, CES-D, T-AI, and GAD-7), to get the anxiety and depression scores as the tagging data for modeling; Third, we extracted 6125 features from each participant’s speech data and 1602 features from the gait data; Finally, we used a variety of machine learning algorithms to train and test the predictive models. The results show that the max Pearson Correlation Coefficient between the predicted scores and the questionnaire scores of the speech-based model is 0.43, and the max Pearson Correlation Coefficient of the gait-based model is 0.73, which is much better than the speech-based model.