End-to-end multilingual speech translation (ST) directly models the mapping from the speech in source languages to the text of multiple target languages. While multilingual neural machine translation has been proved effective in modeling the general knowledge with shared parameters and handling inter-task interference with language-specific parameters, it still lacks exploration of when and where parameter sharing matters in multilingual ST. This work offers such a study by proposing a comprehensive analysis on the influence of various heuristically designed sharing strategies. We further investigate the inter-task interference through gradient similarity between different tasks, and improve the parameter sharing strategy in multilingual ST under the guidance of inter-task gradient similarity. Experimental results on the one-to-many MuST-C dataset have shown that the gradient-guided sharing method can significantly improve the translation quality with a comparable or even lower cost in terms of parameter scale.