Person head detection is still a challenge due to the large variability in heads’ sizes and orientations, lighting conditions and strong occlusions. Small heads require local information contained in low level layers instead of semantic features of upper layers. But most of these fine details are lost in the early convolutional layers of the deep convolution neural networks (DCNN). In order to improve the overall detection accuracy, it is important to utilize local information from lower layers into the detection framework. In this letter, we use multi-scale representation fusion of DCNN as a way to incorporate lower layers with upper layers for detection. Our proposed model is based on the recent object detection network Single Shot MultiBox Detector (SSD). VGG16 is used as the base network. Batch normalization (BN) layers are used in our proposed multi-task learning method to accelerate training process and improve the robustness. Compared to state-of-the-art methods, our proposed detector achieves superior person head detection performance on the HollywoodHeads dataset (81.0 AP) and Casablance dataset (78.5 AP).