ing at the problems of deep network depth, low utilization rate of feature relationship and low time efficiency in existing pedestrian recognition algorithm based on deep learning, this paper proposes an improved method based on squeeze and excitation residual neural network (SE-ResNet) and feature fusion. By introducing the squeeze and excitation (SE) module, the features are compressed and excited on the feature channels, and then weights are assigned to each channel to enhance the useful feature channels and suppress the useless feature channels to reduce the depth of the network model. In order to improve the recognition accuracy and computing efficiency, shallow features and deep features are used, and feature extraction modules are deleted. The relationship between the size of convolution kernel and the running time and recognition accuracy is modeled to find the best balance point. Experimental results show that compared with ResNet50, the recognition accuracy of this algorithm is 4.26 percentage points higher, mean average accuracy value is 17.41 percentage points higher. Compared with other classic algorithms, the recognition accuracy of this algorithm has also been improved to varying degrees, and the robustness is better.