In this paper, the response of fiber Bragg gratings (FBGs) subjected to the ultrasonic wave has been theoretically and experimentally investigated. Although FBG sensors have been widely used in the ultrasonic detection for practical structural health monitoring, the relationship between the grating length and ultrasonic frequency is not yet to be obtained. To address this problem, an ultrasound detection system based on FBGs is designed and the response sensitivity of different lengths gratings are detected. Experimental results indicate that the grating with 3 mm length has a higher sensitivity when detecting high frequency ultrasonic wave, and the amplitude can be up to 0.6 mV. The 10 mm length grating has better detection sensitivity for low frequency ultrasonic wave and the amplitude is 0.8 mV. The results of this analysis provide useful tools for high sensitivity ultrasound detection in damage detection systems.