Brillouin microscopy, which maps the elastic modulus from the frequency shift of scattered light, has evolved to a faster speed for the investigation of rapid biomechanical changes. Impulsive stimulated Brillouin scattering (ISBS) spectroscopy has the potential to speed up measurement through the resonant amplification interaction from pulsed excitation and time-domain continuous detection. However, significant progress has not been achieved due to the limitation in signal-to-noise ratio (SNR) and the corresponding need for excessive averaging to maintain high spectral precision. Moreover, the limited spatial resolution also hinders its application in mechanical imaging. Here, by scrutinizing the SNR model, we design a high-speed ISBS microscope through multi-parameter optimization including phase, reference power, and acquisition time. Leveraging this, with the further assistance of the Matrix Pencil method for data processing, three-dimensional mechanical images are mapped under multiple contrast mechanisms for a millimeter-scale polydimethylsiloxane pattern immersed in methanol, enabling the identification of these two transparent materials without any contact or labeling. Our experimental results demonstrate the capability to maintain high spectral precision and resolution at a sub-millisecond integration time for one pixel. With a two-order improvement in the speed and a tenfold improvement in the spatial resolution over the state-of-the-art systems, this method makes it possible for ISBS microscopes to sensitively investigate rapid mechanical changes in time and space.