We experimentally investigate the effects of the surface roughness of gold thin films on the properties of surface plasmon resonance. By annealing at different temperatures, film samples with different surface morphologies are obtained. Specifically, due to the diffusion of the gold atoms towards the films’ surface, the surface root-mean-square roughness decreases with the increasing annealing temperature. Then, we measure the surface plasmon resonance of the samples. The results show that the resonance angle of the surface plasmon resonance is sensitive to the root-mean-square roughness, and it gradually decreases by reducing the surface root-mean-square roughness.