The Zn_(1-x)MgxO thin films were deposited on sapphire substrates by reactive electron beam evaporation deposition (REBED). The X-ray diffraction (XRD) measurement demonstrates that these films undergo phase transition from hexagonal to cubic with increasing the Mg concentration. Absorption coefficients at 532 nm of the samples were obtained from the absorption spectra. Using optical Kerr effect, the third-order susceptibilities of the ternary films over a wide range of Mg concentrations were determined. The magnitude of χ(3) of the ternary Zn_(1-x)MgxO films is order of 10^(-11) esu at λ = 532 nm. The sample with phase mixture of both hexagonal and cubic structures shows the largest third-order susceptibility. The difference observed in the magnitude of χ(3) of Zn_(1-x)MgxO films is attributed to the different microstructures of the ternary films, such as crystalline phase separation and crystal grains that enhance stimulated scattering.