A model of the alignment of azobenzene molecules in polymer film induced by all-optical poling is proposed and verified by experiment. We found that when the writing beams of frequencies ω and 2ω are both linearly polarized with their polarization directions parallel to each other, azobenzene molecules tend to reorient to the direction perpendicular to the writing beams polarization. At the end of the writing process, more molecules orient to the direction perpendicular to the writing beams polarization than those orient to the parallel direction. The alignment of molecules parallel or perpendicular to the polarization of the writing beams is characteristic of polarity or no polarity, respectively. The alignment of molecules along the polarization of writing beams results in the second order nonlinearity in the polymer film. According to the model, a new method to improve the optical poling efficiency is put forward.