Overview: Due to the large dynamic range and high precision, the optical 3D measurement technology based on phase calculation is widely used in aerospace, automobile manufacturing, biomedicine, cultural relics protection, and other fields to measure a type of surface. For example, fringe projection profilometry and phase measurement deflectometry are used to measure diffuse and specular surfaces, respectively. With the development of advanced manufacturing technology, the measurement of one type of surface cannot meet the current situation. In the existing research on diffuse and specular composite surfaces, 3D topography restoration of large gradient and discontinuous composite surfaces has been achieved. The composite surface measurement system based on fringe projection and fringe reflection can obtain absolute phase rapidly through the multi-color channel of the camera. However, the multi-color channel of the camera not only realizes the rapid measurement but also introduces the errors such as crosstalk and chromatic aberration into the system, which limits the accuracy of 3D topography restoration of composite surface objects. Crosstalk mainly comes from the process of simultaneously shooting different color stripes projected by the projector to the diffuse part of the object and displayed by the transparent display screen on the mirror part of the object. The color difference mainly comes from the different color stripes displayed by the double screen transmission of the mirror part, and there is a phase difference between the absolute phase of the two colors. Therefore, this paper studies the crosstalk elimination method based on the matrix and the color difference elimination method of absolute phase corresponding pixel deviation. Based on the crosstalk matrix, the crosstalk matrix of the projector and the transparent screen is calculated respectively according to the color light intensity relationship between the color camera, projector, and transparent screen. The stripes of different colors projected on the diffuse surface and mirror of the composite object are separated, and the absolute phase of the two parts is obtained. The absolute phase in the horizontal and vertical directions of each color channel was obtained by color orthogonal stripes, and the relationship between phase difference and pixel deviation between color channels was established to complete the pixel matching of different color channels. Finally, the pixel matching points are converted into the matrix obtained by two-dimensional interpolation to realize the pixel deviation correction of each pixel point and eliminate the influence of color difference. The experimental results show that the proposed method reduces the root mean square error of the composite step from 0.479 mm to 0.030 mm, and improves the measurement efficiency and accuracy.