We present detailed analysis of calibration process error for electro-optical detection systems, which can be simplified as the plane rotation around a non-orthogonal axis. By means of octonions it firstly proves that the plane rotation around a non-orthogonal axis can be decomposed into rotations around two perpendicular axes. The rotation is further divided into three steps, and the calibration error is hence discussed and obtained. The simulation and test results indicate that there are large calibration errors in calibration process. The pointing error can be effectively improved after separating error components, which provides a more accurate set data for further compensation.