The output intensity variations of the laser used in a prism coupling system are observed and found to be induced by the exteal optical feedback, which comes from the reflection on the prism. The intensity variations are explained with laser theory. The trough in the intensity variation corresponds to the position of the prism when the output light beam propagates perpendicularly to the prism. Based on the trough a new method for rotating the prism and reading out the step numbers is proposed, by which the angle 0° in the system need not to be calibrated. It is proven by experiment that the new method would improve the accuracy of the refractive index up to ±0.00001 and thickness to ±1 nm.