Nonlinear forces and moments caused by ball bearing were calculated based on relationship of displacement and deflection and quasi-dynamic model of bearing. Five-DOF dynamic equations of rotor supported by ball bearings were estimated. The Newmark-β method and Newton-Laphson method were used to solve the equations. The dynamic characteristics of rotor system were studied through the time response, the phase portrait, the Poincar? maps and the bifurcation diagrams. The results show that the system goes through the quasiperiodic bifurcation route to chaos as rotate speed increases and there are several quasi-periodic regions and chaos regions. The amplitude decreases and the dynamic behaviors change as the axial load of ball bearing increases; the initial contact angle of ball bearing affects dynamic behaviors of the system obviously. The system can avoid non-periodic vibration by choosing structural parameters and operating parameters reasonably.