Construction of high resolution images from low resolution sequences having rigid or semi-rigid objects with unified motions is often important in surveillance and other applications. In this paper a novel object-based super resolution reconstruction scheme was proposed, in which a six-parameter affine model-based object tracking and registration method was first used to segment and match objects among a sequence of low resolution frames. The motion model was then further extended to the traditional maximum a posterior (MAP) super resolution algorithm. The proposed object tracking and registration method was evaluated by both simulated and real acquired sequences. The results have demonstrated the high accuracy of the proposed object based method and the enhanced reconstruction performance of the extended approach.