Study of the moving small and big target detection and tracking is one of the hot research areas in the domain of computer vision.The captured small target image in complex background has the characteristics of low signal-to-noise ratio and low contrast, therefore preprocessing before detection is needed.Target detection is the process of finding and extracting target from the image sequence.Atter target detection, efficiently tracking the moving target in each frame is needed in the image sequence.On the basis of reading relevant literature, image preprocessing, target detection and target tracking algorithms have been researched and compared, image pre-processing and target detection are the research focuses, in addition, the software part of photoelectric imaging target tracking system has been completed, it is performance verification platform of different algorithms.　　Effective background suppression is the premise of target detection and decides to the performance of the system.At the beginning, several typical background suppressing algorithms are recommended, the background suppressing method based on correlation between adjacent pixel block is discussed in detail and deeply.Through the experiment, the real images are processed by the proposed algorithm in this paper, and the other methods about the image preprocessing are compared, and the performance of different algorithms is evaluated by signal-to-clutter ratio and signal-to-clutter ratio gain.The experimental results indicate that the proposed algorithm in this paper can efficiently suppress the complex background.　　The two different video files (*.avi) was given for this researching, the first video shows moving single plane on the sky with simple background and the second is the moving tank on the ground with complex background.The task is to detect the targets on image sequence and track them with various functions and algorithms.　　In research, some important image pre-processing methods are presented, which include background suppression method, such as morphological operation, high pass filter and median filters.These methods are the basis to the digital image preprocessing.Then after pre-processing methods, the proposed segmentation based on maximum entropy value and Otsu methods are run, which is needed for further computing the geometric centre of target.　　The main researching of this paper focuses on making accurate and effective detection and tracking for small and big targets on various backgrounds.The proposed algorithm for the small target detection on not complex background is Local Entropy Algorithm (LEA), and its improved method for reducing operation complexity is present as Box-Filter method.For the big target with complex background, the combined method is proposed, which includes Mean-Shift algorithm and Kalman filter to solve many tracking problems and overcome the disadvantages of Mean Shift algorithm.　　For realization and implementation of the given algorithms, the programming languages, such as Matlab, Visual Studio 2005, and OpenCV, are used.