Extraction of moving objects is an important and fundamental research topic for many video ap-plications. This paper addresses an unsupervised spatio-temporal segmentation scheme to extract moving objects from video sequences. The temporal segmentation localizes moving objects by comparing the motion vector of each block in each frame with the corresponding global motion vector estimated by an outlier rejection (OR) based method. Furthermore, the temporal compensation utilizing the temporal coherence of moving objects is considered in the temporal segmentation to solve the temporarily stopping problem. The detected moving regions usually have discontinuous boundaries and some holes. These regions are then compensated in the spatial domain. In the spatial segmentation, the watershed algorithm considering the global information improves the accuracy of segmentation in the spatial domain. The modified mean filter is presented to suppress some minima. By using a fusion module, moving objects are extracted. Experiments on various sequences have successfully demonstrated the validity of the proposed scheme.