Motion estimation is an important and intensive task in video coding applications. Since the complex-ity of integer pixel search has been greatly reduced by the numerous fast ME algorithm, the computation overhead required by fractional pixel ME has become relatively significant. To reduce the complexity of the fractional pixel ME algorithm, a directionality-based fractional pixel ME algorithm is proposed. The proposed algorithm efficiently explores the neighborhood positions which with high probability to be the best matching around the minimum one and skips over other unlikely ones. Thus, the proposed algorithm can complete the search by examining only 3 points on appropriate condition instead of 17 search points in the search algorithm of reference software. The simulation results show that the proposed algorithm successfully optimizes the fractional-pixel motion search on both half and quarter-pixel accuracy and improves the processing speed with low PSNR penalty.