An improved rate distortion optimization (RDO) algorithm in JPEG2000 is proposed. The proposed algorithm is suitable for integrated circuit (IC) implementation and can reduce 30% computational cost. A hardware architecture which includes control unit, memory, divider, data converter is also given to implement the algorithm. The circuit based on the improved algorithm is tested on FPGAs and integrated in a JPG2000 chip codec core.