Low-field nuclear magnetic resonance magnet (2 MHz) is required for rock core analysis. However, due to its low field strength, it is hard to achieve a high uniform B0 field only by using the passive shimming. Therefore, active shimming is necessarily used to further improve uniformity for Halbach magnet. In this work, an equivalent magnetic dipole method is presented for designing shim coils. The minimization of the coil power dissipation is considered as an optimal object to minimize coil heating effect, and the deviation from the target field is selected as a penalty function term. The lsqnonlin optimization toolbox of MATLAB is used to solve the optimization problem. Eight shim coils are obtained in accordance with the contour of the stream function. We simulate each shim coil by ANSYS Maxwell software to verify the validity of the designed coils. Measurement results of the field distribution of these coils are consistent with those of the target fields. The uniformity of the B0 field is improved from 114.2 ppm to 26.9 ppm after using these shim coils.