Ion pickup by a monochromatic low-frequency Alfvén wave, which propagates along the background magnetic field, has recently been investigated in a low beta plasma (Lu and Li 2007 Phys. Plasmas 14 042303). In this paper, the monochromatic Alfvén wave is generalized to a spectrum of Alfvén waves with random phase. It finds that the process of ion pickup can be divided into two stages. First, ions are picked up in the transverse direction, and then phase difference (randomization) between ions due to their different parallel thermal motions leads to heating of the ions. The heating is dominant in the direction perpendicular to the background magnetic field. The temperatures of the ions at the asymptotic stage do not depend on individual waves in the spectrum, but are determined by the total amplitude of the waves. The effect of the initial ion bulk flow in the parallel direction on the heating is also considered in this paper.