A linear theory on the propagation of ultrashort pulses including only a few cycles in underdense plasmas is presented. It is shown that the dispersion in plasmas causes severe distortions in the pulse shape, including pulse chirping and spreading. The analytical calculations coincide very well with those obtained by particle-in-cell (PI3simulations. The upper limit of the peak amplitude of the pulses, above which the linear theory breaks down due to the setting in of nonlinear effects of both the relativistic electron-mass increase and ponderomotive force, is also examined by PIC simulations. At certain high amplitudes, it is found that the ultrashort laser pulses can propagate like solitons.