Early and accurately detecting faults is crucial for the mod manufacturing system. We proposed a novel Deep fault diagnosis (DFD) method for rotating machinery with scarce labeled samples. A spectrogram of the raw vibration signal is calculated by applying a Short-time Fourier transform (STFT). Several candidate Support vector machine (SVM) models are trained with different combinations of features in the feature pool with scarce labeled samples. By evaluating the pretrained SVM models on the validation set, the most discriminative features and best-performed SVM models can be selected, which are used to make predictions on the unlabeled samples. The predicted labels reserve the expert knowledge originally carried by the SVM model. They are combined together with the scarce fine labeled samples to form an Augmented training set (ATS). Finally, a novel 2D deep Convolutional neural network (CNN) model is trained on the ATS to le more discriminative features and a better classifier. Experimental results on two fault diagnosis datasets demonstrate the effectiveness of the proposed DFD.