Marginal Fisher analysis (MFA) not only aims to maintain the original relations of neighboring data points of the same class but also wants to keep away neighboring data points of the different classes. MFA can effectively overcome the limitation of linear discriminant analysis (LDA) due to data distribution assumption and available projection directions. However, MFA confronts the undersampled problems. Generalized marginal Fisher analysis (GMFA) based on a new optimization criterion is presented, which is applicable to the undersampled problems. The solutions to the proposed criterion for GMFA axe derived, which can be characterized in a closed form. Among the solutions, two specific algorithms, namely, normal MFA (NMFA) and orthogonal MFA (OMFA), are studied, and the methods to implement NMFA and OMFA are proposed. A comparative study on the undersampled problem of face recognition is conducted to evaluate NMFA and OMFA in terms of classification accuracy, which demonstrates the effectiveness of the proposed algorithms.