How to extract robust feature is an important research topic in machine leing community.In this paper,we investigate robust feature extraction for speech signal based on tensor structure and develop a new method called constrained Nonnegative Tensor Factorization(cNTF).A novel feature extraction framework based on the cortical representation in primary auditory cortex(A1)is proposed for robust speaker recognition.Motivated by the neural firing rates model in A1,the speech signal first is represented as a general higher order tensor,cNTF is used to le the basis functions from multiple interrelated feature subspaces and find a robust sparse representation for speech signal.Computer simulations are given to evaluate the performance of our method and comparisons with existing speaker recognition methods are also provided.The experimental results demonstrate that the proposed method achieves higher recognition accuracy in noisy environment.