Motivated by the problem of expanding the single-trace tree-level amplitude of Einstein-Yang-Mills the-ory to the BCJ basis of Yang-Mills amplitudes,we present an alternative expansion formula in gauge invariant vec-tor space.Starting from a generic vector space consisting of polynomials of momenta and polarization vectors,we define a new sub-space as a gauge invariant vector space by imposing constraints on the gauge invariant conditions.To characterize this sub-space,we compute its dimension and construct an explicit gauge invariant basis from it.We propose an expansion formula in this gauge invariant basis with expansion coefficients being linear combinations of the Yang-Mills amplitude,manifesting the gauge invariance of both the expansion basis and coefficients.With the help of quivers,we compute the expansion coefficients via differential operators and demonstrate the general expan-sion algorithm using several examples.