The hidden Z2 symmetry of the asymmetric quantum Rabi model (AQRM) has recently been revealed via a systematic construction of the underlying symmetry operator.Based on the AQRM result,we propose an ansatz for the general form of the symmetry operators for AQRM-related models.Applying this ansatz we obtain the symmetry operator for three models:the anisotropic AQRM,the asymmetric Rabi-Stark model (ARSM),and the anisotropic ARSM.