In this paper, the clustering and resource allocation problem in device-to-device (D2D) multi-cast transmission underlay cellular networks are in-vestigated. For the sake of classifying D2D users into different D2D multicast clusters, a hybrid intelligent clustering strategy (HICS) based on unsupervised ma-chine learning is proposed first. By maximizing the to-tal energy efficiency of D2D multicast clusters, a joint resource allocation scheme is then presented. More specifically, the energy efficiency optimization prob-lem is constructed under the quality of service (QoS) constraints. Since the joint optimization problem is non-convex, we transform the original problem into a mixed-integer programming problem according to the Dinkelbach algorithm. Furthermore, to avoid the high computational complexity inherent in the traditional resource allocation problem, a Q-Learning based joint resource allocation and power control algorithm is proposed. Numerical results reveal that the proposed algorithm achieves better energy efficiency in terms of throughput per energy consumption.