回顾了基于回报函数逼近的学徒学习的发展历史,介绍了目前的主要工作,总结了学徒学习的一般方法,讨论了线性和非线性假设条件下的回报函数求解,比较了逆向增强学习(IRL)和边际最大化(MMP)两类逼近方法.基于IRL的学徒学习是一种通过迭代的方法用基回报函数的线性组合来逼近真实回报函数的过程.MMP方法可以看作是一类基于梯度下降的最优化方法.综合采用滤波及将策略函数概率化等方法可以降低对专家演示的最优要求.最后指出了该领域存在的问题,提出了未来的研究方向,如在部分可观察马尔可夫决策过程框架下的学徒学习及对不确定策略的学习等. This paper reviews the history of apprenticeship learning based on the approximation of reward functions, introduces the main work at present, summarizes the general methods of apprenticeship learning, solves the reward functions under the linear and nonlinear assumptions, and compares the IRL (Reverse Reinforcement Learning) And marginal maximization (MMP) .Approach-based learning based on IRL is a process of approximating the true reward function by a linear combination of basis-reward functions by an iterative method.MMP can be considered as a class based on gradient descent The optimization method is adopted.It can reduce the optimal requirements for expert presentation by using filtering and probabilistic method of strategy function.Finally, the problems in this field are pointed out and the future research directions are proposed. For example, Apprenticeship learning under the framework of decision-making and learning of uncertain strategies.