Computational models of eye-movement control during reading provide precise quantitative descriptions of the perceptual, cognitive, and motoric processing that guide readers′ eyes, but are based on numerous equivocal a priori theoretical assumptions. This article describes an alternative approach to understanding eye-movement control: Using reinforcement learning to examine how complex eye-movement behaviors emerge from the requirement to identify words rapidly in the context of known psychological and physiological constraints(e.g., limited visual acuity). An example simulation is reported, as are key results from an fMRI experiment that demonstrates that structures implicated in reinforcement learning support the learning of eye-movement behavior in humans.