The source-sink relationship determines the ultimate grain yield. We investigated the genetic basis of the relationship between source and sink and yield potential in rice. In two environments, we identified quantitative trait loci (QTL) associated with sink capacity (total spikelet number per panicle and thousand-grain weight), source leaf (flag leaf length, flag leaf width and flag leaf area), source-sink relationship (total spikelet number to flag leaf area ratio) and yield-related traits (filled grain number per panicle, panicle number per plant, grain yield per plant, biomass per plant, and harvest index) by genome-wide association analysis using 272 Xian (indica) accessions. The panel showed substantial variation for all traits in the two environments and revealed complex phenotypic correlations. A total of 70 QTL influencing the 11 traits were identified using 469,377 high-quality SNP markers. Five QTL were detected consistently in four chromosomal regions in both environments. Five QTL clusters simultaneously affected source, sink, source–sink relationship, and grain yield traits, probably explaining the genetic basis of significant correlations of grain yield with source and sink traits. We selected 24 candidate genes in the four consistent QTL regions by identifying linkage disequilibrium (LD) blocks associated with significant SNPs and performing haplotype analysis. The genes included one cloned gene (NOG1) and three newly identified QTL (qHI6, qTGW7, and qFLA8). These results provide a theoretical basis for high-yield rice breeding by increasing and balancing source–sink relationships using marker-assisted selection.