Measuring early tau accumulation is important in studying aging and Alzheimer disease and is only as accurate as the signal-to-noise ratio of the tracer. Along with aggregated tau in the form of neurofibrillary tangles, 18F-flortaucipir has been reported to bind to neuromelanin, monoamine oxidase, calcifications, iron, leptomeningeal melanocytes, and microhemorrages. Although 18F-flortaucipir successfully differentiates healthy controls (HCs) from subjects with Alzheimer disease, variability exists in the cortical signal in amyloid-negative HCs. We aimed to explore the relationship between off-target binding signal and variability in the cortical signal in HCs. Methods: Subjects (n = 139) received 11C-Pittsburgh compound B (PIB) and 18F-flortaucipir PET scans and a magnetization-prepared rapid gradient echo MRI scan. PET frames were realigned and coregistered to the MR images, which were segmented using FreeSurfer. In amyloid-negative HCs (n = 90; age range, 21-94 y), 7 nonspecific or off-target binding regions were considered: caudate, pallidum, putamen, thalamus, cerebellar white matter, hemispheric white matter, and choroid plexus. These regions of interest were assigned to 3 similarly behaving groups using principle components analysis, exploratory factor analysis, and Pearson correlations for caudate, putamen, and pallidum (also correlated with age); thalamus and white matter; and choroid plexus. In amyloid-negative HCs with 11C-PIB and 18F-flortaucipir scans, correlations were calculated between white and gray matter before and after partial-volume correction. Results: The correlation between white and gray matter disappeared after partial-volume correction in 11C-PIB (r2 = 0) but persisted for 18F-flortaucipir (r2 = 0.27), demonstrating that the correlation between white and gray matter signal in 18F-flortaucipir is not solely due to partial-volume effects. A linear regression showed that off-target signal from putamen and thalamus together explained 64% of the variability in the cortical signal in amyloid-negative HCs (not seen in amyloid-positive HCs). Variability in amyloid-negative HCs but not amyloid-positive HCs correlated with white matter signal (unrelated to partial-volume effects) and age-related off-target signal (possibly related to iron load). Conclusion: The noise in the 18F-flortaucipir measurement could pose challenges when studying early tau accumulation.