Both beta-amyloid (Ab) deposition and decline in white matter integrity, are brain alterations observed in Alzheimer's disease (AD) and start to occur by the fourth and fifth decades. However, the association between both brain alterations in asymptomatic subjects is unclear. Amyloid positron emission tomography (PET) and diffusion tensor imaging (DTI) were obtained in 282 cognitively normal subjects (age 30-89 years). We assessed the interaction of age by abnormal amyloid PET status (Florbetapir F-18 PET >1.2 standard uptake value ratio [SUVR]) on regional mean diffusivity (MD) and global white matter hyperintensity (WMH) volume, controlled for sex, education, and hypertension. Subjects with abnormal amyloid PET (n = 87) showed stronger age-related increase in global WMH and regional MD, particularly within the posterior parietal regions of the white matter. Sporadic Aβ deposition is associated with white matter alterations in AD predilection areas in an age-dependent manner in cognitively normal individuals.