Advancing age is associated with both declines in episodic memory and degradation of medial temporal lobe (MTL) structure. The contribution of MTL to episodic memory is complex and depends upon the interplay among hippocampal subfields and surrounding structures that participate in anatomical connectivity to the cortex through inputs (parahippocampal and entorhinal cortices) and outputs (fornix). However, the differential contributions of MTL system components in mediating age effects on memory remain unclear. In a sample of 177 healthy individuals aged 20-94 we collected high-resolution T1-weighted, ultrahigh-resolution T2/PD, and diffusion tensor imaging (DTI) MRI sequences on a 3T Phillips Achieva scanner. Hippocampal subfield and entorhinal cortex (ERC) volumes were measured from T2/PD scans using a combination of manual tracings and training of a semiautomated pipeline. Parahippocampal gyrus volume was estimated using Freesurfer and DTI scans were used to obtain diffusion metrics from tractography of the fornix. Item and associative episodic memory constructs were formed from multiple tests. Competing structural equation models estimating differential association among these structural variables were specified and tested to investigate whether and how fornix diffusion and volume of parahippocampal gyrus, ERC, and hippocampal subfields mediate age effects on associative and/or item memory. The most parsimonious, best-fitting model included an anatomically based path through the MTL as well as a single hippocampal construct which combined all subfields. Results indicated that fornix microstructure independently mediated the effect of age on associative memory, but not item memory. Additionally, all regions and estimated paths (including fornix) combined to significantly mediate the age-associative memory relationship. These findings suggest that preservation of fornix connectivity and MTL structure with aging is important for maintenance of associative memory performance across the lifespan.