Down syndrome (DS) is associated with a variety of cognitive impairments, notably memory impairments. Due to the high prevalence rates of early-onset dementia associated with DS, it is imperative to understand the comprehensive development of memory impairments beginning in childhood and into adulthood, as this may help researchers identify precursors of dementia at earlier stages of development and pinpoint targets for memory intervention. The current paper provides a systematic, developmentally focused review of the nature of memory difficulties in DS across the lifespan. Specifically, this review summarizes what is known about long-term, short-term, and working memory abilities (distinguishing between verbal and nonverbal modalities) in DS, compared to both mental age-matched typically developing peers and individuals with other forms of intellectual disability (ID) at three developmental stages (i.e., preschool, adolescence, and adulthood). Additionally, this review examines the degree of impairment reported relative to typically developing mental age-matched peers in the existing literature by examining effect size data across memory domains as a function of age. With few exceptions, memory abilities were impaired across the lifespan compared to mental age-matched typically developing peers. Relative to other groups with ID, research findings are mixed. Our review of the literature identified a scarcity of memory studies in early childhood, particularly for STM and WM. In adulthood, research was limited in the LTM and WM domains and very little research has compared memory abilities in older adults with DS to those with typical development. Looking to the future, longitudinal studies could provide a better understanding of the developmental trajectory of memory abilities in DS, and the possible associations between memory abilities and real-world functioning. This research could ultimately inform interventions to improve independence and overall quality of life for those with DS and their families.