A rapid rise in age at parenthood in contemporary societies has increased interest in reports of higher prevalence of de novo mutations and health problems in individuals with older fathers, but the fitness consequences of such age effects over several generations remain untested. Here, we use extensive pedigree data on seven pre-industrial Finnish populations to show how the ages of ancestors for up to three generations are associated with fitness traits. Individuals whose fathers, grandfathers and great-grandfathers fathered their lineage on average under age 30 were ~13% more likely to survive to adulthood than those whose ancestors fathered their lineage at over 40 years. In addition, females had a lower probability of marriage if their male ancestors were older. These findings are consistent with an increase of the number of accumulated de novo mutations with male age, suggesting that deleterious mutations acquired from recent ancestors may be a substantial burden to fitness in humans. However, possible non-mutational explanations for the observed associations are also discussed.