It is widely hypothesized that the evolution of female extra-pair reproduction in socially monogamous species reflects indirect genetic benefits to females. However, a critical prediction of this hypothesis, that extra-pair young (EPY) are fitter than within-pair young (WPY), has rarely been rigorously tested. We used 18 years of data from free-living song sparrows, Melospiza melodia, to test whether survival through major life-history stages differed between EPY and WPY maternal half-siblings. On average, survival of hatched chicks to independence from parental care and recruitment, and their total lifespan, did not differ significantly between EPY and WPY. However, EPY consistently tended to be less likely to survive, and recruited EPY survived for significantly fewer years than recruited WPY. Furthermore, the survival difference between EPY and WPY was sex-specific; female EPY were less likely to survive to independence and recruitment and lived fewer years than female WPY, whereas male EPY were similarly or slightly more likely to survive and to live more years than male WPY. These data indicate that extra-pair paternity may impose an indirect cost on females via their female offspring and that sex-specific genetic, environmental or maternal effects may shape extra-pair reproduction.