Some epigenetic modifications are inherited from one generation to the next, providing a potential mechanism for the inheritance of environmentally acquired traits. Transgenerational inheritance of RNAi phenotypes in Caenorhabditis elegans provides an excellent model to study this phenomenon, and although studies have implicated both chromatin modifications and small RNA pathways in heritable silencing, their relative contributions remain unclear. Here, we demonstrate that the putative histone methyltransferases SET-25 and SET-32 are required for establishment of a transgenerational silencing signal but not for long-term maintenance of this signal between subsequent generations, suggesting that transgenerational epigenetic inheritance is a multi-step process with distinct genetic requirements for establishment and maintenance of heritable silencing. Furthermore, small RNA sequencing reveals that the abundance of secondary siRNAs (thought to be the effector molecules of heritable silencing) does not correlate with silencing phenotypes. Together, our results suggest that the current mechanistic models of epigenetic inheritance are incomplete.