Time is a critical component of episodic memory. Yet it is currently unclear how different types of temporal signals are represented in the brain and how these temporal signals support episodic memory. The current study investigated whether temporal cues provided by low-frequency environmental rhythms influence memory formation. Specifically, we tested the hypothesis that neural tracking of low-frequency rhythm serves as a mechanism of selective attention that dynamically biases the encoding of visual information at specific moments in time. Participants incidentally encoded a series of visual objects while passively listening to background, instrumental music with a steady beat. Objects either appeared in-synchrony or out-of-synchrony with the background beat. Participants were then given a surprise subsequent memory test (in silence). Results revealed significant neural tracking of the musical beat at encoding, evident in increased electrophysiological power and inter-trial phase coherence at the perceived beat frequency (1.25 Hz). Importantly, enhanced neural tracking of the background rhythm at encoding was associated with superior subsequent memory for in-synchrony compared to out-of-synchrony objects at test. Together, these results provide novel evidence that the brain spontaneously tracks low-frequency musical rhythm during naturalistic listening situations, and that the strength of this neural tracking is associated with the effects of rhythm on higher-order cognitive processes such as episodic memory.