It is generally assumed that identical neural regions mediate the same cognitive functions in females and males. However, anatomic and molecular sex differences exist in the brain, including in regions associated with long-term memory, which suggests there may be functional differences. The present functional magnetic resonance imaging (fMRI) investigation aimed to identify the differences and similarities in brain activity between females and males during spatial long-term memory. During encoding, abstract shapes were presented to the left or right of fixation. During retrieval, shapes were presented at fixation and participants made "old-left" or "old-right" judgments. For both females and males, spatial memory hits versus misses produced activity in regions commonly associated with visual long-term memory; however, the activations were almost completely distinct between the sexes. An interaction analysis revealed sex-specific activity for males in visual processing regions, the left putamen, the right caudate nucleus, and bilateral cerebellum, and sex-specific activity for females in the parietal cortex. A targeted anatomic region-of-interest (ROI) analysis identified sex-specific activity for males and females in the left hippocampus and language processing cortex, respectively. A multi-voxel pattern correlation analysis within functional ROIs between all pairs of participants showed greater within-sex than between-sex correlations, indicating the differential activations were due to sex differences rather than other individual differences between groups. These results indicate that spatial long-term memory is mediated by largely different brain regions in females and males. These findings have major implications for the field of cognitive neuroscience, where it is common practice to collapse across sex.