It has been reported that the miR-125 family plays an important role in regulating embryo development. However, the function of miR-125b-2 in spermatogenesis remains unknown. In this study, we used a model of miR-125b knockout (KO) mice to study the relationship between miR-125b-2 and spermatogenesis. Among the KO mice, the progeny test showed that the litter size decreased significantly (p = 0.0002) and the rate of non-parous females increased significantly from 10% to 38%. At the same time, the testosterone concentration increased significantly (p = 0.007), with a remarkable decrease for estradiol (p = 0.02). Moreover, the sperm count decreased obviously (p = 0.011) and the percentage of abnormal sperm increased significantly (p = 0.0002). The testicular transcriptome sequencing revealed that there were 173 up-regulated genes, including Papolb (PAP), and 151 down-regulated genes in KO mice compared with wild type (WT). The Kyoto Encyclopedia of Genes and Genomes (KEGG) and gene ontology (GO) analysis showed that many of these genes were involved in sperm mitochondrial metabolism and other cellular biological processes. Meanwhile, the sperm mitochondria DNA (mtDNA) copy number increased significantly in the KO mice, but there were no changes observed in the mtDNA integrity and mutations of mt-Cytb, as well as the mt-ATP6 between the WT mice and KO mice. In the top 10 up-regulated genes, PAP, as a testis specific expressing gene, affect the process of spermatogenesis. Western blotting and the Luciferase assay validated that PAP was the target of miR-125b-5p. Intriguingly, we also found that both miR-125b and PAP were only highly expressed in the germ cells (GC) instead of in the Leydig cells (LC) and Sertoli cells (SC). Additionally, miR-125b-5p down regulated the secretion of testosterone in the TM3 cell by targeting PAP (p = 0.021). Our study firstly demonstrated that miR-125b-2 regulated testosterone secretion by directly targeting PAP, and increased the sperm mtDNA copy number to affect semen quality. The study indicated that miR-125b-2 had a positive influence on the reproductive performance of animals by regulating the expression of the PAP gene, and could be a potential drugs and diagnostic target for male infertility.