In this paper, an immunosensor using CdTe@SiO2 core-shell nanoparticles as labels was constructed for highly sensitive detection of prostate-specific antigen (PSA). In this approach, CdTe@SiO2 core-shell nanoparticles were synthesized using the sol-gel method. The additional Cd ions and sulfur source in SiO2 shell can greatly enhance the fluorescence intensity of CdTe nanocrystals. The reason is the formation of CdS-like cluster in SiO2 shell, which reduced the quantum size effect. The obtained CdTe@SiO2 nanoparticles also exhibited excellent biocompatibility, which was ideal for applying in biomarker detection. Furthermore, PSA capture antibodies functionalized magnetic Fe3O4 nanoparticles (Fe3O4-Ab1) were utilized in the proposed immunosensor to capture and enrich the PSA. The captured PSA was then immuno-recognized by CdTe@SiO2 labeled with PSA detection antibodies (CdTe@SiO2-Ab2) by forming the sandwich complex Fe3O4-Ab1/PSA/Ab2-CdTe@SiO2. The construction of this immunosensor was confirmed by fluorescence spectroscopy. The proposed immunosensor showed a good linear relationship between the fluorescent intensity and the target PSA concentration ranging from 0.01 to 5 ng/mL, and a detection limit as low as 0.003 ng/mL was achieved. The sensor also exhibited good specificity to PSA. This highly sensitive and specific immunosensor has great potential to be used in other biological detection.