In this work, we report a simple and rapid surface-enhanced Raman scattering (SERS) method for the screening of pesticide residues on fruit peels using a portable Raman spectrometer. Adhesive tapes were used as the sampling media; the effectiveness of different tape brands was examined. Collection efficiencies were found to be 60.2 ± 7.6%, 54.3 ± 5.0%, and 52.3 ± 9.0% on glass, aluminum foil, and fruit peels, respectively. SERS was achieved by applying silver nanoparticles (Ag NPs) to the surface of the tape after analyte collection. Preparation of the Ag NPs was optimized for pesticide detection. The limit of detection of triazophos on apple peels was 25 ng/cm2 with the portable Raman spectrometer. Considering the least favorable conditions, the calculated detection limit was 0.0225 mg/kg, which is an order of magnitude less than the maximum residue limit (MRL, 0.2 mg/kg) in China. The method is sufficiently sensitive for use in field analysis.