The influence of androgen deprivation therapy (ADT) on other-cause of mortality (OCM) was investigated in patients with localized prostate cancer treated with modern high-dose radiotherapy. A retrospective review was conducted on 1125 patients with localized prostate cancer treated with high-dose radiotherapy, including image-guided, intensity-modulated radiotherapy or brachytherapy with a median follow-up of 80.7 months. Overall survival rate was no different between ADT (+) and ADT (-) group in high-, intermediate-, and low-risk groups. OCM was found in 71 patients, consisting of 4% (10/258) in the ADT (-) group and 7% (61/858) in the ADT (+) group (p = 0.0422). The 10-year OCM-free survival rate (OCMFS), if divided by the duration of ADT (ADT naïve (ADT (-)), ADT <2-year, and ADT ≥2-year groups), showed statistical significance, and was 90.7%, 88.2%, and 78.6% (p = 0.0039) for the ADT (-), ADT <2-year, and ADT ≥2-year groups, respectively. In patients aged ≥75 years, 10-year OCMFS for ADT (-), ADT <2-, and ADT ≥2-year groups was 93.5% (at 115.6 months), 85.6%, and 60.7% (p = 0.0189), respectively, whereas it was 90.7%, 89.9%, and 89.0% (p = 0.4716), respectively, in their younger counterparts. In localized prostate cancer patients, treatment with longer ADT for ≥2 years potentially increases the risk of OCM, especially in patients aged ≥75 years.