Physical activity (PA) is a potential modifiable correlate of the age-related decline in sexual function, but no studies have explicitly tested this. This study aimed to examine associations between PA, television viewing (TV) time and sexual activity, problems, and concerns. Data were from 7,038 men and women aged ≥50 years participating in the English Longitudinal Study of Ageing. PA and TV viewing time were self-reported. Sexual behaviour and concerns were assessed by self-completion questionnaire. Covariates included age, partnership status, socio-economic status, limiting long-standing illness, smoking status, alcohol intake and depressive symptoms. The odds of reporting any sexual activity were increased among individuals who participated in moderate (OR = 1.64, 95% CI: 1.24⁻2.15 in men) or vigorous (OR = 2.06, 95% CI: 1.50⁻2.84 in men, OR = 1.42, 95% CI: 1.09⁻1.85 in women) PA at least once a week. Erectile difficulties were less common among men who were active (OR = 0.58, 95% CI: 0.44⁻0.77 for vigorous PA). Women who watched ≥6 hours of TV/day had lower odds of thinking about sex frequently (OR = 0.69, 95% CI: 0.50⁻0.96) or, if they did not live with a partner, being sexually active (OR = 0.40, 95% CI: 0.22⁻0.72). Encouraging older adults to be more physically active could help to improve sexual relationships and, as a result, mental health and wellbeing.