In the present study, we compared the extent to which linguistic comprehension (vocabulary and listening comprehension) and word reading explain reading comprehension differentially for English learners (ELs) and non-ELs with reading difficulties, and we investigated whether different mechanisms of reading comprehension failure exist for each group. Using the simple view of reading as our framework, we tested a model in which vocabulary exerts a direct effect on reading comprehension and indirect effects through listening comprehension and word reading. Results from a multigroup structural equation model with a sample of 446 struggling fourth-grade readers (n = 229 for ELs; n = 211 for non-ELs) demonstrated both similarities and differences in the sources and mechanisms of reading comprehension difficulties for ELs and non-ELs with reading problems. Word reading was an important source of reading comprehension difficulty for both groups. For non-ELs, the effect of word reading was larger than the effects of linguistic comprehension (vocabulary and listening comprehension combined); however, for ELs, the effects of linguistic comprehension were greater than the effect of word reading. Vocabulary had indirect effects via both listening comprehension and word reading for ELs, but it demonstrated a direct effect on reading comprehension for non-ELs. These results suggest that developing a range of linguistic comprehension skills (e.g., word-level and sentence-level language skills) may be important for ELs with reading comprehension difficulties in the upper-elementary grades.