Slow sand filters (SSFs) have been shown to effectively improve water quality. The aim of the present study was to obtain low-cost materials (LCMs) as filter mediums (FMs) to efficiently purify harvested rainwater and to document the relationship between bacterial community structure and water purification. The red clay was mixed with crushed limestone and crushed brick, respectively. The mixtures or brick powder were used as the filter media for SSFs. Laboratory column tests were conducted in conjunction with the monitoring of representative water quality parameters (COD, NH4+, CFU and total coliforms) to estimate the performance of low-cost material slow sand filters (LCM-SSFs), including the time needed for biofilm maturation. The relationship between bacterial community structure and SSF performance was determined using a combination of 16S rRNA gene sequencing and an array of statistical techniques. The results demonstrated that LCM-SSFs perform well in purifying harvested rainwater, and are of superior economic benefit. LCMs had a stronger adsorptivity than quartz sand, which enhanced the purification of harvested rainwater before the biofilms matured, and shorten the time required for biofilm maturation. During the 90-day laboratory experiment, a mixture of crushed limestone and red clay exhibited the best performance. The abundance of Opitutae could be used as a potential indicator of NH4+ removal efficiency by SSFs. Schmutzdecke was characterized by abundant, diverse and evenly distributed bacterial communities that produced rich, stable and robust environmental functions, and that possessed an excellent purifying capacity. Environmental conditions associated with low ecological stress, such as neutral pH filter mediums and lucifugal experimental conditions, were conducive to the diversity and evenness of effluent bacterial communities and improved the performance of LCM-SSFs in purifying harvested rainwater.