Phosphonate is an important phosphorous species in the effluent of wastewater treatment plant (WWTP), contributing to eutrophication and interfering with phosphate removal in WWTP. It is particularly difficult to determine phosphonates in samples of complex solution chemistry, resulting in very limited information on their presence in environmental matrices. Herein, we proposed a sensitive method based on liquid chromatography tandem mass spectrometry (LC-MS/MS) to determine six quantitatively most important phosphonates even at the ng/L level, i.e., 2-phosphonobutane-1,2,4-tricarboxylic acid, 1-hydroxyethane 1,1-diphosphonic acid, nitrilotris(methylene phosphonic acid), ethylenediamine tetra(methylene phosphonic acid), hexamethylenediamine tetra(methylene phosphonic acid) and diethylenetriamine penta(methylene phosphonic acid). Trimethylsilyldiazomethane (TMSCHN2) derivatization of the target phosphonates is pre-requisite since it could greatly increase the sensitivity up to 2-3 orders of magnitude over direct analysis of the virgin ones. The sample pretreatment methods (including ion exchange and solid phase extraction(SPE)), the derivatization procedures, and the LC-MS/MS conditions were systematically optimized. The limits of quantitation for the six phosphonates in the background of tap water ranged from 1.4 μg/L to 57 μg/L for direct analysis, and from 5.0 ng/L to 200 ng/L for SPE enabled pre-concentration analysis, respectively. The reliability of the proposed method was successfully validated by analysis of authentic water samples collected from one river and three WWTPs (0.088-7200 μg/L phosphonates) with satisfactory recoveries (72-126%). To the best of knowledge, this is the first report on quantification of phosphonates in environmental samples in China.