Noninsulin dependent diabetes mellitus is a serious global disease that is treated by inhibiting α-glucosidase to reduce the glucose content in the blood. Several incompletely satisfactory therapeutic drugs are already on the market. In this report, we showed that polyoxomolybdates based on Keggin-type architecture are promising candidates. Kinetic studies indicate that H3PMo12O40, Na4PMo11VO40, Na6PMo11FeO40 and Na7PMo11CoO40 strongly inhibit α-glucosidase with IC50 values of 6.14 ± 0.38 μM, 52.33 ± 1.41 μM, 161.90 ± 7.68 μM and 103.10 ± 2.88 μM, respectively. Moreover, H3PMo12O40, Na4PMo11VO40, and Na7PMo11CoO40 are reversible, competitive inhibitors with KI values of 0.018 mM, 0.146 mM and 0.121 mM, respectively. Na6PMo11FeO40 inhibited α-glucosidase in a reversible noncompetitive manner with KI and KIS of 0.312 mM and 0.412 mM, respectively. Molecular docking simulation suggested that H3PMo12O40 binds into the substrate binding site in accordance with competitive inhibition behavior and offered, in addition, an initial insight into the polypeptide-inhibitor interactions. This work presents a promising new perspective for designing effective α-glucosidase inhibitors and further demonstrates the enormous potential of polyoxomolybdates as enzyme inhibitors.