Next generation immunotherapies have limited efficacy in colorectal cancer. Immune checkpoints inhibitors demonstrated their benefit in mismatch repair-deficient tumors, which also exhibit microsatellite instability (MSI). The Consensual Molecular Subtype (CMS) classification has been recently proposed and highlights specific immune escape mechanisms for each subtype. CMS1 "immune" subtype is hypermutated with a favorable immune microenvironment for immune checkpoints inhibitors activity. Importantly, CMS1 is not restricted to MSI tumors and includes also exonucleasic domain POLE mutated tumors which are good candidates for immunotherapy. The scope of this comprehensive review is to described immune anomalies and propose immunomodulating strategies for each CMS subtype in colorectal cancer. Finally, the potential interest of tumor mutation burden and the Immunoscore® in colorectal cancer is discussed taking into account the molecular classification and obstacles to antitumoral immune activity.