Immune system control is a principal hurdle in cancer evolution. The temporal dynamics of immune evasion remain incompletely characterized, and how immune-mediated selection interrelates with epigenome alteration is unclear. Here we infer the genome- and epigenome-driven evolutionary dynamics of tumor-immune coevolution within primary colorectal cancers (CRCs). We utilize a multiregion multiomic dataset of matched genome, transcriptome and chromatin accessibility profiling from 495 single glands (from 29 CRCs) supplemented with high-resolution spatially resolved neoantigen sequencing data and multiplexed imaging of the tumor microenvironment from 82 microbiopsies within 11 CRCs. Somatic chromatin accessibility alterations contribute to accessibility loss of antigen-presenting genes and silencing of neoantigens. Immune escape and exclusion occur at the outset of CRC formation, and later intratumoral differences in immuno-editing are negligible or exclusive to sites of invasion. Collectively, immune evasion in CRC follows a 'Big Bang' evolutionary pattern, whereby it is acquired close to transformation and defines subsequent cancer-immune evolution.