Metazoan life requires the coordinated activities of thousands of genes in spatially organized cell types. Understanding the basis of tissue function requires approaches to dissect the genetic control of diverse cellular and tissue phenotypes in vivo. Here, we present Perturb-Multimodal (Perturb-Multi), a paired imaging and sequencing method to construct large-scale, multimodal genotype-phenotype maps in tissues with pooled genetic perturbations. Using imaging, we identify perturbations in individual cells while simultaneously measuring their gene expression profiles and subcellular morphology. Using single-cell sequencing, we measure full transcriptomic responses to the same perturbations. We apply Perturb-Multi to study hundreds of genetic perturbations in the mouse liver. Our data suggest the genetic regulators and mechanisms underlying the dynamic control of hepatocyte zonation, the unfolded protein response, and steatosis. Perturb-Multi accelerates discoveries of the genetic basis of complex cell and tissue physiology and provides critical training data for emerging machine learning models of cellular function.