Actinic keratoses (AK) are lesions of epidermal keratinocyte dysplasia and are precursors for invasive cutaneous squamous cell carcinoma (CSCC). Identifying the specific genomic alterations driving progression from normal skin-AK-invasive CSCC is challenging due to the massive ultraviolet radiation-induced mutational burden characteristic at all stages of this progression. Here, we report the largest AK whole exome sequencing study to date and perform mutational signature and candidate driver gene analysis on these lesions. We demonstrate in 37 AK, from both immunosuppressed and immunocompetent patients, that there are significant similarities to CSCC in terms of mutational burden, copy number alterations, mutational signatures and patterns of driver gene mutations. We identify 44 significantly mutated AK driver genes and confirm that these genes are similarly altered in CSCC. We identify the azathioprine mutational signature in all AK from patients exposed to the drug, providing further evidence for its role in keratinocyte carcinogenesis. CSCC differ from AK in having higher levels of intra-sample heterogeneity. Alterations in signaling pathways also differ, with immune-related signaling and TGF-β signaling significantly more mutated in CSCC. Integrating our findings with independent gene expression datasets confirms that dysregulated TGF-β signaling may represent an important event in AK-CSCC progression.