Hao et al. report the cryo-EM structure of the human CbA–Sec61 complex to develop photo-controlled and hypoxia-activated prodrugs. This backbone-engineering strategy overcomes the on-target toxicity of Sec61 inhibition by restricting activity to tumor microenvironments, demonstrating potent anti-tumor efficacy in a colon cancer CDX model.