Lung and colon cancers are among the deadliest diseases worldwide, necessitating early and accurate detection to improve patient outcomes. This study utilizes the EfficientNetB3 model, a state-of-the-art transfer learning approach, to enhance the detection of colon and lung cancers from histopathological images. The research leverages the LC25000 dataset, comprising 25,000 histopathological images evenly distributed across five classes: colon adenocarcinoma, benign colon tissue, lung adenocarcinoma, lung squamous cell carcinoma, and benign lung tissue. The EfficientNetB3 model initially achieved an impressive accuracy of 99.39% across all classes. To further validate and enhance the model’s robustness and generalizability, we augmented the dataset by replacing 1,000 cancerous class images with new Genomic Data Commons (GDC) Data Portal - National Cancer Institute images, simulating more diverse clinical scenarios. This modification resulted in an accuracy of 99.39%, with equally high performance across other metrics, including precision, recall, and F1-Score, all reaching 99.39%, and a Matthew’s Correlation Coefficient (MCC) of 99.24%. The Gradient-weighted Class Activation Mapping (Grad-CAM) technique was utilized to visually interpret the model’s decisions, enhancing its transparency and reliability. These findings demonstrate that EfficientNetB3 is an effective and generalizable end-to-end framework for histopathological image analysis with minimal preprocessing. The promising results underscore the potential of EfficientNetB3 to advance automated cancer detection, thereby contributing to earlier diagnosis and more effective treatment strategies.