Plant diseases remain a major threat to global food security, making reliable and scalable diagnostic systems increasingly important. This study compares three model families, Convolutional Neural Networks (CNNs), Vision Transformers (ViTs), and hybrid CNN–ViT architectures, for plant disease classification. The goal is to evaluate both accuracy and computational efficiency, two factors that heavily influence how suitable these models are for Agricultural Decision Support Systems (ADSS), especially those running on edge devices. Six representative architectures were trained using the same experimental setup, including transfer learning and data augmentation. All models performed well on the controlled dataset, but the hybrid models stood out. They achieved 99.29% accuracy and a 99.18% F1-score by combining local and global feature extraction. ViT models also reached high accuracy (98.92%) but required far more computation, making them less practical for real-time use. Lightweight CNNs had slightly lower accuracy (~97.44%) but were extremely efficient, with fewer parameters and very low FLOPs, which makes them strong candidates for mobile or IoT-based systems. Future directions should include using multispectral data, adding object-level localization to reduce background bias, and adopting Explainable AI to increase interpretability and trust. In conclusion, this work offers a clear comparison of leading deep learning architectures and highlights practical guidelines for selecting efficient and reliable models for next-generation ADSS aimed at early plant disease detection.