Malaria continues to pose a significant global health challenge, with its persistent transmission creating major difficulties for healthcare systems worldwide. Tackling this problem calls for innovative and effective methods to enhance understanding and control of the disease. In this work, we proposed a fractional-order mathematical model to study the dynamics of malaria transmission, integrating essential control measures such as treatment of humans and management of mosquito populations. The model employed three different types of non-integer order differential operators: the Caputo operator, the Caputo-Fabrizio operator with exponential decay, and the Atangana-Baleanu operator with an extended Mittag-Leffler kernel. Using fixed-point theory, we proved the existence and uniqueness of solutions for the proposed model. Numerical simulations are carried out to assess the impact of varying fractional orders on the progression of the disease. The results revealed that increasing the fractional order slows down the spread of malaria, reduces the peak number of infections, and prolongs the duration of outbreaks highlighting the memory-dependent nature of fractional systems. Our findings demonstrated that fractional-order models offer a more accurate and flexible approach to capturing the complex dynamics of malaria transmission. The study underscores the importance of integrating both therapeutic interventions and vector control strategies in reducing disease burden. Based on the findings of this study, we recommended the integration of fractional order modeling into malaria control strategies, as it captures the memory effects and long-term dynamics of disease transmission more accurately than classical models. Public health programs should adopt combined intervention approaches incorporating both effective treatment and vector control measures to significantly reduce infection rates. Furthermore, control efforts should be sustained over time, as fractional models reveal that short-term interventions may not be sufficient in curbing prolonged outbreaks. Policymakers are encouraged to use insights from these models to design adaptive, data-driven strategies that enhance the efficiency and sustainability of malaria control programs.