The Breast cancer affects more women than any other form of cancers. Breast cancer is diagnosed mostly by mammography. Medical data from CT scans, PET scans, and MRIs are among the most widely used types of information. The use of Deep learning and Machine Learning approaches has become essential for efficient and precise cancer prediction and detection since the work of analyzing this massive amount of data has gotten increasingly difficult. Clinically relevant information can be mined from medical photographs to better aid in illness diagnosis and early detection, which is the primary focus of medical image mining. Patients need careful symptom observation and a prediction automatic system that can identify the tumor as benign or malignant in order to receive effective treatment. Traditional AI approaches specifically reinforcement learning methods hit several problems. However, these methods often provide high computational overhead, slow convergence, and suffer from limited interpretability. In this paper, we propose a novel framework that replaces DRL with lightweight and nterpretable Machine Learning (ML) algorithms. This article suggests a Federated Forcart-based prediction model with federated learning to address these issues. To assess the stage of breast cancer, the model uses a fully hybrid method that has been updated and broadened in design to operate with fewer training images and deliver more accurate tumor height and width segmentations.