Advances in machine learning (ML) offer significant potential to accelerate drug discovery. Although mathematical modeling and ML have become crucial in predicting drug-target interactions and properties, the complexity of chemical space and the "black box" nature of ML leave untapped potential in computational drug discovery. In this study, we propose a comprehensive workflow, termed FragOPT, for optimizing molecules for specific targets. FragOPT identifies advantageous and disadvantageous fragments of molecules to be optimized by using a classification model for bioactive molecules and a model interpretability method (SHAP). These fragments are then sampled within the 3D pocket of the target protein, and disadvantageous fragments are redesigned by using a deep learning model. Finally, the advantageous fragments of the original molecule are recombined with the redesigned fragments to generate new molecules with enhanced binding affinity. This method was validated on two distinct protein targets associated with solid tumors and the SARS-CoV-2 virus, respectively. Compared to the other two fragment-based drug discovery methods, the majority of molecules generated by FragOPT exhibited superior synthesizability and enhanced binding affinity. Moreover, MMPBSA and FEP calculations indicated that the novel molecules with the best docking scores possess lower binding free energies than the original ligands. Overall, the results indicate that guiding the molecular generation workflow with QSAR models and the interpretability method can significantly optimize the initial drug discovery process, providing a more precise and efficient pathway for developing new therapeutics. The FragOPT is available at https://github.com/WxyChem/FragOPT.