A multi-tasking learning model based on interpretable guided CNN}%\authorCHN{劉佳, 姚蘭}%\authorENG{LIU Jia}%\affiliationCHN{% 湖南大學數學學院,長沙　410082 %}%\affiliationENG{% School of Mathematics, University of Hunan, Changsha 410082 %

• 1、School of Mathematics, University of Hunan, Changsha 410082

Abstract：Deep learning method performs well in the classification and segmentation tasks of medical images, but its clinical application is limited due to the lack of interpretability in the decision-making process. This paper uses interpretable methods to guide convolutional neural networks (CNN) to propose an end-to-end, interpretable multi-task learning model (IGMTL). IGMTL can complete gastrointestinal endoscopic lesion image classification and segmentation. The model first uses CNN and interpretability methods to obtain the attention map of the input image, then generates the mask image of the input image based on the attention map, and finally feeds the mask image into the CNN. A new loss function is proposed to train the IGMTL model. This paper verified the effectiveness of the IGMTL model on two types of images, polyps and normal cecum, in the public dataset Kvasir. For the polyp recognition task, the recall and accuracy of IGMTL were 4\% and 2\% higher than those of the CNN without interpretable guidance. For the weakly supervised segmentation task of polyp images, the IoU of IGMTL was 11.79\% higher than that of the FCM clustering model. From the qualitative results of weakly supervised segmentation, IGMTL can accurately complete the weakly supervised segmentation task of polyp images. Therefore, the IGMTL model can not only be used for lesion image recognition, but also can assist doctors in clinical diagnosis or guide non-professionals to annotate the lesion areas of endoscopic images.

Keywords： Deep learning Interpretable method Image classification Image segmentation Medical image