MultiMed-Fusion: Privacy- Preserving Multimodal AI for Medical Summarization

Anil Panday (Northwest Missouri State University), Sai Srinivas Kanneganti (Northwest Missouri State University), Charan Reddy Chilipireddy (Northwest Missouri State University), Sheng Chai (Northwest Missouri State University)

ABSTRACT: Healthcare professionals regularly work with patient information from multiple sources, including written medical records, medical images, and audio recordings generated during clinical care [2][3]. Reviewing such heterogeneous data manually is time-consuming and may delay clinical decision-making [3]. One of the main challenges in using AI (Artificial Intelligence) in healthcare is producing accurate summaries of patient data while ensuring privacy, since medical information is highly sensitive [1][4]. This paper introduces MultiMed-Fusion, an AI-based healthcare document summarization system that combines multimodal data sources, including clinical notes, medical images, audio recordings, PDFs, and text documents, into a single unified summary [2][4]. To safeguard sensitive patient data, privacy-preserving methods such as anonymization and differential privacy are implemented prior to automated processing [1]. The proposed system employs a neural multimodal fusion framework using extractive summarization guided by multimodal embeddings. Specifically, embeddings from different modalities are aligned in a shared representation space to identify and extract clinically relevant content for summary generation . The system is evaluated using standard performance metrics such as accuracy, precision, F1-score, and privacy compliance to assess both summarization quality and data protection effectiveness [4]. Experimental evaluation results suggest that MultiMed-Fusion can improve accessibility to essential patient information and reduces the effort required for medical record review while maintaining strong privacy safeguards [1][2]. REFERENCES: [1] H. Khalid, S. Ahmed, A. Malik, “Privacy-preserving techniques for healthcare data using federated learning,” Patterns, Elsevier, vol. 4, 2023, pp. 1–10. [2] M. Demrozi, C. Daza, M. Islam, “Multimodal AI for healthcare: Improving diagnostic accuracy through data integration,” Artificial Intelligence in Medicine, vol. 145, 2024, pp. 102–115. [3] S. Zhang, J. Li, “Summarizing medical records using deep learning techniques: A survey,” Journal of Healthcare Informatics, vol. 33, no. 2, 2024, pp. 34–49. [4] R. Patel, A. Kumar, “AI-driven medical document summarization with privacy- preserving methods,” Journal of Artificial Intelligence in Healthcare, vol. 12, no. 4, 2024, pp. 122–135.