DESCRIPTION
Machine learning and deep learning models are the main engines in many multimodal AI applications, which are characterized by the fusion of multiple modalities of data streams. In this lecture, we highlight the trust and robustness challenges of machine learning that arises from data fusion. To do so, we present several case studies demonstrating how multimodal applications exacerbate existing challenges of trustworthy and robust machine learning. In a first case study, we investigate the impact of fusion depth on the robustness of multi-modal machine learning models, observing that model architecture could impact robustness. In a second case study, we investigate the impact of fusion modality on the robustness of multi-modal machine learning models, observing that fusion models are only as robust as their most susceptible modality. In another case study, we explore the impact of weight quantization techniques on the robustness of multimodal models, observing the need for modality-based quantization schemes. Through these case studies, we hope to share some perspectives on the unique trust and security challenges that arise in AI machine learning models in typical multimodal applications and offer insights to fortify such systems in real-world scenarios.
DETAILS
Course type: Invited lecture (in person delivery)
Duration: 1 hour
Level: Postgraduate/PhD
Institution of lecturer: Department of Computer Engineering, Rochester Institute of Technology, Rochester, NY
Notes: no exams
LECTURER
Dr. Alexander Loui
Alexander C. Loui received his B.A.Sc. (Honors), M.A.Sc, and Ph.D. all in Electrical Engineering from the University of Toronto, Canada. He is a Professor of Practice in the Department of Computer Engineering at Rochester Institute of Technology, and leads the Multimodal Analysis and Perception (MAP) Lab. Prior to that, he spent over 28 years in a number of technical and research leadership positions with Kodak Alaris, Kodak Research Labs, and Bell Communications Research. His research interests include computer vision, machine learning, image/video understanding, image management, as well as AI and multimedia applications. Dr. Loui has published over 100 refereed journal and conference papers and has been granted 100 US patents in these areas. He has been a Senior Area Editor of IEEE Transactions on Image Processing and a Senior Editor of SPIE/IS&T Journal of Electronic Imaging. He had served as an associate editor of IEEE Transactions on Circuits and Systems for Video Technology and IEEE Transactions on Multimedia. He was Chair of the IEEE Rochester Section, and Chair of the Rochester Chapter of IEEE Signal Processing Society. Dr. Loui was named a Kodak Distinguished Inventor for his contributions to image understanding and management technologies. He is a recipient of the IEEE Region 1 Technological Innovation Award and a Fellow of IEEE and SPIE.