DESCRIPTION
Human behavior, both for face (e.g., expressions) and body (e.g., actions) has been studied in detail (for example for expression / action classification and prediction), but there have been few works exploring generation of novel behaviors. Generating novel sequences of human facial expression, talking heads, or body to form a natural and plausible action with continuous and smooth temporal dynamics is a challenging problem. These motions can either simulate full body movement, like for gait, or part specific movement, like in playing the guitar or phone call, or involve facial expressions, action units or even mouth movement when a person speaks or reads a text. With the advent of powerful generative models such as GANs or Diffusion models, novel data generation paradigms have become possible, and these networks have shown to be powerful in many image generation tasks. However, many issues remain to be solved especially when passing from the static to the dynamic case and new research problems emerge.
We expect generating synthetic and realistic static and dynamic data of humans can have a big impact in several different contexts. A straightforward outcome that developing such techniques could have, is that of generating an abundance and variety of new data that could be otherwise difficult, very expensive and time consuming to obtain from reality. Such data can be essential in simulation, virtual and augmented reality, in training more robust learning tools, to cite a few. For example, we could expect new applications in the game and movie industry, where fully synthetic actors could be used in the near future, without the need of explicit modeling.
In this talk, we will address some recent works in this domain for generating facial expressions, talking heads and body animation of 3D human avatars.
DETAILS
Course type: Invited lecture (online delivery)
Duration: 1 hour
Level: Postgraduate / PhD
Institution of lecturer: Department of Information Engineering & Media Integration and Communication Center, University of Florence, Italy
Notes: There will be a multiple-choice exam for students interested to have credits recognized
Course link: The lecture will be registered and made available
LECTURER
Prof. Stefano Berretti
Stefano Berretti is an Associate Professor at the Department of Information Engineering of the University of Florence, Italy. Since 2017 he got the habilitation as full professor. He was a visiting professor at the University of Lille, France, and at the University of Alberta, Canada, and also Adjunct Professor at the Kerala University of Digital Sciences, Innovation and Technology (India), 2021-2023 and member of the Board of Studies (BoS), School of Computer Science and Engineering (SoCSE), Kerala University of Digital Sciences, Innovation and Technology (Digital University Kerala (DUK)). The research interests of Stefano Berretti focus on 3D computer vision methods for face and facial expression recognition, biometrics, human behavior understanding, face reconstruction and generation in 3D and 4D, geometric methods for shape analysis, and modeling. On these themes, he has published over 230 articles in peer reviewed international journals and conference proceedings. He organized workshops on “Learning with few or without annotated face, body, and gesture data” (LFA at IEEE FG 2023 and 2024), “Generation of Human Face and Body Behavior” (GHB at WACV 2021, ICIAP 2023). He has been a general chair of The Eurographics Symposium on “3D Object Retrieval” (3DOR) 2022, and of the Conference on “Smart Tools and Applications in Graphics” 2021 and has served as an area chair for ACM Multimedia (2020, 2021, 2022, 2023, 2024), IEEE Face and Gesture Recognition (2019, 2020), and program chair for the Int. Conf. on Smart Multimedia (2022, 2024). He organized special issues on the Computers & Graphics journal, ACM TOMM, IEEE TII, IEEE TCE, IEEE JBHI, and ACM TALLIP. He is the Associate Editor in Chief for Digital Communications of the IEEE Trans. on Circuits and Systems for Video Technology, an Associate Editor of the ACM Trans. of Multimedia Computing, Communications, and Applications (ACM TOMM), and of the IET Computer Vision journal. He was also the Information Director of ACM TOMM. He is a Senior member of IEEE.