Realtime | Eloi Du Bois - Résumé

Robust facial animation from video using neural networks

Tue, 04 Jun 2024 00:00:00 +0000

Overview

url: https://patents.google.com/patent/US12002139B2

Implementations described herein relate to methods, systems, and computer-readable media to generate animations for a 3D avatar from input video captured at a client device. A camera may capture video of a face while a trained face detection model and a trained regression model output a set of FACS weights, head poses, and facial landmarks to be translated into the animations of the 3D avatar. Additionally, a higher level-of-detail may be intelligently selected based upon user preferences and/or computing conditions at the client device.

Audiovisual Inputs for Learning Robust, Real-time Facial Animation with Lip Sync

Wed, 15 Nov 2023 00:00:00 +0000

Systems and methods for animation generation

Tue, 10 Jan 2023 00:00:00 +0000

Overview

url: https://patents.google.com/patent/US11551393B2

Systems and methods for animating from audio in accordance with embodiments of the invention are illustrated. One embodiment includes a method for generating animation from audio. The method includes steps for receiving input audio data, generating an embedding for the input audio data, and generating several predictions for several tasks from the generated embedding. The several predictions includes at least one of blendshape weights, event detection, and/or voice activity detection. The method includes steps for generating a final prediction from the several predictions, where the final prediction includes a set of blendshape weights, and generating an output based on the generated final prediction.

Fast Facial Animation from Video

Fri, 06 Aug 2021 00:00:00 +0000