Microsoft Swears VASA-1 Deepfake Tool Isn't for Creating Deepfakes

Microsoft’s artificial intelligence researchers on Tuesday announced VASA-1, a model capable of generating impressive deepfakes with extremely little data. Though the tool ostensibly exists to create “virtual characters,” it was trained on real people’s images and voice audio, raising questions about its potential role in an internet increasingly rife with misinformation. 
VASA-1 is the brainchild of Microsoft Research Asia, a collection of laboratories encompassing the research, incubation, and development stages for various forms of technology. While some of the division’s previous projects center on robotics and neuroscience, its labs have unsurprisingly embraced AI with open arms. One of its most eye-catching projects from this realm, VASA-1 can reportedly create hyper-realistic deepfakes with just a single photograph and one audio clip.
In a press release and a preprint paper shared to the arXiv, Microsoft Research Asia demonstrates how VASA-1 (short for “visual affective skills animator”) uses limited data to create “lifelike audio-driven talking faces.” After receiving a static image, a machine learning model assesses it alongside the provided audio clip to see how it can match the image with the person’s voice. Users can specify which emotion, gaze direction, and head distance they want the product to display. The resulting video depicts an extremely realistic “human” saying whatever the provided audio clip contained, complete with sideways glances and accidental stutters. Videos generated via VASA-1’s offline batch processing mode are 512×512 and 45 frames per second (fps); the frame rate dips to 40fps with online streaming mode, with a latency of just 170 milliseconds. 
VASA-1’s eerie faculties come from the VoxCeleb2 dataset, a collection of “over 1 million utterances for 6,112 celebrities.” Because these utterances were pulled from YouTube videos back in 2018, its options—as far as time-sensitive misinformation goes, at least—might be a bit outdated. VASA-1 can also work with 2D illustrations and other unrealistic works. Still, deepfakes are where the tool impresses—something Microsoft knows, based on the little amount of real estate its materials offer these “out-of-distribution generalizations.” 
While VASA-1 doesn’t create audio like Meta’s Voicebox, it can easily be paired with imitation audio to generate clips of people saying or doing things they haven’t said or done. Minimal streaming latency also means the tool could be used during virtual interviews and other video conferencing applications. According to Microsoft, “forgery detection” methods might be required to stave off VASA-1 misuse. 
“Our research focuses on generating audio-driven visual affective skills for virtual AI avatars, aiming for positive applications,” the researchers’ paper reads. “It is not intended to create content that is used to mislead or deceive. However, like other related content generation techniques, it could still potentially be misused for impersonating humans. We are opposed to any behavior to create misleading or harmful contents [sic] of real persons.”
ExtremeTech supports Group Black and its mission to increase greater diversity in media voices and media ownerships.
© 2001-2024 Ziff Davis, LLC., a Ziff Davis company. All Rights Reserved.
ExtremeTech is a federally registered trademark of Ziff Davis, LLC and may not be used by third parties without explicit permission. The display of third-party trademarks and trade names on this site does not necessarily indicate any affiliation or the endorsement of ExtremeTech. If you click an affiliate link and buy a product or service, we may be paid a fee by that merchant.

source