This is probably the strangest thing you’ll see all year: watch in awe as former US president George W. Bush appears to mimic the facial expressions of a man suffering from a severe case of facial tics. No, this isn’t how George normally behaves. OK, maybe it is. However, in this case anyway, it’s actually the handiwork of a group of researchers from Stanford University, the University of Erlangen-Nuremberg and the Max Planck Institute for Informatics.
The researchers take a normal YouTube video, and a video of an actor. They combine the expressions of the actor with the face of the unwitting candidate, and after much technical jargon and re-rendering of stuff etc, we have the reenacted video.
Bush isn’t the only politician to suffer at their hand either; Russian president Vladimir Putin’s and US presidential aspirant Donald Trump’s facial expressions get manipulated like putty. You could do many bad things with tech like this; apparently without much more than a cheap webcam and a YouTube video…and the software and know-how of these researchers…and audio manipulation software, of course.
Time to rob a bank and fake the video confession of an annoying neighbor?
At the end of the video, a recording of US president Barack Obama is played alongside a mimicked version. There aren’t many differences between the two; the areas that need to be re-rendered are minimal, mostly the mouth, mouth interior and eye region. The re-rendering is done rather well. Perhaps too well.
This is what their abstract says:
“We present a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion.
To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit.
Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time.”
Sources: Stanford
This article (Watch as Imposter-Researchers Convincingly “Manipulate” Presidents on TV) is a free and open source. You have permission to republish this article under a Creative Commons license with attribution to the author(CoNN) and AnonHQ.com.
So this program allows you to manipulate the facial expressions shown through T.V. and Youtube. Hmmmmmm. Well if their mouth is closed, and then they open it, how can the program detects the teeth and tongue? Does that mean that the program would have to display the (possibly) tongue and teeth of whoever if manipulating the facial expressions? Perhaps this could be a way of detecting wether manipulation is involved. I could very well be wrong but at least take this into consideration until there’s a definite answer 😀