Deepfakes: the next gen­er­a­tion of coun­ter­feits

Making up facts, bending the truth – this has been happening since the internet began. Whether it’s ma­nip­u­lat­ed images, alleged news, or made up in­for­ma­tion, the world wide web is full of it, making it in­creas­ing­ly difficult to dis­tin­guish between reality and fiction. Now coun­ter­feits have reached a new level: deepfakes.

Deepfakes first appeared in December 2017 on Reddit. One user had managed to put the faces of several celebri­ties into porn movies in such a way that made it look shock­ing­ly real. For a short while, creating celebrity porn became the thing to do, but soon fake videos like these were banned on Reddit as well as other platforms such as Twitter and Discord. It didn’t stop them from spreading elsewhere though. But the question is: What makes deepfakes so special?

Usually these fakes require a lot of work and expertise. They aren’t nec­es­sar­i­ly always created for shady reasons – it’s not uncommon for Hollywood movies to exchange faces, for example. Experts in the areas of cutting tech­nol­o­gy and CGI are normally hired for these tasks. Deepfakes, however, are created by the computer itself without any ad­just­ments being made by hand.

The word “deepfakes” is a port­man­teau of “deep learning” and “fake.” Deep learning is one form of machine learning. Al­go­rithms are required for ex­chang­ing faces or objects with deepfakes. For deep learning to work, the al­go­rithms are fed with a very, very large amount of image or video data. The more material you have of a person, the better the result should be.

Around 300 images with the chosen person’s face (at best from all possible per­spec­tives) should be enough to get a decent result. The deepfakes code contains a neural network, a so-called au­toen­coder: the network is trained to compress data in order to de­com­press it again. During de­com­pres­sion, the au­toen­coder tries to achieve a result that is as close as possible to the original. To achieve this, the networks learns to dis­tin­guish between important and unim­por­tant data during the com­pres­sion process.

By feeding the algorithm with images of dogs, the ar­ti­fi­cial neural network learns to focus only on the dog and ignore anything in the back­ground (noise). The au­toen­coder can then create its own dog from the data. This is also how face swaps work with deepfakes: the neural network learns what the person’s face looks like and can then create it in­de­pen­dent­ly – even if the face and mouth are moving at the same time, for example.

To ef­fec­tive­ly swap faces, two heads need to be rec­og­nized: the face that appears in the original material and the one that you want to exchange it with. So, one input (the encoder) and two outputs (the decoders) are used. The encoder analyzes all the material while the two decoders each generate a different output: face A or face B.

In the end, it works in such a way that the algorithm doesn’t insert face A into the video, but face B instead, even though it doesn’t belong there at all. This is the dif­fer­ence between other fakes where the face is cut out of an image, retouched, or adjusted, and inserted into another image. Regarding deepfakes, however, the image material isn’t copied into another image: totally new material is created. This is the only way to match the facial ex­pres­sions of the original face.

This explains why some errors occur with deepfakes: the neural networks reach their limit when it comes to unusual facial movements. If there isn’t enough material from the relevant per­spec­tive, the frame will appear blurry. The algorithm tries to generate an image from the little source material it has, but will un­for­tu­nate­ly leave it lacking in detail.

Deepfakes first orig­i­nat­ed on Reddit. The website is known for also housing curious topics in its sub forums, which are known as sub­red­dits. A Redditor with the name “deepfakes” created a subreddit in December 2017 and published porno­graph­ic videos featuring celebri­ties. To do this, the anonymous user had built the afore­men­tioned algorithm, which is based on other tech­nolo­gies such as the open source libraries, Keras and Google’s Ten­sor­Flow.

Within a very short time, the subreddit had over 15,000 followers. Reddit quickly put a stop to the forum and, like other companies (including the porno­graph­ic video platform, Pornhub), banned the dis­tri­b­u­tion of fake porn. But that wasn’t enough to stop deepfakes, since the code used is open source and is available to everyone. On GitHub, you can find several repos­i­to­ries where de­vel­op­ers work on the al­go­rithms. This is how a deepfakes app, called FakeApp, was created.

The program enables even those with little computer knowledge to perform face swaps. To create deepfakes via app, you only need a powerful graphic card from Nvidia. The program uses the graphics processor (GPU) for the cal­cu­la­tions. Apart from FakeApp, deepfakes can also be created with a computer’s CPU, but this usually takes much longer.

In the meantime, the network community has found further uses (other than pornog­ra­phy) for face swaps based on machine learning. As you know from the internet, this tech­nol­o­gy is used to a very large extent to create funny nonsense. It is often used to put actors in films in which they have never appeared. In a short clip from the film adap­ta­tion “Lord of the Rings,” for example, users replaced every actor with Nicholas Cage, and Sharon Stone was replaced by Steve Buscemi in her notorious scene from “Basic Instinct.”

Jokes like those mentioned above are harmless, but the new pos­si­bil­i­ties of video ma­nip­u­la­tion pose several chal­lenges to society. Firstly, there is the question of legality. The women appearing in these porn videos have never given their consent. Apart from the fact that this is more than ques­tion­able from a moral point of view, these videos could also wreck the person’s rep­u­ta­tion.

Apart from causing distress to in­di­vid­u­als, deepfakes can also cause social changes. In recent years, the problems caused by so-called fake news have already started to become apparent. It is becoming in­creas­ing­ly difficult to dis­tin­guish between real facts and false claims. Up to now, video evidence was con­sid­ered a reliable in­di­ca­tion of whether a statement was correct or not, but since deepfakes came on the scene, this is no longer the case. De­cep­tive­ly real ma­nip­u­la­tions can now be created with rel­a­tive­ly little effort – and not only for the purpose of en­ter­tain­ment.

Coun­ter­feit­ing is and always has been an important pro­pa­gan­da tool. Deepfakes can be used to influence politics slightly. While a video of Trump’s face covering German chan­cel­lor Angela Merkel’s face is obviously fake, other politi­cians could be brought into sit­u­a­tions in which they have never been. Since machine learning can even recreate a person’s voice in a rel­a­tive­ly credible way, it’s scary to think what deepfakes will be able to do in the future. Fakes like these could in­evitably influence election campaigns and in­ter­na­tion­al relations.

For our society, this means that we can’t really trust the media, es­pe­cial­ly the internet media. People are already skeptical when it comes to news, although there are others that believe every social media post they see even if it’s lacking any factual basis. In the future, we will no longer be able to believe what we’ve even seen with our own eyes!

But not all deepfakes de­vel­op­ments are de­struc­tive and foolish: deep learning can rev­o­lu­tion­ize the creation of visual effects. At the moment, it requires a lot of work to stick the faces of actors onto the bodies of other people. For the Star Wars film “Rogue One,” the young princess Leia was created with visual effects since the actress, Carrie Fisher, was already 60 years old when the movie was released. Saying this, an internet user achieved a similar result with the help of deepfakes – according to him, it took half an hour on a normal PC. Deepfakes have the power to make visual effects in en­ter­tain­ment media faster and cheaper.

There has been spec­u­la­tion that deepfakes and the sim­plic­i­ty as­so­ci­at­ed with the new types of fakes could give viewers more choice. For example, if you watch a movie in the future, it would be in­ter­est­ing to be able to select which star should play the main character. All that’s needed is a quick click before the movie begins. Something similar could also happen in the ad­ver­tis­ing industry. Soon, celebri­ties won’t need to fly to pho­to­shoots to wear the latest designer clothes or pose for the newest perfume, they will just sell a license to allow others to use their face.