Face Authentication

As artificial intelligence continues to advance, the ability to distinguish between real human faces and those generated by AI has become increasingly important. This project aims to develop an image classification model that can accurately identify whether a face belongs to an actual person or is an AI-generated synthetic image. Additionally, we're working on an auxiliary model that can pinpoint the specific AI architecture used to generate the synthetic face, including cutting-edge models like DALL-E, Imagen, and others.

While researching existing solutions, we couldn't find any publicly available models specifically designed for this task. Although commercial offerings like Sensity.ai claim to detect deepfakes, they don't disclose the underlying models or provide a way to evaluate their effectiveness independently.

Through this project, we hope to contribute a reliable and transparent solution to the growing challenge of discerning AI-generated synthetic faces from real ones, enabling better detection and prevention of potential misuse or deception.

First we combined multiple datasets to get artificially generated images from multiple sources

• 1millon fake faces: This dataset contains a millon fake faces generated by the StyleGan model developed by Nvidia.
• FakeFaceDB: This dataset is a collection of 87000 images which are again generated by the styleGan model but are transformed with the GANprintR approach. GANprintR is a tranformation to remove the GAN fingerprint from the images
• DeepFakeFace: This dataset contains fake face images which are generated by diffusion based models. This dataset contains images generated by Stable Diffusion Impainting model, Insightface toolbox and Stable Diffusion V1.5.

We trained our model with the combination of all the fake images i.e, training set had images from iFakeFaceDB set, deepFakeFace set and diffusion models generated images. Other than this, we also trained our model on each of these specific datasets. The thought process behind this was the know how biased a model is getting to one particular dataset if the model is given exclusively those pictures or how well can the model generalize with just a single source of images.

To investigate the clusters of latent images, we made the tSNE plots for the embedded images. We removed the last layer of our trained transformer, then passed 50 set of images from each class to it.