Disentangling images of a faces to separate private features from target information

During my internship, I explored the use of beta-VAEs (Variational AutoEncoders) as a tool for disentangling the latent representations of images. I knew I wanted to work in the latent space and that VAEs were a good tool for this task, but I didn’t know exactly how to apply it.

Research Intern: Elisa Fraticelli 

Project supervisor: Tariq Elahi, Marc Juarez   

PhD Tutor: Aradhika Bagchi 

Do you agree to give away all your personal information associated with your face when submitting a picture for a simple age check? No, but that’s the risk associated with data in our world: it’s entangled, all linked together. As humans, we can’t imagine a picture of someone we can guess the age of, without seeing their gender, ethnicity, identity... But we can imagine the face of a completely different person of the same age. So, how do we change everything about a picture without messing with age? 

During my internship, I explored the use of beta-VAEs (Variational AutoEncoders) as a tool for disentangling the latent representations of images. I knew I wanted to work in the latent space and that VAEs were a good tool for this task, but I didn’t know exactly how to apply it. Over the course of 8 weeks, I learned a lot about machine learning through trial and error, reading many papers, and core discussions with my supervisors. I started by training a beta-VAE to learn reconstruct privatised images from scratch. Although my model did learn the image features important for age, and did anonymise the images, it was limited to producing average images (Figure 1). 

This was a proof of concept for me, showing that beta-VAEs are applicable in this type of problem, but it also showed me that building a competent model from scratch was way beyond the scope of my internship. For the last part of it, I tried to focus on disentangling pre-processed embeddings of the images, which proved much faster, but leaves with a problem of re-constructing an image from the modified embeddings (Figure 2). 

I definitely learned a lot about training machine learning models, fine-tuning to find the right parameters and hyper-parameters. I first-hand saw what posterior collapse was, and developed my problem-solving skills while looking for ways to make all the different aspects of a problem work together. I also learned how to use SLURM to submit processing jobs on the GPU clusters. 

Overall, I am very grateful for the time I had to properly dedicate to this complex problem, and thank Aradhika and Tariq for letting me be part of their project. I hope this has opened the project up to an alternative pathway for approaching the problem: separating the data as opposed to targeted noise.