To begin morphing images, we need to first define corresponding points across the two images. Using this tool, I manually selected corresponding points in the same order across two images. I used key facial features such as the eyes, nose, and mouth in order to base my correspondences.
We can now create midway faces! Using the labeled correspondence points, I ran the Delaunay algorithm to create triangular regions.
We can now use the triangles to create midway faces! We first warp the face shape to an average shape. This involves averaging the correspondence points across both images, then computing a triangulation for the average points. This enables us to compute a transformation matrix between the average shape and the shapes of both images. Using this transformation matrix to inverse warp color values from each of the two source images, we can then average the two values per pixel in order to cross-dissolve the images.
There are some pretty noticeable differences between each of the two images. Relative to me (Geoffrey), Vincent's mouth is more open (less teeth), and his eyes are tilted upwards. Sadie has long hair. These differences lead to slight discrepencies in the midway images that may not look super natural. The majority of the features across the faces do seem to be properly morphed together though! I got a slightly more natural-looking result from these two EECS faculty members:
With a very similar process used to compute midway faces, we can now create a morph animation between two faces! Instead of one average image, we now make 50 intermediate images, each warping and cross-dissolving an incremental amount between the two images. The results use 1/50 increments for warping and cross-dissolve, leading to 50 frames. This creates a seamless transition between each source image to destination image.
Now, given a large amount of labeled faces from a popuation, we can get the average face. Similar to part 2 when we computed the midway face between two images, we can scale this process for hundreds of images. We first find the average set of points to compute Delaunay triangles, then warp each image's triangluation to the average triangulation we just computed. Then using the same inverse warping process to extract pixel values from each of the hundreds of images, we can average all of those values per pixel to get the "mean face" of the population! I used the FEI dataset to compute the average face and separated the population into smiling and neutral expressions. We can see that the area around the faces is pretty blurry, likely due to how the images have greatly varying backgrounds and details outside the facial region.
Taking a closer look at individual images within the datset, we can warp an image to the average facial expression for the corresponding subpopulation. Keep in mind the average of all of the warped faces led us to our "mean faces" earlier.
We can go even further with this and morph my own face with the average face of the FEI dataset. Since I'm smiling in my photo, I chose to morph with the smiling subpopulation. These are the results of morphing my face to the average smile, and morphing the average smile to my face.
Again using the FEI dataset mean faces, we can produce caricatures through extrapolation. By giving the correlation points in my image a weight of 1.5 and the correlation points in the average FEI smile a weight of -0.5, I can warp my face into an image that more heavily accentuates the features of my face compared to the FEI dataset smiles. Since the dataset is composed to Brazilian faces and I'm Chinese, the resulting image definitely exaggerates the differences.
The FEI dataset conveniently allowed us to compute the average neutral facial expression. Since the current image of myself is smiling, we can use that to wipe off my smile. We can achieve this with a direct warp with the average neutral expression. However, we can go further and cross dissolve with the average neutral expression (which just morphs the appearance). Combining the two both removes my smile and changes my appearance to be closer to Brazilian. Since the neutral expression generally involves no teeth and my original image has teeth, there will still be remnants of them in the morphed image.
