Pulling Faces 

Face Generator (unreleased, 1980s)

Back in the Last Days of the Atari ST, one of the first things I prototyped with my shiny new 386PC was a face generator program.

The idea was that you'd input exact measurements of certain proportions of the human face, and the program would them "cartoon" them as lineart. Once you had a small library of face metrics, you'd be able to get the program to generate additional faces as tweened versions, randomised versions of particular populations, or exaggerated versions of the difference between given faces. 

Since the program was only generating lineart (and a little bit of eyeball shading), it was easy  to produce real-time changes, and the program quickly got a few manually-programmed expression algorithms.  If you wanted the character to look right, or up, or down, it wouldn't just be the eyeball alignment that that would change, parts of the mesh overlay for the eyelids would be "pulled" in sympathy.

The idea was that the program would have alternative versions of the same face showing different expressions, and would isolate and save the differences as "expression" files, and then it'd accumulate a library of expressions that could be applied by varying amounts, to any of the other faces. I was also working on finding the absolute minimal set of additional features that would be useful for 3D. The idea was that if you were, say, a comicbook artist drawing the same character over and over again, then the program would give you they key features of that face, from any  given angle, showing any given expression or combination of expressions. The artist could then put in the surrounding profile details themselves. It'd basically be doing for the face what those wooden artist's mannequins do for body poses. It'd also help people who couldn't really draw to produce half-decent comics. :) 

The prototype worked satisfyingly well, but I ended up abandoning it because it didn't have an obvious market other than people who needed help keeping comicbook characters fresh but consistent, or as the basis of a "talking head" for PC-based news delivery. Long-term I was playing with the idea that I could perhaps also produce a speech synthesis library, and have a PC download the daily news as a streaming text feed with embedded gestural information, with the character of your choice as a virtual newsreader in the corner of your screen. With more advanced hardware than was generally available in the 1980's one might be able to package a mapped high-resolution face image along with its facial metrics, and have the program morph the photographed face real-time to match the spoken text. Choose your own Newsreader! But it was all a bit nebulous, and the project was sucking up a lot of R&D time without an obvious financial payoff. So I pulled the plug.

Singular Inversions' FaceGen

Anyhow, doing a bit of googling on the subject in 2008 for old times sake, it seems that there are now a few different companies that've written custom face-generating programs, which now output fully-rendered 3D images and/or 3D data for modeling packages. There's now a market for developing computer gaming characters or online avatars that didn't exist when I was tinkering with the idea.

The program that I just had a little play with was one called FaceGen, which has a free export-disabled evaluation version. 

For serious work, you're supposed to feed in front and side views of a face, mark a few key points, and it'll generate a corresponding 3D shape. However, FaceGen is smart enough to make a pretty good guess using just a single front-facing (or roughly front-facing) image. Subsequent manipulations of the eye-mesh assume that the subject was looking directly forwards when the photo was taken.

Here's what it managed to do with a few single images of distinctive celebrity faces. Some of the profile views aren't so convincing, but that's going to be at least partly because I didn't give the program "profile" photos to work from (as you're supposed to).     

		Albert Einstein
		FaceGen did a decent job from a badly-colorised source picture. The tip of the nose should have been a little more bulbous, and the shape of the head looks wrong (to me), but a finished image would normally have hair, so no biggie.

		Angelina Jolie				Britney Spears
		Any webpage on computer graphics and faces HAS to include at least one attempt at Angelina Jolie. It's The Law.				I tried two different Britney pics and both default FG models were a bit grim, for different reasons. But a 50:50 average of the two "Mutant Britney" files, using the FG "tween" function, worked surprisingly well, and managed to distill out the common features of both photos.
		Danni Minogue				Jennifer Love Hewitt
		A few manual parameter tweeks helped this one no end. For this test, I switched the texture off, saw how close I could get to a recognisable stylised representation, and then switched the texture back on again.				If you were doing this properly, you'd want to erase the overlapping hair in the source image first.  Other than that, it came out pretty well.
		Zhang Ziyi				Janet Jackson
						FG doesn't ask you to identify the eyebrow line, it works it out for itself.  In this case, FaceGen got thrown by the very heavy eye makeup, and decided that the top of Janet's eyelids were actually her eyebrows. The composite image looks adequate,  but if you switch the texture map off ... ew! It's gone horribly wrong ...
		Shaznay Lewis				Kimberley Davies
		FaceGen likes Shaznay! To see if it was a fluke, I tried a second picture, and it liked that, too. [[.fg file]. Good lighting helps.
		Lynda Carter				Jeri Ryan ("Seven of Nine")
		The result of using a rough, bitty image, with teeth showing. Unless there's some trick that I've missd, FG doesn't seem to like teeth in an image, and doesn't have an obvious way of letting you set a lip line "edge" (see earlier comments about not being able to manually define eyelid cutouts).				I thought that the "Borg eyepiece" might confuse FG, but it cleverly ignored it.

Here are a few "before and after" pictures of what FaceGen does. On the left is a small thumbnail of the original source image, then we have a view of the output FaceGen face, a smoothed view of just the surface, the surface plus the source mesh grid, and the mesh grid with the "texture" overlay derived from the photograph. The file for these images is here. As before, click on the "bordered" images to see larger versions.

Charisma Carpenter
source image	FaceGen output	smoothed surface	mesh without texture	texture and mesh

FaceGen then lets you morph, tween, tweek, and shift a face by age, or ethnicity, or a variety of other averaged trends, and can also apply some facial expressions. 

The program seems to be hampered by the lack of a decent expression library, but I guess that if you're producing faces that'll be rendered as a tiny piece of screen in a shoot-em-up, perhaps there's not so much justification in developing a proper set of convincing expressions, beyond cartoon shock and "Grrrrr!". It also has "phoneme-specific" expressions, presumably for people who need to animate talking faces.

FaceGen is a commercial program, but the free demo version is okay for "play" as long as you don't mind losing its ability to export .obj object files. It'll still export "[.fg" files in its own format, which consists of the face parameter set, manual parameters, and the texture map. Some games can apparently import fg files, so if you're a game enthusiast and want to create your own face for your online avatar, or you think that it'd be funny to have the cast of Casablanca shooting aliens or the panel of X Factor running amok in Grand Theft Auto, then FaceGen might be a reasonably fast way to get results.  If you want to export your [.fg files to more generic computer-modelling mesh files so that you can use them in "pro" 3D graphics apps, then you'll probably need to get the full commercial version.   

Looxis Faceworx

Another freebie is FaceWorx from Looxis. This one does export object files, but it's only free for non-commercial use.

Looxis have more of an emphasis on 3D portraiture, and they have booths that let you sit down, be photographed, and have your head computer-modelled and laser-etched into a glass block. Consequently their program is designed more as a front-end for generating accurate 3D face surface data, for more conventional computer-modelling applications. 

FaceGen vs Faceworx

The two companies are coming at the problem from different angles. Looxis are focused on generating standard, accurate 3D files for export to other programs, and the Looxis program lets you spend a lot of time fine-tuning the details of a face by moving guide marks. If you want to, you can use a lot of guide marks to produce a very faithful 3D mesh-map of the given face. It'll tween and smooth the data, and project the original photograph over the surface as a texture map, but if you really want a truly convincing rendering of a face's contours, it's really up to you to spend some time clicking very accurately.

The emphasis of the SI  programme is more on automatic analysis and recognition of facial structure. FaceGen already knows a lot about faces. Although it renders 3D maps and textures in the same way as Faceworx, FaceGen stores faces as combinations of physiological parameters (nostril tilt, nose length, etc.). FaceGen's "photofit" function only needs a very small number of points - sides of the nose and mouth, centre of the eyes, etc  -- and it then tries to recreate a face by analysing the photo(s) provided and tilting, sizing, and randomising its model until it finds a combination of metrics that it thinks might produce the best match to those photographs, when appropriately angled and lit. It then generates a mesh that is texture-mapped in a similar way to Faceworx. The computer's doing all the serious work, but all that analysis means that if your computer is slow and doesn't have a graphics accelerator, it might take fifteen minutes for it to analyse a face. Once FaceGen has finished, you'll get a realistic-looking face model, but it might veer slightly away from the original in terms of any unusual aspects that are difficult to replicate using that stack of  parameters. The texture map created from the original photo will usually do a decent job of putting these missed details back, but there might be a few unusual shaping subtleties that the software can't quite mimic by varying its standard parameter set.  The use of a fixed  parameter set means that extreme "caricature"  variations can still look reasonably natural, but it won't always give enough variables to perfectly emulate the precise line of a distinctive nose, or an eyelid crease. FaceGen also looks as if it's trying to "rotate" eyes by distorting a mesh rather than by implementing proper swivellable eyeballs as spheres behind the face. 

Summary:

• Faceworx allows you to use front and side images to manually place accurate points, and so it tends ot produce crisper detail and hopefully a more recognisable, more accurate final 3D contour map of the subject ... but since it's relying on you to place all those control points, it takes a certain amount of skill, and any lack of accuracy on the user's part can result in a face that has subtle wonkiness or jagginess, fracturing the illusion. And once you've generated your static 3D model, there's not a lot you can do with it other than light it and look at it from other angles, or export it into another program for more powerful processing.
• FaceGen does almost all the work for you, and can make a brave attempt at a 3D map from even a single image ... but some aspects of the resulting face profile might be a little "generic". It's good at matching a lot of aspects of bone structure, and will tend to get the right sort of jawline and head shape, but if you zoom on on an eyelid, you'll find that there don't seem to be enough "eye shape" parameters for it to always be able to recreate a "distinctive" eye properly within the 3D model, without the help of the overlaid "photographic" layer ... if you're modelling a "known" face, you'll tend to find that the contours of the eye region in the output file are slightly standard-looking, and that the distinctive detail is instead being superimposed by the projected texture map. If you take a recognisable face, and switch that texture map off, so that you're just looking at a pure set of computer-generated contours, the "known" face isn't always recognisable. And because it's relying on the projected texture map, some features can look a bit fuzzy and imprecise. It manages some subtle features surprising well, but other aspects like lip line, eyelid detail or the fleshy tip of a nose are more difficult to parameterise.   

Barack Obama	A few sample FaceGen ".fg" files (for more, see the facebank page):   Anna Friel | Claudia Black | Freema Agyeman  | Morena Baccarin | Kaley Cuoco | Monica Bellucci | Melanie Sykes | Shaznay Lewis

Disclaimers

• I don't have any relationship with any of these other companies or their programmers. I just thought that the programs were cool.
• I used celebrity faces in these "tutorial" examples because a lot of people will be familiar with them, and unless you're familiar with a face, it's difficult to judge how well a program works. Educational use. This doesn't mean that it's okay to use other people's faces in commercial products or advertising or promotions. It isn't. If you start using one of these files to produce materials in a way that would normally require the subject's permission for the use of an image, and you don't have that permission, you're still liable to get sued by them. 
• FaceGen and Faceworx are both primarily aimed at applications in which the operator has full control over the posing and lighting of the original photograph (front and side view, eyes front, even lighting, blank expression). Some of the other photographs that I tried, with side lighting or strongly tilted faces, didn't work as well as the images above. If you're thinking of buying FG, try the free version first, with some material that's representative of what you'll be working with.

External Links

• FaceGen – @  Singular Inversions – "smart" face creator/manipulator, highly automated. Free version lacks .OBJ export
• Faceworx – @ Looxis.com – "manual" face-transcription software. 
• Poser – a full-figure modelling and animation program. Poser is probably the the best-known "computer mannequin" program
• DAZ 3D – a free program that reads Poser files. DAZ make their money from selling add-in computer models for people to use within the program..
• Blender – a free generic 3D modelling program (but with a notoriously long learning curve due to the "quirky" user-interface).
• Wikipedia: Anatomical simulation – a useful category that I created on Wikipedia
• ErkDemon Blog – for feedback or discussion of these pages