Architecture Machine Group Projects: Interactive Frame Buffer Animation 1980

LIPSYNC was originally funded at M.I.T. by the a DARPA contract (not industry, not entertainment, not art) to investigate teleconferencing; studying the feasibility and reliability of digital transmissions in lieu of face-to-face meetings. As an animator, my first contribution was to give the Votrax (a voice synthesizer) a face, or at least, a mouth. I digitized sixteen lip positions and matched them to the Votrax library of phonemes. Using the zoom and pan capabilities of an AED Frame Buffer, I was able to display the animated lips pronouncing words typed in “real time” at the workstation. It wasn't long before I pasted these lips onto a face.

The results were crude – closer to “Clutch Cargo” than PIXAR animation – but chillingly effective. Although there the “man behind the curtain” was revealed, literally typing to put words in this mouth, it was hard not to believe that the face was actually talking. Animators exploit this; we are conditioned to suspend disbelief and accept that a message coming from a talking head originates from the mind represented by the image. We improved the image only slightly with a series of photographs, each posed with the required phonemes.

It was 1981 when an officer from the Office of Naval Research looked at our demonstration and made the request, “Can you make it say, ‘Hello Breshnev?'” He then made the prescient comment, "From now on, no one will ever be able to push the button based on a transmitted image.”