Face MOCAP Light Leak Generator

An interactive audio visualization that generates a light leak effect based on facial motion capture data in TouchDesigner.

My goal

For this project my goal was to create an audio visualization in Touchdesigner that generates a light leak controlled by facial motion capture data. I was inspired to create this effect after noticing how live audio visualizations can create a perfect light leak effect that matches the mood of a sound.

My process: Step 1

  • First, I set up the audio spectrum and then I brought in the audio and normalized it so that it goes from 0-1. 
  • I connected the audio to an audio spectrum and sorted out the sounds (spectrum and size) along with setting up a Math for it. 
  • I selected the two transforms x,y by assigning the ty and tx to two noises and played around with their transform units and period size. 
  • I merged the 4 factors together and set up an instance so that I could pass it onto the visual.

My process: Step 2

  • I set up the geo and camera.
  • I connected the 4 factors set up earlier to the translation and scale of the geo instance (in order to have them move and resize based on the sound).
  • I set up the spec to control the color of the circles in instance2. 
  • I added a bunch of TOP (feedback, transform, level, ramp and blur) to make the circles more nuanced and look like they are moving toward the camera. 

My process: Step 3

  • At this point, I played around with the circle SOP and the transforms until I got an effect that reminded me of the light leak effect on a film camera.
  • I then used OSC to take in data from the FaceOSC app.
  • I set it up so that when the user’s face is far away, the camera input will look blurry – giving the camera focus effect. 
  • I also set it up so that the width and height of the user’s mouth controls the black noise size that appears from the corner – giving the illusion of film distorting and burning.

My process: Step 4

  • At this point, I played around with the circle SOP and the transforms until I got an effect that reminded me of the light leak effect on a film camera.
  • I then used OSC to take in data from the FaceOSC app.
  • I set it up so that when the user’s face is far away, the camera input will look blurry – giving the camera focus effect. 
  • I also set it up so that the width and height of the user’s mouth controls the black noise size that appears from the corner – giving the illusion of film distorting and burning.
Close Menu