Wednesday 19 May 2010

Visual/Music synchronization

Ok here is a good idea I had that will make my (and hopefully your) life easier:

Lets assume that your visuals are ahead of the music (in terms of production time). How do you make sure that they fit perfectly? I'm not talking about the easy case of a "beat every other second with a screen flash" but of longer segments that may need to fuse and they are off by a couple of seconds or more.

This is what we do: lets say that the music is slower than the visuals. You introduce a small "delay" factor that is a function of time. So:

DeltaT*=1.0-0.1*sin(max (0,min (1, (time-20)*0.1)));

this will slow down time for the visuals by the integral of this function all over 10 seconds. It is not noticeable. (Yeah I know, it is not really all that framerate independent, maybe you have a better way of doing so?).

Once you digest this piece of information I will continue with the rest.


  1. ehm, *PI in there.. but you get the idea anyway

  2. I input a progress-value to each segment, that is scaled depending on the length of the given segment. Basically setting a start- and end-keyframe for the scene. E.g.

    float starttime = 10.0f, endttime = 20.0f;
    scene01.addInterpolator( starttime, endtime, 0.0f, 1.0f );

    The scene then animates using the input going from 0.0->1.0.

  3. yes. But sometimes the segments are quite large (over a minute). What then?

  4. I like hornet's suggestion, basically have each segment last "one unit" of time, and simply scale the timer for that section by the real time so it lasts as long as you want.

    I recommend having your segments (and effects within segments too) render using their own timers btw, if you're not doing that. Your rendering method becomes something like renderAtTime(time).

    This way it's really easy to loop effects, change the timing, cut back and forth to different sections, play it backwards.. Changing the keyframes becomes as simple as keyframing the timer instead of reworking the animation sometimes. And synching is becomes really easy - you just scale the timer by the BPM of the song - set you white flash for every "1 second" and it magically becomes every "1 beat". (not that I'm expecting white flashes and hypnoglow ;)

  5. psonice - for all things blinky: midi-input ftw! \o/