Generate animation timing events during audio playback

I want to play audio files in Anvil client with synchronised animation of analysis plots. After exploring various options for generating playback timing events during playback, I found a solution that may be of interest to other Anvil users. It employs the Buzz JavaScript library, which can be imported as an asset of the Anvil app, referenced in Native libraries. In a custom JavaScript file (also an asset), a buzz sound object can be instantiated and controlled.

Two approaches are shown here. Firstly, the buzz object itself can generate timeupdate events a few times a second during playback. A handler can be bound to the timeupdate event of this sound object, giving the playback time of the event:

var sound   
  
function play_audio(audio_file_link){
  sound = new buzz.sound(audio_file_link);
  sound.bind("timeupdate", function () {
    var floatTime = this.getFloatTime()
    anvil.call($('#referenceElementForJScalls'), 'timeupdate_event_handler', floatTime) });
  update_UI_with_repaints();
  sound.play();
}

You can see that this calls a getFloatTime() method to read the playback time at the point when the timeupdate event is raised. The original Buzz script only provides a getTime() method, which returns integer seconds – too slow for animation. If Buzz script has been imported as an asset, a finer resolution method, getFloatTime() can be added:

this.getFloatTime = function() {
   if (!supported) {  return null;   }
   var floatTime = this.sound.currentTime;
   return isNaN(floatTime) ? buzz.defaults.placeholder : floatTime;
 };

Buzz only raises about four timeupdate events each second. For the smoothest possible animations, we want the playback time to be updated whenever the client window is repainted by the browser. The second approach achieves this by running a parallel JS function throughout the playback, which uses the window.requestAnimationFrame() method to call the sound object’s getFloatTime() method before each repaint:

let continue_raising_playback_time_events = false;

// Start regular timing events before every re-paint, until stopped
function generate_playback_time_events(){
  continue_raising_playback_time_events = true;
  window.requestAnimationFrame(playback_timing_loop);
}

function playback_timing_loop(){
  // Only update if the sound is actually playing
  if (!sound.isPaused()){ raise_playback_timing_events(); }
  // repeat if appropriate
  if (continue_raising_playback_time_events) { window.requestAnimationFrame(playback_timing_loop); 	}
}

function stop_playback_timing_events(){
	continue_raising_playback_time_events = false;
}

function raise_playback_timing_event() {
  var floatTime = sound.getFloatTime();
 	anvil.call($('#referenceElementForJScalls'), 'repaint_event_handler', floatTime);
}

Whichever approach is used to generate playback timing events, I found it necessary to add a div to the custom HTML of the form (id=“referenceElementForJScalls”).
Timing events are raised on this div, to be picked up by event handlers in client python. More experienced Anvil users may find a better technique.

Here is a demo. It simply uses the timing events to update time displays in the client, but it could be developed to update animations.:
Audio demo

3 Likes