Glaze! is an interactive media framework in Divooka that features a Processing-like interface.

Overview

Today, I'm laying the groundwork for audio APIs for our Glaze! framework.

Divooka Glaze! draws inspiration from SFML and Processing, so we need to understand how those two handle audio first.

How SFML Handles It (SFML.Net)

To play audio in SFML.Net, we use the SFML.Audio module, which provides two main classes:

1. Sound + SoundBuffer for short, fully-loaded sounds (like sound effects).
2. Music for long, streamed audio (like background music).

Playing Short Sounds

Short audio files (WAV, OGG, FLAC, etc.) can be fully loaded into memory:

// Load the entire file into a buffer (only do this once per sound).
SoundBuffer buffer = new SoundBuffer("hit.wav");

// Create a Sound instance and assign the buffer.
Sound hitSound = new Sound(buffer);

// Optional settings
hitSound.Volume = 75f;    // 0–100
hitSound.Pitch  = 1.2f;   // playback speed
hitSound.Loop   = false;  // play once

// Play it
hitSound.Play();

// Make sure the program doesn't exit immediately,
// otherwise the sound will be cut off:
while (hitSound.Status == SoundStatus.Playing)
    System.Threading.Thread.Sleep(10);

Key points:

• Reuse the same SoundBuffer if you need to play the same sound multiple times.
• Keep the Sound (and its SoundBuffer) in scope until playback finishes.

Streaming Music

For large files (e.g., full-length tracks), use the Music class, which streams from disk:

• It doesn't load the entire file into RAM.
• Suitable for long tracks, though with slightly higher CPU usage due to streaming.

// Create a Music instance; it will stream the file on the fly.
Music backgroundMusic = new Music("background.ogg");

// Optional settings
backgroundMusic.Volume = 50f;
backgroundMusic.Loop   = true;

// Start playback
backgroundMusic.Play();

// In a game loop we'd typically:
// while (window.IsOpen()) { /* update, draw, etc. */ }

Common Pitfalls

• Missing Native DLLs: If you get DllNotFoundException, ensure all CSFML native libraries are present.
• Scope & Lifetime: If Sound or Music goes out of scope (is garbage collected), playback stops.
• Program Exit: If the application ends before the audio finishes, you won't hear the full sound - keep the main thread running or integrate playback into the render loop.

How Processing Handles It

Processing offers sound support through the built-in Sound library and the legacy Minim library.

Below are the two most common approaches.

The Built-in Sound Library

Installing/Importing

1. In the Processing IDE, go to Sketch → Import Library… → Add Library…
2. Search for “Sound”, then click Install (if not already installed).

Placing Audio Files

• Create a folder named data in the sketch's directory.
• Copy audio files (e.g., music.mp3 or beat.wav) into that data folder.

In the sketch, import and declare:

import processing.sound.*;
SoundFile song;

void setup() {
  size(400, 200);
  // Load a file named "music.mp3" from /data
  song = new SoundFile(this, "music.mp3");
  song.play();        // start playing once
}

void draw() {
  background(0);
  // (Optionally visualize or control playback here)
}

Looping, Volume, Rate

void setup() {
  size(400, 200);
  song = new SoundFile(this, "music.mp3");

  song.loop();               // loop indefinitely
  song.amp(0.5);             // set volume (0.0–1.0)
  song.rate(1.2);            // change playback speed (1.0 = normal)
}

Minim

Minim handles things similarly, albeit through AudioPlayer:

import ddf.minim.*;
Minim minim;
AudioPlayer player;

void setup() {
  size(400, 200);

  minim = new Minim(this);
  // loadFile(filename, bufferSize)
  player = minim.loadFile("music.mp3", 2048); // Place MP3/WAV files inside the sketch's `data` folder.
  player.play();

  // Looping & Controlling
  player.loop();        // loop playback
  player.setGain(-10);  // lower volume: gain in dB
  player.setTempo(1.5); // speed multiplier
}

void draw() {
  background(50);
  // ...
}

• Where to Store Files: Always place audio files in the sketch's data/ folder so Processing can find them.
• File Formats:
  • Sound library: supports WAV, AIFF, MP3 (depending on platform).
  • Minim: supports MP3, OGG, WAV.
• Latency / Real-Time: For low-latency playback or live audio input/processing, Minim (or advanced libraries like Beads or TarsosDSP) may be preferable.
• Visualizations: Both libraries support real-time audio analysis (waveforms, FFT), which is great for interactive visuals.

Observations

Those two statements are crucial:

// Load a soundfile from the /data folder of the sketch and play it back
file = new SoundFile(this, "sample.mp3");
file.play();

We can observe a few things regarding API design:

1. It's procedural and imperative - we create resources first, then set properties, then play/stop via the resource reference. Importantly, it's necessary to keep a reference for later control.
2. Resources might require explicit (or scope-based) release.

On the other hand, I'm more interested in a "functional" style API, like this:

The Approach

We could implicitly load and manage resources, releasing them at the end of the application lifetime (on window close), if no explicit reference is needed. However, distinguishing between SoundBuffer and Sound (both are IDisposable) can be tricky when deciding which to persist. At the very least, this is what we should expose via PlaySound() and PlayMusic().

private void ReleaseResources()
{
    // Sounds
    foreach (Sound item in _soundInstances)
        item.Dispose();
    foreach ((string key, SoundBuffer item) in _soundBuffers)
        item.Dispose();
    foreach (Music item in _music)
        item.Dispose();
}

In fact in the current design we cannot share the underlying instance - since Glaze! is implementation-agnostic - unless we introduce strongly typed types for SoundBuffer, Sound, and Music, so all implementations can recognize them. Otherwise, we can't return any instance objects.

// IInteractiveServiceProvider, which is ignorant of SFML.Net's existence
#region Audio Engine
void PlaySound(string filePath);
void PlayMusic(string filePath, bool loop = true);
#endregion

A slightly inelegant shortcut is to expose them as untyped "object handles":

/// <summary>
/// One-shot sound.
/// </summary>
object PlaySound(string filePath);
/// <summary>
/// Loopable BGM etc.
/// </summary>
object PlayMusic(string filePath, bool loop = true);
/// <param name="volume">0–100f</param>
void SetVolume(object soundOrMusic, float volume);
/// <param name="pitch">0–1f</param>
void SetPitch(object soundOrMusic, float pitch);
void SetLoop(object music, bool loop);

While it might seem "unsafe" on the paper, it works well in Divooka's graphical interface:

(This is partly thanks to Divooka's support for return value naming via XML doc comments, e.g., <returns name="Music">A music instance.</returns> for PlayMusic().)

Conclusion

So far so good! There's a lot we can enhance in the audio engine - more parameters, DSP support, etc. For example, Processing has very neat functions for low-pass/high-pass filters.

But for now, we can play sounds - and that's a solid start!

References

• Processing Sound library documentation: here
• Divooka Glaze! framework documentation on Methodox Wiki