[RFC] Audio and Spectral Processors

[kunitoki]

Spec: AudioProcessorBase / DomainProcessor / SpectralProcessor Hierarchy

Overview

The AudioProcessor class has been refactored into a three-tier hierarchy:

AudioProcessorBase                        — name, params, latency, sample rate, listeners, preset/state
    ↑
DomainProcessor<Context<float>, Spec>     — typed processing: prepareToPlay(spec), processBlock(ctx), flush()
    ↑
AudioProcessor (DomainProcessor<AudioProcessContext, AudioSpec>)
                                          — AudioBusLayout, MIDI buses, hasEditor(), createEditor()
SpectralProcessor (DomainProcessor<SpectralProcessContext, SpectralSpec>)
                                          — AudioBusLayout, MIDI buses
    ↑
SpectralBridge (: AudioProcessor)         — STFT bridge: forward FFT → SpectralProcessor → inverse FFT + COLA overlap-add

SpectralBridge is itself an AudioProcessor, so it plugs directly into AudioGraphProcessor without any graph-level changes. No separate spectral graph module or SpectralGraphProcessor exists in this branch.

---

Tier 1 — AudioProcessorBase

File: modules/yup_audio_processors/processors/yup_AudioProcessorBase.h
Implementation: modules/yup_audio_processors/processors/yup_AudioProcessorBase.cpp

Extracted from the former monolithic AudioProcessor. Owns all domain-agnostic infrastructure:

Members

Category	Members
Name	processorName, getName()
Parameters	parameters, parameterMap, parameterHostIDMap, addParameter(), getParameters(), getParameterByID(), getParameterByHostID(), getParameterIndexByHostID()
Playback config	sampleRate, samplesPerBlock, getSampleRate(), getSamplesPerBlock(), setPlaybackConfiguration() (virtual, non-pure)
Latency	latencySamples (atomic), getLatencySamples(), setLatencySamples() (notifies listeners on change)
Process lock	processLock (CriticalSection), processIsSuspended (atomic), getProcessLock(), isSuspended(), suspendProcessing(), ScopedProcessSuspension
Offline mode	offlineProcessing (atomic), isOfflineProcessing(), setOfflineProcessing()
Listeners	Listener (with audioProcessorChanged(AudioProcessorBase*, ChangeDetails&)), addListener(), removeListener(), updateHostDisplay()
ChangeDetails	Flags: latencyChanged, tailChanged, parameterValuesChanged, parameterInfoChanged, nonParameterStateChanged; builder methods withLatencyChanged(), withTailChanged(), etc.
Tail	getTailSamples() (virtual, default 0)
Presets	getCurrentPreset(), setCurrentPreset(), getNumPresets(), getPresetName(), setPresetName() (all pure virtual)
State	supportsDataTreeState(), loadStateFromDataTree(), saveStateIntoDataTree(), loadStateFromMemory(), saveStateIntoMemory() (all virtual with default implementations)

---

Tier 2 — DomainProcessor (header-only template)

File: modules/yup_audio_processors/processors/yup_DomainProcessor.h

template <
    template <typename FloatType> class DomainProcessContext,
    class DomainSpecification>
class DomainProcessor : public AudioProcessorBase

Design rationale

The first template parameter is a template template (DomainProcessContext<FloatType>) so each domain can define its own Context<float> / Context<double> pair. The second parameter is a plain class (DomainSpecification) because specs are not typed on float precision — they describe static configuration (sample rate, block size, FFT size, etc.).

Members

Category	Members
Processing entry	processBlock(DomainProcessContext<float>&) — pure virtual, single-precision
Double precision	processBlock(DomainProcessContext<double>&) — virtual, default no-op; supportsDoublePrecisionProcessing(), setProcessingPrecision(), getProcessingPrecision(), isUsingDoublePrecision()
Bypass	processBlockBypassed(DomainProcessContext<float>&), processBlockBypassed(DomainProcessContext<double>&) — virtual, default no-ops
Lifecycle	prepareToPlay(const DomainSpecification&) — pure virtual; releaseResources() — pure virtual
Flush	flush() — virtual, default no-op
ProcessingPrecision	Enum class: singlePrecision, doublePrecision

---

Tier 3 — AudioProcessor

File: modules/yup_audio_processors/processors/yup_AudioProcessor.h
Implementation: modules/yup_audio_processors/processors/yup_AudioProcessor.cpp

class AudioProcessor : public DomainProcessor<AudioProcessContext, AudioSpec>

AudioSpec (the DomainSpecification)

File: modules/yup_audio_processors/processors/yup_AudioSpec.h

class AudioSpec {
    float sampleRate = 44100.0f;
    int maxBlockSize = 1024;
    int numChannels = 2;
};

AudioProcessContext (the DomainProcessContext)

File: modules/yup_audio_processors/processors/yup_AudioProcessContext.h

template <typename FloatType>
struct AudioProcessContext {
    AudioBuffer<FloatType>& audio;     // in-place I/O buffer
    MidiBuffer& midi;                  // MIDI events, sorted by samplePosition
    ParameterChangeBuffer& params;     // parameter automation events
    AudioPlayHead* playHead = nullptr; // host play-head (null if unavailable)
};

AudioProcessor additional members (beyond DomainProcessor)

Member	Purpose
busLayout (AudioBusLayout)	I/O bus configuration
getBusLayout(), getNumAudioInputs(), getNumAudioOutputs()	Bus introspection
acceptsMidi(), producesMidi()	MIDI bus presence checks
hasEditor()	Pure virtual
createEditor()	Returns nullptr by default
getNumVoices()	Returns 0 by default
setPlaybackConfiguration(sampleRate, samplesPerBlock)	Overrides base; calls releaseResources(), then AudioProcessorBase::setPlaybackConfiguration(), then prepareToPlay(AudioSpec(...))

Backward compatibility

Existing AudioProcessor subclasses that override prepareToPlay(float, int) must be updated to prepareToPlay(const AudioSpec&). The processBlock signature changed from (AudioBuffer<float>&, MidiBuffer&) to (AudioProcessContext<float>&). All existing code in the repo (examples, tests, graph) has been migrated.

---

Tier 3 — SpectralProcessor

File: modules/yup_audio_processors/spectral/yup_SpectralProcessor.h
Implementation: modules/yup_audio_processors/spectral/yup_SpectralProcessor.cpp

class SpectralProcessor : public DomainProcessor<SpectralProcessContext, SpectralSpec>

SpectralSpec (the DomainSpecification)

File: modules/yup_audio_processors/spectral/yup_SpectralSpec.h

class SpectralSpec {
    float sampleRate = 44100.0f;
    int maxBlockSize = 1024;
    int numChannels = 2;
    int fftSize = 2048;
};

SpectralProcessContext (the DomainProcessContext)

File: modules/yup_audio_processors/spectral/yup_SpectralProcessContext.h

template <typename FloatType>
struct SpectralProcessContext {
    SpectralBuffer<FloatType>& bins;   // in-place frequency-domain buffer
    ParameterChangeBuffer& params;     // parameter automation events
};

SpectralProcessor additional members

Member	Purpose
busLayout (AudioBusLayout)	Bus configuration (same as AudioProcessor)
getBusLayout(), getNumAudioInputs(), getNumAudioOutputs()	Bus introspection
acceptsMidi(), producesMidi()	MIDI bus presence checks
setPlaybackConfiguration(sampleRate, samplesPerBlock)	Override: calls releaseResources(), then AudioProcessorBase::setPlaybackConfiguration(), then prepareToPlay(SpectralSpec(...))

Note: SpectralProcessor does NOT have hasEditor() or createEditor() — these are audio-domain concerns.

---

SpectralBuffer

File: modules/yup_audio_basics/buffers/yup_AudioSpectralBuffer.h

template <typename Type>
class SpectralBuffer { ... };
using AudioSpectralBuffer = SpectralBuffer<float>;

Design

• Backed by AudioBuffer<Type> internally (2 × numBins samples per channel)
• Each channel stores interleaved [real, imag, real, imag, ...] matching FFTProcessor::performRealFFTForward output convention
• Realtime-safe read/write operations (no allocation)

Key API

Method	Description
SpectralBuffer(int numChannels, int numBins)	Sized constructor
setSize(numChannels, numBins, ...)	Resize with optional content preservation
clear()	Zero all bins (no-op if already cleared)
getNumChannels(), getNumBins()	Dimension accessors
getWritePointer(channel)	Raw interleaved pointer for vectorized operations
getWritePointer(channel, bin)	Pointer to specific bin's real component
getReadPointer(channel), getReadPointer(channel, bin)	Const pointer accessors
getBinRef(channel, bin)	Returns BinRef — lightweight mutable reference to a single bin's real/imag
copyFrom(destCh, src, srcCh)	Channel-to-channel copy
copyFrom(destCh, destStart, src, srcCh, srcStart, numBins)	Ranged copy
makeCopyOf(other, avoidReallocating)	Resize and copy (supports cross-type)
fill(channel, real, imag)	Fill single channel
fill(real, imag)	Fill all channels
hasBeenCleared(), setNotClear()	Cleared flag management
operator==, operator!=	Comparison operators

BinRef (inner class)

Method	Description
real(), imag()	Mutable references to real/imaginary components
set(real, imag)	Set both components
magnitude()	std::sqrt(real² + imag²)
phase()	std::atan2(imag, real)
operator std::complex<Type>()	Implicit conversion

---

SpectralBridge — Audio-to-Spectral STFT Bridge

File: modules/yup_audio_processors/spectral/yup_SpectralBridge.h
Implementation: modules/yup_audio_processors/spectral/yup_SpectralBridge.cpp

class SpectralBridge : public AudioProcessor

Design

SpectralBridge is an AudioProcessor that internally performs:

1. Per-channel input ring buffer accumulation
2. When enough samples accumulate (≥ hopSize): extract fftSize windowed samples → forward FFT → build SpectralProcessContext<float> → call spectralProcessor->processBlock() → inverse FFT → COLA overlap-add into output ring buffer
3. Copy from output ring buffer to audio output

This means SpectralBridge plugs into AudioGraphProcessor as a regular audio node — no graph changes needed. It is NOT paired with a separate spectral graph.

Key API

Member	Description
SpectralBridge(name, busLayout)	Constructor; calls allocateResources(), sets latency
setSpectralProcessor(shared_ptr<SpectralProcessor>)	Assigns the spectral sub-processor; registers its parameters
getSpectralProcessor()	Returns current spectral processor or nullptr
setFFTSize(int)	Must be power of 2; updates latency immediately
getFFTSize()	Current FFT size
setOverlapFactor(int)	hopSize = fftSize / overlapFactor (≥ 1)
getOverlapFactor()	Current overlap factor
setWindowType(WindowType)	Window function for analysis/synthesis (default: hann)
getWindowType()	Current window type
setWindowParameter(float)	Window parameter (Kaiser beta, Gaussian sigma, Tukey alpha, etc.)
getWindowParameter()	Current window parameter

Lifecycle

Method	Behavior
prepareToPlay(AudioSpec)	Stores sample rate/block size via AudioProcessorBase::setPlaybackConfiguration(), calls allocateResources() (sizes rings, FFT, windows), calls spectralProcessor->prepareToPlay(SpectralSpec(...))
releaseResources()	Deallocates rings/windows/buffer, calls spectralProcessor->releaseResources()
flush()	Zeroes all ring buffers and counters, calls spectralProcessor->flush()
processBlock(AudioProcessContext<float>&)	Input → rings → processAvailableFrames() → output; no allocation

Internal: processAvailableFrames()

1. Clears spectralParams buffer
2. While nextFrameStart + fftSize ≤ samplesWritten:
   • For each channel: extract fftSize samples with analysis window → fft.performRealFFTForward() → copy numBins×2 floats into spectralBuffer
   • If spectralProcessor is set: build SpectralProcessContext<float>{spectralBuffer, spectralParams} → spectralProcessor->processBlock()
   • For each channel: copy modified bins back → fft.performRealFFTInverse() → overlap-add with colaCompensation into output ring
   • Advance nextFrameStart by hopSize

COLA compensation

Analysis/synthesis windows are computed via WindowFunctions<float>::generate(). The COLA compensation array ensures perfect reconstruction for the given overlap factor: colaCompensation[n] = window[n] / sum(window[i]² for i in overlap positions at n).

Thread safety

• processBlock acquires ScopedTryLock on processLock; clears output if lock fails
• Suspended flag checked after lock acquisition; clears output if suspended
• Configuration setters (setFFTSize, setOverlapFactor, setWindowType, setWindowParameter) use ScopedProcessSuspension and call reconfigureIfPrepared()

[kunitoki]

This has been mostly implemented (with some changes) in #116