Voice Memo

Development profile compiled from project source and retained records in October 2026. Dates and test results describe those records; they do not imply a new build, live acceptance test or public release.

Biography

The problem Voice Memo solves is the gap between a quick capture and a usable production object. A memo is easy to record and hard to do anything with: it has no grid, no lanes, no idea what instruments it contains and no record of where it came from. The module answers each of those in turn, and because it lives inside AssistantApp it answers them with engines the rest of the app already uses.

Recording comes first. The recorder runs its own AVAudioEngine path (input, capture mixer with a file tap, monitor mixer, output) separate from playback, so monitoring latency does not depend on file-write buffer size. It measures the input chain latency (device latency, safety offset, I/O buffer, first stream latency) and shifts audio takes earlier onto the grid by that amount; a manual trim stays available. A 30 fps playhead clock drives only an isolated marker view, after a re-render storm was traced to the whole window redrawing on every tick. Live display samples are capped in a ring buffer because the authoritative take is on disk, which closed a memory runaway that reached 1.4 GB an hour.

Arrangement uses the app-neutral ResearchControls TimelineBlockEditor. Since 3 October 2026 the Quantized Grid arrange is one editor that lays out the app's track headers and ruler itself, so headers and lanes cannot drift apart. The arrange tempo is the tempo detected from the recording, a user decision recorded in the notes. Track mute and solo mirror between the editor's document and the app's per-session audio keys; a project with audio on two or more lanes plays through a stereo project mixer with equal-power pan, fades and a soft limiter, covered by fifteen XCTest cases.

The stem splitter is the module's deepest piece. A StemSeparator protocol sits in front of two authored Accelerate-only engines: a four-stem DSP separator (STFT, harmonic/percussive separation, soft masks) and a pitch-informed vocal refiner whose default pitch tracker is a Kalman-smoothed harmonic-sum estimator from SemanticCompute. Twenty-five further DSP operator files cover median HPSS, informed ratio masks, NNLS harmonic demixing, NMF, LPC envelopes, Griffin-Lim and PGHI phase reconstruction, DUET and MVDR spatial separation, constant-Q chroma, source-event tracking with Hungarian assignment, and a break splitter that classifies percussion hits and detects sped-up breaks. A Core ML seam exists for trained models; no trained model is shipped. The honest measurement, taken on real audio against Demucs reference stems with a bench tool in the repo, is that the pitch-informed refiner separates about twice as well as the four-stem engine on SIR (+6.8 dB versus +3.1 dB), that the drums stem is worse than doing nothing, and that an "improvement" the synthetic tests loved scored −15 dB on real audio until it was gated. Those numbers are in the evidence and should be quoted, not rounded up.

Groove Studio attaches the arrange's regions to ResearchControls' durable studio document: capture regions as alternate takes, analyse transients, audition highlights, quantise into a preview with one monotonic warp per channel, compare original and corrected, and commit immutable CAF renders with JSON recipes and a single undo transaction. Musical positions use checked 64-bit microticks (983,040 per quarter note). Phase analysis between a reference and a candidate track measures delay and polarity across bands and can apply a bounded Hilbert phase rotation. Pitch-preserving WSOLA quantise previews were added later with a 0.5–2× ratio guard.

Provenance closes the loop. A scene rendered in the Audio Scene canvas can be promoted into Voice Memo with a versioned provenance packet (scene id, fingerprint, producer, claim) that survives the project's directory rescans and is visible in the block inspector. The reverse path, "Open in Audio Scene", bakes a region to its own file and queues it for the canvas. Clips carry a kind tag (audio, scene, MIDI, automation, annotation) and one function decides whether a block sounds and from which file.

A concrete journey: open Voice Memo, arm a lane, press record; the input meter shows level and clip latch; stop, and the take lands on the grid already shifted by the measured latency. Right-click the region, choose Isolate Vocal, and a Vocal and a Backing lane appear grouped on a bus, collapsible into a track stack. Capture regions as takes, analyse, quantise highlights, audition original against corrected, commit. Select the vocal region and open it in Audio Scene.

The audience is the author and anyone who wants a capture-to-arrangement path with measured, not marketed, separation. There is no screenshot on disk, which matters for a website: the module is better described through its evidence and its bench numbers than through images.

Features a visitor can read about (nothing to try remotely)

• Recording with latency compensation, input meter, project mix playback. (Authored.)
• Arrange page on the ResearchControls block editor; tempo from the recording; M/S mirror; harmonic copy between regions by ⌥-drag. (Authored on an authored package.)
• Stem splitter actions: Split Stems → track stack; Isolate Vocal; Split Instruments with Quality × Masking menu. (Authored DSP; Core ML seam empty.)
• Break splitter with percussion classes and sped-up detection; break breakdown view. (Authored; not yet wired to a live editor action.)
• Groove Studio takes, quantise, commit; phase analysis; audio journeys with HSEQ ownership summaries. (Authored; HSEQ from SpectralProcessor.)
• MIDI input capture, timeline MIDI output, SXP host, synth matching panel, Audio Scene handoff. (Authored.)

Technical foundations

Swift 6/SwiftUI, AVFoundation, Accelerate (vDSP), CoreMIDI, CoreML seam; packages: ResearchControls (the app links the project archive, not the stale vendored copy), Kalman, FIRFilters (Farrow resampler), SemanticCompute (GPU matmul/YIN/spectral Kalman behind canImport), SpectralProcessor (HSEQ), Imager. Code: AssistantApp/Sources/Media/Audio/VoiceMemo/ (24 files), VoiceMemo/EditorWindow/ (33 files), Media/Audio/StemSplitter/ , Media/Audio/Workspace/VoiceMemoPromotion.swift. Probes: Tools/SemanticGrooveTimingProbe (swift test), Tools/ResearchControlsProbe (off-screen editor renders), Tools/GAWBench/run.sh (SDR/SIR/SAR against Demucs).

Maturity

Roadmap, limits, FAQs

Roadmap (documented): Core ML separators behind the existing seam ; GAW stage B (source-range plumbing through the scene engine); wire break breakdown to a live action and colour transient markers by class; RC ruler tempo decisions; ⌘Z routing between app menu and editor tool row. Limits: macOS 26 (ResearchControls floor); one lead voice per refiner; dense polyphony and orchestral demix are out of reach for the DSP engines, as the notes state; drums stem weak; arrange features verified in probe renders rather than on the live window in several cases; the recorder stacks un-armed takes on the selected lane. FAQ: Is this a DAW? It is an arrange editor with recording, mixing, separation and groove tools, not a plugin-hosting DAW; DAW Link and Logic control-surface mapping are separate panels. How good is the stem splitting? Measured, modest: see the SIR table; neural separation lives in Acoustician and AA Music, not here. Does quantising change pitch? Not by default; WSOLA previews preserve pitch within a 0.5–2× ratio. Where do renders go? Immutable CAF files with JSON recipes under Application Support, undoable in one transaction. Can I use it on iPhone? Only a basic memo screen.