Inside the project
Prepared with the project’s development agent from source and retained evidence. This is a project-owner assessment, not an independent customer review. Dated checks were not rerun for this website.
AssistantApp is a private macOS research workstation whose audio modules share one set of engines. Four of them are far enough along to review as working tools. Actor Rehearsal Studio is useful for rehearsing a scene alone: partner lines are rendered from saved neural voices, placed in a virtual room, and the partner waits for your actual words before answering. Its strongest working parts are the script parser, the spatial sound stage with recorded effects, and the turn tracker; its real limitation is that a full human-run scene has not yet been accepted end to end, so the finished demos are offline productions, not microphone sessions. Voice Memo is useful as a capture-to-arrangement path: record with measured latency compensation, arrange on a block editor, split stems, quantise takes with pitch preserved. Its DSP separation is honest and modest, measured against a neural reference rather than claimed. AA Music is the most complete piece of software in the family: an installed 0.48.1 player that turns a link into an album, separates six instruments with a cached neural model, mixes them live with EQ and plugins, cues songs on the bar and runs a vocal lab with de-reverb, de-ess, pitch correction and voice conversion, all with self-tests and receipts per build. AudioIOKit is the shared audio host package underneath; it is useful to developers, not visitors, and its device-level acceptance is still open. Across all four, next priorities are the same: close human acceptance where it is missing, commit untracked work, and resolve voice-consent documentation before any voice-based demo is published. This is the project owner's review, not a customer endorsement; no ratings or user feedback exist.
What stands out
- Actor Rehearsal: script-conditioned turn-taking that completes a turn on recognised final words plus rate-scaled silence, with protected lines and scripted early entrances (281 component checks 12 Oct; 279 with audio 13 Oct).
- Actor Rehearsal: finished binaural scene productions with full receipts, cue timelines and a dry A/B copy (The Last Light 55.7 s; The Last Transmission 6:09).
- Voice Memo: measured input-latency compensation, project mix with 15 XCTests, and a stem splitter benchmarked on real audio against Demucs (refiner SIR +6.8 dB vs four-stem +3.1 dB, 2 Sep 2026).
- AA Music: installed native app with per-build receipts; live stereo-to-stems crossfade with compensated HSEQ latency; beat-mapped layering; vocal lab measured on real stems (de-esser cuts up to ~9.6 dB on strong sibilants, bypass sample-exact).
- AudioIOKit: narrow host plus 18 optional modules; October reliability upgrade adopted by AssistantApp with three schemes building green (4 Oct 2026).
Current limits
- Actor Rehearsal human whole-scene acceptance failed on 12 Oct and has not been re-run; all 'passed' host receipts are controlled-input runs; all 13 Oct work is untracked (branch ends at dd992e69d).
- Voice Memo has no screenshot or audio asset on disk; several arrange features were verified only in off-screen probe renders; the four-stem drums output is worse than the mixture.
- AA Music is not distributed (ad-hoc signed, macOS 26 floor, not a git repo); receipts for builds 75–83 were not located in bounded reads, so the 0.48.1 install date is unknown; de-esser prepare/audition acceptance was still pending in the latest doc read.
- AudioIOKit package changes are uncommitted; interruption, media-services reset and route-change acceptance on real devices is undone.
- Voice-based demos: only the owner's own profile has documented consent in my notes; two imported profiles need clearance before publication.
Features & evidence
Typed script parser + screenplay reader
installed-testedPaste or import a script; sluglines, cast specs, dialogue, movements and compound sound cues are extracted with line numbers; unrecognised lines are listed, not dropped.
Recorded evidence
ActorRehearsalScriptModel.swift, ActorRehearsalScreenplayReader.swift; screenplay-checks.swift; installed 'Screenplay 2026-10-13' app
Live turn-taking
installed-tested (controlled input); human whole-scene: not acceptedSpeak your line; on-device recognition and prosody evidence decide the handoff; manual 'Actor Line Delivered' fallback.
Recorded evidence
Tools/ActorRehearsalChecks/LiveTurnTaking.md; Evidence/2026-10-12/host-acceptance-ready-final.json (18 checks, 12 Oct); Acceptance/2026-10-12-LiveTurns/README.md (receipt failed)
Spatial stage and recorded SFX
installed-testedVoices and cues placed around a listener with HRTF; rain/wind beds continue, knocks alternate recordings; procedural stand-ins are labelled.
Recorded evidence
StudioAudioFixes-20261013/verification.json (full build exit 0, 279 checks, installed audio check passed, 13 Oct); run.sh --audio --library 111 checks (12 Oct)
Per-character saved neural voices + expression hierarchy
component-tested; installed (Expression 2026-10-12 app)Each role picks a saved voice; a per-line score (intent, focus, arc) is compiled from a clean reference; default render preserves pitch and timing.
Recorded evidence
Tools/ActorRehearsalHierarchy/README.md (14 checks); Clean Source Expression/Listening Notes.md (13 Oct)
Delivery grammar
installed-testedBraced directives before a line ({pace=0.85x; feel=reflective; interrupt=hold}) and {pause=450ms} inside it; invalid directives block analysis and are never spoken.
Recorded evidence
ActorRehearsalDelivery.swift; ActorRehearsalChecks/README.md step 2 (12 Oct)
Offline scene bounce
source + produced outputs (tooling, no in-app export button)Render a whole scene to 48 kHz/24-bit stereo with manifest and audio verification.
Recorded evidence
Tools/ActorRehearsalBounce/README.md (on branch); bounce folders' Audio Verification.json (12–13 Oct)
Recording with latency compensation
source, built greenRecord a take; it lands on the grid shifted by the measured input-chain latency; input meter with clip latch.
Recorded evidence
memory voicememo-recording-mixer.md (17 Jul 2026 pass; Aug commit b98f38d68)
Arrange page (ResearchControls block editor)
source, built green; UI seen in probe renders onlyLanes, ruler, mute/solo mirror, tempo from the recording, harmonic copy by ⌥-drag.
Recorded evidence
memory voicememo-arrange-lane-alignment.md (3 Oct 2026, build-7 green)
Stem splitter: Split Stems / Isolate Vocal
component-tested (56/56 self-tests); built; real-audio benchmarkedRight-click a region to split into four stems or lead vocal + backing, grouped on a bus as a collapsible track stack.
Recorded evidence
memory voicememo-stem-splitter.md; Tools/GAWBench (2 Sep 2026); commits 506a99bc9, a370df5ba
Groove Studio takes, quantise, commit
source; probe tests pass per doc; physical output = manual gateCapture regions as takes, analyse transients, quantise with one monotonic warp per channel, audition original vs corrected, commit immutable renders with undo.
Recorded evidence
Docs/Audio/SemanticGrooveStudio.md; Tools/SemanticGrooveTimingProbe
Provenance + Audio Scene handoff
source + 9 XCTestsScenes promoted into Voice Memo carry a provenance packet visible in the inspector; regions can be sent back to the Audio Scene canvas.
Recorded evidence
memory gaw-p1-status.md (P0/P1 done; stage B not done)
Album import and ordering
installed; live-testedPaste a YouTube link or search; the album is identified via MusicBrainz and downloaded in release order; uncertain matches stay visible for manual choice.
Recorded evidence
README.md live acceptance 10 Oct 2026 (Creepin', 10 tracks); 61 Python tests at 0.20
Six-stem live mixer
installed; self-testedSelect a track; separation runs in the background and crossfades from original into six instruments with EQ, HSEQ, AU inserts and meters.
Recorded evidence
README 0.11/0.15; --continuity-self-test, --handoff-audio-self-test; compact-stem-cards-65…70.png
Beat cue, layer, Q tempo authority, tap/downbeat
installed; self-testedCue a second song on the bar, layer its instruments on the leader's beat map, choose the steady master, tap one bar to correct the grid.
Recorded evidence
README 0.14–0.22; --beat-mix-self-test, --stem-layer-self-test, --tap-tempo-self-test, --tempo-authority-self-test
Vocal Lab: de-reverb, de-esser, pitch correction, singing conversion, saved versions
installed; measured on real stems; some native gesture acceptance pendingProcess the vocal stem and audition Processed / Removed / Original; restore saved conversions from exact archives.
Recorded evidence
docs/VOCAL_DEREVERB_0.43.0.md; docs/VOCAL_PITCH_CORRECTION_0.45.0.md; docs/SINGING_CONVERSION_0.44.0.md; test-output/deesser-real-song-69.json; docs/COMPACT_STEM_CONTROLS_AND_VOCAL_ACCEPTANCE.md (build 74)
Exact stem archives and portable sessions
installed; self-testedStems archived as verified WavPack chunks; sessions export with plugin versions; cold restore without re-separation.
Recorded evidence
README 0.20–0.25; docs/SESSION_AND_STREAMING.md
iOS companion
built, device-installed for developmentPair by AirDropped file; browse the Mac library; save albums offline.
Recorded evidence
ios/README.md; dist/AA Music iOS 0.6.0.ipa
AudioIOHost + AudioDSPCore
source + package tests; adopted by AssistantApp (built green 4 Oct)Engine lifecycle, endpoint replacement, callback gates, SPSC ring buffers.
Recorded evidence
AuioIO/Package.swift; AudioIOUpograde.md; AUDIOIO_ADOPTION.md
Optional modules (offline, plugin host, ARA, timeline, MIDI, convolution, ML/Python, Metal viz, release readiness)
source + per-module test targets; several are contracts/metadata rather than enginesImport only what an app needs.
Recorded evidence
README.md 'Current status'
Real-device acceptance
planned / not doneInterruptions, media reset, route changes, voice processing under load.
Recorded evidence
AUDIOIO_ADOPTION.md 'OUTSTANDING'
Review updates & corrections
- AA Music: installed version remains 0.48.1 (83) per Info.plist; the latest doc read (COMPACT_STEM_CONTROLS_AND_VOCAL_ACCEPTANCE.md) describes build 74 as installed, so builds 75–83 happened after that doc; their receipts were not located in bounded reads (large test-output folder). No 0.47/0.48 release notes exist in docs/.
- AA Music: the de-esser, listed yesterday as roadmap, is implemented and measured on real stems (builds 69–74); prepare/audition native acceptance was still pending at build 74.
- AA Music: genuine compact-module captures exist (compact-stem-cards-65…70.png); build 70's was inspected and contains no library content.
- Actor Rehearsal: a generated Stage Plan.png for The Last Transmission exists and is safe; the actor-assets session (dd1b4a9a) confirmed SFX licensing (Freesound CC0 enforced twice; Sonniss permitted in finished productions, not as a library) and flagged screenplay authorship as unconfirmed.
- Actor Rehearsal: no new builds or evidence since 13 Oct; installed apps on disk are dated 12–13 Oct only.
- Correction to my 17 Oct media note: AA Music was described as '0.45.x installed'; the plist says 0.48.1.
- Voice Memo and AudioIOKit: no material change found.
Workflow walkthrough
A guide to the native workflow. These steps describe the app; only the recordings and images embedded on this page are available as website demonstrations.
- Actor Rehearsal (native, macOS; not playable on the website): 1) Open Creative Apps › Actor Rehearsal Studio and pick 'The Last Light' in Demo Scenes. 2) Set Your character = Alex; assign Morgan and Casey to two saved neural voices. 3) Enable scene sound effects, put on headphones, press Analyze Rehearsal. 4) Press Start Scene; Morgan speaks from far left. 5) Speak Alex's line; the status shows recognised words and coverage, then Casey answers from the right as rain continues overhead. 6) Press Stop. A native capture of this does not exist; the existing demos are offline bounces.
- Voice Memo (native; proposed capture only): 1) Open Voice Memo, arm a lane, press Record, speak, Stop. 2) The take lands on the grid, shifted by the measured latency. 3) Right-click the region › Isolate Vocal; Vocal and Backing lanes appear on a bus. 4) Choose Capture regions as takes › Analyze takes › Quantize highlights. 5) Audition Original vs Corrected, Commit. No capture exists; Tools/ResearchControlsProbe can render the real editor off-screen if the owner runs it.
- AA Music (native; existing captures of steps 3–4 exist): 1) Paste a YouTube link › Add song & album; the seed plays while the album assembles. 2) Click a track; the waveform shows beat ticks and BPM. 3) Open Mix stems; the original crossfades into six instruments (compact-stem-cards capture). 4) Double-click Vocals › Vocal Lab › De-reverb or De-ess › audition Processed/Original. 5) Open Layer library, pick another song, enable drums and bass, play on the next bar. 6) Tap the pulse on beat one to correct the grid.
- AudioIOKit (developer-only; no visitor demo): 1) Add the package by path. 2) import AudioIOHost; create a host; set a render provider. 3) host.setInputReceiver(_, captureEnabled:) for capture. 4) Read diagnostics (desiredRunning, isRunning, callback counters). Nothing to capture for the site.
Planned captures · not yet available
- Voice Memo arrange page (real editor rendered off-screen by Tools/ResearchControlsProbe).
- Actor Rehearsal live turn (screen + mic) showing a recognised line and partner handoff.
- None of these modules can run in the browser; the existing site player for The Last Light is the only interactive element.
Development biography · earlier archive
Development profile compiled from project source and retained records in October 2026. Dates and test results describe those records; they do not imply a new build, live acceptance test or public release.
Biography
AssistantApp began as an AI personal assistant for macOS and iOS: a Swift 6, SwiftUI, Swift Concurrency app with tasks, notes, mail intelligence through Microsoft Graph, calendar and connectors, chat with cloud and offline models, and an "app HUD spine" that shows what the assistant is doing. The repository's own guidance files describe it that way, and the assistant shell, Tasks, Mail and Notes modules are still its backbone. What distinguishes it is what happened next: the same codebase became the host for a long run of creative, clinical-adjacent and accessibility experiments, each added as a registered panel that shares infrastructure rather than as a separate app.
The problem it addresses is fragmentation. The work spans disciplines that usually require separate tools, engines and files. AssistantApp’s answer is a single filesystem-synchronized Xcode project where modules import the same local packages: Kalman (estimation and prosody), FIRFilters and RemezEngine (filter design), SpectralProcessor (the HSEQ harmonic equaliser), Imager (stereo phase scope), ResearchControls (timeline block editor and research sliders), NoiseColour packages (noise synthesis, denoise, audio runtime), SemanticCompute (CPU/GPU parity substrate), ExpressiveNeuralVoiceKit (voice creation contracts) and AudioIOKit (engine lifecycle). A measurement made in one module is reusable in another because the types are shared.
The environment serves creative and technical exploration: rehearsal, audio analysis, knowledge work, imaging research and adaptive interaction. It is a private research environment with different validation and distribution states for each module.
Three distinctive ideas run through the codebase. First, evidence over assertion: modules write receipts (JSON verification files, acceptance logs, listening notes) and documentation carries a doc-status tag stating what fraction of the described scope is actually built. Second, "declared is not wired": the project's own working rules treat a feature as absent until a caller exists and a runtime check has observed it. Third, parity: numerical work is checked across CPU and GPU implementations through SemanticCompute before it is trusted.
A concrete user journey, as the app exists today on the author's Mac: open the app, use the Creative Apps gallery in the middle pane to open Actor Rehearsal Studio, paste a scene, cast two saved neural voices, put on headphones and rehearse a scene with partners who wait for your line. Switch to Voice Memo to record a take against a beat grid, split it into stems, and hand a region to the Audio Scene canvas. Open NeuroAtlas to load a DICOM series, run a segmentation and attach the finding to the research evidence graph. Open Diplopia Correction to calibrate a per-eye display offset from the camera. Each of these is a real, built path with evidence on disk, and each has documented limits.
The recordings on this page demonstrate selected creative workflows. They are retained productions from the native tools; the website is a companion to the research environment.
Maturity is uneven by design. The macOS app builds green from the main checkout; the iOS app target has been verified building (1 July 2026) with a separate iPhone experience; NoiseColour has reached TestFlight; most research modules are "source implemented, built, locally installed, partially accepted". A large share of the source tree is untracked by git, which is a project-management fact worth stating rather than hiding: the working copy, not the commit history, is the product.
Main modules a visitor can see, hear or try
- Assistant shell: tasks, notes, mail workflow, daily briefing, weekly review, chat, HUD. (Authored.)
- Creative Apps gallery: Audiobook Studio, Expressive Neural Voice, Actor Rehearsal Studio, Conference Theatre, Voice Studio/Designer/Setup, Voice Memo, Instruments, DAW Link, Semantic Timing, MIDI Fidget, Maschine MK3, NoiseColour, Semantic Monster, App Audio Router, Sound Shelter, Acoustician, Audio Scene Canvas, Creative Research gallery. (Authored; engines from local packages.)
- Hearing and health: Hum Masker/Sonics, Hyperacusis suite (26 panels incl. HackRF RF analysis, with active-RF operations explicitly blocked), Therapeutic Environment, Ward Quiet Support, Regulation Assist, Rehab Coach, Perception-Action Training. (Authored.)
- Imaging and vision: NeuroAtlas (DICOM, Twin Brain, pharmacology), Diplopia Correction, Focus Lab, Ocular Spatial Interaction. (Authored; OpenJPEGKit and nnU-Net models are dependencies.)
- System: Display Load Governor, Drive Quarantine, Energy Graph, Routing Matrix, Clouds, Video Editor, ChatGPT Archive, Messaging (hash-chain E2E chat, Multipeer + Firebase transports). (Authored; Firebase, GoogleSignIn, AudioKit, KDTree are dependencies.)
Technical foundations
Swift 6, SwiftUI, Swift Concurrency; AVFoundation/AVAudioEngine; Metal and MPS via SemanticCompute; CoreML where models exist; Speech framework; CoreMIDI and MIDI 2.0 UMP; Firebase (Firestore) and GoogleSignIn; Microsoft Graph OAuth with PKCE. Thirty-two local package references in the project. Important authored local packages: ResearchControls, SpectralProcessor, Imager, Kalman, FIRFilters, RemezEngine, SemanticCompute, ExpressiveNeuralVoiceKit, NoiseColour* (seven packages), AudioIOKit (folder AuioIO), WasmBridge, ForensicsCore, NodeBGGen, SemanticControlCore, OpenJPEGKit, AdHocMeshCore, LogicProProject, ARAIntegration. MCP servers expose app control, DICOM analysis and tasks/notes to agents.
Maturity (umbrella level)
| Area | State |
|---|---|
| macOS app build | Green from main checkout, repeatedly (latest cited: 13 Oct 2026 full build exit 0 for the rehearsal install) |
| iOS app target | Built green 1 Jul 2026 (generic/platform=iOS); iPhone UI in Sources/iPhone/ |
| Unit tests | Tests/AssistantAppTests has ~175 Swift files; whole-suite runs on the build host are documented as unreliable; targeted suites have run (e.g. RehabSafetyGovernorTests, 21 tests, 12 Jun 2026) |
| Release | None public. NoiseColour uploaded to TestFlight (build 2026092901, 29 Sep 2026). No App Store listing. |
| Multi-platform | Roadmap doc status "Not Done 10%": macOS-first; no tvOS/visionOS targets |
Roadmap, limits, FAQs
Roadmap (documented): platform compatibility layer and iOS/tvOS/visionOS targets ; Actor Rehearsal audition platform ; SDK adoption; task panel phase 8. Limits: private build; macOS 26 floor via ResearchControls; several modules have "declared but unwired" components by the project's own audits; outbound network on the build host is restricted so cloud features are inert there. FAQ: Is it available? No; it is a private research build. Is it one app or many? One Xcode project, one main app target, plus iOS companions and AUv3 extensions. Does it send my data anywhere? Modules state their boundaries; the local-first default is explicit, and secrets live in Keychain/1Password. Why so many modules? They share engines and evidence; that is the point. Can I run it on an iPhone? The iOS target builds, with a reduced experience; it is not distributed.
Wider module map
AssistantApp’s scope extends beyond the modules with dedicated project stories. The following map records substantial areas identified by its project agent in October 2026. It describes the source inventory and development direction; it does not imply that every area has a finished release or a verified live demonstration.
| Area | Scope and relationship |
|---|---|
| Audio Scene Canvas and Audio Scene Rack | Scene composition, instrument racks, an audio scene engine and offline rendering; a companion creative workflow to Voice Memo. |
| Expressive Neural Voice, Voice Studio and Voice Designer | Voice enrolment, saved voice profiles, voice variations and local rendering, supported by ExpressiveNeuralVoiceKit and NeuralVoiceTrainer. Actor Rehearsal uses this voice infrastructure. |
| SoundBox and room acoustics | Object placement, spatial sound, room modelling and dereverberation research. |
| Sonics and Hum Masker | Hearing-oriented sound tools, runtime contracts and calibration workflows, related to NoiseColour. Their presence is not evidence of clinical efficacy. |
| Hyperacusis and RF analysis | Research panels for sound sensitivity and signal investigation. Active RF operations remain blocked in the described research configuration. |
| Therapeutic Environment and Ward Quiet Support | Research into environmental support, with related web and native implementations. Clinical validation is not established by this inventory. |
| Perception-Action Trainer and Mirror Spar | Movement and response training interfaces, figure rendering and supporting component checks. The agent identified substantial source work rather than a distributed training product. |
| Diplopia Correction and ocular interaction | Guided per-eye calibration, display correction experiments and eye-mediated spatial interaction. These remain research surfaces. |
| Messaging | Local and network communication work, including paired-device, encrypted messaging and iPhone interfaces. Availability depends on the individual transport and build. |
| Notes and canvas | Markdown, mathematical notation, a freeform canvas, document editing and note-intelligence work. |
| ContextAssistant and developer tools | Context routing, agent interfaces, MCP and language-server connections, and a developer console. |
| Rehab Coach and gait companion | Gait capture and rehabilitation-oriented research, including an iOS companion scheme. A source target or successful build does not establish clinical acceptance. |
Shared foundations also include SemanticControlCore, NodeBGGen, OpenJPEGKit, WasmBridge, AdHocMeshCore, LogicProProject and ARAIntegration. These sit alongside the separately profiled audio, numerical and interaction packages. Some are supporting infrastructure; others may warrant dedicated case studies as their product scope and demonstration evidence mature.