Kalman

Development biography · earlier archive

Development profile compiled from project source and retained records in October 2026. Dates and test results describe those records; they do not imply a new build, live acceptance test or public release.

Source-backed biography for the entertrainment showcase. Scope: the project archive. Read-only inspection; no builds run in this session. Everything below is grounded in repository source; unknowns are marked. No marketable assets were invented.

Biography

The problem. Consumer laptops hear the world through a single, noisy built-in microphone. Turning that stream into decisions an assistive-audio app can act on — is this a quiet room, a steady mains hum, a café, a sudden transient? how confident are we? how aggressively should we mask or cancel? — demands estimation that is robust to noise, numerically stable over long runs, and, above all, interpretable. Kalman exists to be that estimation layer. AGENTS.md states the purpose directly: "estimation, tracking, and state-inference infrastructure for a laptop-mic-driven adaptive acoustics app," explicitly oriented to "reduce cognitive load, improve regulation, and help users who are sensitive to particular sonic conditions," including neurodiverse and overload-sensitive users.

Origin & motivation. The repository's documented history shows a telling evolution. Sources/README.md and Documentation/Prosody.md present the package's first life as an audio-prosody and harmonic-tracking toolkit (YIN-style F0, STFT magnitudes, multi-sinusoidal EKF, vibrato/tremolo, speaker-style profiling). Over successive refactors it was reorganised from a flat monolith into a layered set of modules and reframed around an acoustics-support product. The Documentation/NextLevelAcousticEstimationRoadmap.md captures that pivot — the stated next step is "not add more filters" but productising a coherent acoustic-state contract. The git history reflects real, dated engineering: a modular baseline (a374483), Accelerate BLAS dense linear algebra (48b3e78), an incrementally-fed streaming prosody pipeline (7e58cca), and prosody-accuracy fixes for pitch windowing and vibrato/tremolo timing (bc26e97, 1099360).

Intended audience. Primarily the author's own host app(s): developers wiring microphone-derived feature frames into adaptive masking, soundscape and cancellation behaviour. Secondarily, anyone needing a clean, inspectable Swift (or C++) Kalman/IMM/EKF stack for audio or general latent-state estimation.

Distinctive ideas. Three stand out. First, interpretability as a product requirement — estimators expose innovation, covariance diagnostics, mode probabilities and confidence, rather than hiding behind opaque scores (AdaptiveKalmanStepDiagnostics, EKFStepDiagnostics, InnovationConvergenceTracker). Second, numerical honesty — square-root filters (SquareRootKalmanFilter, SquareRootExtendedKalmanFilter, SquareRootAdaptiveKalmanFilter) are first-class for covariance-fragile paths, and the IMM uses log-sum-exp normalisation to avoid underflow. Third, a deliberate stable/experimental boundary — speculative methods (UKF, particle filter, an alternate "Experimental" DKF formulation) live in KalmanExperimental, isolated from the stable app path.

What it does. Feature frames in (AcousticFeatureFrame), interpretable state out. The documented Sonics flow : extract app-side features → BroadbandEnvironmentEstimator.step(frame:) for room/background stabilisation → NarrowbandInterferenceTracker.step(frame:) for mains hum / narrowband tones (seeded near 50/60 Hz) → optional nonlinear hum/harmonic tracking via SinusoidalEKFModel / ComplexSinusoidalEKFModel / MultiSinusoidalEKFModel → IMMFilter for scene/regime inference, converted to an AcousticSceneSummary → SupportControlEstimator.step(frame:scene:) producing a smoothed SupportControlOutput (masking intensity, soundscape density, cancellation aggressiveness, adaptation rate, fail-safe hint). AcousticFeatureAdapters converts app features into the package's measurement types so callers never hand-build matrices.

A concrete user journey (developer). A Sonics engineer imports KalmanCore and KalmanIMM. Each hop, they fill an AcousticFeatureFrame from their front-end and step the broadband and narrowband trackers; the narrowband tracker locks the mains hum and reports amplitude and per-tone confidence. They run an IMMFilter across scene models (quiet / steady-hum / speech-present / transient-heavy / broadband-clutter / overload-risk) and read AcousticSceneSummary.sceneConfidence — documented as an IMM decisiveness measure, not a guaranteed classifier probability. They feed frame + scene into SupportControlEstimator; when confidence drops, SupportControlOutput.fallbackRecommended is raised as a conservative safety hint for the app to honour. Nothing touches the audio render thread the package doesn't own; the app keeps all I/O.

Prosody lineage. The original capability remains live and maturing: FrameProsodyAnalyzer (YIN-like F0, RMS, pause detection), STFTFrontEnd, KalmanProsodyAnalyzer (end-to-end audio→profile), the newer KalmanProsodyStream (incremental), and EKFProsodyFeatureExtractor/SpeakerProsodyProfiler for vibrato, tremolo and speaker-style scores.

Where it sits. This is honest research-grade infrastructure: broad, tested at the numerical-behaviour level, and candid in its own docs about what is solid (the math core) versus still soft (product contracts, real fixtures). It is a serious estimation foundation rather than a finished consumer feature.

Main modules / features

Authored functionality (all first-party Swift unless noted). A visitor cannot "see/hear" output directly — it is a headless library — so these are described as capabilities exercised via tests/host app:

• KalmanCore — dense linear algebra facade over Accelerate ; AdaptiveKalmanFilter + SquareRootAdaptiveKalmanFilter; ExtendedKalmanFilter / AdaptiveExtendedKalmanFilter + EKFModel protocol; SquareRootKalmanFilter, SquareRootExtendedKalmanFilter; robust updates (RobustKernel, MEstimatorRobustEKF); RTSSmoother; HampelFilter/F0Deglitcher; innovation-adaptive Q; example EKF models (SinusoidalEKFModel, LogSpectralEnergyEKFModel, CochlearEnergyEKFModel, AudioDOAEKFModel); and the app-facing acoustic contract (AcousticFeatureFrame, BroadbandEnvironmentEstimator, NarrowbandInterferenceTracker, SupportControlEstimator, AcousticSceneSummary, SupportControlOutput, AcousticFeatureAdapters).
• KalmanHarmonics — MultiSinusoidalEKFModel/Config, ComplexSinusoidalEKFModel, phase-wrap utilities, and HarmonicFreqEKFModel (adds a direct frequency-bearing measurement so angular frequency is observable).
• KalmanIMM — generic IMMFilter + EKFMode; acoustic-scene helpers (AcousticSceneTransitionBuilder, acousticSceneSummary()); IMM frame/mode snapshots; innovation-convergence monitoring.
• KalmanProsody — prosody front-end, STFT, end-to-end and streaming analysers, speaker-style features.
• KalmanDKF — SpectralKalmanDenoiser (canonical spectral denoiser), discriminative AudioDKF/AudioDKFProcessor, NoiseCanceller.
• KalmanExperimental — UKF + square-root UKF, ParticleFilter, DMatrix, and the renamed Experimental DKF formulation (deliberately not* the canonical DKF).
• Kalman — umbrella re-export .

Dependencies: Apple Accelerate (BLAS/linear algebra); AVFoundation optional, guarded by #if canImport(AVFoundation). No third-party package dependencies in Package.swift.

Technical foundations

Adaptive Kalman filtering as the workhorse; EKF for nonlinear harmonic/frequency models; square-root forms for conditioning-sensitive and long-running states; IMM for scene switching; RTS smoothing for offline passes. Linear algebra runs on Accelerate with a BLAS dense path (48b3e78). The C++17 mirror (CppKalman/) re-implements adaptive full-covariance and square-root Kalman filters, template EKF/adaptive-EKF, multi-sinusoidal model, IMM + scene transitions, and the acoustic smoothing wrappers, built via CMake/CTest. Local/internal packages only — this repo is self-contained.

Maturity by feature (dated evidence)

• Source implemented: Yes, broadly — 7 library targets, 38 Swift sources under Sources/ (ccx repo map, HEAD 1099360, 2026-10-17).
• Tests passing: swift test → 42 tests, 0 failures, verified first-party on 2026-07-15 (same repo). Since then commits 48b3e78, 7e58cca, bc26e97, 1099360 added/changed tests (AccelerateBackendParityTests, StreamingKalmanProsodyTests, FrequencyTrackingPipelineTests) and prosody behaviour; the suite was not re-run in this read-only session, so the current exact count/pass state is unverified as of 2026-10-17. 14 test files present across all 6 test targets.
• Verified capabilities (by dedicated tests): frequency observability (HarmonicFreqObservabilityTests), IMM log-sum-exp stability (IMMLogSumExpTests), NEES/NIS consistency + adaptive-R convergence (AdaptiveConsistencyTests), square-root UKF equivalence (SquareRootUKFTests), Accelerate backend parity (AccelerateBackendParityTests).
• C++ build/test: CMake configure/build + 1 CTest passed, verified first-party 2026-07-15; not re-run 2026-10-17.
• Built as a product / released / deployed: No. No executable/demo target in Package.swift; not published to any registry; no license. Runtime inside a shipping app = unknown/unverified from this repo.
• App-facing contract hardening: Not done. Contract types (AcousticFeatureFrame, summaries, SupportControlOutput) are Sendable only — grep for Codable/schemaVersion/version in AcousticSupport.swift returns no matches (2026-10-17). No units/ranges/versioning/serialization or missing-data quality companion yet.

Roadmap, limitations, FAQs

Roadmap : harden the app-facing contract (units, ranges, versioning, Codable, a feature-quality/missing-data type); dedicated HumHarmonicTracker and narrowband-interference bank; explicit acoustic-scene descriptors replacing name-based mapping; real feature fixtures (quiet room, HVAC hum, fan, charger whine, speech contamination, café clutter); realtime tiers + allocation audits; Swift↔C++ parity tests; and an experimental research track (variational-Bayes adaptive filtering, change-point detection, multi-tone tracking) gated behind fixture wins.

Known limitations: app-facing contract still soft (no units/versioning/serialization/quality); validation leans on synthetic/numerical tests rather than real recorded fixtures; realtime-safety tiers not yet formalised per estimator; Sources/README.md is outdated ; not released/licensed.

FAQs:

• Does it record or process audio? No — it owns no audio I/O. You feed it feature frames / spectral measurements; the host app keeps capture and playback.
• Swift or C++? Both. Swift is primary (macOS/iOS 26, Accelerate); CppKalman/ is a standalone C++17 CMake mirror of the estimation core for non-Swift deployments.
• What does "scene confidence" mean? An IMM decisiveness-style measure, not a guaranteed classifier probability .
• Is it production-ready? It is research-grade and test-backed at the numerical level; the product contract and real-fixture validation are still in progress. Treat defaults as sane but tune Q/R and thresholds per domain.
• Stable vs experimental? Stay on the stable targets (Core/Harmonics/IMM/Prosody/DKF). KalmanExperimental (UKF, particle filter, alternate DKF) is isolated and not on the default app path.