Diplopia / Ocular Panel — entertrainment

PROJECT REVIEW · 18 October 2026

Inside the project

Prepared with the project’s development agent from source and retained evidence. This is a project-owner assessment, not an independent customer review. Dated checks were not rerun for this website.

Two linked research strands inside AssistantApp, both explicitly pre-clinical. Diplopia Correction asks whether a display can be reshaped to ease double vision: one bounded set of display adjustments (horizontal/vertical offset, rotation, a second 'ghost' layer and its opacity, type scaling) delivered through three scopes — a single view preview, every window in the app, and the whole Mac desktop via ScreenCaptureKit re-projection. The part that works most convincingly is the least glamorous: a manual override in the Diplopia Correction Studio that drives all three scopes with no camera involved at all, so the display geometry can be tuned and seen immediately. The sensing side is deliberately held to a higher bar than the rendering side. A capture harness scores the raw per-frame eye metrics for signal integrity before anyone asks whether a deviation estimate is clinically right, because a healthy-looking agreement statistic computed on a metric that secretly tracks the face bounding box is measuring the artefact. Eleven of those checks are mutation-tested — swapped eyes, a shared normaliser moving both apertures together, 'not measured' collapsed into 'measured zero' — and the derived layer is a pure fold, so the same capture replayed under swept constants must give a byte-identical result. That determinism is what allows constants to be tuned offline from one recording instead of asking a person to re-perform a protocol for every candidate value. Ocular Spatial Interaction is the forward-looking half: gaze and hand input driving spatial surfaces, with a trial engine and an evidence board. Its software is built and tested, its panel is wired into the Creative Apps gallery, and it has never once been run with a camera and a person — every measured acceptance gate is still open. Honest summary: strong engineering discipline around measurement and delivery, real numerical evidence for the geometry and the GPU shader, and no validated clinical or human-performance result yet.

What stands out

Camera-independent manual path: the Studio's offset/rotation/ghost sliders drive the per-view, app-wide and whole-desktop scopes immediately, so the correction geometry is demonstrable without any sensing or personal data.
Exact numerical evidence, dated and reproducible: production float64 geometry vs the prism-distance reference, 90 values, zero mismatches at 1e-9 tolerance (max abs error 2.27e-13); the production Metal shader run on GPU over a synthetic impulse image across two backing scales and two ghost opacities, 32,768 values, zero mismatches at exact tolerance. Harnesses are checked in, not just reported (2026-09-11).
Signal integrity is scored separately from, and before, clinical agreement — with 11 mutation-tested checks and a byte-identical replay determinism gate. One check detects the image-side vs anatomical eye swap automatically from a single stream.
Anti-double-correction is handled explicitly: when the whole-desktop overlay is on, the app's own root modifier is fed a neutral adjustment, so app windows are not corrected twice.
Epistemic honesty is built into the artefacts rather than added afterwards: requested poses are recorded as requested, not as observed ground truth; Optionals distinguish 'not observed' from zero, and that distinction is itself a tested contract.
The iOS/iPad TrueDepth path yields genuinely measured interpupillary distance, distance and vergence, rather than the population-prior pinhole reconstruction the Mac webcam path must fall back on.

Current limits

Not a clinical product, diagnostic system or validated treatment surface — the module README says so in those words. No clinical validation, no independent evaluation, no user feedback or ratings exist.
Ocular Spatial Interaction has never been launched with a camera or a person. Package tests and a macOS app build pass; every measured exit gate remains open and needs the owner at the camera.
A hard sensor limit, measured 2026-08-31: the macOS Vision landmark model will not close its eye polygon, so a shut eye and an open eye in downgaze are not separable in that feature at any threshold. Absolute lid 'openness' is therefore not recoverable from landmarks; monocular closure is recovered from the ratio between the two eyes instead, gated on eye width for head yaw. Bilateral closure is invisible to that ratio.
The ocular pipeline measured roughly 3 frames per second on the development Mac while its timing constants had been written for 6–7. Gates were re-expressed against the measured frame interval on 2026-08-31 and the spurious trip rate fell from ~75% to ~5%, but the underlying reason for the low frame rate was never diagnosed. At that rate a 100–150 ms blink is shorter than one sampling interval and is structurally unobservable.
macOS supplies no camera intrinsics, so the field of view is an assumption (~60–66°) unless a calibration has set it; this scales every pixel-to-degree conversion.
Resolution budget is tight by optics, not by code: at 60 cm on a 1080p webcam the pupil traverses the whole 16-inch panel in roughly 14 × 9 camera pixels, so half a pixel of noise is about 1.15° of gaze. 'Median error under one degree' sits at the noise floor.
Distribution boundary: the OcularSpatialInteraction framework and its tests are not in the Main checkout — only the Creative Apps panel is. Landing requires a file-list plan agreed by the owner.
Essentially macOS-only: of the whole DiplopiaCorrection module only two files carry any iOS branch. The iPad route historically fell through to a stub; the measured TrueDepth work lives outside the module.

Features & evidence

Diplopia Correction Studio (manual override)

source

Before/after preview with offset, rotation and ghost-opacity sliders; toggles for app-wide and whole-desktop delivery, with permission and display-count status.

Recorded evidence

Source present: App/DiplopiaCorrectionStudio.swift, App/LiveCameraCorrectionCoordinator.swift. Offset logic unit-covered via pure functions; recorded 26/26 assertions through a standalone harness because xcodebuild test could not run on this host (2026-07-23). Not independently re-verified today.

Three delivery scopes over one bounded adjustment

source

The same DisplayAdjustments value drives a single-view preview, every app window, and the whole desktop via ScreenCaptureKit capture and re-projection.

Recorded evidence

Contract documented in Docs/OcularCorrection_Delivery_Validation_2026-09-11.md: bounded output ±28 logical points at 60 points/second, unbounded estimate exposed separately, prism dioptres and angular deviation never summed. Requires Screen Recording permission.

Delivery geometry and GPU shader numerical verification

component-tested

Checked-in harnesses compare the production geometry and the production Metal shader against references at float64 and exact tolerance.

Recorded evidence

2026-09-11: 90 geometry values, 0 mismatches, 1e-9 tolerance, max abs error 2.2737e-13; GPU parity 32,768 values, 0 mismatches, exact, max error 0. Tools/OcularCaptureHarness/verify_delivery_{geometry,gpu,sc}.swift.

Guided calibration protocol and pipeline probe

installed-tested

A spoken-cue protocol walks a seated person through baseline, per-eye occlusion, ductions, gaze, head-pose, lid and expression blocks, exporting measurements, the exact emitted script, per-step completion and optional stills/video.

Recorded evidence

Source verified today at Modules/Features/Accessibility/Validation/{OcularGuidedCalibration,OcularPipelineProbeWindow,OcularCalibrationRunner}.swift. Export contract and a shorter single-eye-recovery variant documented 2026-09-11. Real captures exist but are personal biometric material and are excluded here.

Capture harness — signal integrity scoring

component-tested

Two scorers: raw per-frame metrics, and the derived alignment/trust/gating layer. Verdicts distinguish pass, regression, known-failing and unanswered-for-want-of-instrumentation.

Recorded evidence

Tools/OcularCaptureHarness/README.md: 11 checks, all mutation-tested (swapped eyes, common-mode aperture, zero-collapsed Optionals, drifted IPD anchor, pinned trust, dt-gating, broken determinism gate). Standard library only; exit status 1 on regression.

Deterministic replay and offline constant tuning

component-tested

The world-state derivation is a pure fold reading dt from the sample data, so a capture can be replayed under swept constants and asserted byte-identical.

Recorded evidence

score_replay.py --compare asserts byte-identical world sequences as the Phase-4 gate; BayesianOptimizer.swift sweeps constants offline. Registry of sweepable constants with exclusions is machine-readable.

Frame-relative timing gates

source

Dropout and discontinuity gates expressed as multiples of the measured frame interval rather than absolute seconds, with a bounded estimator and a clamped filter variance.

Recorded evidence

2026-08-31: macOS build succeeded and the edited files were confirmed recompiled. Replayed against three recorded captures, the torsion gate's trip rate fell from 73–75% to 4.5–5.8% and spurious discontinuities from 10–14% to under 2%. No camera run since.

iOS/iPad TrueDepth ocular metrics

source

ARKit face tracking supplies measured distance, interpupillary distance, vergence and blink signals, with an optional ~10 Hz stream to the Mac over the companion bridge.

Recorded evidence

Verified present today: AssistantApp/Sources/iPhone/TrueDepthOcularMetrics.swift (ARFaceTrackingConfiguration). Recorded 2026-07-23: both macOS and iOS destinations built successfully; adaptive-distance scaling 10/10 on a standalone check. No device session recorded.

Ocular Spatial Interaction — panel and framework

component-tested

Gaze and hand driven spatial surfaces with a trial engine, carousel and landscape layouts, a diagnostics overlay and an evidence board; an iPad companion can act as a gesture surface.

Recorded evidence

Recorded 2026-09-30: 579 package tests, macOS app build succeeded (5d08a1c94); scripted runs with simulated input correctly report 'cannot assess' on every gate. Verified today: only the Creative Apps panel (3 Swift files) is in Main; the framework core is not. Never launched with a camera.

Nine-point gaze calibration map

source

A fitted quadratic feature-to-screen map persisted between sessions, used by the gaze-reveal surface.

Recorded evidence

DiplopiaCorrection/Modules/Features/Calibration/9Point/NinePointCalibrator.swift, persisted to dc.calibration.ninepoint.json. Its quality figure is a training residual, not held-out. Recorded lesson: leave-one-out on a 3×3 grid is blind to a separable between-level bend, so interleaved validation targets are mandatory.

Screen-coverage probe (geometry kit)

component-tested

Command-line probe computing pupil travel and panel-pixels-per-camera-pixel for the attached display, with a measured 16-target run.

Recorded evidence

35 XCTest pass, 7 mutants killed (2026-09-22). The live run cannot be started from an agent session — macOS denies camera access — so the measured 16-target run remains owed and must be started by the owner from their own terminal.

Review updates & corrections

No 17 October biography exists for this slug — verified by listing 2026-10-18/../2026-10-17/responses/, which contains no diplopia entry. Nothing here is a correction to a previous entry; this is the first packet for the family.
Overlap to de-duplicate during curation: the gaze/pinch work shared with DICOM Lens belongs in the neuroatlas family, and Mirror Spar / dodge-reaction material belongs to perception-action. This packet deliberately excludes both.
One correction to an internal claim, for accuracy: it was previously recorded that no ARKit existed anywhere in this work. That is true only of the DiplopiaCorrection module itself; a measured TrueDepth path does exist, outside it, at Sources/iPhone/TrueDepthOcularMetrics.swift.
The Creative Apps wiring for the ocular spatial panel is now applied — its own WIRING.md was auto-marked superseded on 2026-09-24 because the diff it contains no longer applies.

Workflow walkthrough

A guide to the native workflow. These steps describe the app; only the recordings and images embedded on this page are available as website demonstrations.

Open AssistantApp on macOS and go to the Diplopia section of the Cognitive HUD, then open 'Preview & Tune' to show the Studio sheet with its side-by-side uncorrected and corrected panes.
Leave the mode on Manual — no camera, no personal data — and drag the horizontal offset slider. The corrected pane separates into a primary and a ghost layer; the uncorrected pane is a true baseline because the sheet is not transformed by the window's own correction.
Raise ghost opacity from zero and add a degree or two of rotation, to show that the ghost layer is an explicitly chosen display parameter rather than anything inferred from a face.
Turn on the app-wide toggle: every window in the app picks up the same bounded adjustment instantly, because the manual patch disarms continuous (camera-driven) arming.
Turn on the whole-desktop toggle to show the ScreenCaptureKit path re-projecting the entire display, and point out that the app's own windows stop being corrected twice at that moment.
Close with the checked-in GPU parity report, which is the evidence that the shader doing the above matches its reference exactly over 32,768 values.

Planned captures · not yet available

Proposed 60-second screen recording of the six walkthrough steps above, captured on a synthetic test card with no face in frame.
Proposed still of the harness verdict table from a scored capture, with per-eye measurement columns redacted.