Ocula — Webcam Eye-Controlled Cursor for Assistive Input

Gaze signal. Face Mesh runs with refineLandmarks set to true, which is what exposes the iris landmarks (indices 468 and 473). For each eye I take the outer and inner corners, the top and bottom lids, and the iris centre, then express iris position as an offset from the eye centre normalised by eye width and height. Normalising by socket size is what makes the signal roughly invariant to how far the user is sitting from the camera. The two per-eye vectors are averaged, and x is negated because the video preview is mirrored.

Calibration as a least-squares fit, not a lookup. Five targets sit at the four corners inset to 10 and 15 percent plus the centre. Each target samples for 1500 ms, discarding the first 600 ms so the eye has time to settle, then commits the median of the samples rather than the mean so a single blink or saccade cannot drag the point. The five gaze-to-screen pairs feed an affine solve (screenX = a·gx + b·gy + c, same for Y) built from 3x3 normal equations and Cramer's rule. A determinant below 1e-10 means the gaze samples were degenerate, and the app says calibration failed instead of producing a cursor that lies.

Smoothing versus latency. Raw per-frame gaze is far too jittery to dwell on a target. I used a single-pole exponential filter with alpha 0.20, so each new frame contributes 20 percent and 80 percent carries over from the previous smoothed position. That trades responsiveness for stability, which is the correct direction for dwell input: the user is holding still on purpose, and overshoot costs a misfire.

Dwell that tolerates drift. A naive dwell timer resets the moment the cursor moves one pixel, which never completes on a noisy signal. Ocula anchors the dwell at the position where the target was first entered and only resets if the cursor wanders more than 55 pixels from that anchor or moves onto a different element. Completion fires the click, then a 600 ms cooldown suppresses the next dwell so the user does not double-fire a tile they are still looking at.

Hit testing without the cursor in the way. The gaze cursor is a DOM element, so document.elementFromPoint would return the cursor itself. The update loop hides it, samples the element under the point, and restores it in the same frame. Activation calls the element's own click() method, meaning the gaze layer works with ordinary click handlers and needs no special component API.

State machine and failure modes. The app moves through welcome, loading, calibrate, tracking and error states stamped onto a body data attribute that CSS keys off. Distinct errors are surfaced separately: no camera API in the browser, MediaPipe failing to load, NotAllowedError when permission is denied, and calibration that was too noisy to fit. If no face has been seen for 700 ms the status indicator flips and a warning appears, because silently freezing the cursor is the worst thing an assistive tool can do.

Calibration invalidation. A window resize changes the coordinate space the affine map was fitted in, so the resize handler discards the fitted function and prompts a recalibration rather than continuing with a stale mapping. Recalibration is also reachable by dwelling on a control or pressing R, without reloading the page.