An apartment that runs itself.
A year-long buildout of an always-on command center for one apartment. Lights, music, voice, vision, and game-day celebrations — all reacting to what's actually happening in the room, all running on a single Latitude 7420 in the corner.
60 seconds of the apartment behaving.
Mode-switching, light response, the journal page, and the analytics SectorBoard — recorded straight off the kiosk.
What's actually in there.
Snapshots from the current build — every number is grounded in a shipped feature, not a wishlist.
One backend, many surfaces.
A FastAPI core fans state out over WebSocket and serves a SvelteKit static build. Services for Hue, Sonos, ML, and the automation engine plug in as long-running async tasks. The PC agent and camera report inward; the Alexa Lambda reaches in through a Cloudflare Tunnel.
Four problems worth talking about.
ConfidenceFusion: blending four signals into one mode.
The apartment has four ways to guess what's going on — what processes are running on my PC, what the camera sees (zone + posture + lux), what YAMNet hears in the room, and a small bank of learned rules. Each emits a confidence-scored opinion every second.
ConfidenceFusion blends them with weights that bend toward whichever signal has been freshest, then applies a suppression layer for known footguns: late-night dev tools get weighted down, stale process reports yield to fresher camera evidence, and a stamp system keeps user-initiated overrides from getting steamrolled by autonomous pushes.
The autonomy gate: five ways the apartment changes its own mind.
Five autonomous mode-setters can override what the user is doing — each one cost a small amount of trust to ship, and each one has a kill switch.
Late-night rescue catches dev-tools-still-foreground-at-midnight and flips to relax. The zone+posture rule reacts to "bed + reclined" sustained for 180 seconds. A watching-sleep-guard fires when you've been reclined in bed under the projector for 90 minutes (caught me asleep with YouTube on more than once). Two camera-at-desk vetoes block the autonomous pushes while I'm actually present.
Game Day: from ESPN poll to bedroom lamp in <1s.
On Colts game days the apartment auto-flips into a celebration palette 30 minutes before kickoff. An ESPN poll watches play-by-play; scoring plays, big plays, and high-WPA momentum swings fire a custom celebration — light sequences, TTS commentary, and a shifted color base on the bedroom lamp.
The celebration orchestrator runs custom sequences per play type with an 8-second cooldown to prevent stacking. Spent enough time tuning latency that the lamp visibly reacts before the play-by-play commentary catches up.
Living on a Latitude 7420: deploy.sh, systemd, and a kiosk.
The whole thing runs on one Dell Latitude in the corner. Ubuntu 24.04, the backend as a systemd user service, Firefox in kiosk mode via GNOME autostart, Pi-hole v6 in Docker, a cloudflared tunnel exposing one subdomain to the public internet for Alexa.
The deploy script pulls fast-forward, conditionally reinstalls deps + rebuilds the frontend, restarts the service, and health-checks. The kiosk auto-reloads when the WebSocket reports a new build_id. The whole pipeline runs from my dev box on the LAN in about 20 seconds.
What it's built with.
Source on GitHub
Full backend + frontend + docs. Spec docs live in docs/; the working guide is in .claude/CLAUDE.md.