An apartment that runs itself.
Year-long buildout of an always-on command center for one apartment. Lights, music, voice, vision, and game-day celebrations. Everything reacts to what's happening in the room. The whole thing runs on one Latitude 7420 in the corner.
60 seconds of the apartment behaving.
Mode-switching, light response, the journal page, and the analytics SectorBoard. Straight off the kiosk.
What's actually in there.
Snapshots from the current build. Every number is grounded in a shipped feature.
One backend, many surfaces.
A FastAPI core fans state out over WebSocket and serves a SvelteKit static build. Services for Hue, Sonos, ML, and the automation engine plug in as long-running async tasks. The PC agent and camera report inward. The Alexa Lambda reaches in through a Cloudflare Tunnel.
Four problems worth talking about.
ConfidenceFusion: blending four signals into one mode.
The apartment has four ways to guess what's going on. Processes running on my PC, what the camera sees (zone, posture, lux), what YAMNet hears in the room, and a small bank of learned rules. Each emits a confidence-scored opinion every second.
ConfidenceFusion blends them with weights that bend toward whichever signal has been freshest. A suppression layer handles known footguns. Late-night dev tools get weighted down. Stale process reports yield to fresher camera evidence. A stamp system keeps user-initiated overrides from getting steamrolled by autonomous pushes.
The autonomy gate: five ways the apartment changes its own mind.
Five autonomous mode-setters can override what the user is doing. Each one cost a small amount of trust to ship. Each one has a kill switch.
Late-night rescue catches dev tools still in the foreground past midnight and flips to relax. The zone+posture rule reacts to "bed + reclined" sustained for 180 seconds. A watching-sleep-guard fires when you've been reclined in bed under the projector for 90 minutes (caught me asleep with YouTube on more than once). Two camera-at-desk vetoes block the autonomous pushes while I'm actually present.
Game Day: from ESPN poll to bedroom lamp in <1s.
On Colts game days the apartment auto-flips into a celebration palette 30 minutes before kickoff. An ESPN poll watches play-by-play. Scoring plays, big plays, and high-WPA momentum swings fire a custom celebration. Light sequences, TTS commentary, and a shifted color base on the bedroom lamp.
The celebration orchestrator runs custom sequences per play type with an 8-second cooldown to prevent stacking. I spent enough time tuning latency that the lamp visibly reacts before the play-by-play commentary catches up.
Living on a Latitude 7420: deploy.sh, systemd, and a kiosk.
The whole thing runs on one Dell Latitude in the corner. Ubuntu 24.04. The backend runs as a systemd user service. Firefox runs in kiosk mode via GNOME autostart. Pi-hole v6 lives in Docker. A cloudflared tunnel exposes one subdomain to the public internet for Alexa.
The deploy script pulls fast-forward, conditionally reinstalls deps and rebuilds the frontend, restarts the service, and health-checks. The kiosk auto-reloads when the WebSocket reports a new build_id. The whole pipeline runs from my dev box on the LAN in about 20 seconds.
What it's built with.
Source on GitHub
Full backend + frontend + docs. Spec docs live in docs/; the working guide is in .claude/CLAUDE.md.