Projects

Started from an existing model that could generate music but couldn't be steered or trusted to have actually improved. The real bug turned out to be greedy decoding, not the model — always taking the highest-probability note walks one path and loops — so it was replaced with temperature sampling calibrated against the model's own measured confidence. Key and mood control were added as an additive bias at the sampling step, since no labeled data existed to train a conditioning input. Extended beyond piano by integrating MusicGen and Magenta's MelodyRNN, then closed the loop with a blind A/B arena and Elo scoring, since validation loss measures corpus fit, not whether a person enjoys listening.

A trained checkpoint saved under Keras 3 couldn't be loaded into the project's pinned Keras 2.10 — rather than retrain, wrote a custom weight loader that reads the arrays out of the archive and assigns them layer-by-layer, shape-checked so a mismatch fails loudly instead of generating silent noise. Every generated note was coming out as a quarter note: the MIDI writer's Stream.append() recomputes offsets from duration, but duration was never being set, so the model's predicted rhythms were discarded after inference. The blind listening test wasn't actually blind — model identity leaked through three separate channels (the API payload, the streamed job log, and a metadata sidecar reachable by guessing a URL) — all three had to be closed for the Elo comparison to mean anything.

Temperature-calibrated sampling cut the repetition rate from 0.151 (greedy) to 0.024 and corpus KL-divergence from 0.498 to 0.147, averaged over four seeds. Key conditioning raised the in-key note fraction from 0.786 to 0.930 while tightening the spread across seeds from ±0.145 to ±0.039. Extended the system with two external generative models — MusicGen for text-to-audio and Magenta for in-browser melody continuation — without touching the core evaluation harness. Shipped as a genuine full-stack system: FastAPI backend running background training/generation jobs, a Next.js/React frontend polling job state via TanStack Query, and in-browser MIDI synthesis via Tone.js that removed the need for any native audio library.

Started from the hardest constraint first: form feedback has to happen live, while someone is mid-rep, which ruled out sending video to a server for processing at all. Built the entire pose-to-feedback loop to run client-side in WASM via MediaPipe, so latency is bounded by the device, not the network. Rather than writing one scoring function per exercise ad hoc, architected each of the 21 exercises as an isolated module behind a shared interface — adding a new exercise means writing new joint-angle geometry rules, not touching the pipeline that feeds it. Applied the same discipline to the AI-generated workout feature: treated the LLM as an untrusted input source rather than a trusted writer, parsing every response against a nested Zod schema covering program structure, per-day layout, and per-exercise set/rep/rest bounds before anything reaches the database.

Raw pose landmarks are noisy frame-to-frame — a single misread joint angle could flip a rep-phase detector and miscount a rep. Solved with hysteresis-gated state transitions smoothed by an exponential moving average, so one bad frame can't flip the state machine on its own. The same exercise performed from different angles or positions (front vs. side, standing vs. prone vs. seated) would break a single fixed rule set — solved with a stability gate that auto-detects camera orientation and body position across multiple frames before a session is allowed to start scoring. LLM output is inherently unpredictable in shape and content — solved by validating every response against a strict Zod schema and failing closed (rejecting the output) rather than risking malformed data reaching Postgres.

Shipped 21 hand-built exercise-scoring engines, each combining joint-angle geometry, bilateral symmetry checks, and tempo analysis into a live 0–100 form score. Real-time pose estimation runs at up to 30fps fully client-side with zero video transmitted to a server. Enforced authorization at two layers — middleware-level route protection checked server-side before any protected page renders, and per-query scoping so every data-access function is bound to the authenticated user. Shipped as an installable PWA with a versioned service worker that precaches the app shell for offline use. CI-gated by GitHub Actions running lint and build checks on every push, independent of Vercel's deploy pipeline — a broken build surfaces as a failed check before it ever reaches production.

Architected the backend with strict separation of concerns from the start — thin routers delegate to a services layer for business logic, Pydantic schemas handle validation, and SQLAlchemy models own persistence, so any single concern can change without rippling through the others. Rather than trusting Supabase Auth's session model directly, layered a custom HS256 JWT scheme on top of it — decoupling the API's own session lifecycle (access/refresh rotation, role claims) from the auth provider underneath. Built job search on native PostgreSQL full-text search instead of introducing a dedicated search engine, hand-writing parameterized tsvector/ts_rank queries with SQL-aggregated facets. For AI coaching, built a provider-abstraction layer that routes between a frontier model and a cheaper one depending on task value — a deliberate cost/quality tradeoff rather than defaulting to one model everywhere.

Needed the API's session model to be independent of its auth provider — solved by minting and verifying its own JWTs (with a role claim for RBAC) rather than relying on Supabase's session tokens directly, so switching or extending the auth backend later wouldn't touch route-level authorization logic. Registration needed to preserve email verification, but the more powerful Supabase service key auto-confirms accounts and silently skips that flow — solved by deliberately using the anon key for registration instead, trading convenience for a correct verification path. Faceted, relevance-ranked job search needed to work without adding infrastructure — solved by pushing ranking and facet aggregation into PostgreSQL itself (ts_rank ordering, GROUP BY-based facet counts) rather than standing up Elasticsearch or a similar service.

Shipped a 14-model relational schema (User, PixelSection, Payment, Subscription, JobPost, CoachingSession, and others) with UUID primary keys and Alembic-managed migrations, behind 11 modular API routers with RESTful semantics and auto-generated, environment-gated OpenAPI docs. Built a 650-line strongly-typed API client on the frontend as the single boundary to the backend, with auto-restoring session state mirroring the backend's own token-rotation model. Integrated Stripe for both one-time billboard purchases and recurring subscriptions with webhook handling, and hardened the API with per-endpoint rate limits (5/min on auth-sensitive routes, 120/min on search), strict CORS/CSP, and HTML sanitization on all user input. Delivered two distinct authenticated experiences — a seeker dashboard and a recruiter portal — from one shared, role-gated codebase.