home / 02 — projects / motif
Motif
Reverse-engineers what makes short-form video work in an industry, and turns it into an evidence-backed production brief.
2nd place · Track 2 winner, Forward: AI in Business
- status
- shipped
- when
- sep 2026
- took
- 48 hours (hackathon)
- team
- team of 4
about
Marketing teams can see which short videos perform, but not why. Motif breaks top-performing videos in an industry down into their structure: when the hook lands, the pacing, the order in which information is revealed and where the call to action (CTA) sits. It then turns those patterns into a scene-by-scene production brief covering shots, lighting, on-screen text and the exact CTA line. Each brief shows its evidence: how many real videos backed it, how closely they matched, and links to the reference videos.
I built it with a team of four in 48 hours. I owned the data pipeline and the backend: ingestion, niche classification, retrieval-augmented generation (RAG) and the API. The app is live, uses real login, and runs on a library of 44 analysed videos with 46.1M combined views.
stack
- frontend
- React, Vite, Clerk auth (deployed on Vercel)
- backend
- Python, FastAPI, Pydantic, psycopg with connection pooling, per-user rate limiting (deployed on Railway)
- data
- Supabase Postgres with pgvector
- ai
- GPT-4o for frame-plus-transcript structural extraction and brief generation; GPT-4o-mini for niche disambiguation; text-embedding-3-small for embeddings
- ingestion
- YouTube Data API, youtube-transcript-api, yt-dlp, ffmpeg
what I learned
- Measure in code, interpret with the LLM. Hook timing, words per minute and CTA position come straight from transcript timestamps. GPT-4o is only used for what needs judgment, and only on videos that pass classification. This made results more reliable and cheaper.
- Use the right-sized model for each job. GPT-4o handles extraction and generation, where quality matters most. GPT-4o-mini handles classification, which is the highest-volume and lowest-stakes step.
- "Not sure" should be a real answer. The niche classifier compares each video against the average embedding of human-checked examples for each niche. A high score is accepted, a borderline score goes to an LLM check, and a low score is rejected. Nothing gets forced into the closest category.
- Make the evidence traceable. When there isn't enough evidence, retrieval widens in a fixed order: niche + content type, then content type only, then niche + platform, then niche only, then fully open. The brief reports which step its evidence came from, which is what lets it "show its work".
- The data model mattered more than the prompts. The biggest quality gain came from adding content type (demo, testimonial, culture piece) as a second axis alongside niche. Two culture videos from different industries have more structure in common than a demo and a culture video from the same industry.
- Setting the team up to work in parallel. I defined the API schemas and test fixtures on day one so the frontend and backend could be built at the same time. I also handled the PR merges and deployment.
what was hard
- Silent correctness bugs. A CTA position of 0.0 (the CTA is at the very start) was treated the same as "unknown", which quietly skewed the industry benchmarks. Nothing crashed, so I only found it by running the full pipeline on real data.
- Bad labels led to bad insights. At first, videos were sorted by who posted them: a tech company's basketball interview landed in "tech" and produced meaningless benchmarks. Moving to topic-based classification fixed it, and is the reason the classifier exists.
- Small niches. A niche with one or two examples has a very narrow average embedding, so a correct match scored only 0.43 and was rejected. Lowering the thresholds for niches with few examples let them grow.
- Labels drifting apart. The LLM kept inventing near-duplicate labels such as "product demo" and "product_demonstration". I fixed this with lookup tables and by matching new labels to existing ones by embedding similarity instead of exact text.
- Cost against coverage in 48 hours. Each video takes a download, several GPT-4o calls and embeddings. I chose a hand-picked URL list fed through a reliable classifier over automatic discovery, and designed it so a scheduled discovery job can plug into the same classifier later.