A no-answer beats a confident wrong one

A woman in one of our Netflix research studies said something I still think about. If Netflix says a show is highly recommended and she hates it, fine, they got one wrong. If Netflix says it's a 92% match and she hates it, now she wonders what's wrong with her.

That's a confident wrong answer doing what it does. The product made the mistake, and the person on the couch paid for it. I'd pushed for a holdback test on that score, and when a big group of members stopped seeing it, not one metric moved. It had sat on our titles for years looking scientific and doing nothing, except, for her, making things worse.

I'm a product designer, and most of my work has been deciding what a product is allowed to say to you, and when. At Netflix that ended with the first words an LLM ever wrote to members. On my own time it's bre.ai, an assistant I built on Gemini. Either way, the thing I design with is the line itself: what the model may say, what it has to prove first, and what it does when it can't.

Your Netflix home screen is about forty rows of titles, every one picked for you, and Netflix almost never said why. Fewer than half of members felt their home page was personalized to them. After the match score was gone, I led the design of Personalized GenAI Explanations: an LLM reads your taste and a title and writes one sentence about the real thread between them. Watch if you liked the small-town mystery and eerie supernatural twists in Dark. It launched in May 2025, the first time Netflix put AI-written words directly in front of members.

The model could phrase anything, and that was the problem. Sometimes it made things up. Sometimes it used a sensitive topic like addiction to chain two titles together. Every line needed a check, and no team of people can read a catalog the size of Netflix's.

So before anything shipped, my content designer and I wrote down what good means, in words a reviewer or a model could apply the same way twice. We landed on four principles: clear, relevant, trustworthy and engaging. Under them sat hard rules. We could say what you watch, never who you are, so nobody got called a fan. No sensitive topic as the link between two titles. No pairings across a maturity line or inside the same franchise, and nothing that just repeats the synopsis.

This is the part of the job I love most. Taste is easy to have and hard to hand off. "Would a friend who knows you say this?" is a real test, but only I can run it. "Is every shared trait true of both titles?" is a test anyone can run, a model included. The work is turning the first kind into the second without losing what made it worth asking.

Netflix taught me how to hold a bar at scale. bre.ai taught me what it's like to be on the wrong side of my own bar.

bre.ai is a personal assistant I designed and built, with AI coding agents writing most of the code. It runs on Gemini 2.5 Flash, with a scripted fallback for when the model is down. Woobo's rule is baked into its loop: check that every claim traces back to a record, then answer with sources or say it doesn't know. I ran its voice like any design project: forty real prompts, typed the way I actually type, each answered three ways, and four rounds of picks that became the voice guide the model reads as its instructions.

Then I wrote a rubric, and a script that grades every live answer from 1 to 5 on voice, specificity, leading with the finding, a real receipt, and abstaining gracefully when it should. Every axis has to average 4 or better, with nothing under 3. I chose a script over a model judge on purpose, so the grade comes out the same every run and my answers never get sent anywhere a second time.

The answers passed. They also felt awful. My voice rules said lead with the number, and the rubric gave points for a number from my records, so the chat learned to hand me statistics. A feeling like "I'm tired" could come back as a note that something wasn't logged yet, plus a stat. Every check was green. I was annoyed, mostly at myself, because I'd written the rules it was following. What I typed into the chat was: "Numbers are not that helpful for me."

So I rewrote the rules. A number earns its place only when it's the clearest way to say the thing. A feeling routes to care, never to a task. One line is a complete answer when one line is true. I went after the opposite failure too. Once, when I asked it to log three minutes of cleaning, it told me no habit matched, which made me so mad. An honest stop is for a real gap. Refusing when the data is right there is its own way of being wrong.

How a line is judged, combining the content principles and hard rules from Netflix's Personalized GenAI Explanations with the five axes from bre.ai's live quality rubric. The Netflix rows are what our team's judges and reviewers held every explanation to. The bre.ai rows are scored 1 to 5 by a script on every live answer.

The bar on bre.ai: every axis averages 4.0 or better over at least 200 live answers, and no answer scores under 3 on any axis. Anything under 3 is fixed in the prompt, routing or composition, never by changing the rubric to move the number.

Netflix: what happens to every AI-written explanation

1. Generate. The recommendation system picks the title. An LLM reads the member's taste (what they finished, what they rated, the themes they keep coming back to) and the title (cast, tone, the shape of the story), and writes one sentence about the real thread between them.
2. Judge. LLM judges each ask one question. Is it true? Is it sensitive in a way we should avoid? Do these two titles belong together? The pairing question has its own dedicated model.
3. Pass every check, and the explanation is shown.
4. Fail any check, and the explanation is dropped. No weaker version of it is shown just because one exists. The title keeps its art and its usual synopsis, and the space for a reason stays empty on purpose. An empty slot costs a little. A wrong reason costs the member's trust in every reason after it.
5. Keep it rare enough to mean something. If two of the eight titles on your home page carry a real reason, it feels personal. If all eight do, it reads as noise.

bre.ai: how the assistant abstains

1. Plan, retrieve, act through tools. The model only acts through tools that exist. If there's no tool for something, it can't do it.
2. Check every claim against a record. A sentence with a number no record carries is cut, and only that sentence, so one bad number doesn't wipe out a good answer. A sentence that claims something about the person's feelings, relationships or actions that no record states is cut too.
3. Answer with sources, or say what's missing. A good stop names the gap in the person's own words and offers one thing they can ask or do instead. A generic stop is the last resort and scores lower.
4. Never stop when the data is there. If the person says they did something, it gets logged, even when it matches nothing they track. Refusals that replace answers are treated as bugs.
5. A feeling gets care first. "I'm tired" or "I don't want to go" routes to a warm reply, never to a stat or a task correction.
6. Say when the model is off. If Gemini is unreachable, a scripted fallback answers and an "AI off" indicator says so with the reason. The chat never quietly gets dumber.