Building Recommendations from the "Centroid Vector" of Watch History — A Three-Tier Fallback That Never Leaves Even Zero-History Members Empty
Introduction
I added member-facing “recommendations” to a self-built, all-in-one platform for video and music. Not a popularity ranking, but a personalized recommendation chosen from that person’s own watch/rental history.
And it reuses, as-is, the index I built for semantic search. The story of implementing semantic search without adding a dedicated vector DB is in Searching by “meaning,” not “surface text”. This article is about repurposing that same index for “recommendations,” and about the three-tier structure that keeps the screen from going empty even for members with no history.
Here’s what it actually looks like on screen. In the member’s recommendation list, the green “For You” badge on each title shows that it was chosen from that person’s own history (not a popularity fallback).
What grounds a “recommendation”?
Plenty of shelves carry the label “recommended for you.” But build one naively and the contents turn out to be just a popularity ranking. Popularity is easy — you can show the same thing to everyone. But it never looks at “you” at all.
What I wanted was to infer a person’s taste from what they’d actually watched or rented, and recommend titles they haven’t seen yet. The problem is “how do you represent taste from history” — stumble here, and you end up falling back to a naive rule like category matching.
[Popularity] Same shelf for everyone … easy, but never looks at "you"
[Personalized] Chosen from the person's own history … looks at "you". but how do you quantify taste?
The “average vector” of watched titles becomes that person’s taste
This is where the index built for semantic search pays off. Every title is already a semantic vector (the title’s description text converted into a sequence of a few hundred numbers; titles close in meaning land close together), loaded into memory since startup.
The way to represent taste is simple. Sum up the vectors of every title the person has watched, and average them. Call this average vector the “centroid.” The centroid is a single point at the “middle” of the titles that person has watched — it represents the center of their taste. From there, just return unwatched titles close to this centroid. You simply ask the index for “closest to the centroid.”
Average the semantic vectors of watched titles = the "centroid" of taste
Title A ●
Title B ● ─▶ average (centroid) ─▶ ★ the center of your taste
Title C ●
│
└─▶ return "unwatched" titles close to the centroid = recommendations
In code, the part that builds the centroid is just an average. Titles missing from the index (vector not yet generated) are silently skipped, and if none are left, it returns empty — and when empty comes back, the caller decides “can’t produce this from embedding” and moves to the next step.
// Average of the embeddings of watched titles (centroid) = that person's taste vector
List<Double> profileVector(List<Integer> filmIds) {
double[] sum = null;
int count = 0;
for (int filmId : filmIds) {
List<Double> v = getVector(filmId);
if (v == null) continue; // skip titles with no vector yet
if (sum == null) sum = new double[v.size()];
for (int i = 0; i < sum.length; i++) sum[i] += v.get(i);
count++;
}
if (count == 0) return List.of(); // none at all → empty (= signal to fall back to the next tier)
List<Double> centroid = new ArrayList<>();
for (double s : sum) centroid.add(s / count);
return centroid;
}
Once you have the centroid, all that’s left is to pass it into the exact same topSimilar (returns results ordered by closeness to a query vector) used for semantic search. Feed in “your centroid” instead of a query string — that’s personalization.
Some members will always be unable to produce a centroid
Here’s the catch. A centroid can only be built for someone who has history.
- A brand-new member has zero history (cold start — no history, no clue yet about that person).
- Even with history, the vector for a given title may not have been generated yet.
For members like these, embedding-based personalization returns nothing. So do you show an empty “recommendations” shelf? That would be the worst outcome. If the very first shelf a new member opens is empty, they’re not coming back.
So I built it as: “if you can’t produce it at full quality, degrade and produce something another way, always” — a three-tier fallback. Fallback here means: when the tier above can’t be used, degrade and switch to the tier below. Try from the top, and stop as soon as one succeeds.
GET /api/me/recommendations
│
▼
① embedding: unwatched titles close to the centroid of history
│ succeeded ─────────────▶ return (method="embedding")
│
└─ zero history / embedding not yet generated → degrade
│
▼
② taste_tag: chosen by taste-tag rules
│ (ranked by same category + number of shared tags)
│ succeeded ─────────────▶ return (method="taste_tag")
│
└─ no history at all → degrade
│
▼
③ popularity: popularity ranking (the last resort, for everyone)
always ────────────▶ return (method="popularity")
The third tier’s popularity ranking doesn’t look at “you,” but it’s far better than an empty screen. The ideal of quality (①) and the responsibility to always return something (③) are connected by a staircase of graceful degradation.
I also added one more thing — small, but effective. The response carries an internal indicator called method (“embedding” / “taste_tag” / “popularity”). This lets me measure afterward which tier the recommendation was actually served from. How often ① fires tells me what fraction of members are actually getting personalization that works. It’s scaffolding for watching quality with numbers instead of building it and walking away.
And when returning from tier ①, after taking results in order of closeness to the centroid, I always exclude titles already watched. Recommending a movie someone’s already seen again is pointless. Since exclusion shrinks the count, I fetch more than the target number up front and then trim — the related topic of “dropping items at the exit that are only close in meaning but shouldn’t be there” is covered in Vector search freely mixes in things that are just “close in meaning”. Filtering out music CDs that sneak into a video recommendation shelf is the same reasoning.
One index, wearing three faces
The best part of building this was that the index I set up for search turned, as-is, into “recommendations” too.
- Feed in a query string → semantic search
- Feed in a title → “titles like this one”
- Feed in the centroid of history → “recommended for you”
Only the entry point differs; underneath, the same single topSimilar runs every time. The index that lightly implemented semantic search showed a third face with no additional DB and no additional model. “Personalization” tends to conjure images of heavy-duty infrastructure, but it stands up just by adding one move — averaging history — on top of a mechanism you already have for measuring “closeness in meaning.”
And the other thing: don’t finish the job with just one high-quality tier. Lay down ② and ③ for members ① can’t serve. Holding both the ideal recommendation and the promise that no one’s screen goes empty, at the same time — that’s the shape I chose for this feature.
Related articles
- The semantic vector index this “recommendation” repurposes is covered in Searching by “meaning,” not “surface text” — implementing semantic search without adding a dedicated vector DB.
- The story of dropping, at the exit, a different category that’s merely close to the centroid (like a music CD mixed into video results) is covered in Vector search freely mixes in things that are just “close in meaning” — the trap of different-kind catalogs sneaking into recommendations.
- The overall vision for this platform is in From DVD Rental to a DMM-Style Platform.