Netflix

Summary

Netflix recommendation is a system that optimizes page (row × column) composition over a curated catalog (thousands to tens of thousands of titles), not "per-source candidate allocation." Since 2015 it has generated pages with ML and no templates verified, and in 2025 it centralized member representations in a Foundation Model (FM) verified. In 2026 a generational shift is under way toward the LLM ranker GenRec and GenPage, which generates the whole page with a single Transformer verified.

↑ Contents

Generation breakdown — read this first to avoid confusing the sources

Netflix recommendation sources span four generations: Classic, Unified, FM, and LLM-native verified (table below). In the LLM-native generation, the LLM ranker GenRec won a 10% traffic A/B on batch-compute surfaces (surfaces whose recommendations are precomputed in bulk rather than computed when a request arrives) verified (GenRec section). GenPage (RecSys 2026), which generates the homepage, has also appeared verified (Re-ranking, diversity, and filters section). There is still no full rollout statement verified (table below). The source this document calls TMIS 2015 is the paper The Netflix Recommender System (2015) by Gomez-Uribe and Hunt.

Most market analyses take the Classic generation, that is, the 2015 paper, as their baseline. This is the same pattern as X's fixation on the 2023 snapshot.

↑ Contents

Feed surfaces

The Netflix homepage is a two-dimensional grid of rows and columns. It has about 40 rows, each holding up to 75 titles, and the counts differ by device verified TMIS 2015.

• Row types are genre rows (PVR), Top Picks (Top-N ranker), Trending Now, Continue Watching, and Because You Watched (BYW, sims) verified TMIS 2015. Each row having its own ranker is a hallmark of the Classic generation.
• Recommendations influence about 80% of viewing time, and search accounts for the remaining 20% verified TMIS 2015. Keep in mind that this is a 2015 figure.
• The savings from personalization and recommendation are "more than $1B/year" verified TMIS 2015. This is Netflix's own estimate.
• In 2025-05 the TV home was redesigned to include recommendations that respond to a moment's mood and interest. On mobile, a vertical clip feed is being tested verified About Netflix 2025.

↑ Contents

Candidate generation (retrieval)

As of TMIS 2015, there is effectively no retrieval layer. PVR "sorts the whole catalog (or a genre-filtered subset) per member" verified TMIS 2015. Because the corpus is on the scale of thousands to tens of thousands, the whole catalog can be scored, and there is no reason for per-source caps like X's 500/400 to appear. The allocation problem shifts from item selection to row selection (→ candidate-allocation.md).

• Row candidates reach tens of thousands per member ("tens of thousands of rows"), and the row selection algorithm picks and orders them by relevance and diversity verified TMIS 2015.
• Since 2015, templates have been removed, allowing "a home with no BYW rows at all, or a home that is half BYW" verified TMIS 2015.
• In the FM era, member embeddings and title embeddings are produced in daily batches into an embedding store. These embeddings are features for downstream models rather than for search, and they are also used as a candidate source verified FM unification blog 2025. The blog does not disclose whether it uses ANN for candidate generation observed 2026-10-19 (1 blog post body).

↑ Contents

Ranking

In production, the Foundation Model (FM) builds member representations, and downstream models use them in three ways verified (Foundation Model section). The LLM ranker GenRec improved both short-term and long-term metrics in a large-scale A/B on batch-compute surfaces, but neither the blog post nor the paper discloses a full rollout verified (GenRec section, paper section). As of the 2024 long-term satisfaction blog post, the objective is not a weighted sum of action probabilities but a proxy reward with retention as the north star verified (Objective function section).

Foundation Model (2025, production)

FM is a model that turns a member's interaction history into a token sequence and trains on it with GPT-style autoregressive next-token prediction verified FM blog 2025-03. Downstream models use FM in 3 ways, all in production verified The result of combining the two posts.

• Connects directly to multiple prediction heads (FM blog 2025-03).
• Used through the embedding store (unification blog 2025-11).
• Used as a subgraph and through fine-tuning (unification blog).

Here is what is known about the model design and scale.

• When converting interaction history into tokens, adjacent actions are merged. This approach can be compared to BPE verified FM blog.
• Each token carries heterogeneous features, with request-time features separated from post-action features verified FM blog.
  • Request-time features are time, device, and location.
  • Post-action features are title and watch time.
• The objective is weighted, and multi-token prediction is added to prevent myopic optimization verified FM blog.
  • The blog's wording is "a 5-minute trailer play and a 2-hour completion cannot carry the same weight."
• There is a statement that scaling laws were confirmed verified FM blog. The blog describes it as "consistent improvement as data and model size increase."
• The parameter count of the production FM is undisclosed. The 2M→1B scale study (2026) is at the production-shadow stage verified arXiv 2605.23312.
  • Production-shadow is a stage in which the model runs on real traffic but its results are not shown to members.

GenRec (2026, large-scale A/B)

GenRec is a ranker built by post-training an internal LLM on Netflix data verified GenRec blog 2026-07. It unrolls the input into sentences instead of feature vectors and scores the whole candidate set in a single forward pass verified GenRec blog. In a large-scale A/B run on batch-compute surfaces, both short-term and long-term metrics improved significantly verified GenRec blog.

• GenRec has 2 phases verified GenRec blog.
  • The first phase (Phase 1) is a Netflix-adapted LLM and is updated infrequently ("updated relatively infrequently").
  • The second phase (Phase 2) is ranking post-training and is updated more often ("refreshed more frequently").
• It uses verbalization instead of feature vectors verified GenRec blog.
  • The blog puts it as "The context window becomes our new 'feature budget'".
  • The user message consists of the following elements: "verbalized context, profile, history, item metadata, and task".
• Scoring is not text generation. It is a structure that attaches a learned item-embedding scorer to the LLM backbone, not a generative model verified GenRec blog.
  • A pooled hidden state h is extracted from the backbone: "we extract a pooled hidden state h".
  • The learned embedding eᵢ of item i and the scoring head ϕ combine to produce sᵢ, and softmax ranks the items. An example of the combination: "(e.g., via dot product or small MLP)".
  • The three parts are trained together: "All parameters — the backbone, scoring head, and item embeddings — are trained jointly."
• It uses only prefill (the step that reads the prompt and builds only the internal state), and it is 1 pass for the whole candidate set, not 1 pass per candidate verified GenRec blog.
  • The blog explains: "the model consumes the prompt once and scores the entire candidate set in a single forward pass, with no token-by-token decoding".
  • It states 3 serving cost drivers (model size, context length, prefill vs autoregressive) and uses vLLM.
• There are 4 context engineering rules verified GenRec blog.
  • High-signal actions such as long plays and thumbs-up are all kept (retain in full).
  • Low-signal actions such as very short plays and quick hovers are dropped (omit).
  • Repeated actions such as binge-watching are summarized or compressed (summarize or compress).
  • New releases and cold-start items are selectively described in detail (elaborate selectively).
  • The blog describes the compression result this way: "roughly one-third of the original budget with negligible degradation in offline ranking metrics". ⚠ The negligible degradation holds for offline metrics only.
• It aligns to long-term satisfaction with a reward-weighted loss. It is not full RL; GRPO, a reinforcement learning method, was left as future work because of "higher cost" verified GenRec blog.
  • One source of the weights is a long-term satisfaction proxy: "return behavior, catalog exploration, or sustained engagement".
  • Another source is rebalancing by content type and release stage: "games vs. movies, new releases vs. evergreen titles".
• It used "roughly 40×" fewer Phase-2 labels, and offline MRR (the mean reciprocal rank of the correct item) was "about +1.6%" verified GenRec blog.
  • Separately, the blog states that this matches or exceeds production, as follows: "10–40× fewer Phase-2 labeled examples, depending on configuration".
  • 40× should be used only in connection with the MRR statement.
• In the online A/B on batch-compute surfaces, both short-term and long-term metrics improved significantly verified GenRec blog.
  • The experiment scale is as follows: "we ran a large A/B test on batch-compute recommendation surfaces, covering ~10% of Netflix traffic over ~4 weeks".
  • ⚠ The condition batch-compute surfaces only is stated explicitly.
  • The post neither confirms nor denies applying it to real-time serving surfaces inferred. Reading it as "not yet in real time" is our inference, not the source.
• The ranking backbone size is: "~1B to ~10B parameters". This is a first-party statement, among the first in the industry, that an LLM entered the organic feed ranking path. verified GenRec blog (→ revised the claim in llm-in-recsys.md)
• There is no Limitations/Future Work section observed rechecked 2026-08-09. Sentences about the outlook are scattered throughout the body.
  • "substantial headroom as we further scale data and input signals"
  • "opens the door to natural-language steering for future experiences"
  • "text-generation use cases" such as explaining the reason for a recommendation

GenRec serving layer — disclosed in a separate blog post (2026-07)

A separate serving blog post half-reveals which infrastructure GenRec runs on verified In-House LLM Serving at Netflix, 2026-07-17. By summer 2025, Netflix had switched its inference engine from TensorRT-LLM to vLLM, and it scores every model from XGBoost to LLMs with the same gRPC call verified serving blog. We secured the original text as an archive.org snapshot.

• The evidence is the timing of the engine switch and the breadth of the workload verified serving blog.
  • It started with TensorRT-LLM and switched to vLLM at the point marked "By summer 2025".
  • The decisive reason was the widening of the workload: "embedding generation, prefill-only inference for ranking and retrieval, autoregressive decoding, and custom models with non-trivial per-step constraint logic".
  • GenRec appears explicitly as 1 of the 4 main workloads that became the criteria for choosing the serving stack.
• It uses a common serving path regardless of model type verified serving blog.
  • Model Scoring Service (MSS) is the shared inference backend: "the shared inference backend, supporting XGBoost, TensorFlow, PyTorch, and LLMs behind a unified interface".
  • The structure is a Java control plane on top of Triton.
  • The design principle is also stated explicitly: "A key design goal of our system was that LLM models should NOT be special snowflakes. Every model — XGBoost ensemble or large-scale LLMs — is scored via the same gRPC call".
  • The OpenAI-compatible API was attached only as an additional frontend.
• Cold start is solved with infrastructure verified serving blog.
  • Downloading a large LLM from S3 or HF at boot time exceeds the scheduler timeout.
  • So models are materialized in advance on a high-performance file system at model announcement time: "We materialize models on Amazon FSx at the time of model announcement".
  • Netflix's distinctive property that "with a curated catalog, item inflow is an event" also shows up unchanged in the serving layer.
• The scaling problem of constrained decoding set the upper bound on batch size verified serving blog.
  • Constrained decoding is an approach that restricts, by rules, which tokens can be chosen during decoding.
  • In vLLM V0, the custom logits processor ran sequentially per request: "the GIL prevents Python from parallelizing the per-request work. CPU time in logit processing therefore grows linearly with batch size".
  • V1 (Q4 2025) restructured this processing to the batch level and rewrote the hot path in multithreaded C++: "reimplemented the hot path in C++ with multi-threading to step around the GIL".
  • As a result, logits processing time did not grow as the batch grew: "logits processing time staying flat as batch size grows".
  • This is why GenRec's "whole candidate set in 1 pass" scoring holds at batch scale.
  • Two operational cautions follow. The state machine must be resynchronized on partial prefill and preemption.
• Deployment uses Red-Black and Versioned together verified serving blog.
  • Red-Black is used when the interface is stable, and Versioned when the schema changes: "Versioned solves that gap by maintaining an independent deployment for every (modelId, modelVersion) pair".
  • In GenRec generation changes where the schema changes, the structure pays in GPU cost for the period when the old and new models serve at the same time.
• Absolute values for GPU generation, QPS, batch size, and latency are also missing from this post observed original text rechecked 2026-08-10.
  • No Model FLOPs metric or throughput figures appear either.
  • To calculate an upper bound on GenRec's batch serving cost, this item must stay an open question.

GenRec arXiv paper — figures the blog post did not disclose (2026-08-10)

The GenRec paper is the academic version covering the same system as the blog post. Nothing in it explicitly contradicts the blog post, and quantitative detail, the loss formula, and the team structure are new verified GenRec: An LLM-Backed Recommendation Ranker at Netflix (arXiv 2608.10257), 2026-08-10. There are 6 authors (Ying Li, Shradha Sehgal, Arjun Rao, Rein Houthooft, Yaochen Zhu, Ashish Rastogi), and §8 lists all contributors by team.

• It discloses measured values of the online core metric for the first time verified §5.1.
  • The paper's sentence: "we observe a +0.006% relative improvement on our core online metric, which is statistically meaningful at Netflix scale".
  • The blog post said only "statistically significant" and gave no absolute value. The paper itself characterizes this value as "meaningful at Netflix scale."
• It breaks down the contributions of Phase 1 and Phase 2 separately verified §5.3.
  • Phase 1 is +10–20% in offline MRR over the OSS backbone, and Phase 2 is +35–50% over Phase 1 at its latest point in time.
  • Two weeks after Phase 1 is trained, Phase 2's relative gain widens to ~80%. The paper gives this reason: "increasing staleness of the less frequently updated Phase-1 backbone".
  • It newly emerged that the time axis determines the relative value of the two phases.
• An absolute example of context compression appeared, and the figure of 1/3 serving cost was added verified §5.4.
  • Context sentence: "the context length can be reduced to roughly a third of the original token budget (for example, from about 5,000 tokens down to about 1,700 tokens) with only negligible degradation"
  • Cost sentence: "serving cost is reduced to roughly a third of original cost as well"
  • An absolute token example was attached to the blog's "roughly one-third", and a quantitative claim was added that cost also falls to 1/3.
• It discloses the loss function formula verified §4.4-§4.5.
  • ℒ = α·ℒ_ranking + β·ℒ_language + γ·ℒ_miscellaneous, with α+β+γ=1. The paper does not disclose the α·β·γ values.
  • The ranking loss is softmax cross-entropy over the whole catalog (or candidate set), and it is replaced with sampled softmax when the catalog is too large.
  • At inference only the ranking head is used, and the LM head is kept for training regularization.
  • It also explicitly names text generation as a future extension: "future text generation use cases (e.g., natural-language explanations)".
• The training data is in a user/assistant conversation format verified §4.2, §4.4.
  • The user message contains context, profile, history, item metadata, and task. The blog listed only this side.
  • The assistant message contains the actual engagement (play, play duration, abandon, thumb, etc.).
  • The positive label is defined as follows: "high-quality engaged events (e.g., high-duration plays, strong explicit feedback), with denoising logic and thresholds that vary by content types".
• GRPO actually did better verified §4.6.
  • The blog post treated it as "future work", but the paper writes: "In preliminary experiments, RL-style methods (e.g. GRPO) showed additional gains over supervised finetuning, but their high training overhead makes them better suited for future work".
  • They tried it and confirmed improvement over SFT, but finally adopted the reward-weighted loss because of cost.
• In a sweep growing Phase-2 data from 1x to 20x, both the ~1B and ~10B backbones improved monotonically verified §5.2.
  • The scaling patterns of the two backbones were also similar. The absolute curves and slopes are not disclosed.
• Serving cost is roughly proportional to the product of model size and context length verified §4.7.
  • The paper's sentence: "inference cost in practice is roughly proportional to the product of model size and context length"
  • There are two cost optimization axes: using a smaller or distilled model, and context compression.
  • Distillation (a method that trains a smaller model to imitate the outputs of a larger one) was not mentioned in the blog post.
• The paper explicitly positions GenRec as not in the Semantic ID (SID) family verified §2.
  • Semantic ID is a token sequence that represents an item as a hierarchical code.
  • Sentence at the end of §2.2: "Most LLM-based generative retrieval systems rely on autoregressive decoding with beam search at inference time... Our work avoids these limitations by augmenting the base LLM architecture with a catalog-aware ranking head, so that it can rank a large candidate set in a single forward pass"
  • The paper itself lays out the differences from SID/generative families such as Google PLUM, Kuaishou OneRec-Think, Spotify GLIDE, and Yandex Gryphon (→ topics/llm-in-recsys.md, the generative retrieval thread in radar.md).
• The team structure was revealed verified §8. A collaboration among 3 teams that was absent from the blog post appears in §8.
  • AI for members (22 people): includes the lead authors and Justin Basilico.
  • AI platform and serving (13 people): owns the serving infrastructure.
  • Product: 4 people.
  • Justin Basilico is known as Netflix's head of personalization. This is the first time his name has been confirmed in a document adjacent to GenRec.
• The paper also does not affirm a full rollout verified §7.
  • Sentence in §7: "GenRec represents an initial step toward a more LLM-centric recommendation stack at Netflix. Our results suggest that this is a viable and promising path for large-scale personalization in at least some scenarios"
  • Expressions such as batch-compute surfaces only, "some scenarios", and "initial step" follow.
  • The canon's open questions (whether there is a full rollout, whether real-time surfaces are covered, the source of the candidate set) are not answered in the paper either, so they stay as they are.
• Items still not public in the paper include:
  • α/β/γ values
  • Absolute values for GPU generation, QPS, and latency
  • Quantitative speedup of prefill vs autoregressive
  • Details of Phase 1's training data and objective function
  • Identity of the base OSS model (Llama? Mistral?)
  • Specific names of the A/B surfaces
  • secondary metrics/guardrails
  • The party that builds the candidate set

Objective function — retention family, not a weighted sum

Netflix's north star for recommendation is retention, but what it actually optimizes is a proxy reward verified long-term satisfaction blog post 2024.

• It treats the problem as a contextual bandit (a framework that picks an action given the context, then updates the policy from the reward received) verified long-term satisfaction blog post.
• It cites "1 signal per account per month", noise, and the difficulty of attribution as reasons it cannot optimize retention directly verified long-term satisfaction blog post.
• The published proxy reward has this form: r(user, item) = f(play, complete, thumb) verified long-term satisfaction blog post.
• Delayed feedback is handled by the Delayed Feedback Prediction Model. This model predicts p(final feedback | observed feedback) and injects it into the reward verified long-term satisfaction blog post.
• The blog post states explicitly that "CTR over-optimization encourages clickbait." This is structurally different from the Σ w·P(action) weighted-sum framing of X and Meta verified long-term satisfaction blog post (→ multi-task-ranking.md).
• Weights and rewards are set through a reward engineering loop (hypothesis → reward revision → bandit policy → A/B) verified.

Classic-generation rankers (for reference, 2015)

The classic generation had a dedicated ranker for each row type verified TMIS 2015.

• PVR: sorts the whole catalog and blends popularity with personalization verified TMIS 2015.
• Top-N: optimizes only the head verified TMIS 2015.
• Trending: adds personalization to trends on a scale of minutes to days verified TMIS 2015.
• Continue Watching: distinguishes resuming from abandoning. It uses elapsed time, stopping point, partial viewing, and device verified TMIS 2015.
• sims: similarity is non-personalized, and row selection is personalized verified TMIS 2015.

UniCoRn (2024) in the unified generation unified search, pre-query, and More Like This into a single contextual ranker. With personalization, offline metrics rose 7% in search and 10% in recommendation verified RecSys 2024.

↑ Contents

Re-ranking, diversity, and filters

Netflix's diversity lives in the page construction stage, not in list re-ranking. It optimizes row selection "for relevance and diversity" verified TMIS 2015 (→ feed-diversity.md).

• Calibrated Recommendations (Steck, RecSys 2018) is a re-ranking approach that preserves the genre proportions of the watch history in the recommendation list, and its formula is published verified. However, there is no production deployment claim inferred. Stating that deployment status cannot be known is the accurate state (paper).
• We could not find a primary source for the rule that allows or caps the same title appearing in multiple rows. Duplication may work as a "different reason to recommend" rather than as a bug inferred.
  • Combined with a different theme per row and artwork personalization, duplicate impressions may be by design.
  • This has to be observed directly on a real Home, and we have not yet done so.
• Artwork and video preview personalization are a separate means of adjusting perceived diversity. Even for the same title, each member sees a different image and video preview.
  • The generation changed from the 2017 bandit to the 2026 MediaFM tri-modal foundation model (see the Artwork and video preview personalization section).
• GenPage (2026) replaces page construction itself with autoregressive generation by a single Transformer. End-to-end latency fell 20% and core engagement metrics improved. It is still at the experiment stage verified arXiv 2606.31031.

↑ Contents

Artwork and video preview personalization — MAPS (Multimodal Asset Personalization at Scale, 2026-08 production)

Netflix shows each member a different artwork and video preview even for the same title, and as of 2026-08 a production system called MAPS handles this verified (CLIP section, MediaFM section). The MAPS blog post groups three products under this name: CLIP artwork, query-aware artwork, and MediaFM video preview.

The generation moved from the 2017 bandit (Artwork Personalization, full rollout) to the 2026 CLIP two-tower and MediaFM. A two-tower is a structure that encodes the member and the item separately and then matches the two to produce a score. The figures are in the 2026-08-18 arXiv paper (Multimedia Asset Personalization via Multimodal Embeddings at Netflix), and the 2026-08-28 blog post disclosed the system name MAPS and the architecture context. There are two expansion directions: the unified CLIP model on the artwork side and MediaFM on the video preview side.

CLIP-based unified two-tower — one model for 5 canvas types

Previously, a separate model was trained and served for each of the 5 artwork canvas types (billboard / vertical box / horizontal panel / short panel / landscape panel) verified §3. This structure was unified into a single two-tower that concatenates the CLIP image embedding (768-d) to the item tower verified §3.

• Reward-weighted training: balances the per-canvas training examples with reward weights so that training does not skew toward a particular canvas.

• Cold start: new titles and assets are served from "day zero" with only the CLIP embedding, even without interaction history. This removes a limitation that ID-only models inherently had.

• Online A/B result: the core discovery metric rose by a relative +0.127% (p<0.005) verified §5.1.

  • Against the Eclipse UI holdback (a control group left without the new change), discovery rose +0.233% and streaming hours rose +0.184%, and the authors estimate an additional ≈350M streaming hours per year.

• Turning on V1 and V2 each alone did not move the online metrics verified MAPS blog post 2026-08-28.

  • V1 applied only the image embedding, and V2 applied only the canvas unification. Both were flat and non-significant, offline and online.
  • Only V3, which combined the two ideas, obtained a statistically significant lift and was deployed to production.
  • On short-panel, V3's offline IPS relative lift (+5.691%) was larger than the sum of the V1 and V2 lifts.
  • IPS is an offline metric often used in off-policy evaluation; it estimates a new model's counterfactual performance from logged random-exposure data.
  • This is a case where the observation "turning on a single content feature alone did not move anything" is stated explicitly.

  "Neither idea moved our online core member metrics on its own. V1 and V2 were both flat and non-significant, and only V3 won a statistically significant lift." — MAPS blog post, 2026-08-28

• In the holdback run during the 2025 TV home screen overhaul, the lift was larger than in the steady-state ablation verified MAPS blog post, 2026-08-28.

  • It was a cold-start situation in which the UI overhaul suddenly made short-panel the dominant canvas.
  • V3 was deployed just before the overhaul, and a holdback A/B ran for a month. A small control group kept the previous per-canvas models.
  • It obtained a statistically significant lift in both the core discovery metric and streaming hours.
  • This is a case where signal transfer across canvases in a single model was measured.

• It also extended to search. Reusing CLIP's joint text-image space, it applied query-aware artwork to the search canvas, and playthrough rate rose +0.36% (p<0.05) verified.

  • Ranking score = α · personalization + (1−α) · cos(text_emb(query), image_emb(asset)) as a blend, with α tuned through online A/B.
  • The blog post states that additional modeling effort is effectively 0 (because the CLIP embedding is already in the asset representation) verified MAPS blog post.

MediaFM — in-house tri-modal foundation model (for video previews)

MediaFM is a shot-level tri-modal FM that Netflix trained in-house, and it is served in production verified §4. A shot is one continuous stretch filmed without a camera cut.

• Modalities: visual is encoded with SeqCLIP (a CLIP-style frame encoder), audio with wav2vec 2.0, and timed-text with text-embedding-3-large.
  • For each shot, the three are concatenated into a 2,304-d unit-normed vector.
• Encoder: a BERT-style 3-layer / 8-head transformer.
  • It does self-supervised pretraining with Masked Shot Modeling (20% of shots masked) and cosine distance loss.
  • The preview representation is obtained by mean-pooling the contextualized shot embeddings.
• Training corpus: it uses a large-scale shot corpus drawn from the Netflix show catalog. The specific scale was not disclosed.
• Applied surface: applied to video preview personalization on all device platforms. Preview length is 30–90 seconds, and the gain was largest on the TV UI surface.
• Online A/B result: the streaming metric rose by a relative +0.193% (p<0.02) verified §5.2.
  • The offline IPS metric rose 0.380%.
  • Tri-modal being better than the visual-only baseline was confirmed both offline and online.

Popularity-Based Winner — offline proxy task

Netflix introduced a new offline proxy task to gate MediaFM checkpoint releases verified §5.4. This proxy gates every new MediaFM checkpoint, and a checkpoint moves on to A/B only after the candidate space has been reduced.

• Definition: it predicts "which asset would have won if exposed under an unpersonalized, uniform serving policy" using only the asset content embedding as input. It is a binary classification in the linear probe style (training only a linear classifier on top of fixed embeddings).
• Why it is valid: it can rank embedding quality without end-to-end integration or an A/B test. A correlation with online outcomes was also observed.
  • The ranking by probe accuracy, the ranking by offline IPS, and the ranking by online A/B result agreed across all three signals.
• Figures: MediaFM probe accuracy is +25.90 percentage points higher than pick-at-random. SeqCLIP alone is +18.75 pp higher. The tri-modal uplift was reproduced on the shot-level proxy as well.
• Why it matters: this is a case that partly sidesteps Netflix's long-standing problem that "offline metrics do not predict A/B", through a single task that looks only at embedding quality.

Netflix Embedding Store — infrastructure layer

The platform layer consists of 3 stages of precomputation and caching verified §4.3. This 3-stage caching made low-latency serving and cheap screening possible at the same time.

• Content embedding: precomputed at the moment an asset enters the catalog (ingestion time) verified §4.3.
• Item-tower projection: cached per asset verified §4.3.
• Daily materialization: precomputes the selection of the top ~500 titles per profile verified §4.3.

Batch production of member embeddings was already in production in the FM era (see the candidate generation section), and MediaFM established the item-side embedding pipeline in the same way. The two flows correspond to each other.

The 2026-08-28 MAPS blog post named this store the Netflix Embedding Store and stated two properties explicitly verified.

• There is no training-serving skew verified MAPS blog post.

  • Exactly the same embeddings are served at training time and at online inference time.
  • Because the Embedding Store is the only source of truth, there is no room for the training pipeline and the serving pipeline to see different values.

• Foundation model updates and personalization model deployments are decoupled verified MAPS blog post.

  • When a new embedding or a new version of an existing one appears, it is registered, backfilled across the whole catalog, and self-validated. After that, downstream can switch with one line of config.
  • Embedding generations are switched without changing the training or serving code of downstream models.

  "Its key property is that it decouples foundation-model updates from personalization-model deployments. A new embedding, or a new version of an existing one, can be registered, backfilled across the catalog, and validated entirely on its own, without touching the training or serving code of any model that consumes it." — MAPS blog post, 2026-08-28

The blog post explicitly states that as a next step it will unify image, text, and video into one shared semantic space, opening up cross-modal retrieval and unified asset ranking verified. This means the member-side FM and the item-side MediaFM will likely keep flowing through the same store.

Implication — artwork and previews became a separate axis of the personalization stack

The 2026-08 MediaFM paper established artwork personalization as a separate production system. Until then, this canon mentioned this axis only in one line under Re-ranking, diversity, and filters, because there had been no major update since the 2017 bandit. Four things changed:

1. The architecture was unified from per-canvas models to a single model (5→1).
2. The modality was widened from visual-only to tri-modal. It was verified quantitatively that audio and timed-text contribute to the uplift.
3. The place where cold start is handled moved from inside the ranker to pretrained embeddings (day-zero CLIP embeddings).
4. Offline gating was incorporated into the production process as a precondition for A/B (popularity-based winner).

This can be seen as the item-side counterpart to Netflix's move to "centralize member representation with an FM". The member side is the FM (2025) and the item side is MediaFM (2026). The two embeddings are heading toward being combined into a common substructure for the ranker, retrieval, artwork, and previews.

↑ Contents

Cold start

New titles are a programmed catalog whose release dates are known in advance, so this is a problem of exploration and promotion, not of "index coverage". The FM initializes new title embeddings from metadata verified FM blog post 2025.

• New titles (billboard): an incrementality bandit maximizes P(play|impression)−P(play|no impression). This means it does not waste a surface on a title the member would watch anyway verified Kawale & Amat 2018 slides. We could not confirm whether it was fully rolled out.
• New titles (FM): initializes the new title embedding as a weighted combination of metadata, and mixes the ID embedding and the metadata embedding with entity-age-based attention verified FM blog post 2025. This is the opposite of X, which treats new items by ID only.
• New content types: there is a case study on live event cold start, and live engagement rose +20% in a global experiment verified RecSys 2025.
• New members: a title picker survey is given at sign-up. The survey is optional, and if it is skipped, a "diverse and popular" set is given verified TMIS 2015 §5.5 and help.netflix.com.
  • We confirmed that the survey is given, from the 2015 paper through the current help document.

↑ Contents

Feedback signals

Implicit signals dominate. Netflix logs what was watched, how it was watched (device, time, day of week, intensity), where it was discovered, and even what was shown but not played verified TMIS 2015.

• Explicit signals changed from star ratings to thumbs up and down in 2017. Up means "more like this" and down means "stop recommending". Rating activity rose 200% compared with star ratings verified About Netflix 2017.
• In 2022 it added Two Thumbs Up ("really liked it"), and this rating is reflected more specifically verified About Netflix 2022.
• % Match is a display score computed "from viewing habits and behavior analysis only" verified 2017 presentation. The same presentation says thumbs are used to learn taste further and improve recommendations verified. There is no statement that explicit ratings go into the UI's % Match (→ survey-signals.md).
• There is also a mechanism against clickbait. Artwork labels exclude "induced play but low-quality engagement" verified 2017.
• It states explicitly that demographics (age, gender) are not used verified help.netflix.com.
• Session intent is handled by FM-Intent. It predicts the next intent and item at the same time, and is in production verified blog post 2025.

↑ Contents

Training cadence and evaluation

The FM retrains from scratch monthly and fine-tunes daily. This is the most concretely disclosed cadence among big-tech recommendation models verified unified blog post 2025-11.

• Classic-generation sources mention "data up to two days ago" as an example of the offline experiment protocol verified TMIS 2015.
  • This is not a statement about the production training cadence. The cadence itself is not public.
  • The classic-generation system has a 3-layer (offline/nearline/online) structure verified 2013 blog post.
• Evaluation has 2 stages. After narrowing candidates by interleaving, it moves to an A/B with ~2 million people per cell over 2-6 months verified 2017 blog post and TMIS 2015.
  • Interleaving mixes the results of two rankers in one list and checks which side's results are chosen more.
  • Netflix mixes with the team-draft method and judges by share of watch time. It achieves "95% power with 100× fewer people" than A/B.
• It candidly acknowledges the limits of offline metrics. In 2015 it said offline metrics "do not predict A/B results as well as we want", and in 2024 it said "offline improvements are flat/negative online". This has been the same admission for 10 years verified.

Counterfactual observability framework (2026-09, RecSys 2026)

Netflix published a paper on a framework that observes deployed recommender systems through counterfactual measurement verified Beyond Raw Engagement: A Counterfactual Observability Framework for Recommender Systems at Netflix (arXiv 2609.22747), 2026-09-19, RecSys 2026 accepted. Counterfactual measurement estimates what would have happened without a specific content item or model decision. It is a framework that Netflix "built and deployed on multiple recommenders over several years", and this paper is its first academic statement. All 4 authors are affiliated with Netflix Los Gatos.

Offline gating (MediaFM's popularity-based winner) is pre-deployment selection, while this framework is the post-deployment observation (observability) axis. The two sit in different places.

• It redefines observability as a counterfactual measurement problem verified paper.

  • The paper's definition: "estimating what the recommender would have done, and what engagement would have followed, in the absence of a specific content item or model decision"
  • There are two audiences. One is content creators, and the other is model developers.

• There are three measurement axes (§3.1) verified paper.

  • Bias reduction is done with SNIPS (self-normalized IPS).
  • Relativity is measured by policy-versus-policy comparison and by position-adjusted item-versus-item comparison.
  • Incrementality is measured by counterfactual leave-one-out.
  • The authors emphasize as the unifying point that the three axes support both audiences on one measurement foundation.

• It was extended to cascading recommenders (§3.3) verified paper. A cascading recommender is a recommender that narrows candidates through multiple stages.

  • The metric for upstream model developers treats the target action as a set and redefines the match function with set similarity such as Jaccard.

  • The metric for content creators decomposes item i's whole-pipeline contribution into two values:

    Incrementality(i) = Irreplaceability(i) · Universality(i)

    Irreplaceability is the conditional causal lift when i is in the candidate pool (a self-normalized Hájek IPS estimator), and Universality is the probability that i is eligible.

• Row Curators A/B alignment R²=0.8212: the value from comparing the offline incrementality measurement with the actual A/B ground truth verified §4.2.

  • A Row Curator is a title collection row that content curators create on the Netflix homepage.
  • The ground truth A/B removed that row series from the recommender and measured the loss in user engagement.
  • R² = 0.8212 across 11 A/Bs.
  • It also disclosed production details. It clips propensity to [0.0001, 0.9999], requires at least 100 observations per row, and aggregates across homepages of different lengths.

  "Across 11 A/B tests, the R² between our proposed row-incrementality measurement and the A/B-test ground truth is 0.8212, providing strong validation of our approach." — arXiv 2609.22747, 2026-09-19

• There is a case where label misattribution was detected automatically verified §4.3.

  • Incrementality time-series monitoring detected a situation where similar content was stable but only a specific content item A dipped in performance.
  • The root cause turned out to be positive label misattribution in the model.
  • This is a case where the framework caught a finding that raw engagement would have missed.

• Exploration is a precondition verified §3.1. The framework works on a contextual bandit / RL setup and requires logged propensity.

  • Netflix states that this condition is already met by most of its own recommenders.
  • However, a constraint comes with it: it cannot be applied as is to lines served without exploration (e.g., a pure Foundation Model shadow deployment) inferred (the paper does not address this constraint explicitly).

• What the paper did not disclose observed close reading of the paper.

  • The names of the recommenders the framework is deployed on. Only Row Curators was named explicitly.
  • Absolute values of core online metrics
  • Relationship to the FM, GenRec, GenPage, and MediaFM/MAPS generation lines. Whether this framework was used to verify GenRec's batch-compute A/B results is a follow-up item.
  • Individual results of the 11 A/Bs. They were aggregated into R² alone.
  • Countermeasures for recommenders without exploration

Among the tracked services, Netflix is the only company in this repository that has steadily disclosed the limits of offline/online alignment; after "does not predict A/B results as well as we want" (2015) and "offline improvements are flat/negative online" (2024), the counterfactual measurement framework (2026) has now appeared. If the earlier statements were an admission of the problem, this source is an in-house methodology for one axis of that problem, "production observability". Within this repository's corpus, it is the first statement by a company-year combination to publish a recommender measurement methodology as a standalone paper. Its purpose (gating vs monitoring) and its timing (before vs after deployment) differ from MediaFM's popularity-based winner, so the two axes are complementary.

Measuring the value of recommendations — choice model + small-scale nudge A/B validation (IJIO paper, EC '26)

This paper estimates the value of personalized recommendations with a choice model instead of a large experiment that turns personalization off, and validates it with a small-scale nudge A/B verified The Value of Personalized Recommendations: Evidence from Netflix (arXiv 2511.07280), v1 2025-11-10 · v5 2026-06-08, paper. The summary blog post is Measuring the Impact of Personalized Recommendations. We could not confirm the blog post's publication date, and the RSS date 2026-10-14 is a bulk republication value. 6 authors are affiliated with Netflix.

• Instead of a large experiment that turns recommendation personalization off, the paper estimates daily viewing choices with a low-rank choice model and adds a "recommendation effect" term for recommended titles. Identification comes from natural variation such as algorithm exploration verified (blog).

• For validation, it used a randomized A/B that slightly raises or lowers the impressions of a specific category. Measuring the diversion ratio in this A/B and comparing it with the model prediction gives a correlation of 0.86 and an R² of 0.73 verified (blog).

  • The diversion ratio shows where viewing shifts when a set of titles is shown less.

• In the counterfactual comparison, engagement relative to the current recommendations is −16% for random, −12% for popularity-ordered, and −4% for matrix factorization. Popularity-ordered and MF recommendations increased viewing concentration verified (blog, abstract).

• Of the viewing difference produced by recommendations, mechanical exposure is about 7%, and most of it comes from selection and targeting verified (blog).

  • Selection is the effect of showing titles to people who would have watched them anyway, and targeting is the effect of showing them to people whose decisions the recommendation changes.
  • The targeting effect is about 7 times the exposure effect and is largest for mid-popularity titles.

  "Relative to the current system, random recommendations reduce engagement by 16%, popularity-based recommendations reduce engagement by 12%, and matrix factorization reduces engagement by 4%." — Measuring the Impact of Personalized Recommendations (blog)

• If the counterfactual observability framework above is an operational tool that measures the increment of "individual item and model decisions", this paper is an economics measurement of the value of the "whole recommender system" inferred.

  • The basis is that the two sources use different measurement units.
  • The paper did not disclose whether the "current system" used as the comparison is the FM generation or an earlier one verified (absence).

↑ Contents

Ads

The only things public about Netflix ads are the event pipeline (frequency capping, Flink/Druid) and the targeting products. The targeting products consist of 17 categories and 100+ interests. As of 2026-08, there are no public materials on the engineering of the ads ranking model verified (we confirmed the absence). It is currently impossible even to describe the organic lineage and the ads lineage separately, as is done for Meta.

• The ads business started with the 2022-11 Microsoft partnership, went through the 2024-01 in-house decision, and in 2025 the Ads Suite went live in all 12 countries verified Upfront 2025 and pipeline blog post.

↑ Contents

Open questions

• Have GenRec and GenPage been fully rolled out? As of 2026-08, statements go only as far as the A/B stage. observed Rechecked on 2026-08-09: netflixtechblog has no follow-up post on GenRec. However, the index is JS-rendered, so we relied on search results.
• Has GenRec been applied to real-time serving surfaces as well? The public A/B is limited to batch-compute surfaces. There is no statement on whether the cost of 1 prefill pass holds at real-time candidate scale.
• Where does GenRec's candidate set come from? The post does not disclose who builds the candidate set that is scored (FM embedding retrieval or the existing per-source candidates).
• Has Calibrated Recommendations been deployed in production?
• How many parameters does the production FM have?
• What is the primary source for the rule that allows duplicates across rows?
• What is the structure of the ads ranking model? There are no public materials.
• Does a unified system named "Hydra" actually exist? There is only secondary reporting, and we could not confirm a primary source.
• Does Netflix use semantic IDs? There is no primary source. The PRS 2025 semantic tokenization talk was Spotify's, which is commonly confused.

↑ Contents

Sources

• The Netflix Recommender System (Gomez-Uribe & Hunt) — 2015, paper (production description)
• Learning a Personalized Homepage — 2015, blog post
• To Be Continued (Continue Watching) — 2016, blog post
• Recommending for the World (global single model) — 2016, blog post
• Artwork Personalization — 2017, blog post
• Interleaving — 2017, blog post
• Goodbye Stars, Hello Thumbs — 2017, talk
• AVA — 2018, blog post
• Calibrated Recommendations (Steck) — 2018, paper
• Bandit infrastructure — 2019, blog post
• Search: Netflix Case Study (SIGIR) — 2019, paper
• DL for RecSys: A Netflix Case Study — 2021, paper (only the abstract was verified)
• Two Thumbs Up — 2022, talk
• Consolidating ML Models — 2023, blog post
• Reward Innovation (RecSys) — 2023, paper
• UniCoRn — 2024, paper
• Long-Term Member Satisfaction — 2024, blog post
• Foundation Model — 2025, blog post
• FM unified 3 modes — 2025, blog post
• FM-Intent — 2025, blog post
• Embedding stabilization (RecSys) — 2025, paper
• Live cold start (RecSys) — 2025, paper
• The Value of Personalized Recommendations: Evidence from Netflix — 2025-11 (v5 2026-06), paper (IJIO, EC '26)
• Measuring the Impact of Personalized Recommendations — blog post, publication date unconfirmed (shown as 2026-10-14 because of the RSS bulk republication)
• Live real-time recommendation delivery — 2025, blog post
• Ads pipeline — 2025, blog post
• Upfront 2025 (Ads Suite) — 2025, talk
• New TV experience and conversational search beta — 2025, talk
• 1B scaling study — 2026, paper (shadow stage)
• GenRec — 2026, blog post
• GenPage (RecSys) — 2026, paper
• Artwork LLM Post-training — 2026, paper (research stage)
• In-House LLM Serving at Netflix — 2026-07-17, blog post. It covers GenRec's serving layer. vLLM, Triton, the Java control plane, MSS, and constrained decoding scaling appear
• GenRec: An LLM-Backed Recommendation Ranker at Netflix (arXiv 2608.10257) — 2026-08-10, paper. It is the academic version of the GenRec blog post. It contains figures the blog post does not disclose: the loss formula, the Phase 1/2 contribution breakdown, the context 5000→1700 example, the +0.006% core metric, GRPO preliminary results, and the team structure
• MAPS: Netflix's Multimodal Asset Personalization at Scale — 2026-08-28, blog post. It gives the name MAPS to the same system as arXiv 2608.18322 above. The figures are in the paper, and the blog post newly adds the following five things
  • The system name and the grouping of three products (CLIP artwork, query-aware artwork, MediaFM video preview)
  • Measured holdback from the 2025 TV home screen redesign
  • V1·V2 alone flat vs V3 significant ablation
  • The name Netflix Embedding Store and its two properties (no training-serving skew, FM updates decoupled from personalization deployment)
  • A plan to expand to a shared image·text·video semantic space as the next step
• Multimedia Asset Personalization via Multimodal Embeddings at Netflix (arXiv 2608.18322) — 2026-08-18, paper. It covers the production deployment of the artwork CLIP two-tower and the MediaFM tri-modal model (SeqCLIP, wav2vec 2.0, text-embedding-3-large), the +0.127%/+0.193% A/B lift, and gating checkpoints with a popularity-based winner proxy
• Beyond Raw Engagement: A Counterfactual Observability Framework for Recommender Systems at Netflix (arXiv 2609.22747) — 2026-09-19, paper (RecSys 2026 accepted). It is a counterfactual-measurement-based observability framework. It covers three axes (bias reduction/relativity/incrementality), support for single-stage and cascading setups, Row Curators A/B alignment R²=0.8212, and a case of automatic label misattribution detection

↑ Contents