2026-08-30

Other changes reviewed today:

• In the same commit, X attached a reranker head tag to requests going to Phoenix (the ranking model of X For You). No server-side logic reads it, and it is presumably a head-selection switch for A/B experiments.
• Phoenix serving moved the gRPC callback for large embedding downloads, which had been holding a tokio runtime thread, to the blocking pool.
• Netflix published a blog post that names its artwork and video preview personalization system MAPS. It also includes measured holdback results from the 2025 TV home screen overhaul.

1Refactored visibility rules with no policy change, added 90-case golden corpus — X

• Gen4+4: wired up xai-vf-shadow mirroring, which duplicates requests to the shadow service
• Gen4+7: wired up the reference_compare.rs harness, which compares the VF rewrite's verdicts against the actual verdicts
• Gen4+9: removed all of the media fallback cache in TesHydrator, which calls TES (X's internal tweet metadata lookup service)

This Gen4+11 commit (bc8e5f0) is the next step: a refactor that thins out the expression of individual rule files and puts a regression corpus inside the code.

• The context that rules use to check conditions was split into its own file and exposes about 25 domain methods verified — visibility-filtering/rules/context.rs.

  • What was split out is RuleContext, and the new file is 214 lines.
  • Previously, each rule file accessed data fields directly, as in context.candidate().nsfw.user.
  • Now they are unified in a language that states the basis for a verdict, such as context.has_tweet_nsfw_user_flag(), context.viewer_country_in([...]) and context.legal_takedown_in_viewer_country().
  • The WORLDWIDE_COUNTRY_CODE = "xx" constant and the takedown reason extraction function moved from tes_rules.rs to this file.

• Individual rule files kept their logic and only became thinner in expression verified.

  • For example, in tes_rules.rs, DropStaleTweetsRule shrank from three early returns (the retweet check, the source_tweet_id.is_some() check and the is_stale check) to two lines.

    if context.is_stale_tweet() && !context.is_retweet() {
        return VfAction::Drop(FilteredReason::UnspecifiedReason);
    }
    

• The callbacks of the flag rules and user rules were also changed to take the context instead of data verified.

  • In tweet_flag_rules.rs and user_rules.rs, the fn(&TweetFeatures) -> bool callbacks became fn(&RuleContext<'_>) -> bool.
  • The closure bodies moved from |t| t.nsfw.user to |context| context.has_tweet_nsfw_user_flag(), for example.

• A golden corpus (a set of test cases with fixed expected verdicts) of 90 cases in 13 categories was added, and three assertions run against it verified — visibility-filtering/rules/golden_corpus.rs.
  • The categories are filter_all, baseline, author_state, relationship, tweet_label, tweet_shape, age_gating, exclusive_content, interstitial, oon_media, oon_tweet_label, oon_user_label and interaction.
  • The cases are listed as a table of the form Case { name, level, viewer, candidate, expected_action, expected_decided_by }.
  • golden_corpus_pins_policy_verdicts checks that every case still yields the same action and the same decided_by.
  • every_wired_rule_decides_a_corpus_case checks that subtracting the set of names the corpus used as deciders from the set of wired rule names leaves the empty set. Every rule wired into Policies must have at least one decider case in the corpus (a case where that rule issued the verdict), and CI enforces this.
    • For this assertion, a new helper, #[cfg(test)] pub(crate) fn wired_rule_names(level) -> Vec<&'static str>, was added to registry.rs.
  • corpus_case_names_are_unique prevents case names from overlapping.
• Of the −298 in registry.rs, about half appears to be individual rule tests moved into the corpus, and the other half the removal of repeated inline setup such as Viewer::LoggedIn(999) inferred.

The two Phoenix changes are in a separate context from the VF refactor. The reranker head tag (RerankerHeadTag) starts as a home-mixer parameter, and its default is 0. The value goes out as a field in the two recsys.proto files, Rust and Python, and then comes back in the scoring response. This commit has no logic showing how the server actually consumes the tag (the head routing logic).

• The tag is wired from the home-mixer parameter through the Phoenix request to the proto fields, and no logic consumes the tag on the server verified — home-mixer/params/param.rs.

  • The parameter name is RerankerHeadTag, newly defined as param!(RerankerHeadTag, i64, "rust_home_mixer_reranker_head_tag", 0).

  • A pub reranker_head_tag: Option<u32> field was added to PostCandidate.

  • PhoenixScorer puts this value into the request with reranker_head_tag: Some(query.params.get(RerankerHeadTag) as u32).

  • phoenix_request.rs also attaches this value to scoring_sequence with sequence.reranker_head_tag = Some(...).

  • The same field was added symmetrically in two places in the proto. The Rust proto and the Python proto were updated at the same time with the same field number.

    message ScoreInfo {
      ...
      optional double rewardRerankSlotProb = 5;
      optional uint32 rerankerHeadTag = 6;
    }
    
    message UserActionSequence {
      ...
      repeated Mask masks = 8;
      optional uint32 rerankerHeadTag = 9;
    }
    

  • This commit has no head routing logic on the server side that reads RerankerHeadTag.

The emb_table (the embedding table module of the Phoenix serving engine) gRPC callback change is part of the ongoing process in which the infrastructure serving large embedding tables refines the boundary between the tokio runtime and CPU-bound work. It follows the same line as the load_tensor_into core split in the 2026-08-20 commit (Gen4+5) and the copy_port shuffle in Gen4+10.

• Split the copy work that the gRPC callback was doing into the blocking pool verified — phoenix/crates/serving/xai-recsys-engine/src/emb_table.rs.

  • The change in this file is +79/−12.

  • Previously, the gRPC proto callback itself performed the non-temporal copy and the Adler-32 computation.

  • The new code pushes 4 MiB chunks into a sync_channel(2), and the consumer runs inside tokio::task::spawn_blocking.

    const GRPC_COPY_PIECE: usize = 4 << 20;
    let dest_ptr = buf.as_mut_ptr() as usize;
    let dest_len = buf.len();
    let (tx, rx) = std::sync::mpsc::sync_channel::<bytes::Bytes>(2);
    let copy_join = JoinOnDrop::new(tokio::task::spawn_blocking(move || {
        let dest = unsafe { std::slice::from_raw_parts_mut(dest_ptr as *mut u8, dest_len) };
        let mut pos = 0;
        let mut checksum = 1u32;
        while let Ok(chunk) = rx.recv() {
            install_grpc_chunk(dest, &mut pos, &mut checksum, &chunk);
        }
        (pos, checksum)
    }));
    

  • In the RDMA path as well, everything up to the final Adler-32 summation was moved to spawn_blocking.

  • The new helper JoinOnDrop<T> collects the result with join().await on the normal path, and when dropped it always terminates the blocking task with block_in_place and block_on(handle). Its purpose is presumably to prevent handle and file descriptor leaks inferred.

• An idiom repeated in the copy_port client was extracted into a single helper, with no change in behavior verified.
  • The file is copy_port_client.rs, and the new helper is join_transfers.
  • What join_transfers replaces is the four-line join_all(...spawn) → collect → map_err idiom that appeared in two places.
  • The test names that were added when yesterday's commit (Gen4+10) introduced the deterministic shuffle ("_despite_spawn", "_between_spawned_batches") were cleaned up.
• The proto parser gained a new callback type that can take chunks by ownership transfer verified.
  • In proto_parser.rs, a Bytes callback was added next to the earlier Str callback.
  • The signatures are FnMut(usize, &[u8]) for Str and FnMut(usize, Bytes) for Bytes.
  • In this commit emb_table still uses the Str signature, so a true zero-copy path is a checkpoint to watch in a follow-up commit.

"```rust param!(RerankerHeadTag, i64, "rust_home_mixer_reranker_head_tag", 0);

— [`home-mixer/params/param.rs`](https://github.com/xai-org/x-algorithm/blob/bc8e5f0/home-mixer/params/param.rs), 2026-08-28

• In the safety (visibility-filtering) section, added the subsection "Gen4+11 (2026-08-28, bc8e5f0) — visibility-filtering rule layer refactor + 90-case golden corpus". The subsection covers the list of RuleContext domain methods, the 13-category corpus, the assertions that tie rule wiring to the corpus, and examples of individual rules made shorter.
• In the Phoenix training and index section, added the subsection "Gen4+11 (2026-08-28, bc8e5f0) — Phoenix serving: rerankerHeadTag parameter wiring + emb_table gRPC receive moved to the blocking pool".
• In the open questions section, put a "Resolved by the Gen4+11 release / New" block above the Gen4+10 block.
• In the sources section, listed the bc8e5f0 sha and nine representative files. The listed files are context.rs, golden_corpus.rs, fixtures.rs, registry.rs, param.rs, recsys.proto, emb_table.rs, copy_port_client.rs and proto_parser.rs.

• On the VF side, the early Gen4 "removal of safety nets" (reference_compare harness → media fallback cache removal) has now continued into "accumulating regression-defense assets inside".
  • The policy is unchanged. However, it now makes the basis of a verdict readable in domain language, alongside the file names, whenever rules are added or fixed later.
  • It also blocks "wiring without tests" at the source, at the CI level.
  • Deletions are 1,175 lines, but the commit as a whole is a net gain of 775 lines, and regression-defense coverage also grew.
  • This direction matches the practice X has repeated throughout August of "put it on first, then remove it while accumulating hardening" (the BSR-V2 3→7 journey, the Grok 4.6 10%→30%→100% dial, the TES media cache being introduced and then removed, etc.).
• The Phoenix side is different. RerankerHeadTag is still a switch the server does not read.
  • However, because the connection is complete through the home-mixer parameter, the proto fields and the round trip, an A/B experiment becomes possible immediately once the next commit opens the serving consumption logic.
  • Training several reranker heads and routing to them by tag at serving time is a pattern that commonly appears in the A/B maturity phase. This pattern is likely to combine with Gen4's other dial-up practice (shadow deployment → dial-up commit → production promotion).
  • The field being attached symmetrically in two places (ScoreInfo and UserActionSequence) also suggests that the request and response are symmetric. If so, the request side is an instruction to "score with this head," and the response side is a confirmation that "this head was actually used."
• Splitting out the emb_table gRPC callback is a narrow, clear infrastructure optimization.
  • If the gRPC callback performs the non-temporal copy and the Adler-32 computation for a long time on a tokio runtime thread, other tasks on the same runtime (request routing, admission decisions, health checks) are delayed.
  • With this change the gRPC receive thread only forwards chunks, and the CPU-bound copy goes out to the blocking pool.
  • The 2026-08-20 commit (Gen4+5) split the emb_table core to be pyo3-free, and Gen4+10 equalized peer load with the copy_port shuffle. Together with this blocking pool split, these changes are part of the ongoing refinement of the boundary between the tokio runtime and CPU-bound work in the infrastructure serving large embedding tables.
  • Introducing the Bytes callback type leaves open the possibility that this line leads to a true zero-copy path in the next commit.
• The checkpoints to watch:
  1. When and in what form the server-side consumption logic for RerankerHeadTag opens (the point when a head routing function is added to the Phoenix Python serving code)
  2. When emb_table consumes the Bytes callback (the true zero-copy path)
  3. The channel that verifies whether the 90 golden corpus cases exactly match prior production verdicts. If the Gen4+7 reference_compare harness is still running after this refactor, then in effect real-traffic verification is happening.

2Named the artwork and video preview personalization system MAPS — Netflix

The previous system treated assets only as opaque IDs. It is the 2017-2018 Artwork Personalization bandit lineage. The model could not see what was in the image. New titles and new assets have no interaction history, so cold start relied on exploration and a popularity fallback.

MAPS is a production system that uses CLIP and MediaFM embeddings to let the model "see and hear" the content of an asset, so that personalization works even before interaction history builds up. Here, multimodal is used at two levels.

1. CLIP is an image-text shared space.
2. MediaFM is tri-modal, handling visual, audio and text together.

What this blog post newly establishes relative to the arXiv paper is not quantitative figures but architectural context and a description of the infrastructure. There are five new axes.

• The system name MAPS itself
• Measured holdback results from the 2025 TV home screen overhaul
• The ablation of V1/V2 alone (flat) vs V3 (significant)
• The naming of the Netflix Embedding Store and its two properties
• The plan to extend to a common image, text and video semantic space

• The blog post bundles CLIP artwork, query-aware artwork and MediaFM video previews into one system, MAPS verified — MAPS blog post.
  • So far the canon described "artwork personalization" and "video preview personalization" as separate axes.
  • The common underlying infrastructure is the Embedding Store.

• V1 and V2, which each turned on one idea, were flat both offline and online verified — MAPS blog post.
  • V1 turns on only the image embeddings, and V2 turns on only the canvas unification.
  • A canvas is the screen format in which artwork is placed, such as a billboard or a short panel, and previously each canvas had its own model.
  • Both the offline IPS (inverse propensity scoring, a method that corrects the bias of the logged impression policy to evaluate a new policy offline) and the online core member metric were non-significant.
• Only V3, with both turned on together, got a significant lift and was deployed to production verified (MAPS blog post).
  • In short-panel, V3's offline IPS relative lift was +5.691%, larger than the sum of V1 and V2.
  • This is a case where the observation "turning on just one content feature does not move the metrics" is stated explicitly.
• In the holdback measurement during the 2025 TV home screen overhaul, V3 also got significant lifts in both the core discovery metrics and streaming hours verified — MAPS blog post.
  • It was a cold-start situation in which the UI overhaul suddenly made short-panel the dominant canvas.
  • V3 was deployed just before the overhaul, and a holdback A/B ran for a month. A small control group kept the previous per-canvas models.
  • Both metric lifts were statistically significant, and larger than in the steady-state ablation.
  • This is a case where signal transfer across canvases in a single model was measured.

• The additional modeling effort for query-aware artwork is effectively zero verified — MAPS blog post.
  • The ranking score is blended as α · personalization + (1−α) · cos(text_emb(query), image_emb(asset)), and α is tuned with an online A/B test.
  • Because CLIP embeddings were already in the asset representation, no new model needed to be trained.
  • Playthrough rate rose by +0.36% (p<0.05).

• The video preview A/B ran for 5 weeks on all device platforms; MediaFM got a significant lift on core streaming metrics and was deployed to all platforms verified — MAPS blog post.
  • The ranking is MediaFM > SeqCLIP > ID-only.
  • The gain was largest on TV.
• The linear probe proxy was adopted as the gate for every new MediaFM version release verified — MAPS blog post.
  • A linear probe trains only a linear classifier on top of frozen embeddings, to measure the information contained in the embeddings.
  • This probe is a binary classification that predicts from embeddings alone "which asset will win under an unpersonalized policy". The label is the debiased popularity winner from exploration data.
  • It was adopted for gating because all three signals agreed: probe accuracy, offline IPS and online A/B.
  • Only after the candidate space is pruned does it move on to A/B.

• The Netflix Embedding Store has no training-serving skew and decouples FM updates from personalization deployments verified — MAPS blog post.
  • FM is short for foundation model (a model pretrained at large scale and reused by many downstream models).
  • The Embedding Store is an AI Platform component that hosts dense embeddings of titles, games, member profiles and multimedia assets.
  • First property: it serves exactly the same embeddings at training time and at online inference time, so there is no skew.
  • Second property: after a new embedding (or a new version of an existing one) is registered, backfilled across the catalog and self-validated, downstream can switch to it with a single config line.
  • Embedding generations switch without changing the training or serving code of downstream models.
• As a next plan, it explicitly states that it will extend the Embedding Store to a common image, text and video semantic space, opening up cross-modal retrieval and unified asset ranking verified — MAPS blog post.
  • This means the picture in which the 2025 member-side FM and the 2026 item-side MediaFM flow through the same store will be maintained going forward.

• The blog post's concealment of the online A/B figures is notable observed 2026-08-30.
  • The blog post describes its online A/B results only qualitatively, as in "statistically significant lift" and "gains largest on TV".
  • The figures (+0.127% and others) are in the arXiv paper (2608.18322) and are already folded into the canon.
  • The online panel in the last figure hides its values with "values withheld". The blog post shows only offline IPS relative lifts in the figure.

"Neither idea moved our online core member metrics on its own. V1 and V2 were both flat and non-significant, and only V3 won a statistically significant lift. It is what runs in production today." — MAPS: Netflix's Multimodal Asset Personalization at Scale, 2026-08-28

"We shipped V3 ahead of the launch and measured it with a month-long holdback A/B test, keeping a small control group on the prior per-canvas model. V3 absorbed the shift immediately, with statistically significant gains on both our core discovery metric and streaming hours, and larger gains than in the steady-state ablation." — MAPS: Netflix's Multimodal Asset Personalization at Scale, 2026-08-28

"Its key property is that it decouples foundation-model updates from personalization-model deployments. A new embedding, or a new version of an existing one, can be registered, backfilled across the catalog, and validated entirely on its own, without touching the training or serving code of any model that consumes it." — MAPS: Netflix's Multimodal Asset Personalization at Scale, 2026-08-28

1. The section header became ## Artwork and video preview personalization — MAPS (Multimodal Asset Personalization at Scale, 2026-08 production), and the blog post URL was added next to the intro paragraph.
2. In the CLIP two-tower (a two-tower structure that embeds members and items separately) subsection, added the ablation of V1/V2 alone (flat) vs V3 (significant) and a quote of the original text. Also added the measured holdback results from the 2025 TV home screen overhaul, the query-aware artwork blend formula, and the explicit statement that additional modeling effort is effectively zero.
3. In the Netflix Embedding Store subsection, added the name the blog post newly gave it and its two properties (no training-serving skew, FM updates decoupled from personalization deployments). Also included the quote of the original text on decoupling and the next plan (common semantic space).
4. In the sources section, added the MAPS blog post entry and stated its relationship to the arXiv paper (figures vs architectural context).

• First, three products were bundled into one system.
  • So far the canon described "artwork personalization" and "video preview personalization" as separate axes. The blog post bundles CLIP artwork, query-aware artwork and MediaFM video previews into one system, MAPS, and the common underlying infrastructure is the Embedding Store.
  • This updates the earlier view that "the member-side FM was unified, but the item side still has several axes".
• Second, the Netflix Embedding Store was named as a separate component of the FM stack.
  • The picture in which the embeddings of the FM (2025, member) and MediaFM (2026, item) flow through the same store becomes clear. The earlier observation that member-side embedding batch production and the item-side embedding pipeline are symmetric is settled by a single infrastructure name.
  • The decouple property is especially important. If a new embedding generation can be deployed with only register, backfill and validate, the update cadence of the FM stack and the deployment cadence of personalization models are completely separated.
  • This means that the member-side FM's "monthly retraining + daily fine-tuning" cadence (see the training cadence section of the canon) and the item-side MediaFM release cadence can each be optimized independently.
• Third, the ablation of V1/V2 alone (flat) vs V3 (significant) is empirical evidence for the observation "turning on just one content feature does not move the metrics".
  • That the blog post states this explicitly is itself new. It squarely acknowledges the opposite of the "won offline but flat online" case common in recommender systems (offline and online both being flat).
  • This becomes a case that can be cited as evidence when similar experiment results appear later.
• It is reasonable to regard the 2017 bandit as superseded. As evidence, the blog post states the limitation of the earlier generation explicitly with "for years, our models answered the first question well and the second poorly", and says outright that V3 is production.
• GenRec and GenPage are the ranking and page-generation axis, and MAPS is the asset axis, so they do not directly overlap. However, because all three systems are FM stacks centered on member and item embeddings, the architectural direction is the same.

What to check next