2026-09-23

Other changes reviewed today:

• In the same commit, X began recording which retrieval source each candidate came from and introduced a new path that publishes 5% of requests to Kafka.
• X's visibility verdict engine (VF) was rewritten so that verdicts are expressed as a two-branch tree of hide and show.
• Netflix published a RecSys 2026 paper that treats recommender-system observability as a counterfactual measurement problem. In the Row Curators verification, R² against A/B was 0.82.

1Re-ranking service VMRanker is ready to compute the value model — X

VMRanker is home-mixer's post-processing ranker, a gRPC service that until now served only as a DPP re-ranker (a probabilistic model that avoids selecting similar candidates together). With this commit it is ready to expand into a "service that computes the value model itself" (the value model produces a candidate's final effective value).

• VMRanker calls now split into three modes: the existing DPP-only re-ranking, a mode that sends value model inputs for shadow evaluation, and a mode that actually computes the new value model score verified — home-mixer/scorers/vm_ranker.rs, new home-mixer/scorers/vm_ranker_request.rs (+74).
  • The mode is selected by two new parameters, VMRankerSendValueModelInputs and VMRankerComputeValueModel. Both are false by default.
  • dpp_only is the existing behavior.
  • send_inputs sends the viewer context and candidate policy flags to run a shadow evaluation.
  • compute_value_model actually computes the new value model score.
  • The VMRanker request payload carries the 12 fields of the new domain type FsRecipientInputs.
    • user_id, country_code, language_code, client_app_id, client_version, user_roles, datacenter, has_phone_number, resurrection_time_ms, product, now_ms, fs_overrides
  • It also carries 8 per-candidate fields and 26 PhoenixScores fields.
  • A sixth cluster, Experiment6, was newly registered.
• Two policy decisions made by author cold start handling are now attached to candidates as tags verified — home-mixer/scorers/author_cold_start.rs (+85/−20).
  • AuthorColdStart::apply_with_decisions() is new and returns ColdStartOutcome { scores, author_policy_zeroed: Vec<bool>, lift: Option<ColdStartLift> }.
  • author_policy_zeroed[i] records whether this candidate was masked to a zero score by the MoE ranking policy. MoE (mixture of experts) is a model structure that selects and uses some of several expert sub-models.
  • lift records which candidate was lifted to which rank.
  • Once RankingScorer fills these two fields into PostCandidate, downstream components (VMRanker and the new Kafka pipeline) can see them.

home-mixer gained a new attribution pipeline for analyzing candidate contributions per retrieval source offline. Each candidate records, as a vector, which retrieval source, which cluster, and which rank it entered with, and 5% of requests are published to Kafka together with selected/non-selected labels.

• A new side effect publishes the retrieval candidate list of sampled requests to a Kafka topic verified — new home-mixer/side_effects/retrieval_candidates_kafka_side_effect.rs (+209).
  • It is gated by two new parameters, RetrievalCandidatesKafkaSamplePercent (default 5.0) and RetrievalCandidatesKafkaMaxCandidates (default 200).
  • For each request, one RetrievalCandidateBatch proto is published to RETRIEVAL_CANDIDATES_TOPIC. The proto holds request_id, viewer_id, request_time_ms, sample_rate, ranker_cluster, experiment_buckets, and the candidate list.
  • It is enabled only inside is_prod(), and in-network-only requests and cached requests are excluded.
• Candidates now keep a record of the retrieval source that fetched them verified — home-mixer/models/candidate.rs.
  • The new field is PostCandidate::retrieval_sources: Vec<RetrievalSource>. RetrievalSource is {served_type, cluster: Option<PhoenixRetrievalCluster>, score: Option<f32>, position: Option<u32>}.
  • The three retrieval RPC sources, Phoenix, MoE, and Topics, migrated candidate creation to the new helper candidates_from_retrieval_response(), which assigns a rank (1-based) by descending score per served_type.
  • The Simclusters source also attaches RetrievalSource::from_served_type(ForYouSimclusters).
  • For duplicate candidates coming from several sources, the dedup filter (filters/drop_duplicates_filter.rs) merges the retrieval_sources vectors, leaving multiple source records on a single candidate.
  • Corresponding aggregate metrics were wired up as well. record_retrieval_source_contribution in side_effects/scored_stats_side_effect.rs emits the following three families.
    • Phoenix.RetrievalContribution.Selected(source·mode=credited/inclusive)
    • PhoenixTopK(k ∈ {10, 100, 200, 500, 1000})
    • SelectedTotal

visibility-filtering (VF) is the service that decides, for each post, whether it may be shown to this viewer. This commit completely rewrote the VF rule engine. The verdict changed from a single flat action to a binary verdict tree, and rules moved from functions to data (a predicate list, an audience, and an action).

• The verdict is now a tree with two branches, hide and show. Hiding includes drop and tombstone (a treatment that leaves only a notice in place of the post) verified — new visibility-filtering/models/verdict.rs (+62), visibility-filtering/treatment.rs (+446).

  • The previous verdict was a flat VfAction (3 variants: Allow/Drop/Interstitial). The new verdict is a binary Verdict tree (Withheld{Drop|Tombstone} | Shown{media?, engagement?}).

  • The new definition:

    pub enum Verdict {
        Withheld(Decided<Withholding>),
        Shown { media: Option<Decided<MediaInterstitial>>, engagement: Option<Decided<LimitedEngagement>> },
    }
    pub enum Withholding { Drop(FilteredReason), Tombstone(TombstoneReason) }
    

  • Withheld is terminal (a policy or regulatory drop, or a tombstone).

  • Shown is non-terminal, so the two restrictions, media interstitial and limited engagement, can overlap at the same time.

  • Each decision records the rule name separately in by: &'static str.

  • The enum slots TombstoneReason::LocalRegulations and LimitedEngagementReason::ConversationControl have a wire format but no rule triggers them yet, so they are marked #[expect(dead_code)]. Local-regulation replacement rendering and conversation-control policy are preparation to come into these slots.

• Rules are now written as data, not functions. A rule consists of a name, a condition list, an audience, and an action verified — visibility-filtering/rules/rule_spec.rs (+240/−69).
  • The previous RuleSpec::{Tweet, Author, Custom} enum was replaced by the struct RuleClause { rule_name, when: &[Condition], applies_to: Audience, action: ActionSpec }.
  • Predicates are cataloged in 4 families: 11 TweetPredicate, 8 AuthorPredicate, 5 ViewerPredicate, and 5 RelationshipPredicate.
  • There are 3 audiences: Everyone / ExceptAuthor / ExceptAuthorAndFollowers.
  • ActionSpec has 4 variants: Drop / Tombstone / Interstitial{legacy, media} / LimitedEngagement.
  • The new helper tweet_timestamp_ms(tweet_id) extracts the tweet creation time from the Snowflake epoch (1288834974657 ms, 22-bit shift).
  • Policy assembly happens in rules/registry.rs. The order of the 30 rules for TimelineHome and the 56 rules for TimelineHomeRecommendations is locked by the regression test wired_rule_order_matches_pre_migration_sequence.
• The NSFW interstitial (a warning screen laid over the content) is applied for different reasons depending on when the tweet was created verified.
  • The new rule NsfwHighPrecisionAdultInterstitialRule branches on the predicate CreatedAfter(1705536000000) (2024-01-18 UTC).
  • Tweets created after that are blurred with InterstitialReason::Nudity(true), and earlier tweets with the existing InterstitialReason::Sensitive(true).
  • We infer that the semantics of the NSFW_HIGH_PRECISION label changed by policy on 2024-01-18, and that the rule handles the two periods separately by point in time instead of reinterpreting the old label inferred.
• A new SafetyLevel was introduced for the hydration stage. At this level only 6 tweet-only rules run verified.
  • SafetyLevel is the unit that determines the rule list (policy) VF applies. The new level TimelineHomeHydration was introduced as the 4th of 4 variants.
  • In the hydration stage, only tweet-only rules run, without the author-state and socialgraph rules.
    • TWEET_LABEL_DROPS / EXCLUSIVE_TWEET_DROP / TAKEDOWN_DROPS / SENSITIVE_VIEWER_DROPS / NSFW_MEDIA_INTERSTITIALS / NSFW_AUTHOR_INTERSTITIAL
  • The Thrift TIMELINE_HOME_HYDRATION enum (value 4) is now recognized. In the 2026-09-18 commit (Gen4+25), this value was a stub marked unimplemented.

• In the Phoenix xrex serving code, the bloom filter and topic filter that lived in separate files were unified into a single model_runner verified — phoenix/xrex/inference/model_runner.py (+328).
  • The class that absorbed both filters is RetrievalModelRunner.
  • The deleted files are phoenix/xrex/inference/serving_filters_runner.py (−543) and phoenix/xrex/models/recsys_two_tower_serving_filters.py (−173).
  • The topic filter loads parquet files for 6 options (1fav_topic_option_{0..5}_1day.parquet) and picks one per request. Option 0 is the default, and 1–5 are A/B alternatives.
  • The bloom filter allocates mask buffers in pinned host memory per bucket, and Rust (RetrieveRequestBatch::build_eligible_mask_into_shards()) fills them.
  • If either of the two is enabled, live post-embeddings hotswap is disabled and serving falls back to the drained finalize path.
  • This fallback is presumably to avoid a stale mismatch, because the pinned mask is strongly coupled to all_post_ids inferred.
• An option to quantize the post embedding table to int8 was added verified — new phoenix/xrex/inference/int8_post_table.py (+11).
  • The quantize_post_table() function converts bf16 to int8 with per-post absolute-max scaling. The scale is float32.
  • When the RetrievalModelRunner.enable_int8_post_table flag is on, it requantizes and caches whenever the post embedding object changes (cache[0] is not x identity check).
  • RecsysTwoTowerModel.forward() newly takes a post_scales argument and performs the matmul after the int8 → bf16 conversion.
  • With the new sharding pspec (("stage", "expert", "replica", "data"), ("seq", "model")), the first axis of the post embedding expanded to 4-axis sharding. Pipeline and data parallel axes were added.

• The Phoenix serving engine's copy_port client was heavily refactored so that one slow transfer no longer blocks the next batch verified — phoenix/crates/serving/xai-recsys-engine/src/copy_port_client.rs (+879/−93).
  • The copy_port client is the client with which a serving node downloads embedding tables and dense weights in parallel from the trainer or peers.
  • The batch-by-batch approach is gone, replaced by streaming continuation based on tokio::task::JoinSet, so a slow-tail transfer does not block the next batch.
  • There are two new env vars. COPY_PORT_TRAINER_CONNS_PER_SOURCE is the number of parallel connections opened to one trainer (clamped to 1–16). XAI_RECSYS_RDMA=1 is RDMA mode, in which connection multiplexing is disabled.
  • The new spawn_striped_trainer_downloads() splits one trainer's list of owned files evenly by the number of connections and pipelines them.
  • This is presumably an optimization to reduce checkpoint hot-swap latency inferred.
• Two constants hardcoded in home-mixer were moved to FS (FeatureSwitch, a configuration switch whose value can be changed at runtime) parameters verified.
  • The NEW_USER_OON_WEIGHT_FACTOR constant moved to NewUserOonWeightFactor (FS rust_home_mixer_new_user_oon_weight_factor, default 0.00001).
  • The ANN_MAX_POST_CANDIDATE_AGE_HOURS constant moved to SimclustersMaxCandidateAgeHours (FS rust_home_mixer_simclusters_max_candidate_age_hours, default 48).
  • The Simclusters cache key was extended to signal_id → (signal_id, max_candidate_age_hours), so experiments with different ages get different cache entries.
• A kill-switch decider that can turn off the Phoenix MoE candidate source immediately was added verified — home-mixer/sources/phoenix_moe_source.rs (+66/−19).
  • When the new decider disable_home_mixer_phoenix_moe_source is on, the MoE source is disabled immediately.
  • An FS takes time to roll out, whereas a decider propagates within seconds.
  • PhoenixMoeColdStartMaxResults (default 0) was also introduced, so the cold-start slice cap introduced for non-MoE sources in the 2026-09-18 commit (Gen4+25) is now exposed for MoE as well.
• Grox (X's LLM annotation pipeline) now hands replies that contain a quoted tweet from its simple reply scorer to the full scorer, and repairs LLM output JSON in 3 steps verified — grox/flows/reply_spam/classifier_simple_reply_scorer.py (+103/−3).
  • The simple scorer is SimpleReplyScorer. A reply with a quoted tweet (quoted_post) skips the simple path and is delegated to the full ReplyScorer path (counter task.spam_comment_detection.full_scorer.count).
  • The LLM output JSON parser defends in the following 3 steps.
    1. Extract with re.search(r"\{.*\}", output, re.DOTALL).
    2. If that fails, use json_repair.repair_json().
    3. If it still fails, pull out only the number with the regex r'"score":\s*([\d.]+)'.
  • Parse failures are counted in 3 families: task.reply_ranker.invalid.count{reason=invalid_format|invalid_score_format|score_out_of_range}.
  • The new histogram ranked_replies_scores observes the score distribution in 4 buckets (0.0, 1.0, 2.0, 3.0).

"The framework treats recommender-system observability as a counterfactual measurement problem: estimating what the recommender would have done, and what engagement would have followed, in the absence of a specific content item or model decision." — Beyond Raw Engagement (arXiv 2609.22747), 2026-09-19 (quoted from the Netflix source; see the section below)

• VMRanker is ready to compute the value model.
• The retrieval attribution pipeline records candidate origin on the candidate itself and produces a 5% Kafka sample along with aggregate metrics.
• The VF verdict tree rewrite can express overlapping restrictions and opened the Tombstone/LimitedEngagement slots.
• In the Phoenix xrex serving layer as well, the bloom filter and topic filter that were separate files were unified into model_runner, and int8 post table quantization came with them. This is an axis of large-scale readiness for retrieval serving.
• There are three checkpoints to watch: when VMRankerComputeValueModel=true is turned on in an experiment, which offline job takes the retrieval attribution Kafka data as input, and in which rule the Tombstone/LimitedEngagement slots are first filled.

2RecSys 2026 paper treating recommender observability as a counterfactual measurement problem — Netflix

• Netflix deploys measurement along three axes in its own recommenders verified §3.1.
  • Bias reduction corrects exposure propensity with SNIPS. SNIPS is the self-normalized variant of inverse propensity score (IPS, an estimate that weights observations by the inverse of the exposure probability). The alignment between the policy $\pi_e$ and the logging policy $\pi_0$ is gauged by a match function $m$, and the paper states explicitly that exact matching is a special case of unbiased estimation.
  • Relativity separates algorithm performance from content performance. It uses the SNIPS difference for policy-to-policy comparison and position-adjusted SNIPS for item-to-item comparison. To mitigate position bias, it compares each position lift of content $c_i$ against random content and then aggregates by position weight.
  • Incrementality measures the real contribution of content by asking counterfactually "how much value is lost if this item is removed." Netflix presents the methods for this axis in two cases, single-stage and cascading.
• The framework takes an Exploration & Exploitation module as a precondition verified §3.1.
  • The framework operates on a contextual bandit or RL setup. Arm $a_j$ is selected with probability $\pi(a_j|x_j)$ according to policy $\pi$, and that probability is logged.
  • In other words, the framework cannot be applied as is to a recommender that does no exploration and has no logged propensity.
  • Netflix states that this condition is already met by most of its own recommenders.

• Single-stage incrementality is measured with Leave-One-Out measurement verified §3.2.
  • It places a "Leave-One-Out policy" on top of the Exploration policy, simulating a counterfactual in which a specific item is removed, and aggregates the SNIPS difference of the two policies by position.
  • The match function is 1 under one of the following two conditions.
    • The explored arm is the top pick of the Exploration policy and is not the item being removed.
    • The explored arm is the second-best and the top is the item being removed.
  • To reduce variance, the logged probability is clipped from below at the 10th percentile.
• In cascading recommenders, the method differs by audience verified §3.3.

  • In a cascade the candidate pool is filtered upstream, so low engagement may be due to upstream filtering.

  • For upstream model developers, the question is "was it good to put this item in the candidate pool," and Leave-One-Out is reused. However, the target action is a set rather than a single top pick, so the match $m$ is redefined with a set similarity such as Jaccard.

    • If the lift is large for many items, it means the upstream really adds value to the downstream.
    • If not, it means the pool is interchangeable from the downstream's perspective.

  • For content creators and platform observability, each item's contribution across the whole pipeline is decomposed into two values.

    Incrementality(i) = Irreplaceability(i) · Universality(i)

    • Irreplaceability is the conditional causal lift when i is in the candidate pool, estimated with the self-normalized Hájek IPS.
    • Universality is the probability that i is eligible.

• The alignment between Row Curators' offline incrementality measurement and real A/B was R²=0.82 verified §4.2.

  • A "Row Curator" on the Netflix homepage is a title collection row created by content curators.
  • The offline incrementality measurement for this row was compared with the real A/B ground truth. The A/B removed the row series from the recommender and measured the loss in user engagement.
  • Across 11 A/Bs, the alignment was R²=0.8212.
  • Netflix cites robustness in heterogeneous propensity environments as the reason for choosing the self-normalized Hájek estimator.
  • There are three production details.
    • Propensity is clipped to [0.0001, 0.9999].
    • At least 100 observations per row are required.
    • Results are aggregated across homepages of different lengths.

  "For the Irreplaceability estimate, we tried different estimators and eventually selected the self-normalized Hájek estimator for robust observability signal under such heterogeneous propensities. In practice, we clip the propensities to [0.0001, 0.9999] to further reduce variance, require at least 100 observations for each row, and aggregate across homepages of varying lengths. Across 11 A/B tests, the R² between our proposed row-incrementality measurement and the A/B-test ground truth is 0.8212, providing strong validation of our approach." — arXiv 2609.22747, 2026-09-19

• In the model developer observability case, label misattribution was detected automatically verified §4.3.

  • Incrementality time-series monitoring detected a performance dip for a specific content A while other similar content stayed stable.
  • Tracing the cause showed it was "positive label misattribution in the model."
  • The authors conclude that the framework makes discoveries that raw engagement would miss and lets model developers reach the root cause faster.

• The paper does not state the following five things observed close reading of the paper.
  • It does not name which recommenders the framework was deployed on. The homepage row curation is the only clear case.
  • The absolute values of the core online metrics are not public.
  • Its relationship to generation lines such as FM, GenRec, and GenPage is not stated. It is unclear whether this framework was used for GenRec's A/B alignment or is its own line.
    • FM is Netflix's Foundation Model that centralizes member representations. GenRec is an LLM ranker, and GenPage is a model that generates a whole page with a single Transformer.
  • The individual results of the 11 Row Curators A/Bs are not presented in the body of the paper, and are aggregated only into a single R².
  • It does not mention countermeasures for recommenders the framework does not yet cover (such as those without exploration).

• In the repository canon, this framework sits alongside MediaFM's popularity-based winner (an offline proxy for checkpoint gating), but its role is different. The winner is selection before deployment, and this framework is observation after deployment.
• Whether this framework was used to verify the A/B results in the GenRec paper is a follow-up item.
• Because it presumes exploration-based logged propensity, it cannot be applied as is to a non-exploring recommender (for example, a pure Foundation Model shadow deployment).
• Netflix's release of its recommender measurement methodology as a separate paper is the first company statement on the counterfactual measurement line in this repository's corpus. Its position and purpose are separate from the offline gating of MAPS (Netflix's artwork and video preview personalization system).

Radar

Today I triaged 30 new arXiv cs.IR papers and passed 3, and 2 of them were added to radar.md. The remaining 1 that passed is the Netflix source, which section 2 above covers in detail and which was folded directly into the service canon, so it is not added to the radar. The number of parsed <entry> elements is 30, and the target period is 2026-09-18–09-21.

• UNIQUE is a Baidu·Beihang·CityU paper that unifies retrieval and ranking in an industrial mobile feed and uses single-layer flat quantization verified — arXiv 2609.23718, 2026-09-20 v1.
  • It was accepted at RecSys 2026.
  • The author affiliations were confirmed in the HTML version: Baidu Inc., Beihang University, Hong Kong Institute of AI for Science.
  • The paper identifies two problems: hierarchical quantization instability in candidate retrieval and information loss between the retrieval and ranking stages.
  • It claims that unifying generative code-based retrieval and target-aware ranking in an early-fusion single architecture aims at both end-to-end training and efficient candidate generation.
  • Baidu is not a canon service (a service this repository tracks with a canon document), but it operates the Baidu App, a large-scale industrial mobile feed.
  • Whether the paper states production deployment and A/B figures at the abstract level requires a close read. It was added to the radar as awaiting deep dive.
• MuSeR is a Kuaishou multi-interest sequence retrieval that compresses user histories of 10^4–10^5 scale with hierarchical temporal compression on top of the deployed MGS verified — arXiv 2609.23677, 2026-09-20 v1.
  • The HTML version confirmed that the author affiliation is a Kuaishou domain.
  • The approach keeps recent actions at full resolution and compresses older segments by progressive pooling, so that lifelong history fits within a fixed serving budget.
  • Kuaishou already has several threads on the radar, such as SWIM, UniRec, and DrEM.
  • MuSeR sits adjacent to Tencent's ChronicleRec, and both pre-compress ultra-long behavior sequences. MuSeR adds one data point to the radar's lifelong user modeling angle.

The other 27 were rejected and are not under watch. They did not satisfy the requirement triangle: platform-company authors, a production A/B statement, and novelty in mechanism or scale. The rejected list is Ascent (clinical MCP), UK-PRBENCH (legal case law retrieval), Semantic ID reproducibility study, Baidu·Google AI search auditing (external audit), Graded-Relevance e-commerce multimodal, publication bias biomedical causal search, UniK universal knowledge, Q-TIE temporal IR, YouTube Music LLM rationale (music domain), UNDRR disaster schema extraction, Redundancy Reduction Sequential Recommendation (academic), PSD LLM agent memory, Answer-Oriented Context AI search, candidate-job matching bi-encoder eval, Brand Visibility AI Search, MM-ContextFold multimodal RAG, EU research policy, Inherit4Rec (Kuaishou parameter inheritance, mechanism-novel but rejected in this batch because it mentions no production A/B), Static-vs-Agentic RAG Taiwan history, Attributable RAG citations, R-GEAN medication change (EHR), Per-Query Gating LLM rerankers, Dense-Sparse Fusion BEIR SciFact, DOA-SORT multi-object tracking (non-IR), GEO score cross-engine, AdaMerge visual doc retrieval, and semantic candidate-job matching hybrid retrieval.

Industry blog scanning is done only on Mondays. Today, a Wednesday, was not a target.

What to check next