2026-09-25

1home-mixer and vm-ranker ranking scores now come from one shared crate — X

• The home-mixer ranking scorer of 1,255 lines was deleted entirely verified — home-mixer/scorers/ranking_scorer.rs.

  • We confirmed ranking_scorer.rs in the previous commit's tree.

  • The only thing taking its place is the new 114-line file home-mixer/scorers/value_model.rs.

  • This file does three things:

    1. weights_for(query) pulls 33 values from the home-mixer FS parameters and builds ValueModelWeights.
    2. scoring_inputs(query, weights, candidate) turns the candidate's Phoenix engagement probabilities and its in-network, mutual-follow and vqv conditions into CandidateScoringInputs.
    3. weighted_score(query, weights, candidate) calls the shared crate's fuse_heads as is.

    pub(crate) fn weighted_score(
        query: &ScoredPostsQuery,
        weights: &ValueModelWeights,
        candidate: &PostCandidate,
    ) -> f64 {
        fuse_heads(weights, &scoring_inputs(query, weights, candidate))
    }
    

    — home-mixer/scorers/value_model.rs, 2026-09-24. What the home-mixer ranking scorer does is now compressed into these three lines. The actual formula lives inside the shared crate's fuse_heads.

• A new shared crate was released with 6 files and 807 lines in total verified — xai-value-model/lib.rs (+14).
  • The crate name is xai-value-model.
  • The other five files are inputs.rs (+35), phoenix_scores.rs (+28), proto.rs (+92), scoring.rs (+444) and weights.rs (+194).
• The symbols the crate exports are listed below verified. The list is exactly what lib.rs re-exports with pub use.
  • ValueModelWeights (33 fields) · CandidateScoringInputs · QueryScoringContext · PhoenixScores (26 fields) · compute_value_scores · compute_value_scores_with_adjustment · fuse_heads · compute_weighted_score · offset_score · post_fusion_multipliers · apply_cold_start_decisions · diversity_multiplier · author_pool_counts · PostFusionMultiplier · ValueScores · the NEGATIVE_SCORES_OFFSET constant

• 136 lines were deleted from the home-mixer parameter file verified — home-mixer/params/param.rs (+2/−136).
  • Deleted parameters: EnableAuthorDiversity·AuthorDiversityDecay·AuthorDiversityFloor·OonWeightFactor·MultiplierPreOffset·EnableOonRescoreForInNetworkRepliesRetweets·TopicOonWeightFactor·NewUserAgeThresholdSecs·NewUserOonWeightFactor·EnableMultiplicativePostUnexplored·MultiplicativePostUnexploredAlpha·PostUnexploredWeightInNetworkOnly·EnableCdwellOnImpr·ContActiveSecs5mResidualNormWeight·WeightPerturbationSigma·WeightPerturbationSalt·EnableVMRanker·VMRankerDppTheta·VMRankerDppMaxSelectedRank·VMRankerSendValueModelInputs·VMRankerComputeValueModel·CachedPostsReuseWeightedScore
  • The sync time in the file header also changed from 2026-09-22T17:39:32Z to 2026-09-23T16:28:43Z.
• The deleted values moved in two directions verified.
  • The value model weights and gating parameters were gathered into the fields of the ValueModelWeights struct. The weight values are defined in two places: vm-ranker/params.rs, which vm-ranker manages (introduced yesterday, 56 parameters), and home-mixer's param.rs:295-296 onward, such as FavoriteWeight and ReplyWeight.
  • home-mixer's weights_for() reads the home-mixer-side values (see A above).
  • The DPP parameters (VMRankerDppTheta, VMRankerDppMaxSelectedRank) were absorbed into the RankingConfig inside the vm-ranker process.

• If the vm-ranker RPC fails, home-mixer now serves the locally computed weighted sum as is verified — home-mixer/scorers/vm_ranker.rs (+84/−76).
  • The previous code propagated the failure as a per-candidate Err.
  • home-mixer's vm_ranker.rs was restructured from "request assembly" to "local-first, RPC-refine".
  • The new function local_scores() first computes a local weighted sum for each candidate with value_model::weighted_score(). It then attaches the cold start lift and author_policy_zeroed decisions with AuthorColdStart::apply_with_decisions().
  • After that it sends a refinement request to the vm-ranker gRPC.

• A popular-candidate slice was newly introduced in Phoenix retrieval (the candidate source that finds OON candidates with Phoenix) verified — phoenix/xrex/data/retrieval_dataset.py (+103/−8).
  • The new slice is named HOME_HOT. It was registered as HOME_HOT = 14, next to HOME_COLD = 13 in the RetrievalDataset enum.
  • Both slices are drawn from the same 1fav_1day.parquet source and are split by two masks in the new phoenix/xrex/data/cold_pool_filter.py (+78).
    • cold_pool_mask(post_ids, meta_path, max_favs=8, max_views=500)
    • fresh_post_mask(post_ids, max_age_seconds): computes the posting time directly from the tweet id using the Twitter Snowflake epoch.
  • HOME_COLD is cold_keep = cold_pool_mask & fresh_post_mask, and HOME_HOT is its complement.
  • The cold pool is now cut at data loading time in the training and serving stages, and only the remaining popular candidates are managed in a separate index.
  • default_cold_pool_metadata_path() is ${PHOENIX_INDEX_BASE:-/data/phoenix_index/prod}/post_author_pairs_metadata/metadata_1day.parquet.
• When a Phoenix bundle is exported, per-target index metadata is attached verified — phoenix/xrex/train/recsys_bundle_export.py (+170/−46).
  • Four fields were added to the RetrievalExport dataclass.
    • dataset_capacities
    • rows_from: decides, per target, which source dataset's rows to reference.
    • max_age_seconds: the maximum age per target.
    • optional_targets
  • The structure in which HOME_COLD and HOME_HOT split the rows of the same source (HOME) is expressed by exactly this rows_from.
  • The manifest exposes two constants, POST_SCALES_KEY and DATASET_RANGES_KEY.
  • The new gating consists of three flags: enable_int8_post_table, enable_bloom_filter and enable_topic_filter. These gates decide whether to include the int8 post table (per-post absolute-max scale) in the export.

• Logic that merges the retweet verdict and the original tweet verdict was added to VF (visibility-filtering, the service that judges per post whether it may be shown to this viewer) verified — visibility-filtering/models/verdict.rs (+60).

  • The new function is Verdict::merge_retweet_verdict(retweet, source). There are four merge rules:

    1. If source is Withheld, source is always adopted. If the original tweet is withheld, the retweet is withheld too.
    2. If retweet is Withheld, retweet is kept. This is the case where the rule that withholds the retweet itself won.
    3. If retweet is Shown{media: None, engagement: None} (no restrictions), it inherits source.
    4. Otherwise, retweet is kept.

    pub fn merge_retweet_verdict(retweet: Verdict, source: &Verdict) -> Verdict {
        match (&retweet, source) {
            (_, Verdict::Withheld(_)) => source.clone(),
            (Verdict::Withheld(_), _) => retweet,
            (
                Verdict::Shown {
                    media: None,
                    engagement: None,
                },
                _,
            ) => source.clone(),
            _ => retweet,
        }
    }
    

    — visibility-filtering/models/verdict.rs, 2026-09-24

  • The flow that actually runs this merge was implemented in visibility-filtering/evaluate_tweets.rs (+157/−18).

    • It collects the evaluated retweets with source_tweet_id.is_some() from the first-pass evaluation results.
    • It splits those source ids with HashSet.contains() into in_batch (already in the same batch) and fetched. fetched are those that need to be looked up again.
    • It calls filter_tweets.run(...) a second time on the fetched list to obtain the source verdict.
    • For each retweet, it looks up the source verdict and applies merge_retweet_verdict.

  • The new metric is evaluate_tweets_retweet_sources{outcome=in_batch|fetched}.

• As a change supporting the merge, pure_cores: TweetHydrationBatch<PureCore> was added to the hydration result tuple in visibility-filtering/hydration/mod.rs (+244/−18).
  • PureCore is thin core information that holds the retweet's original id (source_tweet_id).
  • PureCoreFallbackCache caches ids that failed the first time so they can be retried in the next request.
  • A source_tweet_id: Option<TweetId> field was added to RawCandidate (introduced after Gen4+22) and is filled in during hydration.

• A harmful-text category for age gating was newly added to PTOS, the policy verdict stage of grox (the Grok-based nearline annotation pipeline) verified — grox/flows/ptos/state.py (+6).
  • PTOS is the stage that judges violation categories against policy.
  • The new category AgeGatingHarmfulText was added as the SafetyPolicyCategory.AgeGatingHarmfulText enum.
  • Five subtypes under it were registered in SafetyPolicyType: AgeGatingHarmfulTextAdultContent·EatingDisorder·HarmfulSubstance·DangerousStunt·GraphicDescription.
  • In grox/flows/ptos/classifier.py (+5), this category is registered in the USE_GEMMA_CATEGORIES set, so the first-pass verdict is made in Gemma-4-26b. As of today, the only other categories in this set are Spam and IllegalAndRegulatedBehaviors.
  • The prompt is built by age_gating_harmful_text_policy_prompt() in prompts.py (+5), which renders a separate template, age_gating_harmful_text_policy.j2. The template body is not included in this commit's tree.
  • That is, a policy axis was introduced that sends text of an adult-content nature to age gating rather than deletion.

• In grox reply handling, the GROK_GEMMA_FOLLOWER_SPLIT follower threshold rose from 225,000 to 250,000 verified — grox/flows/reply_spam/constants.py.
  • The constant's name is GROK_GEMMA_FOLLOWER_SPLIT.
  • In the code, this constant splits task eligibility verified — task_filter.py. Conversations where the root or the replied-to account has more followers than this value go to the reply ranking task, and conversations where both are at or below it go to the reply spam task.
  • The constant's name suggests a Grok and Gemma boundary, but the threshold that picks the spam verdict model is a separate one, GEMMA_REPLY_SPAM_MIN_FOLLOWERS = 150_000 verified — same constants.py.
  • This extends an upward series that has run for about 42 days since the Gen4 release (15k → 30k → 40k → 60k → 80k → 100k → 120k → 150k → 180k → 200k → 225k → 250k) verified — per-commit values in task_filter.py and constants.py.
  • As the threshold rises, the range of conversations handled by the reply spam task widens.

• home-mixer's scoring statistics now count how many of the top candidates come from SimClusters (the candidate source that finds OON candidates by cluster similarity) verified — home-mixer/side_effects/scored_stats_side_effect.rs (+35/−4).
  • The new metric family is Phoenix.RetrievalContribution.SimclustersTopK. Its top-k buckets are [50, 100, 200, 400, 600, 800], an axis separate from Phoenix's [10, 100, 200, 500, 1000].
  • The new Phoenix.RetrievalContribution.ScoredTopK measures the top-k source distribution over all selected and non-selected candidates. It observes which sources made the scoring top-k before selection.
  • It is an aggregate metric axis that pairs with retrieval_candidates_kafka_side_effect.rs (raw data sampling), introduced in Gen4+26.
• VF's clock_cache.rs (412 lines) was removed verified.
  • A primitive that had been introduced in the 2026-09-10 commit (Gen4+19) in an "unwired" state was deleted after 14 days.
  • This is a signal that they do not want to bear the maintenance burden of an unconnected primitive. If it is needed again, it will likely be reintroduced then inferred.

• First, X's ranking computation converged on a single library.
  • Previously, home-mixer and vm-ranker implemented the same formula (the sum of Phoenix probabilities × weights) twice, and so the parameters were mirrored twice as well.
  • Even as of yesterday, how consistency between the two processes was maintained was an open question.
  • With today's commit, home-mixer's copy is gone, and the computation lives only inside the crate.
  • The home-mixer param.rs mirror also got thinner (22 ranking-computation parameters deleted). However, the value model weights remain in both mirrors, home-mixer and vm-ranker/params.rs.
• Second, home-mixer now has a degraded mode for vm-ranker RPC failures.
  • Previously, when vm-ranker died, an Err was thrown per candidate to downstream code; now the locally computed weighted sum is served as is.
  • This degraded mode became possible because the shared crate physically exists inside home-mixer.
• Two other things were reflected today:
  • Phoenix retrieval split HOME_HOT, the popular (hot) slice, into a separate index so it no longer mixes with cold.
  • In VF, the last unfinished connection in the rule engine rewrite that has been under way for the past two months was completed. It is a merge that makes a retweet's visibility actually follow the original tweet's visibility.

Paper and blog scan

Of the 14 new papers seen today, 1 passed the watching criteria, and it was added to the radar after checking the original text. The arXiv cs.IR new list had 30 entries (30 <entry> items parsed, submitted 2026-09-21–09-23). Of those, 14 were new since yesterday, submitted on 2026-09-23. These 14 were screened on three criteria: platform company authors, a production A/B claim, and novelty in mechanism or scale.

• The paper that passed is When LLM-Based User Profiling Adds Value in Production Streaming Recommendation by Comcast Technology AI authors verified arXiv 2609.27183, 2026-09-23.
  • In the HTML version we confirmed that 3 of the 4 authors are affiliated with Comcast Technology AI. They are Neeraj Sharma, Sardar Hamidian and Shaghayegh Agah, with affiliation locations in Sunnyvale and Washington DC.
  • The first author, Milad Sabouri, is affiliated with both DePaul University and Comcast Technology AI.
  • Comcast owns Peacock streaming and Xfinity.
  • The paper's content is a factorial comparison of four content-based user profile types on a production dataset. The four types are combinations of (aggregate vs LLM-generated) × (temporal disentanglement on/off).
  • There are two kinds of profiles:
    1. aggregate: averages the user's recent and past item embeddings.
    2. LLM: feeds the user history into an LLM to get a natural-language summary, then vectorizes it with a text encoder.
  • Because the LLM approach is much more expensive than aggregate, the paper asks "when is that cost justified".
  • There are no online A/B results. It reported only offline accuracy and beyond-accuracy metrics (diversity and others).
  • It does not state which Comcast product the production dataset comes from.
  • It was added to radar.md. Comcast/Peacock were not previously on this repository's watch list, so it is recorded as the first signal from a production stack.

The other 13 were dropped from watching.

• MultiVENT-Raw (video retrieval benchmark, academic)
• Beyond a Scalar (watch-time DSI, Shanghai Jiao Tong University, Rice, academic; uses the KuaiRec/WeChat21 benchmarks but has no platform authors)
• Entangle (quantum GitHub collaboration, off topic)
• Dual-Hypergraph Indexing (RAG, academic)
• Turkish LLM eval (domain-specific)
• LLM workflow structural diff (SE tool)
• Flexible Rec System (group recommendation, academic)
• VDR TTA (document retrieval, academic)
• SLP verifiable LLM inference (LLM infrastructure, off topic)
• RoPA Vietnam (data protection automation, off topic)
• Large Knowledge Model (scientific knowledge graph, off topic)
• LatWeave (multi-hop QA, academic)
• BoundaryMORPH (RAG reranking, academic)

The industry blog scan is done on Mondays only. Today (Friday) is not covered.

The latest Meta engineering blog posts are two from 2026-09-21 (Rebalancer open source and Petal subsea cable), and both are infrastructure, so they are skipped. The latest Netflix Tech Blog post is "Leave the Class Path in the Rearview Mirror" (JVM infrastructure) from 2026-09-18, and it is skipped as well.

Radar

The only new radar entry is 1 Comcast paper.

• Comcast Technology AI — LLM-based user profiling for streaming recommendation (offline production comparison): 3 of the 4 authors are affiliated with @comcast.com, and it is the first public signal from the owner of Peacock/Xfinity. There is no online A/B yet.

What to check next