0x00 한

Research / Daily reports

  • Sources read: 1
  • Canon files updated: 2
  • Radar entries: 0
In one lineX added a per-user debiasing correction to VMRanker that removes each viewer's own engagement tendency before scoring. The correction is off by default, and only the input sending is on.

1A correction for per-viewer engagement tendency was added to ranking scores — X

Source xai-org/x-algorithm@2f25214 — xAI, 2026-10-27, code. In these notes' numbering, this is Gen4+Gen50.4+N is the number these notes assigned, in order, to commits after the 2026-08-13 Gen4 release).

Background X For You's ranking model Phoenix (a Transformer ported from Grok-1) predicts action probabilities such as like and reply for each candidate, and VMRanker (the service that handles the weighted sum and DPP slate re-ranking) multiplies those probabilities by weights and adds them. However, a viewer who likes often gets a high like probability on any post. If probabilities mixed with this viewer habit are summed as they are, the weight of the actions that viewer does unusually often effectively grows. This commit added a path to VMRanker, off by default, that predicts the viewer's baseline tendency separately and removes it from the probabilities, and it turned on the switch that sends the inputs this path uses.

What we learnedA. VMRanker can now re-fit Phoenix probabilities with a separate model that predicts the viewer's baseline tendency

  • The correction rewrites the Phoenix probabilities just before the weighted sum and recomputes the weighted sum from the corrected probabilities verified — vm-ranker/scoring/value_model.rs.

    • compute_or_fallback first calls compute_value_scores with the original probabilities, and if debias::recenter returns values, it calls it again with those as input.
    • The correction is not applied to requests that reuse a weighted-sum score stored in the cache (it returns immediately when cached_weighted is set).
  • The debiasing model produces a per-viewer logit and a per-surface global logit for each head, from the viewer's recent impression sequence and the request context verified — vm-ranker/scoring/debias.rs.

    • The sequence is user_sequence_arrow_ipc, which home-mixer sends in the experiment payload of the VMRanker request as Arrow IPC.
    • The sequence columns in the test are impression time, dwell time, first video length, time zone, surface, media presence, follow status, position in the feed, media type, and a 64-action multi-hot.
    • The request context is time, time zone, app ID, and surface, and an empty surface value is treated as Home (1).
    • The model file is a binary with a UDB1 header, and it is reloaded every 5 minutes and replaced if it changed (--user_debias_reload_secs, default 300).
  • The debiased probability is a mix of the original probability and the corrected probability with a blend ratio α verified — debias.rs:221-235.

    • A probability below 1e-7 or above 1−1e-7 is left untouched.
    • The correction function debiased_probability lives in the crate xai_user_debias, which is not in the repo, so the exact formula is not public.
    let debiased = debiased_probability(p, user_logits[head_idx], global_logits[head_idx], strength.clip);
    ((1.0 - strength.alpha) * p + strength.alpha * debiased).clamp(PROB_FLOOR, 1.0 - PROB_FLOOR)
    
  • The correction direction lowers the probability of actions whose viewer tendency is higher than the surface average verified — vm-ranker/scoring/mod.rs test.

    • The test user_debias_recenters_fav_probability_before_fusion feeds a fav probability of 0.3, a viewer logit of 1.5, a Home global logit of 0.5, and a clip of 2.0, and asserts that the corrected score is lower than the original score.
    • The clip in the name is likely the cap on the difference between the viewer and global logits inferred. The only evidence is that the parameter is named UserDebiasClipLogit and does not allow negative values.
  • By default 14 heads are corrected, and negative heads such as report, block, and mute are not in the default list verified — debias.rs:17-32.

    • The default list is fav, reply, quote, retweet, dwelled, not_dwelled, vqv (a view where the video was actually watched for at least a set time), photo_expand, click, profile_click, bookmark, share, open_link, and video_open.
    • bookmark is in the default list but is not in the score field mapping, so it is not actually corrected.
    • Negative heads can be corrected by changing the list with the parameter UserDebiasHeads.

What we learnedB. The correction itself is off, and only the input sending is on

  • In the home-mixer cron mirror (the parameter file that copies production defaults into the repo every day), the switch that sends debiasing inputs to VMRanker was turned on verified — home-mixer/params/param.rs.

    • VMRankerSendDebiasInputs went from false to true. The mirror sync time is 2026-10-26T16:00:39Z.
    • This switch and the payload file were introduced in the 2026-09-23 commit (Gen4+27), and it had been false by default since then.
  • The debiasing parameters on the vm-ranker side were all introduced with off defaults verified — vm-ranker/params.rs.

    UserDebiasEnabled = false
    UserDebiasAlpha = 0.0
    UserDebiasClipLogit = 2.0
    UserDebiasHeads = ""            // if empty, the default 14 heads
    UserDebiasHeadAlpha = ""        // per-head α as a "head:value" list
    UserDebiasHeadClipLogit = ""
    UserDebiasMaxArtifactAgeSecs = 3600.0
    
  • A model file trained more than 1 hour ago is not used verified — same file and debias.rs:162-167.

    • This age limit presupposes that the debiasing model is retrained at least hourly inferred. The evidence is that the default limit is 3600 seconds and the reload interval is 5 minutes.
  • The test config gives as an example a rule that turns it on with α=1.0 only for viewers in Germany (DE) verified — scoring/mod.rs test YAML. This does not mean the actual experiment targets Germany.

What we learnedC. The Python training and inference code for gen-recs and SID retrieval was removed from the public release

  • 14 Python files for gen-recs (a generative recommendation model) and SID retrieval (an approach that represents a post as a semantic ID, a sequence of hierarchical codewords, and finds candidates by generating that ID) were deleted from the Phoenix release verified — commit diff.
    • The deleted files include xrecsys_gen_recs.py, xrecsys_sid_retrieval.py, xrecsys_sid_variants.py, recsys_gen_recs_model.py, recsys_sid_retrieval_model.py, trainer_gen_recs.py, trainer_sid_retrieval.py, gen_recs_runner.py, and sid_retrieval_runner.py.
    • The Kafka topic settings (SID_TOPIC, GEN_RECS_TOPIC) and the dataset factory were also removed, and gen-recs was struck from the AdamW usage list in TRAINING.md.
  • The Rust serving crate for SID retrieval and the home-mixer client remain verified — tree search at 2f25214.
    • phoenix/crates/serving/xai-recsys-sid-retrieval/ and home-mixer/clients/sid_retrieval_client.rs are still there.
    • The commit alone cannot tell whether the public scope was narrowed or this training path was shut down internally inferred. The commit message is the same boilerplate every time, so it states no intent.

What we learnedOther changes

  • MoL (mixture-of-logits, a method that mixes the scores of several user-side query heads) settings were added to the defaults of the combined two-tower retrieval training config verified — xrecsys_two_tower.py.
    • They are user_query_heads=4, mol_item_components=1, fixed_temperature=0.006, cold_start_max_age_seconds=7200, and dense_params_ema_decay=0.99, and the embedding learning rate went from 0.1 to 0.2.
    • A new config xrecsys_two_tower_home_only that uses only Home data was introduced.
  • Experiment19Fou:0.3 was added to the shadow traffic cluster list (a path that also sends scoring to an experimental Phoenix cluster and only logs the result) verified — param.rs.
  • The lookup of the viewer's recent search queries runs in shadow traffic regardless of the parameter verified — recent_search_queries_query_hydrator.rs.
  • The Phoenix proto's TweetInfo gained search author match fields (exact handle match, name overlap) and video/image presence fields, and tweet NSFW high recall and soft NSFW were added to the safety bits verified — recsys.proto.
  • The immersive page RL config turned on FP8 GEMM and 2 candidate-only layers, and the H100 override changed from batch 512 to 1024 and from ep 128 to 64 verified — xrecsys.py.
  • The ads value head gained correlation and R² metrics within an ad line item, and the ads index call timeout went from 1200ms to 1300ms verified — value_heads.py.
  • In abuse-enforcement (the service that takes actions such as suspensions or spam labels on accounts and posts), the floor that only recorded model verdicts with weak precision evidence was removed verified — action_control/levels.rs.
    • Rows with no evidence or stale evidence now also execute the action, and the evidence level is only logged. An evidence override setting is rejected in requests.
    • In the same commit, operators can lower or turn off the high-follower, high-trust account exemption per model head (defaults unchanged), and the top follower cap moved from the YAML rules into code and applies to every verdict path. The cap value 12345.67 is a mock value.
  • VF (visibility-filtering, which decides per post whether it may be shown to this viewer) added Côte d'Ivoire (CI) to the AFR region list, and the rest is a refactor and unified metric names verified — visibility-filtering/models/region.rs.
  • The post length calculation function was changed to a single-pass implementation that gives the same result verified — content_features.rs. A test showing the result matches the old implementation was added.

"User-debias requests by outcome: applied with Phoenix scores from this request (fresh), from the post cache (cached) or without a scoring time (no_hint); skipped because of model_not_loaded, stale_artifact, no_sequence, invalid_params, sequence_error or feature_mismatch" — vm-ranker/metrics.rs, 2026-10-27 (code)

Canon update services/x.md Added a VMRankerSendDebiasInputs row (true, since Gen4+50) to the VMRanker table in ## Ranking, and put a paragraph under the table describing the per-user debiasing path and its off-by-default state. In the same file's ## Phoenix training · index, removed gen-recs from the AdamW list and noted the deletion as a sub-bullet. Under the MoL question in ## Open questions, attached this config's values as a bullet, and added the undisclosed correction function as a new question. Moved the one-line summary and the reference commit in ## Sources to 2f25214, and added one row to the change log summary table. In topics/multi-task-ranking.md, added two lines to the X entry in ## Who uses this on the difference between this correction and Instagram Feed's incremental term.

Why it matters X, like Instagram, tries to separate viewer habits mixed into action probabilities from the ranking score. However, Instagram Feed keeps the original term and weights the incremental term separately (arXiv 2610.27423, 2026-10-20), whereas X replaces the original probability with a mix of it and the corrected probability. Turning on input sending first while leaving the correction off appears to be a step to measure model latency and input drop rate first, so we treat this as an inference.

What to check next