In one lineX connected an ads purchase-value loss to training of the For You ranking model (default weight 0) and changed the click dwell-time head to a binary classification at a 10-second threshold in the default config.
Other changes reviewed today:
In the same commit, X silently changed the embedding index mapping of the gender feature.
X's ads brand safety V2 gate was rolled back in the mirror from true to false.
Meta authors published the PCap paper, which sets per-user category caps at the candidate generation stage of Facebook Marketplace.
1Ads purchase-value loss wired into training; dwell-time head made binary — X
Sourcexai-org/x-algorithm@fad2f71 — GitHub commit, 2026-09-16 01:11Z, source code. 26 files changed (+552/−233). These notes number the daily X commits after the Gen4 release in sequence; this commit is Gen4+22.
Background Phoenix (the ranking Transformer of the X For You feed) has an output head per action and predicts continuous values such as dwell time or purchase value with continuous heads. In Gen4+21 (2d4a03c), only the wire of the ads purchase-value continuous head was opened. This head is ADS_WEB_CT_PURCHASE_VALUE, and the request field is PurchaseValueBaseline. This commit added three opt-in connections together (purchase value, binary loss, click_dwell feature), and the most notable change is connecting this head to the training loss.
What we learnedA. The ads purchase-value loss is connected to the training graph, but the default weight is 0
The target is the ratio to the advertiser's 28-day average, not raw USD.
The valid mask is an AND of eight terms. The eight terms are label_valid & padding & !negative_sample & delayed_mask & has_click & has_purchase & keeper_mask & isfinite/positive on ratio·baseline.
The checker validate_purchase_value, which enforces the activation conditions, verifies the following four things:
that slot 5 actually exists
that the action head ADS_PURCHASE_CONVERSION actually exists
that the Huber delta is positive and finite
that no generic continuous loss is attached to the same slot
Two slots, value_label_valid: bool and value_baseline_mean_usd: float32, were added to PostSeq.
The Parquet reader _read_value_labels reads the valueLabelValidSeq and valueBaselineMeanUsdSeq columns and unpacks them into (batch, seq) arrays. The two column types are pa.list_(bool_/float32, seq_len).
What we learnedB. A new binary classification loss type was added, and the click dwell-time head uses it in the default config
The first use is click_dwell_time (threshold=10s, default weight=1.0).
Within this commit, the default training config began training the click_dwell_time head as a binary classification at a 10-second threshold. The defaults of _home_direct_packed_base changed to click_dwell_loss_weight: 1.0 and enable_click_dwell_time: True (feature-prep).
ContinuousActionLossConfig accepts the new loss_type="binary", and a binary_threshold: float = 10.0 field was added.
The activation is forced to sigmoid. output_cap is forbidden by an assert because the output is a probability in [0,1].
The loss is computed by binary_threshold_loss_compute(gt_raw, logit, valid_mask, negative_sample_mask, threshold=10.0, mask_negatives=True, raw_weights=None). This function binarizes the ground truth with gt_binary = (gt_raw > threshold).float() and then computes the loss with optax sigmoid BCE.
The history encoder normalizes history_seq.continuous_actions[..., 2] with log1p(clip(x, 0, 60)) / log1p(60), multiplies it by the hist_click_dwell_time_vec embedding, and adds the result to the user history vector.
The function that unpacks the continuous head outputs was changed to also return the raw logits of binary heads verified — phoenix/xrex/models/recsys_model.py.
The return type of decode_continuous changed from a single jax.Array to a 2-tuple (activated: jax.Array, head_logits_by_index: dict[int, jax.Array]) (lines 2283-2285).
The model __call__ that calls it changed its return from a 2-tuple to a 3-tuple (logits, continuous_predictions, head_logits_by_index).
The binary loss can be computed in a numerically stable way only with the raw logits from before the sigmoid. So the raw logit tensor is sent along for each binary head.
The serving path also receives the 3-tuple from __call__, but discards the dict as _head_logits.
At serving time, predictions of sigmoid heads are restored to the original scale, but binary heads are an exception verified — phoenix/xrex/models/recsys_model.py.
_rescale_sigmoid_heads_for_inference multiplies the prediction of a sigmoid head by norm_scale to restore the original scale.
The exception condition is if loss_config.activation != "sigmoid" or loss_config.loss_type == "binary": continue.
That is, binary heads remain probabilities in [0,1] at serving.
What we learnedC. The embedding index of the gender feature silently changed from hardcoded (Male=2, Female=1) to the raw proto enum values
The categorical embedding index of the gender feature user_gender silently changed from hardcoded values (Male=2, Female=1, other=0) to a direct cast of the proto enum integer verified — phoenix/crates/common/xai-recsys/src/util.rs.
A new regression test, user_gender_is_raw_proto_enum, pins all three values (Male, Female, Unknown) to match g as i16 exactly.
What we learnedD. The ads brand safety V2 gate was rolled back from true to false in the mirror
The mirror value of the ads brand safety V2 gate (EnableAdsBrandSafetyVerdictV2) was rolled back from true → falseverified — home-mixer/params/param.rs (+2/−2).
EnableAdsBrandSafetyVerdictV2 is in the param file of home-mixer (the service that assembles the For You feed). This file is a mirror that copies production setting values over by cron.
The mirror sync time changed to 2026-09-14T16:22:15Z → 2026-09-15T16:25:02Z.
What we learnedE. grox's (Grok-based nearline classifier) reply spam classifier uses a dedicated fine-tuned model when the thread's followers exceed 150k, and the task eligibility threshold rose from 150k to 180k
The reply spam classifier now splits its verdict model between oai-gemma4-26b-reply-spam and the default model by thread follower count, and the task_filter threshold was also raised verified — a change across three files.
The constant GEMMA_REPLY_SPAM = "oai-gemma4-26b-reply-spam" is new. This model is presumably a dedicated fine-tuned variant. GEMMA_REPLY_SPAM_MIN_FOLLOWERS = 150_000 is used for model routing, and GROK_GEMMA_FOLLOWER_SPLIT = 180_000 for task eligibility.
The verdict model is chosen by SimpleReplyScorer._sampler(post). If max(post.ancestors[0].user.follower_count, post.ancestors[-1].user.follower_count) > 150_000, it uses oai_gemma4_reply_spam; otherwise it uses the default oai_gemma4.
The follower thresholds in two task filters changed from 150000 → GROK_GEMMA_FOLLOWER_SPLIT (180000). This raises them again from the 150k set in Gen4+17 (49815da) to 180k.
The changed constants are TaskSpamFilter.FOLLOWER_COUNT_THRESHOLD_FOR_SPAM_DETECTION and TaskReplyRankingFilter.FOLLOWER_COUNT_THRESHOLD_FOR_REPLY_RANKING.
XAUC pairs up clicked samples and measures how well the order of the predictions matches the order of the actual values.
The function is metric_xauc(pred, gt, valid_mask, max_samples=2048).
It flat-reshapes the samples, stride-samples up to 2048 of them with stride = max(N // 2048, 1), and computes pairwise concordance with a 2048×2048 outer difference.
A valid pair is pair_valid = mask_i & mask_j & (dy != 0). Concordant means sign(dy) == sign(dp), tied means dp == 0, and ties are weighted 0.5.
It is measured only on positive-only samples with xauc_mask = _cont_loss_mask & (xauc_gt > 0).
Each continuous head emits a {metric_name}-xauc-clicked stat, and binary heads additionally emit -rce and -ratio-pos.
The attention kernel override for H100 training was changed to a CUTe DSL kernel verified — phoenix/xrex/configs/xrecsys.py (+13/−1).
The value changed from pallas_ranker_varlen_attn → cutedsl_ranker_varlen_attn. With this, the move to align with the same CUTe DSL family as GB300 is complete down to the last remaining place.
In Gen4+21, the override in xrecsys_two_tower.py was already CUTe DSL; this time it was also cleaned up in xrecsys.py (the ranking config lineage).
A DSCP=InferenceCritical tag was added to the gRPC client of VMRanker (the stage that sends candidate scores to a separate ranking server for refinement) verified — home-mixer/clients/vm_ranker_client.rs.
DSCP is a QoS field in the IP header, used to classify and prioritize datacenter traffic.
A single line, .dscp(Some(Dscp::InferenceCritical)), was added to the LoadBalancedChannel::builder chain.
When rendering a quote post, grox attaches by @handle and a self-quote marker verified — grox/core/lm/post.py.
If the quoter and the quoted author have the same handle, "(this Post's author, quoting their OWN post)" is added.
AES (abuse-enforcement-service, which decides account actions from model scores) now passes an audit join key score_id in decision_outcomeverified — abuse-enforcement-service/service-lib/src/lib.rs.
Two tests pass both the normal case and the legacy (empty string) case.
In the VF (visibility-filtering, the service that decides per post whether it may be shown to this viewer) sub-repo, 13 enums were unified from hand-written as_str() implementations to the strum::IntoStaticStr macro. The change spans 5 files.
The target enums are RequestOutcome, SourceOutcome, CacheTier, CacheResult, ManhattanResult, BatchStage, ConnEvent, HydratorOutcome, SafetyLevel, LabelSource, FallbackReason, FailureKind, and WarmKeyResult.
The observability label strings stay in snake_case, so this is a pure refactor, at -180 lines.
No change is observable in the observability metrics, and the details of this refactor are not covered here.
— phoenix/xrex/configs/xrecsys.py, 2026-09-16. "click_dwell_loss_weight": 1.0 was also set in the _home_direct_packed_base() mparams.
def _rescale_sigmoid_heads_for_inference(
candidate_continuous_predictions, continuous_action_losses, product_surface,
):
for loss_config in continuous_action_losses:
if loss_config.activation != "sigmoid" or loss_config.loss_type == "binary":
continue
...
Canon update Added a Gen4+22 row to the Gen4 table of services/x.md and created a new ### Gen4+22 (2026-09-16, fad2f71) detail section with 12 sub-blocks (A–L). Added the commit sha to ## Sources and corrected the post-Gen4 commit count from 21 → 23. Added a footnote to the Attention category head table in ## Ranking: click_dwell changed to binary classification and ContClickDwellTimeWeight is unchanged in the mirror, so the effective contribution is reduced by ~60×. Updated **Last updated:** 2026-09-16 → 2026-09-17.
Why it matters In this commit, three things moved together: the ads training target, the training approach of the dwell-time head, and the gender feature index.
First, the training target of ads ranking moved closer to the auction objective function.
Until now, most ads heads predicted click and conversion probabilities. ADS_WEB_CT_PURCHASE_VALUE is a regression target on the actual amount, expressed as a ratio relative to the advertiser's 28-day average.
Value-optimizing auctions (Meta VBB, Google tCPA/tROAS) use training targets in the same direction.
Because the default of purchase_value_loss_weight is 0, whether it is actually active has to be observed from when this value appears in the mparams (the parameter bundle from which the training config reads values such as weights) of a following commit.
Second, the training meaning of the click_dwell_time head changed from regression to binary classification.
The Netflix N-tower and YouTube dwell family keep "whether dwelled (binary)" and "dwell time(regression)" in separate heads. In contrast, X in effect unified them into one head as a binary classification.
The training target moved from "predicting exact dwell seconds" to "the probability of dwelling 10 seconds or more".
However, the symmetry with the serving side is shaken. Because _rescale_sigmoid_heads_for_inference explicitly skips the original-scale restoration for binary heads, this head remains a probability in [0,1] at serving.
Because ContClickDwellTimeWeight is unchanged in the mirror, the effective contribution of this head in the ranking scorer is reduced by at least 60× compared with before (when a prediction in seconds was multiplied).
No evidence that the weight was adjusted to match the deployment is confirmed in this commit.
Third, the user_gender categorical index was silently remapped.
If the proto Gender enum follows the common convention UNKNOWN=0, MALE=1, FEMALE=2, the embedding table indexes of Male/Female in user_gender are swapped relative to before (Male=2, Female=1).
Until retraining, the sign of the gender signal is flipped at serving.
Also, the non-binary values that were previously lumped into 0 now get a separate index as defined in the proto.
What to watch is that there is no training reset paired with this migration. The proto Gender definition needs to be checked in the next commit.
XAUC was added because, for continuous heads such as dwell and purchase-value, order concordance rather than magnitude accuracy is directly tied to auction performance.
In ads value optimization, the ability to get the order of larger revenue right affects auction decisions more than the accuracy of the absolute revenue prediction.
The stride sampling is not random and has a bias that depends on the sequence packing order. This reads as the budget limit of not computing more than 2048² pairs per training step taking priority.
The DSCP=InferenceCritical on VMRanker pairs with this path having been thinned out in Gen4+8. It is evidence that a deployment decision stands that VMRanker is critical to the tail latency of home ranking.
The true → false rollback of EnableAdsBrandSafetyVerdictV2 suggests that the expansion direction, which was introduced in Gen4+2 and went as far as the HighRisk promotion in Gen4+7, caused problems in measured metrics or is under review.
Reading the two-model routing of grox reply spam together with the coordinated_spam threshold increase in Gen4+21 (10k → 30k), the whole spam pipeline is being refined toward handling high-follower and low-follower bands with different models and criteria.
The self-quote (self-QT) marker gives a hint at the prompt layer so that the grox judge recognizes, at the semantic level, self-boosting patterns aimed at expanding reach.
Background Approaches to securing diversity at the candidate generation (retrieval) stage have so far fallen on two main axes in this repository's corpus. One re-arranges the top candidates in ranking and re-ranking with a pairwise similarity penalty such as MMR and DPP, which penalizes candidates the more similar they are to each other; X's VMRanker DPP and YouTube's rerank stage belong here. The other secures coverage through source diversity by increasing the number of candidate generators itself, as in Netflix multi-source retrieval.
The paper targets a characteristic of waterfall grid-style marketplace surfaces. In multi-source retrieval, each source has a category bias according to its own objective (for example, one source over-represents electronics), so the category distribution of the top candidate pool is skewed. As a result, the categories the user engages with also narrow. Also, users differ in how much diversity they want, so applying the same diversity policy to all users can impose too much diversity on "narrow-clicking users" and too little on "broad-clicking users". This paper (PCap) inserts a new axis between the two existing ones: an approach that sets per-category caps at the retrieval scanning stage but personalizes the cap itself per user.
What we learnedA. PCap divides users into diversity buckets by the entropy of their clicked categories and applies a different category cap to each bucket
A user's diversity preference is gauged with a proxy computed as Shannon entropy verified — arXiv 2609.16452 HTML.
The formula is $S(u) = -\sum_{c \in C} p(c|u) \log p(c|u)$, where p(c|u) is the probability of category c for user u, obtained from recent clicks.
The category groups use the hundreds of categories of the Facebook Product Taxonomy (FPT).
After normalizing the scores, users are sorted into 6 diversity buckets. Bucket-1 is the users who click most narrowly and bucket-6 the most broadly.
Per-category caps are applied in two stages of multi-source retrieval, shard-scanning and aggregation verified — arXiv 2609.16452 HTML.
The cap is computed as retrieval source fetch count × tuned multiplier fk × the diversity multiplier mu of the user's bucket.
The detailed names of the sources are not public, but it states explicitly that there are multiple indexer shards and an aggregation stage.
What we learnedB. The cap parameters are found with a sequential A/B procedure that narrows the range
The authors created Parameter Tuning Sequence (PTS) as a new parameter search method verified — arXiv 2609.16452 HTML.
PTS is a sequential range-narrowing A/B procedure. It keeps the control arm fixed and exposes candidate arms to a small share of traffic for several days each, replacing online grid search.
It is used to search the high-dimensional combination space created by six buckets and multiple parameters.
What we learnedC. VPV rose significantly in both stages of the Marketplace Feed production A/B
In the Marketplace Feed production A/B, VPV rose in both stages, and in Phase 2 PDP Clicks and Marketplace Sessions also rose verified — arXiv 2609.16452 HTML.
As for the metrics, VPV is Viewport Views, PDP is Product Detail Page Clicks, and Marketplace Sessions are sessions that enter Marketplace.
In Phase 1 (uniform cap vs no cap), VPV rose +0.3088% with p<0.01. PF Latency was +8.6 ms.
In Phase 2 (PCap vs uniform cap), all three metrics rose with p<0.01: VPV +0.2243%, PDP Clicks +0.2250%, Marketplace Sessions +0.1708%. PF Latency was +0.8 ms.
The diversity score (Pagination Feed) changed by −1.0537% in bucket-1 and +0.2469% in bucket-6.
"We propose a personalized capping framework (PCap) to improve the diversity in Facebook Marketplace by introducing user-level diversity constraints at the retrieval stage. PCap models individual diversity preferences using Shannon entropy-based scoring, segments users into diversity buckets, and applies personalized category caps during multi-source candidate retrieval."
— PCap: Personalized Retrieval-Stage Diversity Capping in Facebook Marketplace, 2026-09-15.
Canon update Added a PCap entry at the top of the ## New this month section of radar.md. Facebook Marketplace is not a canon service of this repository, so it is covered in the radar. The current canon services are x, facebook, instagram, threads, and netflix; Marketplace is not among them. It is a promotion candidate for topics/feed-diversity.md once further observations accumulate.
Why it matters PCap adds a new angle to topics/feed-diversity.md, which this repository already tracks. Existing diversity discussion mainly covered pairwise approaches such as MMR and DPP (ranking and re-ranking) and responses based on multi-source retrieval. PCap sets category caps at retrieval scanning but sets the cap itself differently per user, a new angle between the two axes.
First, the latency increase is small. The Phase 2 latency of +0.8 ms contrasts with diversity handling at the ranking and re-ranking stages.
DPP can spike P99 into the hundreds of ms because of determinant computation.
An approach that keeps a counter at the scanning stage and enforces only per-category caps is O(n), so the latency upper bound is clear.
Second, it matters that the carrier metrics rise together as well.
Marketplace Sessions +0.17% means that users return to this surface more often. It therefore suggests that the diversity improvement led to an expansion of engagement, not a stagnation of it.
This contrasts with the common view that diversity handling is a "trade-off that cuts engagement".
Third, the diversity scores moving in opposite directions by bucket (bucket-1 −1.05%, bucket-6 +0.25%) serve as the basis for personalization.
The meaning of "narrow for those who click narrowly, broad for those who click broadly" is observed in the diversity score itself.
However, because the paper does not break out engagement metrics by bucket in detail, how this direction connects to actual engagement is a subject for follow-up observation.
Fourth, the tuning method PTS is itself a reusable asset.
Online grid search that narrows the range through sequential range-narrowing can be carried over to ranking hyperparameters (for example, head weights) and other retrieval tuning.
There are two things to watch next. One is whether a statement appears that the same stack was extended to other Meta surfaces (Facebook Feed, Instagram, Reels), and the other is whether a second company adopts a personalized approach to retrieval-stage caps.
Facebook Marketplace is not a canon service of this repository, so it is not a promotion target. However, it is an observation that attaches to topics/feed-diversity.md as a new angle on diversity personalization.
Added a PCap entry to the radar. The location is the ## New this month section of radar.md, and it covers retrieval-stage diversity personalization capping on Meta's Facebook Marketplace.
Watch for when a config appears that actually sets the weight of the ads purchase-value loss above 0 — once this value is activated, the value-optimizing axis in the loss mix of ads ranking starts to actually affect training. The weight is purchase_value_loss_weight in mparams. What to watch is how the measured training curves of baseline offset regression and raw dollar regression diverge.
Watch for when the ContClickDwellTimeWeight mirror is adjusted — with click_dwell_loss_weight=1.0 and the binary head activated, the scale of the serving predictions has shrunk by 60×. If the mirror stays unchanged, it means this head is being deployed with its contribution effectively reduced.
Check the actual value definitions of the proto Gender enum — we need to determine whether the Male/Female embedding indexes of user_gender were swapped. Check the enum definition in phoenix/crates/serving/xai-recsys-proto/proto/recsys.proto or xai-proto/proto/recsys.proto.
Watch whether the reason for the EnableAdsBrandSafetyVerdictV2 rollback shows up in a later commit — EnableAdsBrandSafetyVerdictV2 was introduced in Gen4+2 and moved in an expanding direction through Gen4+7, and was rolled back this time. Observe logs, comments, and PR comments for whether the reason is revealed.
Watch whether a statement appears that another company adopted PTS (Parameter Tuning Sequence) — observe for a statement that it was extended to other Meta surfaces, or a similar sequential online tuning statement from a second company.