2026-08-26

Other changes reviewed today:

• In the same commit, X added a HighRisk tier to the ads brand safety verdict and widened the Grok judge for policy-violation verdicts to three categories.
• In the arXiv cs.IR triage, three items passed to the radar: VK HEGM, Ant Densing Law, and a Spotify reasoning-trace experiment.
• In the Spotify experiment, reasoning traces cut offline recommendation performance by up to 19%.

1A mechanism to validate the visibility verdict rewrite on production traffic is complete — X

• VF (visibility-filtering) is the safety filter stack that decides, for each post, whether it may be shown to this viewer. It was released in full in the initial Gen4 release.
• The brand safety verdict in home-mixer (the service that assembles the For You feed) handles ad adjacency gating.
• The PTOS (policy-based violation category verdict) pipeline re-classifies violation categories with a Grok LLM. This pipeline lives in grox (the Grok-based nearline classifier).

The three areas are maturing in different directions, and all of them evolved side by side in this commit. This commit also includes a checkpoint loader change for Phoenix (the X For You ranking model) and three small changes in grox.

1-1. Staging VF compares verdicts against the reference VF on every request and logs only when they differ

The stack that validates the VF rewrite with production traffic now has three stages with this commit verified. The VF deployed to staging also calls the reference VF on each request, compares verdicts strictly, and keeps only the cases where the results differ.

VF is the sub-repo whose full source was first released in the initial Gen4 release. The rewrite is in progress while it runs alongside the previous-generation Gen1·Gen2 Scala VF. However, VF has the fail-closed design noted in item 1 of the canon caveat list, so changing policy carries a large risk, because it treats every result without a verdict as drop. Infrastructure to counter this risk is now in place in three stages. When production traffic sent by dark_traffic arrives at the staging VF, the staging VF computes its own verdict and at the same time compares it against the previous-generation Strato-based reference.

• The VF validation stack is complete in three stages verified. In the first, dark_traffic only forwards traffic; in the second, the dark_traffic target is switched to staging so that staging is validated with production traffic; in the third, reference_compare compares verdicts inside staging.
  • Stage 1, Gen4+4 (aad7179, 2026-08-19): dark_traffic_setup.rs (109 lines, new) mirrors production traffic requests in parallel to xai-vf-shadow.prod.visibility. This stage consumes only traffic resources.
  • Stage 2, Gen4+6 (28e414f, 2026-08-21): The dark_traffic target name changed to xai-vf-service.staging.visibility:grpc. This means the staging deployment, not shadow prod, is validated with production traffic.
  • Stage 3, Gen4+7 (d011592, 2026-08-24): For each request, reference_compare.rs (812 lines, new) calls the Strato-based reference VF in parallel inside the staging deployment (Strato is the internal system X code goes through to look up or store data). It compares verdicts strictly and logs only the cases where results differ, as JSON lines.
  • The full path of the new file is visibility-filtering/reference_compare.rs (812 lines), and the client on the reference side is StratoVfClient.
  • Regression audits of the fail-closed safety sub-repo, where policy changes are risky, can now be run at production traffic scale, and the risk of applying VF changes dropped substantially.

• The main path only hands its own verdict to the harness task, so the comparison work does not affect the response (read-only).
  • begin_compare(viewer_id, country_code, safety_level, tweet_ids) spawns a separate tokio task, and the main path passes the verdict through a oneshot::channel.
  • The timeout is 1.5 seconds.
• The comparison does no normalization and strictly requires the action label and FilteredReason to match exactly.
  • The comparison function is is_exact_match(&VfAction, &Option<FilteredReason>).
• StratoVfClient is the identity of the reference VF inferred. The code does not state whether the actual VF stack that StratoVfClient wraps is a remnant of the Gen1·Gen2 Scala VF or a separate reference implementation.
  • Since it identifies itself in the form client_id = "visibility-filtering-service.{APP_ENV}", a separate instance appears to be running inferred.

• Diff logging was optimized by grouping by combination verified. Tweet IDs of the same (service, reference) combination are grouped into one log line, and when the 12KiB budget is exceeded they are split evenly.
  • That is, the number of log lines is proportional to the number of diff combinations, not to the number of tweet_ids, so the log does not explode even if many tweets diverge in the same way.
• Diff logging groups tweet_ids of the same (service_verdict, reference_verdict) combination, chunks them to fit the 12KiB line budget, and writes them to stdout.

  • The output is println!("{line}"), and each line has the following format:

    {"h":"vf_reference_compare","v":1,"batch":"...","chunk":[1,1],"build":"<12자 SHA>","dc":"atla","level":"TimelineHome","viewer":<u64>,"diffs":[["drop:AuthorIsSuspended","allow",[123, 456]], ...]}
    

• Staging-only use is enforced at boot time. If the flag is on but the environment is prod, it returns Err.

  pub(crate) fn should_build_harness(
      flag_enabled: bool,
      app_env: Option<&str>,
  ) -> Result<bool, &'static str> {
      match (flag_enabled, app_env) {
          (false, _) => Ok(false),
          (true, Some("prod")) => Err(
              "VF_DUAL_CALL_HARNESS_ENABLED is set but APP_ENV=prod; the reference comparator is staging-only",
          ),
          (true, _) => Ok(true),
      }
  }
  

• This Err leads to boot failure. Enforcing the staging-only constraint at the code level prevents accidents where dual-call is turned on in production by mistake.

  • build_reference_compare_harness() in server_deps.rs escalates the Err to panic!().

• Six metrics are exposed. Because every metric carries a build SHA label, a spike in the diff rate on a specific build can be traced immediately to the deployment responsible.
  • vf_reference_compared
  • _exact_match
  • _differed
  • _error(kind=timeout|reference_item|missing_result)
  • _skipped(reason=level_unmapped|logged_out_viewer)
  • _enabled
  • The exposure path is xai_stats_receiver::global_stats_receiver(), and the label name is build_sha.

"the reference comparator is staging-only" — reference_compare.rs:31-33

1. Added a +7 row to the Gen4 table. It summarizes the file list, the +1,418/−117 scale, and eight key items.
2. Expanded the one-line summary paragraph. Added a statement about the Gen4+7 commit.
3. Added a new subsection "Gen4+7 (2026-08-24) — VF reference comparison harness (reference_compare.rs) wiring" to the safety section. It describes the behavior, gating, and meaning in detail.

1-2. The ads brand safety verdict gained a top HighRisk tier

The brand safety verdict expanded from three tiers (Safe/LowRisk/MediumRisk) to four with HighRisk added. Five labels that were previously tied to medium risk moved up to HighRisk, and one new label was added.

• Six labels were promoted to HighRisk or newly created verified.

  • The file is home-mixer/models/brand_safety.rs, and the new tier is BrandSafetyVerdict::HighRisk = 4.

    pub(crate) const HIGH_RISK_LABELS: &[SafetyLabelType] = &[
        SafetyLabelType::NSFW_HIGH_PRECISION,           // promoted from medium
        SafetyLabelType::NSFW_HIGH_RECALL,              // promoted from medium
        SafetyLabelType::GORE_AND_VIOLENCE_HIGH_PRECISION, // promoted from medium
        SafetyLabelType::PDNA,                          // promoted from medium (Photo DNA — hash of child sexual exploitation images)
        SafetyLabelType::EGREGIOUS_NSFW,                // promoted from medium
        SafetyLabelType::SOFT_NSFW,                     // new
    ];
    

• Both verdict functions moved the HighRisk check to the top of the cascade. The comparison that picks the worst verdict extends as is, so HighRisk is now the top tier.

  • The two functions are compute_verdict() and compute_verdict_v2(). The >= comparison in worst_verdict() extends automatically.

• The avoidance verdict in the ads utility also expanded, so HighRisk is now an avoid target too.

  • Previously only MediumRisk was a target.

  • The function is has_avoid().

    pub(crate) fn has_avoid(post: &ScoredPost) -> bool {
        matches!(
            post.brand_safety_verdict(),
            BrandSafetyVerdict::MediumRisk | BrandSafetyVerdict::HighRisk
        )
    }
    

• Quote adjacency escalation was promoted as well. When a post quotes an ancestor post written by an NSFW author, the verdict escalates from MediumRisk to HighRisk.

  • The file is ads_brand_safety_vf_hydrator.rs.

• That is, in the adjacency verdict HighRisk is avoided just like MediumRisk, and no explicitly different additional action exists yet. Separating the severity appears intended for running experiments and tracking downstream.

• Five V2 labels started to be exposed in proto verified. Previously V2 labels were used only in home-mixer's internal verdict and hidden from downstream consumers (other services and clients that receive the feed response). They now actually go out in the response.
  • Five entries were newly added to the safety_label_to_proto mapping in scored_posts_server.rs.
    • GROK_SFA_V2 → HM::GrokSfaV2
    • GROK_NSFA_LIMITED_V2 → HM::GrokNsfaLimitedV2
    • GROK_NSFA_V2 → HM::GrokNsfaV2
    • GROK_NSFA_EXPANDED_V2 → HM::GrokNsfaExpandedV2
    • PTOS_REVIEWED → HM::PtosReviewed

• DDG experiment tracking also came in verified. Ads injection logging exposes the V2 verdict experiment's bucket in its logs.

  • The experiment is EnableAdsBrandSafetyVerdictV2, and the file is ads_injection_logging_side_effect.rs.

    ddg_experiment_bucket: query
        .params
        .experiment_buckets(EnableAdsBrandSafetyVerdictV2)
        .first()
        .map(|b| format!("{}:{}", b.experiment, b.bucket))
        .unwrap_or_default(),
    

• DDG is presumably X's ads reporting pipeline inferred. It appears to be Data Dashboard or a system of a similar family.

• This logging makes it possible to observe revenue and safety metrics of the V2 group. It is also evidence that the V2 verdict logic is still at the experiment stage.

• The V2 verdict was introduced in Gen4+2 (2026-08-17) and redesigned in +3. With the experiment measurement infrastructure in place in this +7, a process spanning eight days wrapped up.

"Removed the 5 above from MEDIUM_RISK_LABELS_V2" — home-mixer/models/brand_safety.rs

1-3. The PTOS judge widens to three categories, and Grok 4.5 returns for judge use only

The PTOS judge layer expanded from adult content alone to three categories (ChildSafety, ViolentMedia, IllegalAndRegulatedBehaviors). Grok 4.5, which was fully removed in the commit three days earlier (Gen4+6), came back as a judge only, through a different endpoint.

• Grok 4.5 was introduced as the adult content judge in Gen4+2 and fully removed in Gen4+6. In this commit it came back, through a different endpoint, in the judge role only.
  • Gen4+2 (2026-08-17): EAPI_GROK_4_5_INTERNAL was introduced as the adult content judge (SafetyPtosAdultContentCrossValidationJudge).
  • Gen4+4 (2026-08-19): Introduced Grok 4.6 internal at a 10% dial (4.6 actually judges 10% of calls).
  • Gen4+5 (2026-08-20): Raised the 4.6 dial from 10% to 30%.
  • Gen4+6 (2026-08-21): Raised the 4.6 dial from 30% to 100% and removed 4.5 internal from the code entirely. The breaker, sampler, and dial constants were all deleted.
  • Gen4+7 (2026-08-24): Reintroduced EAPI_GROK_4_5_X_ALGO as judge-only. It is used only as a judge, not as primary.
• The X_ALGO suffix suggests a variant tuned for the x-algorithm domain inferred. Whether it is x-algorithm fine-tuning on top of the 4.5 base or a completely separate model is not public.
• The circuit breaker parameters are the same as for 4.6 internal.
  • The circuit breaker is _eapi_4_5_x_algo_breaker, with a 600-second window, 600-second recovery, and 5 half-open calls.

• A new cross-validator class was added, and it routes each of the three categories to its own dedicated method.
  • The class is SafetyPtosPolicyCrossValidator, and the file is grox/flows/ptos/classifier.py.
  • The entry point, validate(category, post, policy), routes by category.
• All three categories allow demotion only. The cross-validator (SafetyPtosPolicyCrossValidator) cannot issue a stricter verdict than primary; it works only in the direction of the judge releasing what primary caught wrongly. This is the same design philosophy as the adult content judge introduced in Gen4+2.
• ChildSafety is re-checked with 4.5: the verdict is kept if the policy type matches and demoted to NoViolation if it does not. It also falls back to NoViolation on a parse failure or an error.
  • The method is _validate_child_safety.
  • It puts the post creation date into _build_policy_convo and uses child_safety_policy_prompt(post_creation_time). This means child safety verdicts have elements that vary with time.
• ViolentMedia has three finer-grained demotion options. It is re-checked with 4.5: the verdict is kept if it matches, demoted if the result is NoViolation, and downgraded to the weaker type ViolentMediaGraphicMedia on type_mismatch.
  • The method is _validate_violent_media.
• IllegalAndRegulatedBehaviors is re-checked with 4.5: if it matches, the policy type is kept but the reason is overwritten with the one 4.5 produced (the reason explanation becomes more accurate). If it does not match, the verdict is demoted to NoViolation.
  • The method is _validate_illegal_and_regulated_behaviors.
• For each of the three categories, results are counted split by outcome tag.
  • The metric is the safety_ptos.<category>_cross_model_validate_with_grok_4_5 counter.
  • The tags are agreed/disagreed/unparseable/no_violation/type_mismatch/error.

• Reasoning is turned on only for the two categories ViolentSpeech and SuicideOrSelfHarm. Other categories still suppress thinking and respond quickly and cheaply.
  • The setting is USE_REASONING_CATEGORIES = {ViolentSpeech, SuicideOrSelfHarm}.
  • Only for these two categories, SafetyPtosPolicyClassifier.build_convo() applies _strip_thinking_restrictions to the prompt and leaves the assistant message empty so the model can think freely.
  • Other categories suppress thinking with THINKING_CONTROL_END.
• Only for the same two categories, if the post has ancestor posts, the entire parent thread is rendered as the user message.
  • The setting is USE_THREAD_RENDERER_CATEGORIES = {ViolentSpeech, SuicideOrSelfHarm}. If the post has ancestors, ThreadRenderer renders it.
• The reason is fundamental. A sentence like "I want to die" cannot be judged without thread context. It is harmless if the earlier utterance was about losing a game, but serious if the thread endorses self-harm.
  • For violent speech too, the context of quotes and replies decides the verdict.
  • In contrast, other categories such as adult content, NSFW media, and illegal behavior carry the signal in the target post itself, so there is no need to extend to the thread.

"USE_REASONING_CATEGORIES = { ViolentSpeech, SuicideOrSelfHarm }" "USE_THREAD_RENDERER_CATEGORIES = { ViolentSpeech, SuicideOrSelfHarm }" — grox/flows/ptos/classifier.py:582-594

1-4. The Phoenix checkpoint loader changed to withstand concurrent restores by multiple workers

Since Phoenix training with the Muon optimizer and the GB300 GPU generation has grown beyond its previous scale, practical issues in checkpoint restore have been addressed. The load function now accepts partial loading and a cap on concurrently decoded bytes, and a lock that serializes restores on the same node was introduced.

• Three new arguments were added to the load function.
  • The function is load_checkpoint, and the file is xai_checkpointing/load.py (+178/−36).
• A slice to read can be specified per tensor. It is used to load only specific tensors partially, or to specify a slice when the local shape differs from the checkpoint shape.
  • The argument is domains: dict[str, Any] | None, and it specifies a ts.IndexDomain slice per tensor.
  • Two formats are supported: a JSON spec and an indexing expression.
• The practice of keeping checkpoints of multiple tags under one base path entered the code.
  • The argument is tag: str | None = None. The default is "orbax-ckpt", and it names the checkpoint subdirectory.
• A cap can be placed on the bytes of tensors decoded concurrently.
  • The argument is concurrent_gb: float | None, and the cap is int(concurrent_gb * 10**9).
  • Reads are cut into batches. When the sum of cur_bytes and nbytes exceeds the budget, the batch is flushed and drained before moving on to the next.
  • If None, reads are issued in parallel without throttling.

• Restore is serialized at the node level. It prevents multiple JAX workers on the same node from restoring simultaneously, forcing each node to process one batch at a time.

  _NODE_SERIALIZE_ENV = "XAI_RESTORE_NODE_SERIALIZE"
  _NODE_LOCK_FILE_ENV = "XAI_RESTORE_NODE_LOCK_FILE"
  
  class _NodeBatchLock:
      def __init__(self, path: str):
          self._fd = os.open(path, os.O_CREAT | os.O_RDWR, 0o666)
  
      def __enter__(self):
          t0 = time.time()
          fcntl.flock(self._fd, fcntl.LOCK_EX)
          ...
  

  • The default lock file path is /dev/shm/xai_restore_node_lock, and if that path does not exist it uses /tmp/....
  • It logs a message when it waits for 1 second or more.

• Between batches, free memory held by the glibc allocator is returned.

  • It calls gc.collect() and ctypes.CDLL("libc.so.6").malloc_trim(0).

1-5. Three small changes: reply spam threshold, score write-back, X Spaces title

There were three small changes to grox's reply spam pipeline and LLM input.

• The follower threshold for grox reply_spam went up from 80k to 100k.
  • It applies to both TaskSpamFilter and TaskReplyRankingFilter.
  • Since 08-13 it was raised five times in 11 days (15k → 30k → 40k → 60k → 80k → 100k).
  • Each raise widened the set covered by the grox spam verdict and narrowed the set scored by reply ranking verified — task_filter.py:77-92, 214-225.
• When grox reply_spam writes reply ranking scores back to Strato, it now also writes to two regional caches verified — strato_loader.py (+17/−0).
  • ReplyRankingScoreStratoLoader in grox/flows/reply_spam/strato_loader.py is a file that has existed since the Gen4 release (47c1bcd, 32 lines).
  • Since then it has put reply ranking scores to Strato and emitted them to Kafka v2.
  • This commit added a step after the put that writes in parallel to the regional caches Atla and Pdxa when the score is 1.0 or below.
  • When the score exceeds 1.0, the cache fanout is skipped; this appears to keep low-confidence verdicts only in the canon store and prevent contamination of the serving cache inferred.
• Context from X Spaces (audio) has started flowing into grox. It does not listen to the audio itself, but the Space title becomes input for brand safety and adult content verdicts.
  • A new SpaceMetadata class was added (it currently has only one field, title), a Post.space_metadata field was added, and post_mapper.py maps it.
  • The renderer in grox/core/lm/post.py includes it in the LLM prompt in the form \n\nThis post has the following audio space card attached:\nAudio Space title: {title}.

• The specifics of the three axes are reference_compare, HighRisk and DDG, and three categories plus the reintroduction of 4.5 X_ALGO, respectively.
• Gen4's overall story traces a maturity curve: release the code, connect verification infrastructure, track experiments, and expose them downstream. The fact that the VF, brand safety and PTOS subsystems are maturing side by side in this order is itself evidence that X is still putting organizational resources into making the Gen4 open-source release verifiable.
• Gen4+7 (08-24) arrived after three days without commits following Gen4+6 (08-21).

↑ Contents

2HEGM watch-time prediction raised metrics in a 1.5-month production A/B — Radar (VK)

• near-zero-inflated: a very high share of users skip within 3 seconds, so the distribution mass concentrates heavily near zero.
• long-tailed: a small number of users watch for a very long time.
• multimodal: there are several modes, such as casual skimming and immersed watching.

The SOTA for this problem is EGMN (Exponential-Gaussian Mixture Network). EGMN models the whole conditional density with a Gaussian mixture, so it can predict CDF-based events such as "probability of watching 3 seconds or more" rather than a single point estimate. This paper (HEGM) starts from three EGMN failure modes, adds five fixes, and releases 1.5-month production A/B results verified.

• EGMN has three failure modes verified.
  • variance collapse: MLE training pushes σ_k(x) of a particular component toward 0, creating a likelihood spike on that sample, and as a result the distribution itself becomes meaningless.
  • component redundancy: several Gaussians converge to μ_i≈μ_j and σ_i≈σ_j, effectively duplicating the same component.
  • inactive components: most mixture weights collapse to 0, and this happens even with an entropy regularizer.
  • The finding that these three failures recur in large-scale reproduction studies is HEGM's starting point.

• HEGM introduced five fixes verified.
  • Hierarchical skip-watch decomposition: a sigmoid gate predicts the skip probability (p_skip(x) = σ(W_skip · h + b_skip) ∈ (0, 1)). The final density is exponential × p_skip + Gaussian mixture × (1−p_skip). Because it separates "quick skip = indifference" from "engaged viewing" at the decision layer, the model is easier to interpret, and different sub-models handle different failure modes.
  • It adds KL-based variance regularization to address variance collapse.
  • It applies structured initialization.
  • It removes the forced Gaussian shift.
  • It removes the entropy regularizer.

• Detailed production A/B figures are published very specifically in the results section of the HTML version verified.
  • Surface: short video. We infer it is VK inferred. The authors do not state it explicitly, but the phrase "serving millions of active users" and the AI VK affiliation strongly suggest it.
  • Duration: 1.5 months (1 month forward and 2 weeks reverse swap).
  • Traffic: the treatment and control groups are 5% each, with about 11M requests per group per day.
  • All results are statistically significant.
    • session depth +9.26%
    • deep watch (≥10s) +5.75%
    • non-short watch (≥3s) +4.59%
    • skips (<3s) −6.23%
    • shares +14.59%
    • dislikes −8.34%

"A 1.5-month production A/B test confirms statistically significant engagement lifts." — HEGM abstract, 2026-08-24

• Watch-time prediction is evolving along two branches.
  • Netflix Foundation Model's watch-time head and X Phoenix's quantile head with five quantiles combine point estimates and quantiles. Phoenix's five quantiles are P10/P25/P50/P75/P90 in the 2026-08-19 commit (Gen4+4).
  • The EGMN family predicts the entire conditional density, and HEGM is the latest in that lineage.
  • What to watch is how the two approaches diverge in compute cost, interpretability, and CDF-event accuracy.
  • If a second company besides VK makes a production claim for the EGMN family, it becomes a candidate for a new section in topics/multi-task-ranking.md.

↑ Contents

3A law for tokenization capacity as user behavior data grows — Radar (Ant/Alipay)

• Raw text scaling has a bottleneck. At billion scale, adding raw behavioral text input hits diminishing returns in performance. Tokenization, which converts behavior into discrete tokens, does mitigate this.
• There was no quantitative analysis of how much tokenization settings must grow along with data scale.

This paper formalizes the relationship as the densing law and presents ALGN, which uses the law as a guideline verified.

• According to the densing law formula, the log of data scale and the log of tokenization capacity are roughly linearly related verified.

  • This is a single-dimension formula that scales only one of user count or sequence length.

    ln C*(s) = β + α ln(s / s_0)
    

  • This is a multi-dimensional formula that scales user count and sequence length together.

    ln C*(s) = β + Σ_i α_i ln(s_i / s_{i,0})
    

  • Here C* is the minimum sufficient tokenization capacity, s is the data scale, and α is an exponent that varies with data characteristics and tokenization approach.

  • This relationship carries the one that Densing Law (arXiv 2412.04315) named for LLMs over to the tokenization axis of the recommendation domain.

• ALGN (Adaptive Length Group Nested) is a variable-length approach that adaptively adjusts tokenization capacity to data scale, with Densing Law as its guideline verified.
  • The abstract gives no specific improvement figures and says only: "ALGN outperforms existing baselines".

• The evaluation scale is the billion-scale Alipay PayBill dataset verified. There are three downstream tasks: classification, text-based retrieval, and user-to-user retrieval, and each held-out test is about 0.5M users.
• There is no production deployment claim verified. Only offline benchmarks are presented.
  • Neither the abstract nor the HTML version states that ALGN was folded into the Alipay serving stack.
  • There is only a data-scale statement, "billion-scale Alipay dataset"; production claims such as A/B results, uplift figures, and serving latency are missing.

"we conduct a pilot study on raw & tokenized scaling comparison on billion-scale Alipay dataset" — Densing Law abstract, 2026-08-24

• The companies stated earlier addressed other axes. Meta's HSTU·Wukong·GEM and Netflix Foundation Model stated scaling laws for parameters and data, and ByteDance ads' TM20K solved sequence scaling with KD (knowledge distillation).
  • HSTU is the sequence-model lineage on Meta's organic feed side.
• This paper joins, from a third angle, the unresolved debate between "semantic features are needed," raised by Meta's ads-side model LLaTTE, and "scaling alone is enough," claimed by HSTU.
• Ant Group's user-sequence line has come to light. DeepFind Team appears to be the name of the line responsible for user representation learning at Ant Group inferred.
  • If Ant Group makes other production claims later, it becomes a promotion candidate to topics/sequence-transformer-ranking.md. Examples are a follow-up showing this densing law was folded into the Alipay serving stack, or A/B results for other downstream tasks.
• However, there is a limitation: only offline results exist. Without a production A/B claim, "how much it helps real serving" is not confirmed. Until online results appear in a next paper or tech report, we keep it in the watching tier.

↑ Contents

4Reasoning traces hurt offline recommendation in a controlled test — Radar (Spotify)

• The representative case is Kuaishou PushDualGen (arXiv 2608.07989). It was deployed to production with a two-stage generation structure that first produces the SID and then attaches skippable explanatory text (copy), and recorded an effective play rate of +8.50% and a dissatisfaction rate of −37.70%.
• Other SIDReasoner-family work also claims that traces improve SID inference.

The problem is that these lines mostly support their claims with their own offline metrics or a single production A/B, and controlled comparisons are lacking. This paper fills that gap. It is a 2×2 factorial controlled experiment that independently varies two factors: whether the item representation is an SID or a natural-language title, and whether a reasoning trace is present. The backbone is unified to Qwen3-1.7B, and three Amazon product domains were used.

• The key negative result is that adding descriptive reasoning traces lowers traditional offline recommendation performance under standard SFT and RL training verified.

  "introducing explicit descriptive reasoning traces reduces traditional offline recommendation effectiveness under standard SFT and RL training" — abstract

  • In the Office Products domain, combining the Titles representation with SFT training, adding a trace lowers R@10 (the share of cases where the correct item is within the top 10 recommendations) from .1587 to .1416, about −11%.
  • Across all domains, Titles range from −19% to 0% and SIDs from −9% to 0%.

• Titles produce better traces, but the metrics get worse verified.
  • SID is an opaque learned identifier, so it is hard for an LLM to reason over it like natural language. Only with extensive alignment, additional training that aligns SIDs with natural language, do traces barely become grounded.
  • Natural-language titles yield grounded, interpretable traces even without alignment.
  • However, even when the trace improves, recommendation R@10 drops. In other words, trace quality and recommendation effectiveness are independent of each other.

• A richer reward signal partially mitigates the problem verified. In RL training, if the reward is changed from recommendation R@10 itself to a richer signal, the gap narrows somewhat. However, it still does not fully catch up with the no-reasoning baseline.

• There is no reference to industry cases verified. The SIDReasoner family appears in the related work section, but the paper does not explicitly cite Kuaishou PushDualGen or similar production deployments.
  • The paper's tone is also less about judging whether "this trend is right" and more about summarizing conditional observations from its own controlled experiment.

"R@10 drops from .1587 to .1416" — Office Products domain, Titles condition Section: Results, arXiv 2608.23154 HTML

• What matters is under which conditions the negative result holds. The conditions of the two sources differ.
  • Kuaishou PushDualGen: it used the push surface, a production A/B, and a real catalog, and the reward was designed around production objectives such as effective play and dissatisfaction rate.
  • Spotify: it used Amazon reviews and a Qwen3-1.7B backbone in an offline setting, and the reward was designed as standard R@10.
  • When future production claims combine SID and CoT, conditions such as backbone size, whether the setting is offline or online, and reward design will be key.
• One can speculate why Spotify does not deploy a line that combines SID and CoT to production. Spotify is a company that has already moved seriously into the SID (semantic tokenization) line (see the PRS 2025 semantic tokenization talk).
  • If its own experiments gave a negative result, that result may also have been reflected in the company's production roadmap inferred.
  • We can predict that Spotify's production will adopt SID but not attach CoT.
• The point to watch is a similar-experiment statement from a second company. If one appears, it becomes a candidate for a new, separate section in topics/llm-in-recsys.md titled "SID + reasoning trace: production vs benchmark debate".

↑ Contents

Corrections

None. The three new sources in this batch (HEGM, Densing Law, Spotify's reasoning trace experiment) do not overturn the existing canon. Each attaches as a new data point to a watching thread: watch-time prediction, scaling laws, and LLM-in-recsys. X Gen4+7 is also a natural extension of the Gen4 series.

↑ Contents

Radar

• 3 new:
  • HEGM (VK, arXiv 2608.23356)
  • User Behavioral Densing Law (Ant/Alipay, arXiv 2608.23392)
  • Spotify's reasoning trace negative result (arXiv 2608.23154)

↑ Contents

What to check next

↑ Contents