- Sources read: 3
- Canon files updated: 2
- Radar entries: 2
Other changes reviewed today:
- In the same X commit, the tweet filter pipeline (VF) began filling in retweet originals within a single call, without a second lookup call.
- In the same commit, the retrieval two-tower configuration (a structure that encodes the user and the candidate separately) was registered in the registry as a catalog of six presets.
- The training mask of the ads conversion head moved to the data loader and was split into two axes, web and app.
1Six Home feed cold-start parameters changed and VF lookup was rewritten — X
Source xai-org/x-algorithm@77d431a — xai-org/x-algorithm public repository, 2026-09-30 03:53:42Z (UTC), commit. Line counts are +3,306/−1,489, and 69 files changed. These notes number the X commits after the Gen4 release in order, and this commit is Gen4+32. It came one day after yesterday's commit Gen4+31 a707cc2.
Background The hydration layer of visibility-filtering (VF, X's tweet filter pipeline) flipped three times over four days, from 2026-09-25 to 09-29. Hydration is the stage that fills in the original data needed for a verdict by looking it up in batches from backend services.
- Gen4+29 (2026-09-25, bf7db1b) expanded the
Hydratorenum from 7 to 17 variants and introduced three data-driven plan/execute/sources files. - Gen4+30 (2026-09-26, 4c5cfe8) split the state inside it into
store.rsand the execution intoexecute.rs. - Yesterday's Gen4+31 (2026-09-29, a707cc2) extracted
retweet.rsas a separate module and merged in-batch sources and fetched sources with a double-hydrate pattern. The double-hydrate pattern hydrates the request batch and then fills only the retweet sources with a second hydrate call.
This commit (Gen4+32) simplified the double-hydrate pattern into a single-hydrate with in-flight expansion pattern. In-flight expansion grows the request list by looking at results that arrive first while the lookup is still in progress. It also rewrote the eight Option<HydrationBatch> fields of Store as a new Fetcher<V> trait array, so that calling the same hydrator again does not re-request keys that are already claimed.
At the same time, two axes evolved separately. The xrecsys_two_tower.py of Phoenix (the ranking model stack of X For You, whose training code was released in Gen4) was added to the registry as a formal catalog. Along with this, the SID (semantic ID)-based candidate tower and sequence packing were locked into six presets. SID is a discrete code hashed across several codebook layers instead of a post embedding. Sequence packing concatenates several user sequences into one batch row. ads_head_masking, used to train the ads conversion head, moved from the JAX training graph to the numpy data loader, which made it possible to train two axes, web conversion and app (MMP, Mobile Measurement Partner) conversion, separately.
What we learnedA. Phoenix cold-start retrieval turned on and the post-age cap fell to 2 hours
- Six cold-start parameters of home-mixer (the service that handles For You feed requests) moved in the cron mirror verified —
home-mixer/params/param.rs.- Cold start refers to new users or exploration targets. The cron mirror is a file that syncs and publishes production parameter values daily.
- The Phoenix cold retrieval channel was turned on, from 0 to 200 posts.
- MaxPostAge was made extreme, from 48h to 2h.
- FollowerCap was effectively lifted, from 1000 to 50k.
- Thompson sampling was also turned on. Thompson sampling picks the candidates that fill cold-start slots by drawing samples from a Beta reward distribution.
Yesterday's and today's values of the six parameters:
| Parameter | Yesterday | Today | Multiplier |
|---|---|---|---|
PhoenixColdStartMaxResults |
0 | 200 | 0 → 200 (Phoenix cold-start retrieval output enabled, from 0 to 200) |
ColdStartImpressionThreshold |
1000 | 200 | ×0.2 (only posts with fewer than 200 Home impressions are targeted; narrower target set) |
ColdStartFollowerCap |
1000 | 50,000 | ×50 (author follower cap effectively lifted) |
ColdStartMaxPostAgeSecs |
172,800(48h) | 7,200(2h) | 1/24 (post freshness made extreme) |
LowImpressionsMaxPositionRatio |
0.85 | 0.97 | +0.12 (position boost cap extends to the later part of the feed) |
EnableColdStartThompsonSampling |
false | true | newly ON |
The last sync in the file's comment header was updated from 2026-09-28T16:00:35Z → 2026-09-29T17:02:52Z.
What we learnedB. VF fills retweet originals within a single call, without a second hydrate call
- The double-hydrate pattern that filled retweet sources was simplified into single-hydrate with in-flight expansion verified —
retweet.rs.- The changed file is
retweet.rs(+562/−110). - Until yesterday, it filled the original batch with
filter_tweets.hydrate(request), picked retweet source candidates withsource_ids, and fired a secondfilter_tweets.hydrate(...)for the sources not in the request batch. - Today it calls
filter_tweets.hydrate_with_retweet_sources(request)only once.
- The changed file is
- This single call attaches retweet sources to the request internally at the moment the pure_cores response lands verified —
store.rs:308-330.- pure_cores is tweet core data containing the author id, the body, and the original tweet id.
- The function that runs at this point is
store.join_retweet_sources(). - This function appends those
source_tweet_ids of pure_cores that are not in the request torequest_tweetsand returnsLanding::SourcesJoined.
- Once sources are attached, the executor puts the initial groups back into the ready state and fires an expanded batch only for the fetched sources verified —
execute.rs:112-118.- When
execute.rsreceivesLanding::SourcesJoined, it puts theinputless()groups (the initial groups with no pure_core dependency) back intoready. - Thanks to the claim dedup of Fetcher, keys that were already in the request batch are not re-requested.
- When
- The new
Fetcher<V>trait holds a lookup state per key, picks only the keys not yet present, and returns them as the keys to send in the actual RPC verified —fetcher.rs.- The new file
visibility-filtering/hydration/fetcher.rs(NEW +113) contains theFetcher<V>andAnyFetchertraits. - Internally it is a single
HashMap<u64, State<V>>, and the state isenum State<V> { Pending, Landed(Hydrated<V>) }. claim(keys) -> Vec<u64>marks only the keys not yet present as Pending and returns only the keys to send in the actual RPC. Dedup and the join with earlier calls happen together here.land(&[u64], HydrationBatch)fills responses into the claimed keys and fills keys without a response withHydrated::NotFound.has_incomplete/is_incomplete/has_claimed/is_claimeddecide whether to call again.
- The new file
- All eight fields that Store held were replaced with Fetcher verified —
store.rs:54-84.- What changed is the
Option<...>fields ofstore.rs, and the new type isFetcher<V>. - Each Edge gets its own
Fetcher<bool>, bundled in the arrayedges: [Fetcher<bool>; Edge::VARIANTS.len()]. Edge has eight variants: Follows/Blocks/Mutes/MuteRetweets/BlockedBy/SuperFollows/FollowedBy/SecondDegree. - The new
CallRequest<'p>struct encapsulates the details of actually firing one group. Its fields are group, is_first, keys, key_count, queries, viewer_id, batch_size, and counts. offer(&mut self, group) -> Option<CallRequest>claims the keys of the group, builds a CallRequest, and hands it to the scheduler.- On the second call (is_first=false), the
record_expanded_batchmetric is emitted.
- What changed is the
- The edge representation in the lookup plan was unified into a single eight-variant enum verified —
plan.rs:8-20.Part::Edge(Graph, EdgeDirection)inplan.rsbecamePart::Edge(Edge).Edge::flock()returns the flock RPC parameters asOption<(Graph, EdgeDirection)>. OnlySecondDegreeisNone, and it is routed through Wingman.Hydrator::RootFollowsViewerSecondDegree, which wasPart::Columnyesterday, was promoted today toPart::Edge(Edge::SecondDegree)and is routed uniformly through the fetcher array.
- Groups gained position and reader list fields, so the offer→land→ready schedule is computed in O(readers) per group verified —
plan.rs:172-207.- The new fields are
position: usizeandreaders: Vec<usize>ofGroup.
- The new fields are
- A metric that records retweet sources in three cases was added verified —
filter.rs:62-79.- The metric is
RetweetSources::{NoSource, InBatch, Fetched}. - The test
each_request_kind_labels_its_phase_samplespins the three cases: [2] → NoSource, [1,2,3,5] → InBatch, [1,2,3,4] → Fetched verified —retweet.rs:721-738.
- The metric is
What we learnedC. The functions that convert a verdict into thrift, proto, and metric representations were isolated in one file
- The set of functions that project the pure verdict produced by rules into three kinds of downstream representations was isolated in a new file verified —
treatment.rs.- The new file is
visibility-filtering/treatment.rs(NEW +119), and the input verdict isVerdict(Withheld / Shown{media, engagement}). thrift_action(&Verdict, SafetyLevel) -> Actionprojects to thriftaction::Action(Allow / Drop / Tombstone / ComposedMediaVisibilityResults / Interstitial / LimitedEngagements / TweetInterstitial).thrift_result_state(&Verdict, SafetyLevel) -> TweetFieldsResultStateprojects to the tweet-service response state (Found / Filtered). Only fourFilteredReason::{TweetIsBounced, AuthorBlockViewer, AuthorIsProtected, AuthorIsSuspended}have a separate legacy arm, and the rest are wrapped inSafetyResult.proto_action(Verdict) -> (vf_pb::Action, Option<vf_pb::FilteredReason>)projects to the three-way (Allow / Drop / Interstitial) form for the VF proto.metric_label(&Verdict) -> &'static stremits a six-way label (allow/drop/tombstone/interstitial/limited_engagement/tweet_interstitial).decided_rows(&Verdict)is an iterator that unfolds a verdict into a sequence of (rule, phase) tuples.- The test
dashboard_generator_pins_the_rpc_and_verdict_mix_labelsactually readsscripts/dashboard.pyand asserts thatFT_VERDICT_ACTIONShas the six labels. The metric labels and the dashboard are tied together by golden.
- The new file is
- The age-verification option constant was also pinned in this file verified —
treatment.rs:120-123.- The constant is
AGE_VERIFICATION_OPTIONS = [SELFIE, PERSONA]. blurred_mediaputs these two options into the response only when the blur prompt isAGE_VERIFICATION_PROMPT.
- The constant is
What we learnedD. The VF client now receives a result state instead of a verdict action
- The client contract changed so that the VF response carries a result state instead of a verdict action verified —
evaluated.rs:22-33,vf_client.rs:35-41.- The response was promoted from Thrift
ActiontoTweetFieldsResultState. - The proto changed from
Outcome::ActionThriftCompacttoOutcome::ResultStateThriftCompact, and the constant fromMAX_ACTION_BYTEStoMAX_RESULT_STATE_BYTES. The value is still 64KB. - VF now carries the (Found | Filtered) result state itself, so the caller can build the tweet-service response directly.
- The response was promoted from Thrift
- The check of the tweet.id order in the response is gone, and only the response length is checked verified —
vf_client.rs:35-46.- Until yesterday, the
tweet.as_ref() == Some(expected)assert verified the order and the quote context. - This change is a wire format optimization.
- Until yesterday, the
- A metadata parameter was added to the request header verified —
vf_client.rs:8-25.- The parameter is
metadata: &tonic::metadata::MetadataMap. - It forwards the caller metadata (
twittercontext, etc.) to each chunk RPC.
- The parameter is
What we learnedE. The Phoenix retrieval two-tower configuration was registered in the registry as a catalog of six presets
- The two-tower configuration file was promoted to a catalog in the registry and holds six presets verified —
xrecsys_two_tower.py.- The file path is
phoenix/xrex/configs/xrecsys_two_tower.py. - The presets are in the
MODEL_CFGSdictionary ofxrecsys_two_tower.py. - The preset names are
xrecsys_two_tower/xrecsys_two_tower_immersive_only/xrecsys_two_tower_immersive_only_gb300/xrecsys_two_tower_combined/xrecsys_two_tower_combined_gb300/xrecsys_two_tower_nano. - The common scale shared by several presets is in
_xrecsys_two_tower_combined_baseverified —xrecsys_two_tower.py:466-475.- Sequence length:
history_seq_len=1023·candidate_seq_len=64 - Model size:
num_layers=8·emb_size=1024·emb_table_width=1024·query_heads=16·kv_heads=4(GQA 4:1) - Training samples:
num_global_negatives_per_example=64·total_samples=1e11 - vocab:
item_hash_vocab_size=100_000_000·author_vocab_size=30_000_000
- Sequence length:
- Only SID is used, with no user or post embeddings verified —
xrecsys_two_tower.py:311-318.use_user_embedding=False·use_post_embedding=False·use_post_sid=Truesid_num_levels=6·sid_codebook_size=256·sid_embed_dim=1024·sid_hash_level=True·sid_cross_attn=True
- It uses sequence packing and right-anchored RoPE.
use_seqpack=True·effective_sequence_len=513·BetaLengthDistribution(min=0, max=1023, mean=511, α=β=1.0, block_size=128)
- Home and Immersive share two candidate heads verified —
xrecsys_two_tower.py:293-302.num_candidate_heads=2·head_names=["home", "immersive"]head_dataset_mappingmaps HOME/HOME_COLD/HOME_HOT to head 0 and IMMERSIVE4Day/IMMERSIVE2Day/IMMERSIVENSFW to head 1.- The
positive_actionsof home is only[SERVER_TWEET_FAV].immersive_positive_actionshas eight kinds: FAV / REPLY / QUOTE / RETWEET / VIDEO_QUALITY_VIEW / FOLLOW_AUTHOR / BOOKMARK / SHARE verified —xrecsys_two_tower.py:283-292.
- The hardware presets are separate for H100 and GB300 verified —
xrecsys_two_tower.py:409-413._H100_OVERRIDES = {bs_per_device: 480, ep: 128, attn_impl: "cutedsl_ranker_varlen_attn"}·_GB300_OVERRIDES = {bs_per_device: 960, ep: 64, ...}- On GB300 the per-device batch is doubled and EP is halved.
- The optimizer is Adam, and embeddings get a separate learning rate for rowwise adagrad verified —
xrecsys_two_tower.py:826-831, 271.Adam(b1=0.95, b2=0.98, weight_decay=1e-3)·learning_rate=2e-3·emb_learning_rate=0.1(embedding rowwise adagrad)apply_u2u_and_i2i_loss=False(off by default) ·empty_history_user_dropout_rate=0.1
- The candidate max_posts values differ between combined and immersive verified —
xrecsys_two_tower.py:611-618.- combined 28,672,000 · immersive 40,960,000
- The linear projection setting differs between combined/immersive and nano verified (same source).
enable_linear_projis False in combined/immersive and True in nano.
- The same line range also has an assert that keeps the two settings from being turned on together verified (same source).
apply_u2u_and_i2i_lossandenable_candidate_tower_linear_projare tied by a mutual-exclusion assert.
- The file path is
"Beta length distribution used for sequence packing: BetaLengthDistribution(min=0, max=1023, mean=511, alpha=1.0, beta=1.0, block_size=128)." —
phoenix/xrex/configs/xrecsys_two_tower.py, 2026-09-30
What we learnedF. The training mask of the ads conversion head moved to the data loader and was split into web and app
- The ads head masking code was rewritten from the model-side JAX implementation to a data-side numpy implementation verified.
- The new file
phoenix/xrex/data/recsys/ads_head_masking.py(NEW +87) replacesphoenix/xrex/models/ads_head_masking.py(−67) and rewroteads_head_maskingfrom JAX to numpy. - The patch of the deleted
models/ads_head_masking.pyhasimport jax; import jax.numpy as jnp, while the newdata/ads_head_masking.pyuses onlynumpy as np.
- The new file
- The new function precomputes a numpy bool mask in the data pipe verified —
phoenix/xrex/data/recsys/ads_head_masking.py:44-88.- The function is
build_trained_candidate_mask(actions, sample_source, trained_actions_bits, *, ads_head_masking: bool) -> npt.NDArray[np.bool_]. - The argument names changed:
source_id→sample_source, andraw_targets→ thetrained_actions_bits: uint8bitmask. - The new constant
ADS_FAMILY_SLICE_HEADSholds the representative head indexes of four slices (delayed_website_clicked/delayed_website_non_clicked/delayed_app_clicked/delayed_app_non_clicked). - An MMP app-conversion axis was added, which expanded masking into a web × app × early/late × click/view grid.
- The function is
- The clicked mask of the model was split into web and app verified —
recsys_model.py:1493-1520.masks["clicked"]inrecsys_model.pywas split four ways:website_clicked/non_negative_website_clicked/app_clicked/non_negative_app_clicked.
- A new verification keeps ads head masking and seqpack from being turned on together verified —
recsys_model.py:610-614.- The verification function
validate_ads_head_maskingforbidsads_head_masking and use_seqpack. - In the packed layout, the per-candidate/per-head trained_candidate_mask is not defined.
- The verification function
- The column names of the training batch schema changed verified —
phoenix/xrex/data/recsys/recsys_batch.py:471-479.- In
recsys_batch.py,conversionKeepMask: boolchanged toconversionKeepBits: uint8. - The fallback value when null changed from
TruetoCONVERSION_KEEP_WEB | CONVERSION_KEEP_APP = 0b11. - It now takes a new
sample_sourcecolumn. If absent, it is all zero.
- In
- The MMP action groups, bit constants, enum, and new labels were added to the constants file
constants.pyverified —phoenix/xrex/data/recsys/constants.py.MMP_CLICK_ACTION_INDEX = ADS_MMP_CLICKMMP_CLICK_CONDITIONED_ACTION_INDICESis the seven app click-through actions.MMP_CLICK_VIEW_THROUGH_ACTION_INDICESis the seven app view-through actions. They were separated fromVIEW_THROUGH_ACTION_INDICES, where they were mixed in until yesterday.- The bit constants are
CONVERSION_KEEP_WEB = 1 << 0·CONVERSION_KEEP_APP = 1 << 1. LineItemObjective(enum.IntEnum): WEBSITE_CLICKS = 5promotes to an enum what was the magic numberline_item_objective == 5until yesterday.- The new labels
IsCustomConversion/IsContentViewConversionmap toADS_CUSTOM_CONVERSION/ADS_CONTENT_VIEW_CONVERSION, respectively.
- Two MMP install actions were added to the proto verified —
phoenix/crates/serving/xai-recsys-proto/proto/recsys.proto.- The new actions in
recsys.protoareADS_ATTRIBUTED_MACT_MMP_CLICK_INSTALL = 194andADS_ATTRIBUTED_MACT_MMP_VIEW_INSTALL = 195.
- The new actions in
What we learnedOther changes
- The home-mixer diversity stats helper file was deleted verified.
- The deleted file is
home-mixer/util/composition.rs(−63). - Until yesterday, this file was a diversity stats helper containing the
Composition {size, unique, max_count, hhi, entropy_norm}struct and thefrom_keys/from_counts/unique_ratio/max_shareAPI. - In
response_diversity_stats_side_effect.rs, only the import path changed at the call site, touse xai_candidate_pipeline::component_library::utils::Composition;verified —response_diversity_stats_side_effect.rs. - The shared utilities of candidate-pipeline already have a Composition of the same name, so this is dedup: deleting its own copy and using the shared one.
- It is unrelated to the rules/registry Policy (Gen4+30).
- The deleted file is
- Nine handles were added to the Brazil 2026 election filter, and the list size went from 2,786 to 2,795 verified —
home-mixer/filters/brazil_2026_election_filter.rs:5849-5852.- The added handles (per the comments) are @linoaa, @LPVellozo, @erikakokay, @douglascamargoo, @Anadelso_p, @zeosmaruniao, @siandrade11777, @tiagopiaui22, and @gloriabsoares.
- There is no logic change.
Canon update Created a ### Gen4+32 section in services/x.md that summarizes the facts above, added the commit sha 77d431a to ## Sources, and updated Last updated to 2026-10-01.
Why it matters Three axes overlap.
- The home-mixer cold start was set with production parameters. Gen4+30 added a code path to the HOME_COLD=13 channel, which was introduced in Gen4+28 (44d37eb) together with HOME_HOT=14, and today the parameters were set.
- The Phoenix cold retrieval channel opened from 0 to 200 posts, and MaxPostAge was made extreme, from 48h to 2h.
- FollowerCap was effectively lifted from 1000 to 50k, so the author pool grew sharply.
- ImpressionThreshold was lowered from 1000 to 200, so only posts with fewer than 200 Home impressions became cold-start targets (narrower target set) verified —
author_cold_start.rs:294. Thompson sampling was also turned on. - If cold start has so far been a narrow exploration aimed at tracked authors or new users, this parameter setting combines an extremely strict 2h freshness limit with a widely open author pool.
- Whether this tuning holds up in the A/B is the checkpoint to watch at the next cron sync.
- VF hydration gained the ability to re-expand. The four-day run from Gen4+29 (09-25) to Gen4+31 (09-29) completed the data-driven pipeline skeleton and the
retweet.rsmodule extraction.- Today the
Fetcher<V>trait was added on top, which made claims idempotent. - This makes possible the
Landing::SourcesJoinedflow, which expands request_tweets in flight the moment pure_cores lands and re-fires the inputless groups. - The double-hydrate pattern of
retweet.rswas simplified into single-hydrate with in-flight expansion. - The metric axis (NoSource / InBatch / Fetched) is set as labels, so the actual rate of expanded batches can be observed.
- The VF client response was also promoted from Thrift
ActiontoTweetFieldsResultState, moving to a form where the caller can build the tweet-service response directly. treatment.rsisolating the rules → thrift/proto/metric projection in one file and tying dashboard.py and the six metric labels together with golden tests also completes this simplification.
- Today the
- Phoenix retrieval two-tower was promoted to a catalog. Until now
xrecsys_two_tower.pyexisted as a file and had minor edits across several commits, but the registry catalog itself was not locked.- The minor edits were the introduction of the three options
qk_norm=True,attn_logit_cap=-1, andright_anchored_rope=Truein Gen4+6, and the switch of the H100 attn kernel frompallas_ranker_varlen_attntocutedsl_ranker_varlen_attnin Gen4+21. - Today six presets (the default, immersive-only and its GB300 version, combined and its GB300 version, and nano) were locked into
MODEL_CFGSand migrated to connect to the production data pipe inferred. The evidence is that the only in-file dataset isaggregated_kafkaand that the presets are well defined as hardware-aware. - Notably, SID-only (a 6-level codebook hash instead of post embeddings), seqpack, the H100/GB300 hardware presets, and the sharing of two heads, Home and Immersive, are organized in one file.
- Paired with the "wiring of
sid_codesinto the DPA ads product too" observed in Gen4+13, this gives the SID vocabulary a large potential to unify three uses: organic posts, ads DPA, and the retrieval two-tower.
- The minor edits were the introduction of the three options
- The ads conversion head split along web/app axes. The delayed feedback split introduced in Gen4+6 and the
ads_head_maskingknob unified in Gen4+24 moved today from a float multiplier inside the training graph to a boolean mask precomputed by the data loader.- This secures compile stability by making the graph independent of sample_source.
- It also trains the web conversion head and the app (MMP) conversion head separately, so that different sample subsets can be used for the
keep_web/keep_appaxes. - The mask semantics still keep the same late-window twin suppression logic; it is an exact refactor that changes only where the computation happens.
- The mutual-exclusion assert with
use_seqpackwas newly added because candidate slot correspondence breaks down under seqpack. - Introducing the two actions
ADS_ATTRIBUTED_MACT_MMP_{CLICK, VIEW}_INSTALL = 194, 195in the proto creates a channel to formally fold MMP-side app-install conversion into the training head.
Adding nine handles to the Brazil 2026 election filter is list maintenance with no logic change. Deleting Composition.rs removes duplication by reusing the shared candidate-pipeline library.
2HELIX raised GMV per user by about 6% in a TikTok e-commerce A/B test — Radar
Source HELIX: Purified and Unified - Rethinking Feature Interaction and Sequence Modeling for Large-Scale Recommendation — arXiv 2609.37183 v1, 2026-09-29 (arXiv submission), paper. There are 21 authors. The v1 HTML renders as a 404, but the abstract contains the phrase "Deployed in TikTok's e-commerce recommendation system", so we infer the affiliation is TikTok (ByteDance) inferred.
Background Industrial recommendation ranking models generally scale along two axes.
- heterogeneous feature interaction: combines user / item / context / cross features and lets them interact.
- sequence modeling: encodes long, multi-type user behavior sequences.
The HELIX authors observe that scaling only one axis gives a suboptimal scaling-law slope, and they propose a "unified" ranking architecture that expands both axes together.
What we learnedA. Deployed in TikTok e-commerce, and online A/B raised GMV per user by about 6%
- The abstract states that HELIX was deployed in TikTok's e-commerce recommendation system and consistently improves ranking metrics such as offline CTR AUC and CVR AUC verified (direct quote from the abstract).
- In the online A/B test, e-commerce video GMV per user rose by about 6% verified (direct quote from the abstract).
"Deployed in TikTok's e-commerce recommendation system, HELIX consistently improves offline CTR AUC, CVR AUC, and other ranking metrics. In online A/B tests, it achieves an approximately 6% increase in e-commerce video GMV per user." — HELIX abstract, 2026-09-29
What we learnedB. The core view is that both axes must be scaled together for the scaling law to escape suboptimality
- HELIX is a "purified and unified" ranking architecture that scales the feature-interaction axis and the long/multi-type user-behavior sequence axis simultaneously verified (abstract wording).
- The paper's core view is the observation that the two axes must be expanded together for the scaling law to escape suboptimality verified (abstract wording).
What we learnedC. Service scale is not disclosed
- The paper does not state user counts or request counts.
- We infer that using a GMV per user metric implies large-scale e-commerce traffic inferred.
Canon update Added a 2-line pointer to "new this month" in radar.md. It is not directly tied to the service canon (X·Meta·Netflix).
Why it matters This is the first time TikTok has formally named and published a ranking architecture for e-commerce recommendation that scales feature interaction and long sequences together.
- Until now, releases in this area have continued from the EGA (Elastic Generative Auction, ads auction and ranking) family and the GreenLightning family. GreenLightning is a single inference service.
- The view that the scaling law requires expanding both axes at once suggests that the sequence-only expansion of the Meta HSTU (Meta's generative sequential recommendation architecture) / GEM family and the feature-interaction expansion of the DCN·xDeepFM family are each suboptimal.
- However, this is an abstract-based triage stage, so the detailed architecture (which interaction structure, which sequence encoder, the measured scaling-law slope) is a subject for a deep dive.
- It is not folded into the service canon and stays in the radar.
3Snap's GRP merged retrieval, ranking, and reward into one model — Radar
Source GRP v0.1 Technical Report — arXiv 2609.36688 v1, 2026-09-29 (arXiv submission), paper. The v1 HTML page renders the organization name Snap Inc., so the affiliation is Snap verified.
Background Industrial recommendation systems are generally built as a multi-stage cascade of retrieval → pre-ranking → fine-ranking → reward re-ranking. Because each stage is trained and deployed as a separate model, replacing the whole stack together is difficult. The GRP in this paper (presumably Generative Recommendation Platform) presents a framework that unifies these stages into a single encoder-decoder model. The model generates multimodal Semantic IDs, and a joint MHP (Multi-Head Prediction) doubles as candidate scoring and the frozen reward for RL post-training.
What we learnedA. View time and shares rose in an A/B comparing retrieval only and in an A/B applying bypass and source replacement together
- In the retrieval-only comparison, view time rose by 0.46% and shares by 0.77% relative to production verified (direct quote of the A/B).
- In a separate comparison applying bypass and source replacement together, view time rose by +0.82% and shares by +2.56%, and platform-level guardrails were neutral verified (direct quote of the A/B).
"In a retrieval-only comparison, view time increases by 0.46% and shares by 0.77% relative to production. A separate comparison combining bypass and source replacement yields increases of 0.82% in view time and 2.56% in shares, with neutral platform-level guardrails." — GRP v0.1, 2026-09-29
What we learnedB. With the retrieval bypass combination, views and shares rose and view time fell
- The additional axis of the retrieval bypass combination is these figures: "+0.24% views, −0.24% view time, +2.30% shares (p<0.05)".
What we learnedC. Unifies retrieval, ranking, and reward into a single encoder-decoder
- GRP is a generative recommendation framework that unifies retrieval, ranking, and reward into a single encoder-decoder.
- It generates multimodal Semantic IDs, and a joint multi-head prediction (MHP) doubles as candidate scoring and the frozen reward for RL post-training. The RL post-training is mGRPO (reference-anchored margin).
What we learnedD. Serving optimization reduced retrieval latency
- User counts and request counts are not stated.
- Serving optimization reduced end-to-end retrieval latency by 69%.
- The paper is 26 pages with 11 tables.
Canon update Added a 2-line pointer to "new this month" in radar.md. Snap is outside the tracked services, so it is not folded into the service canon.
Why it matters GRP is a new statement on the "unified serving path for generative recommendation" angle we have been watching since early this month.
- Kuaishou OneTrans-V2 bundled the 3-stage cascade of retrieval, pre-rank, and fine-rank into a single Transformer and gained GMV +9.74% (2026-09-28 radar). Tencent PCG TGR bundled ranking, retrieval, slate generation, and reason tokens into one frame and deployed it on 5 surfaces. GRP goes in the same direction as both.
- GRP adds two axes to this. One is the mGRPO joint MHP axis, which handles reward inside the same model, and the other is the multimodal Semantic ID generation axis.
- The serving latency of −69% is a large reduction relative to the multi-stage cascade, and it signals that whether the encoder-decoder unification justifies the latency budget is the key criterion for adopting industrial-scale generative recsys.
- It is not folded into the service canon and stays in the radar.
Radar
- HELIX (TikTok [inferred]) is a unified ranking architecture that scales both feature-interaction and long user-behavior sequence axes together (arXiv 2609.37183). It was deployed in TikTok e-commerce and gained GMV per user +~6% in an A/B test. The metric is verified and the affiliation is inferred (the v1 HTML rendering returned 404; based on abstract wording). Added to new this month in radar.md.
- GRP v0.1 (Snap) is a generative recommendation framework that unifies retrieval, ranking, and reward into a single encoder-decoder (arXiv 2609.36688). Its components are multimodal Semantic ID, joint MHP, and mGRPO. In the production A/B test it gained view time +0.82% and shares +2.56%, and serving latency is −69%. The affiliation and metrics are verified. Added to new this month in radar.md.
What to check next
- Check how often VF's retweet-source expansion batches actually occur — how the metric labels appear in production shows whether this rewrite had a real effect. The targets are the occurrence-rate distribution of
hydrate_with_retweet_sourcesexpansion batches and theRetweetSources::{NoSource, InBatch, Fetched}labels. If the next commit includes a metric report, we will fold it into the canon. - Check whether the Phoenix
xrecsys_two_tower.pypreset is actually adopted — which of the 6 presets is active in real training jobs, and whether it is only registered inMODEL_CFGSand unused, is the next checkpoint to watch. Currently only the catalog promotion is confirmed. - Check whether the 6 home-mixer cold-start parameters are kept — whether PhoenixColdStartMaxResults 200 and MaxPostAgeSecs 2h remain production values or revert to experiment values will be observable at the next cron sync. We also track yesterday's (Gen4+31) home-mixer weight shift.
- Deep dive on HELIX — the TikTok production claim of GMV +~6% is sufficiently strong evidence, so the abstract, introduction, and architecture sections need a close read. Deep dive needed.
- Deep dive on GRP v0.1 — Snap's encoder-decoder unification frame is the third statement on industrial-scale generative recsys, so its 26 pages are worth a close read. Deep dive needed.
- arXiv cs.IR triage record — 30 items were parsed and 2 (HELIX, GRP) passed. The other 28 are academic RAG / dense retrieval / semantic ID (generative retrieval) / clinical QA / benchmark papers and similar.