Sequence Transformer ranking (in-context candidate scoring)

Sequence Transformer ranking puts a user's behavior history in as a token sequence, appends the candidates to be scored after the same sequence, and scores all of them in a single forward pass. It contrasts with the traditional "feature vector → MLP/DCN/MaskNet" ranker.

These notes cover X's Phoenix (X's current production ranking Transformer) and two Meta lineages together. One Meta lineage started on the organic side (general, non-ads content) with the generative recommendation model HSTU, and the other started on the ads side with DHEN and leads to GEM. Two points run through the whole note.

• Where the two companies independently reached the same conclusion: they put candidates in the sequence and isolate them with a mask, so that scores can be cached. inferred (intersection)
• Where they diverged: Meta has two lineages, and the two do not cite each other. observed (measurement)

Why this structure

This structure replaces hand-built features with attention and lets the candidates share the cost of encoding the history. inferred

• It replaces feature engineering with attention. Instead of hand-building user×item cross features, a candidate token attends directly to the history tokens. inferred — this structure is the basis for X's claim that it removed "all hand-engineered features".
• Packing N candidates into one sequence lets the N candidates share the cost of encoding the history. This costs less than running a separate forward pass per candidate. inferred — it follows from the attention mask design, and it is hard to explain X's reason for placing a candidate block in the sequence any other way.

↑ Contents

Core design elements

X Phoenix puts the user, the history, and the candidates in one sequence, and uses an attention mask so that candidates cannot see each other. verified Most parameters sit in the ID embedding tables. inferred Content such as text or images does not enter the input. verified

1. Sequence layout

X concatenates a user token, S history tokens, and C candidate tokens into one flat sequence. verified xai-org/x-algorithm, 2026 — phoenix/recsys_model.py:617-624

• The order is [user] + [history × S] + [candidates × C].

2. Attention mask — the most important part of the design

X uses the attention mask to keep candidates from seeing each other, which makes a candidate's score independent of what else is in the same batch. However, with the default settings RoPE breaks this batch independence again. verified

Whether candidates may see each other separates pointwise ranking from slate optimization. Pointwise ranking scores each candidate separately, and slate optimization chooses a whole list to show together in one step.

X keeps only the diagonal in the candidate block (self-attend only). verified — phoenix/grok.py:39-71, unit test phoenix/test_recsys_model.py:94-121

        user  h1  h2  c1  c2  c3
user  [  1    0   0   0   0   0 ]
h1    [  1    1   0   0   0   0 ]
h2    [  1    1   1   0   0   0 ]
c1    [  1    1   1   1   0   0 ]
c2    [  1    1   1   0   1   0 ]
c3    [  1    1   1   0   0   1 ]

As a result, a candidate's score does not depend on what else is in the same batch. So scores can be cached, and splitting a list and calling the model several times gives the same results. X states this explicitly in the README as a goal. verified — README.md:328

There are three caveats.

• The user token (position 0) sees only itself because of the causal mask. It is not a token that summarizes the history; it is a prefix/bias token that other tokens read. verified — test_recsys_model.py:105-107
  • A causal mask lets each token see only itself and the tokens before it.
• The README says the user and history tokens see each other bidirectionally, but the code is strictly causal. The README and the code directly contradict each other. verified — grok.py:62 (jnp.tril) vs phoenix/README.md:158
  • The original text of X's phoenix/README.md:158 is "User + History: Full bidirectional attention among themselves".
• Even though the mask guarantees batch independence, RoPE breaks it. With the default, a candidate's score depends on the slot it occupies. verified — grok.py:88-109, recsys_model.py:362, run_pipeline.py:132-162
  • RoPE (rotary position embedding) is a positional encoding that rotates query and key vectors according to token position.
  • With right_anchored_rope=False (the default), candidate slot 0 and slot 40 get different rotations, so the scores differ.
  • A helper that fixes this (right_anchored_rope_positions) exists in the code, but the released pipeline does not turn that helper on.

3. Where the parameters are

99.8% of X Phoenix's parameters are ID embedding tables, and the Transformer is only 0.8M. inferred

• This is a value computed from the released mini config (128-dim, 4-layer, vocab 1M×3). The checkpoint is on Git LFS, so we could not count directly.
• The calculation matches the file sizes stated in phoenix/README.md:222-230. model_params.npz is 3MB and embedding_tables.npz is 1.4GB.

With this ratio, a serving structure that keeps the embedding tables in a separate service or in host memory and runs only the Transformer on the accelerator is natural. In fact, the released code handles embedding lookup outside the model with numpy fancy-indexing. verified — recsys_model.py:103-116, run_pipeline.py:292-297

4. Content does not enter

Neither text nor images enter X Phoenix. The tokens are only hashed IDs and a few categorical/scalar channels. verified — recsys_model.py:238-250, :307-314

• What goes in: 2 post ID hashes, 2 author ID hashes, action multi-hot, product surface (16 kinds), post age bucket, dwell time
• What does not go in: follower count, engagement count, author reputation, text/topic/language, cross features

↑ Contents

The real scope of the "hand-engineered feature removal" claim

The X README says it removed every hand-engineered feature without exception, but taken literally this is false. Several hand-built elements remain inside the model.

The README writes the following. verified — README.md:55

"We have eliminated every single hand-engineered feature and most heuristics from the system"

Going through what remains inside the model one by one:

• The post age bucket divides time up to 4800 minutes (80 hours) into 60-minute units and has overflow/missing sentinels. A person chose all of these values. verified recsys_model.py:33-55
• Dwell normalization clips at 30 seconds and scales linearly. verified recsys_model.py:58-90
• The product surface is a 16-way categorical, and its value is decided by an if statement on the Rust side. verified home-mixer/scorers/phoenix_scorer.rs:73-77
• ID hashing uses a custom-built linear-congruential hash. There are 2 hashes per entity, and the magic offset is pad = 65. verified run_pipeline.py:76-129
• The 19 action vocabulary entries and the 2x-1 signed encoding were also chosen by people. verified recsys_model.py:425

The accurate statement is "content and author features were removed". The time, surface, and hashing handling remain as they were.

And this concerns only the inside of the model. The serving layer outside the model has 17 filters and many hand-tuned constants. The details are in x.md.

---

↑ Contents

Meta — the Transformer lineage is two, not one

Speaking of "Meta's sequence model" in the singular is wrong. There is an organic HSTU lineage and an ads DHEN→GEM lineage, and as confirmed below they do not cite each other.

↑ Contents

Lineage A (organic) — HSTU → ULTRA-HSTU

Where X's Phoenix is "replace the existing ranker with a Transformer", HSTU changes the problem definition itself. It merges all features into one time series and treats retrieval and ranking as different tasks on the same sequence. verified — Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations, ICML 2024 (arXiv 2024-02)

1. All features are merged into one time series

HSTU merges and encodes all features into one unified time series. verified The paper calls such a model a GR (generative recommender).

"In GRs, we consolidate and encode these features into a single unified time series."

How they are merged is also stated concretely.

• For categorical features, the longest time series is taken as the main time series. This is usually the time series of items the user engaged with. verified
• Slowly changing features (demographics, followed creators) are compressed and merged by keeping only the earliest entry in each consecutive segment. verified The paper's wording is "keeping the earliest entry per consecutive segment".

2. It states why numerical features can be dropped

HSTU argues why numerical features can be dropped. On the surface it resembles the X README claim that it removed "all hand-engineered features" (section above).

"the categorical features (e.g., item topics, locations) over which we perform these aggregations are already sequentialized and encoded in GRs. Hence, we can remove numerical features in GRs given a sufficiently expressive sequential transduction architecture coupled with a target-aware formulation ... can meaningfully capture numerical features as we increase the overall sequence length and compute" verified

That is, counters and ratios are aggregates of categorical sequences, so the logic is that the model reconstructs them if the sequence is long enough.

The paper also attaches verification. If a DLRM (an existing deep learning recommendation model that takes a feature vector as input) is given only the same features as the GR, its performance drops sharply. verified The paper's wording is "the performance of DLRMs is significantly degraded".

inferred X's same claim has no such ablation (an experiment that removes or swaps components to compare performance). Even for the same sentence, the amount of supporting evidence differs.

3. Retrieval and ranking become different tasks on the same sequence

In HSTU, retrieval is predicting the next item, and ranking is predicting the next action on a sequence that interleaves item and action tokens.

• retrieval: next-item prediction p(Φ_{i+1}|u_i)
• ranking: item and action tokens are interleaved: p(a_{i+1}|Φ_0, a_0, Φ_1, a_1, ..., Φ_{i+1})

The reason for interleaving is exactly the same kind of problem that X's mask design addresses. The paper explains that the interaction between the target and the history features must happen as early as possible. In a standard autoregressive setup the interaction happens late (for example, via softmax after the encoder output), so it cannot be made to happen early. verified

"'interaction' of target ... and historical features needs to occur as early as possible, which is infeasible with a standard autoregressive setup where 'interaction' happens late (e.g., via softmax after encoder output)"

4. It drops softmax and gives two reasons

HSTU uses "pointwise aggregated (normalized) attention" instead of softmax, and uses SiLU as the nonlinearity. verified The paper gives two reasons for dropping softmax.

• Because of normalization, softmax erases the information "how many times was it seen." The number of past data points related to the target is a strong feature of preference intensity, and the paper says it is hard to capture after softmax normalization. This is fatal if the goal is to predict intensity such as dwell/time-spent.

  "the number of prior data points related to target serves as a strong feature indicating the intensity of user preferences, which is hard to capture after softmax normalization"

• Softmax is robust to noise by construction, but it is less suited to a vocabulary that keeps changing (non-stationary) in a streaming setting. verified

  "while softmax activation is robust to noise by construction, it is less suited for non-stationary vocabularies in streaming settings"

5. The relative position bias carries both position and time

HSTU's relative position bias (a value added to the attention score according to the distance between tokens) carries both position and time. verified The paper writes this term as rab^{p,t} and describes it as a "relative attention bias that incorporates positional" term, and this term also carries a time term. verified

X handled this slot with RoPE, and that became the cause of the broken batch independence (caveat 3 above). The two companies filled the same slot differently, and the side effects differ too. inferred

6. M-FALCON: solving the same problem as X's candidate block mask in a different way

HSTU uses M-FALCON to split candidates into microbatches and uses the encoder-level KV cache to cut cost and tail latency. verified The name comes from "M-FALCON (Microbatched-Fast Attention Leveraging Cacheable OperatioNs)".

• It splits m candidates into microbatches of b_m and adjusts the attention mask and bias so that the attention operations for the candidates within a microbatch are identical. verified

  "modifying attention masks and rab^{p,t} biases such that the attention operations performed for b_m candidates are identical"

• It reuses the encoder-level KV cache across forward passes to reduce cost, or across requests to reduce tail latency. verified

  "leverage encoder-level KV caching ... either across forward passes to reduce cost, or across requests to minimize tail latency"

This is the most important intersection in this note. X isolated the candidate block with a diagonal mask to make scores cacheable, and Meta amortized the history-encoding cost with candidate microbatches and the KV cache. The two companies independently arrived at "put candidates in the sequence, but isolate them from each other." inferred — the basis is that the mask descriptions in the two sources aim at functionally the same properties (independence between candidates and a reusable prefix).

HSTU's results and deployment surface

The main results reported by the paper are as follows.

• 5.3×–15.2× versus a FlashAttention2-based Transformer at length 8192. verified
• It handles a model 285× more complex than a DLRM at 1.50×–2.99× higher QPS. verified
• The production metric improved by 12.4%. verified

⚠ The surface is anonymized. The paper says only "a large internet platform". We could not confirm whether it was applied to Facebook Feed. See open question 5 in facebook.md.

ULTRA-HSTU (2026)

ULTRA-HSTU is the successor to HSTU, and it begins by criticizing the previous generation. The criticism is that many recent approaches rely heavily on cross-attention to address the quadratic computational bottleneck in sequential modeling, which can limit the representational power of self-attention. verified — arXiv 2602.16986, 2026-02-19

"many recent approaches rely heavily on cross-attention mechanisms to address the quadratic computational bottleneck in sequential modeling, which can limit the representational power gained from self-attention."

As results, the paper presents training and inference scaling speed, serving scale, and consumption and engagement gains. verified

• "over 5x faster training scaling and 21x faster inference scaling"
• "serving billions of users daily"
• "4% to 8% consumption and engagement improvements"

inferred The cross-attention criticism reads as a jab at approaches that "compress in advance and attach", like DV365 (Instagram's ultra-long history compression model, section below). It means two strategies for handling ultra-long history are competing inside the same company. However, the paper does not name DV365 directly.

↑ Contents

Lineage B (ads) — DHEN → Wukong → InterFormer → GEM, serving via MARM

The ads side started from a feature interaction lineage (feature interaction: the combined effect of features) and added sequences later. Its starting point differs from HSTU's. inferred — the basis is the release order in the table below (2022 DHEN and 2024 Wukong are interaction, and sequences begin with InterFormer in 2024). GEM is Meta's ads recommendation foundation model, and as of the 2026-10-13 blog its backbone is Kunlun. verified (GEM 2026 generation)

The GEM blog states the lineage directly. GEM extended the Wukong architecture, and InterFormer is a structure that alternates sequence learning and cross-feature interaction layers. verified

"GEM enhances the Wukong architecture to use stackable factorization machines with cross-layer attention connections"

"Our design, InterFormer, employs parallel summarization with an interleaving structure that alternates between sequence learning ... and cross-feature interaction layers."

LLaTTE: taking the sequence model out of the ranker

LLaTTE is the sequence part of GEM, a structure that moves the heavy sequence model out of the ranker. verified — the blog From User Sequences to Scaling Laws, 2026-08-05, and arXiv 2601.20083. However, a few sentences quoted as coming from LLaTTE are not in the blog body. observed

• It does not keep the sequence model inside the ranker but moves it out into an asynchronous upstream user model. verified (LLaTTE blog) This is the same strategy as Instagram DV365, which is covered again in the DV365 section.

  "a two-stage architecture that offloads the heavy computation of large, long-context models to an asynchronous upstream user model"

  • The upstream user model is a model that separates heavy user modeling from lightweight online ranking and runs it separately at an earlier stage.
• ⚠ The two sentences the canon quoted on 2026-08-09 are not in the body of the 2026-08-05 blog. observed Original text rechecked on 2026-08-10. They may be sentences quoted from the paper, but we have not yet re-verified this.
  • The two sentences are "Deployed as the largest user model at Meta" and "4.3% conversion uplift on Facebook Feed and Reels".
• The measured A/B figures in the blog body are cumulative effects and include other model improvements, so they are not the effect of LLaTTE alone. verified
  • The blog figures are "a cumulative lift of 6% in conversions on Instagram, 3% in conversions on Facebook and 3.5% in ad clicks on Facebook".
  • The blog writes "cumulative" and states explicitly that "broader model innovations" are included.
• LLaTTE's core formulation has two parts. One is a multi-stage sequence model that separates heavy offline user modeling from lightweight online ranking. The other is a learning technique that learns feature interactions directly from data with dense tokenization and target-aware attention. verified (LLaTTE blog)

  "(1) a multi-stage sequence model that decouples heavy offline user modeling from lightweight online ranking tasks and (2) a learning technique based on dense tokenization and target-aware attention that efficiently learns feature interactions directly from data"

  • To confirm the scaling law with the word "prerequisite", you have to check the paper (the re-verification item above).
• ⚠ The quote that semantic features are a prerequisite for scaling is also not in the blog body. observed 2026-08-10; the paper must be checked again.
  • The quote is "semantic features bend the scaling curve: they are a prerequisite for scaling". This is a claim that scaling itself does not work without semantic features.
  • It points in the opposite direction from HSTU saying it "goes with raw categorical engagement features alone".
  • The corresponding blog sentence is "Semantic content features from foundation models complement traditional collaborative filtering... especially helpful in cold-start scenarios". It is a softer expression than "prerequisite".
  • The discussion of LLM-derived features is in llm-in-recsys.md.
• The point that LLaTTE is part of GEM (LLaTTE ⊂ GEM) is confirmed by the primary source, the following sentence in the blog body. verified (LLaTTE blog)

  "This unified platform for sequence modeling is a core component of Meta's Generative Ads Recommendation Model (GEM)"

GEM training infrastructure

The 2026-08-03 blog post gave the lineage table's GEM entry concrete figures on the training infrastructure for the first time. Until then, only GEM's architecture unification (Wukong, InterFormer) was known. verified — Training GEM at LLM Scale, 2026-08-03

• GEM is the foundation model for Instagram and Facebook ads recommendation, and it trains at LLM scale on several thousand of the latest-generation GPUs. verified(GEM Training blog post)

  "the foundation model behind ads recommendations across Instagram and Facebook, now trains at LLM scale on several thousand of the latest-generation GPUs"

  • The blog post describes the hybrid architecture as "trillions of sparse embedding parameters and billions of dense parameters".
• Over the past 12 months, it doubled E2E training efficiency while quadrupling total training FLOPs. verified(GEM Training blog post)

  "we doubled GEM's E2E training efficiency to 20-25% MFU while scaling total training FLOPs 4x over the past 12 months"

  • Here the baseline is before the initial GEM release (2025-11), and the metric is end-to-end Model FLOPs Utilization.
• Infrastructure tuned for LLM training does not transfer directly to recommendation models, so custom kernels were needed. verified(GEM Training blog post) The reasons the LLM stack cannot be reused as is are the recommendation system's sparse embeddings, jagged tensors (sequence tensors of uneven length), and heterogeneous attention.

  "AI infrastructure optimized for LLM training (kernels, parallelism, low precision recipes etc.) does not directly transfer"

• The compute-efficiency techniques are custom attention kernels and MXFP8 low-precision arithmetic. verified(GEM Training blog post)
  • The kernels are JFA (Jagged Flash Attention), GDPA (Generalized Dot-Product Attention), and BlockAttention.
  • MXFP8 is used in attention and the MLP.
• On the scaling-efficiency side, it reduces communication overhead with 2D FSDP, Expert Parallelism (dense), Fully Sharded 2D Model Parallelism (sparse), and SM-free collectives. verified(GEM Training blog post)
• DHEN was absorbed, not replaced. verified(GEM Training blog post) In the blog post text, "Deep Hierarchical Ensemble Network experts" remain as GEM's expert module.
  • As of the 2025-11 generation, "GEM = Wukong·InterFormer unification" in the lineage table is valid, and DHEN is part of it. The backbone replacement in the 2026-10 generation is in the section below.

GEM 2026 generation

As of the 2026-10-13 blog post, GEM's backbone changed from InterFormer to Kunlun, and the per-domain models were merged into one central model. verified — Meta's Generative Ads Model (GEM): Scaling the Central Model and Compute Engine for Frontier Ads AI, 2026-10-13 (blog post, no author listed)

• ⚠ Claim changed: the backbone changed from InterFormer to Kunlun. verified(GEM 2026 blog post) "GEM = Wukong·InterFormer unification" in the table above describes the 2025-11 generation. The blog post compares Kunlun with "the previous version (Interformer)".

  "Compared to the previous version (Interformer), Kunlun delivers 2x performance gains with the same computation scaling."

  • Kunlun first appeared in a 2026-02 paper (arXiv 2602.10016), and the paper's abstract also says "Kunlun is now deployed in major Meta Ads models".
  • CompSkip (computation skipping that bypasses some components) is described as "reduce FLOPS by 43% with negligible quality loss".

• It merged the per-domain models into one. verified(GEM 2026 blog post) It keeps shared experts alongside domain-specific experts, and trains on per-surface data mixed virtually rather than physically merged.

  "The 2026 generation of GEM shifts from isolated per-domain optimizations — such as Facebook, Instagram, Reels — to a single central foundation model."

• It states that it increased the length of long-term engagement history in production by 2× and confirmed the capability to extend it up to 25×. verified(GEM 2026 blog post) Model complexity is 2.5× that of the previous GEM.

• Three methods for transferring knowledge to vertical models were laid out. verified(GEM 2026 blog post)

  1. LoopFM is a post-training technique that passes GEM's internal representations and long-term history memory to vertical models. The blog post claims "2x of transfer effectiveness beyond knowledge distillation".

  2. It also uses soft label distillation.

  3. Solaris precomputes GEM embeddings for the user-ad pairs needed by the next request.

     "It speculatively precomputes foundation model embeddings for the user-ad pairs a future request is likely to need, and delivers them just-in-time on the serving hot path."

• The performance figures the blog post gives are as follows. verified(GEM 2026 blog post)

  • Across Meta ad surfaces and objectives: "18.9% conversion improvement and 12% ad click-through rate (CTR) improvement across Meta's ad surfaces and objectives". It does not disclose the period or the baseline for these two figures.
  • Targeted users: "5.8%, 4.1%, and 1.6% conversion uplift on Facebook, Instagram, and off-site conversion for targeted users over the past year". These three figures are cumulative over "the past 1 year".
  • It does not disclose the A/B design verified (the text has no description of the design).

• Relationship to the organic feed: as of the blog post, GEM trains only on ads data. verified(GEM 2026 blog post) For the future, it writes "extend GEM to learn from all of Meta's platform data, capturing behavior across organic and ads engagements".

  • This is the first official statement that lineage A (HSTU) and lineage B could merge at the data level inferred — it is a statement of plans, not a deployment fact.

• Solaris points in the same direction as LLaTTE's "asynchronous upstream user model". Heavy computation is done ahead of time outside the request path, and the ranker only reads the result inferred — the two blog posts do not cite each other, so it is unknown whether they are the same system.

• The blog post from the same week (2026-10-12), Turning Opportunistic AI Inference Capacity Into Better Ads Recommendations, describes using idle inference capacity for ranking. verified Rank More Ads evaluates more ranking candidates when capacity is available, and Serve Better Models routes requests to a larger model. verified It does not disclose effect figures.

MARM does not say it serves GEM

GEM does not appear in the MARM blog post text. It appears only in the sidebar "Related Posts" link (full text checked 2026-08-09). MARM describes itself as per-request routing and O(1T) parameter scaling, and describes the routing as "replaces a 'one-size-fits-all' inference approach with intelligent request routing". "MARM serves GEM" is plausible but has no primary source. inferred — the only evidence is circumstantial: the same organization, the same period, and the same ads surface.

↑ Contents

The two lineages do not cite each other — measurement

The text of three ads-side blog posts states its own lineage but never mentions HSTU once. observed 2026-08-09

• The corpus examined is the 3 blog post texts in the table below. The sidebar and Related Posts were excluded.
• This is the result of fetching each text and counting strings. To reproduce, fetch the text at each URL and count occurrences of HSTU/Wukong/InterFormer/GEM.
• We say nothing about anything outside this corpus (papers, internal documents).

It is 0 in three independent documents.

inferred "Meta has one unified recommendation Transformer" is a common misconception. At least in the public literature as of 2026-08, the organic lineage and the ads lineage have separate paper families and separate citation graphs. However, the absence of citations does not prove that they share no code. Blog posts rarely cite sources to begin with.

↑ Contents

DV365 — compress the sequence in advance instead of lengthening it (Instagram)

Instagram's DV365 does not lengthen the model's sequence; it compresses long history into embeddings offline in advance and feeds them to multiple models. verified — DV365: Extremely Long User History Modeling at Instagram, KDD 2025

Figures and operational detail are in instagram.md. Here we cover only the structural position.

• The history it covers is up to 70,000 interactions, with an average of 40,000 interactions. verified
• Offline, it builds 200 embeddings with multi-slicing and pooling and feeds them to 15 downstream models. verified
• The paper claims that applying it on top of existing attention-family sequence models (including HSTU) is still incremental. That is, it holds that attention has diminishing returns at a length of 40K. verified

Within Meta there are three answers to "ultra-long history". inferred

• HSTU uses the sequence as is, at full length, and amortizes the cost with Stochastic Length and M-FALCON.
• DV365 compresses it offline into features.
• LLaTTE separates it out as an asynchronous upstream user model.

DV365 and LLaTTE share the same form of "moving heavy computation out of the ranker", and ULTRA-HSTU only criticizes recent approaches that rely on cross-attention in general terms without naming either model. All three directions are in production at the same time. inferred

X has no such distinction at all. The history length in Phoenix's public config is much shorter, and the code has no mechanism for compressing history. verified xai-org/x-algorithm, 2026

↑ Contents

How to read the "unified model" claim

Meta's unification of recommendation models has been declared as a goal and has no evidence in the literature yet. inferred The measurement above (0 HSTU citations) and the two statements below point in the same direction.

This is the part most often misquoted while these notes were being written, so we state it explicitly. The original text and details are in facebook.md.

• Zuckerberg said in Q3 2025 that three giant transformers run Facebook, Instagram, and ads recommendation, and that unification was at the "working on" stage. verified Therefore, by the Q3 2025 statement, the three are separate. Also, Threads is not in this list. See threads.md.

  "there are three giant transformers that run Facebook, Instagram, and ads recommendations"

• ⚠ Susan Li's Q4 2025 "consolidation" statement is not about organic unification. verified The sentence mentions "Facebook Feed", but the performance metric is "a 12% increase in ads quality". It is a per-surface unification of ads ranking models, not a unification of organic Feed ranking.

↑ Contents

X vs Meta — contrast on this topic

Three systems carry out the same slogan, "remove dense features", to different degrees. inferred And the ads lineage goes in the opposite direction altogether. LLaTTE says semantic features are a precondition for scaling. Describing them as one industry trend erases this conflict. inferred

---

↑ Contents

Who uses this

The services confirmed with a primary source are X and Meta. verified Other services have not yet been investigated.

• X: Phoenix is the current production ranker. verified xai-org/x-algorithm, released 2026-01
• X (2025 Scala generation): The same name, Phoenix, was also in the Scala home-mixer (the service that assembles the For You feed) as a rescorer. A Transformer ranker first entered alongside the existing heavy ranker and fully took over in 2026. verified twitter/the-algorithm c54bec0, 2025-09-03
  • 7 files besides PhoenixScorer were newly added in this commit.
  • The heavy ranker is the production ranker of the 2025 Scala generation, and appears under the name Navi in x.md.
• Meta (organic): It runs from HSTU (ICML 2024) to ULTRA-HSTU (2026), and ULTRA-HSTU writes "serving billions of users daily". ⚠ The paper anonymizes the surface, so which app it is cannot be confirmed. verified(existence) / inferred(surface) Details are in facebook.md.
• Meta (ads): It runs from DHEN to Wukong to InterFormer to GEM, and the sequence part is LLaTTE. It is stated explicitly that it was deployed to Facebook Feed and Reels ads. As of the 2026-10 blog post, GEM's backbone is Kunlun, and it handles Facebook, Instagram, and Reels with one model. verified
  • There is no primary source that MARM handles serving for this lineage inferred (MARM section).
• Instagram: DV365 feeds 15 product models of Instagram and Threads upstream. It is a form that moves the sequence model out of the ranker. verified Details are in instagram.md.
• Threads: There is no statement about its own sequence model. It is mentioned only as a consumer of DV365. verified Details are in threads.md.
• Other services: Not yet investigated. Pinterest TransAct and Kuaishou are likely to belong to the same family, but this repository has not yet confirmed them with a primary source. This is an open question.

Retraining and freshness (GEM continuous online training, DV365 6-hour refresh) are in model-freshness.md. We do not repeat them here.

↑ Contents

Not yet verified

On the Meta side, we have not yet confirmed HSTU's deployment surface, the relationship between the two lineages, what MARM serves, or the attention mask shape. On the X side, we do not know Phoenix's production sequence length.

• Which surface HSTU runs on. The HSTU paper says only "a large internet platform". ULTRA-HSTU narrowed it to "a large-scale production video serving platform that reaches billions of users daily", which is currently the most specific information. See open question 5 in facebook.md.
• Whether the two lineages actually share code. We measured that there are no citations (table above), but that does not prove infrastructure separation.
• What MARM serves. GEM is not in the text. Its relationship to LLaTTE and GEM rests on circumstantial evidence only.
• The substance of ULTRA-HSTU's "sparse attention" and "model topology". We confirmed only abstract-level statements.
• The shape of Meta's attention mask. For X, the mask can be seen in code, but Meta disclosed only that M-FALCON "modifies" the mask. We could not confirm exactly what form the isolation between candidates takes.
• Phoenix's production sequence length. Only the mini config is public. It cannot be compared directly with Meta's 8192 / 40K.

↑ Contents

Sources

X

• xai-org/x-algorithm — 2026, public source code (Apache 2.0). All of phoenix/.
• twitter/the-algorithm — 2025-09-03 refresh commit c54bec0, public source code.

Meta — organic lineage

• Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations — ICML 2024 (arXiv 2024-02), paper. Primary source for HSTU, unified time series, pointwise aggregated attention, rab^{p,t}, Stochastic Length, and M-FALCON. ⚠ v4 returns 404 (2026-08-09), so v3 was used.
• ULTRA-HSTU — 2026-02-19, paper. Criticizes reliance on cross-attention and presents 5×/21× scaling.
• DV365: Extremely Long User History Modeling at Instagram — KDD 2025 (arXiv 2025-05-31), paper.

Meta — ads lineage

• DHEN — 2022-03-11, paper.
• Wukong: Towards a Scaling Law for Large-Scale Recommendation — 2024-03-04, paper.
• InterFormer — 2024-11-15, paper.
• Meta's Generative Ads Model (GEM) — 2025-11-10, blog post. Primary source for the Wukong·InterFormer lineage statement.
• LLaTTE: Scaling Laws for Multi-Stage Sequence Modeling in Large-Scale Ads Recommendation — 2026-01-27, paper / From User Sequences to Scaling Laws — 2026-08-05, blog post.
• Meta Adaptive Ranking Model (MARM) — 2026-03-31, blog post.
• Training GEM at LLM Scale — 2026-08-03, blog post. GEM training infrastructure (12-month MFU 2×, JFA·GDPA·BlockAttention·MXFP8, 5D parallelism).
• Kunlun: Establishing Scaling Laws for Massive-Scale Recommendation Systems through Unified Architecture Design — 2026-02-10 (v3 2026-06-05), paper. Backbone of the GEM 2026 generation.
• Meta's Generative Ads Model (GEM): Scaling the Central Model and Compute Engine for Frontier Ads AI — 2026-10-13, blog post. InterFormer → Kunlun replacement, single central model, LoopFM·Solaris.
• Turning Opportunistic AI Inference Capacity Into Better Ads Recommendations — 2026-10-12, blog post. Serving policy that uses idle capacity to increase candidate count and model size. No figures.

Statements

• Zuckerberg Q3 2025 / Susan Li Q4 2025 earnings calls — the original quotes are in facebook.md.

↑ Contents