2026-09-28

1Expanded ads lightweight ranking from two-tower to 3-tower — Radar (Pinterest)

The architecture long used to handle this scale is the two-tower. The two-tower builds embeddings for the user side (query) and the item side (Pin) separately and scores with a dot product. Pin-side embeddings can be precomputed and cached outside the request, so scoring itself stays cheap no matter how many candidates there are. The cost is shallow expressiveness. Because the query-Pin interaction is handled by a single dot product, it cannot capture high-order interactions between the user sequence and candidate features. The industry convention was not to use anything beyond a two-tower in the lightweight ranking role, because the candidate count is too large to leave room for a heavier model.

In the earlier post (Part 1), Pinterest deployed a serving stack called Nexus that separated candidate generation from scoring. Nexus set up a "platform on which, in principle, anything from cross-attention to sequence modeling can be served", and this Part 2 is the first modeling statement built on top of it. It settles the first architecture that adds expressiveness on that platform while staying within the serving budget. This post is the first production statement that overturns that convention in the lightweight ranking role.

• The model is a 3-tower structure made of a two-tower, a cross tower, and an inter module verified. It keeps the existing two-tower (query and Pin) as is and adds the cross tower as a third tower.
  • The cross tower takes three kinds of user sequences and four kinds of candidate features as input.
  • The user sequences are organic engagement, ads engagement, and search history.
  • The candidate features are advertiser ID, campaign ID, GraphSAGE embedding, and Pin PinnerSAGE embedding.
  • The inter module merges the query, Pin, and cross embeddings with a shared-bottom MLP (an MLP in which the three tasks share the lower layers) to produce the logits for the three tasks.
  • The authors also tried stacking DCN (Deep & Cross Network) layers and adding MMoE as alternatives. However, they state that the shared-bottom MLP gave the largest latency gain relative to the performance loss.
• The core design is to "turn candidate features into query tokens and compute cross-attention with the user sequence" verified.
  • This lets the query axis of the attention be controlled by the number of candidate features, so serving cost does not blow up with the candidate count.

• The three tasks the model predicts are pCTR, pGCTR30, and pOCTR verified.
  • pCTR is the probability of an ad click.
  • pGCTR30 is the probability of a good click, a click lasting 30 seconds or more. It looks at whether the user actually stays on the ad landing page.
  • pOCTR is the probability of an outbound click that leaves for the advertiser's site.
  • There were earlier signs that Pinterest's ads ranking optimizes all three metrics at once. However, this post is the first to publicly state in production that the three metrics are handled as three heads of one model in the lightweight ranking role.
• The same model produces a fast score with the two-tower dot product for all candidates and an accurate score with the 3-tower path for a selected subset verified.

  • The two predictions have different losses but share parameters and are trained together (co-train).

  • The blog post states the loss function exactly as follows:

    "L = (100 * L_main_ctr + 5 * L_main_gctr30 + 1 * L_main_octr) + (10 * L_cotrain_ctr + 5 * L_cotrain_gctr30 + 1 * L_cotrain_octr)" — Beyond Two Towers Part 2

  • The main (three-tower) CTR is weighted 100 and the co-train (two-tower) CTR is weighted 10. gCTR30 and oCTR are weighted 5 and 1 in main, and 5 and 1 in co-train.

  • Main and co-train are separated so that the offline loss reduction concentrates on the three-tower prediction while the co-train two-tower head stays no worse than the production two-tower model it will replace.

• The embedding dimensions are split by task verified.
  • The query tower produces 192 dimensions, split into 144 for CTR, 32 for gCTR30, and 16 for oCTR.
  • The Pin tower outputs 448 dimensions in total: the corresponding dimensions plus a 256-dimension candidate-feature projection for the cross tower. This projection is cached offline.
  • In serving, the fast CTR score uses only the first 64 dimensions of the 144-dimension CTR segment. The three-tower CTR score uses the entire CTR segment together with the cross embedding.
  • For gCTR30 and oCTR, the fast and slow paths share the entire embedding.

• Serving is a two-stage flow, and the utility topK is 40K verified.
  • Stage 1 scores all candidates with the co-trained two-tower head and computes utility.
  • Stage 2 re-scores only the top 40K by utility with the three-tower.
  • 40K is about the 70th percentile of candidates for Home Feed and Related Pins and the 90th percentile for Search. In most requests, most candidates reach the three-tower, but in P99 requests with an extremely large number of candidates, only the first 40K pass.
  • The two-tower score and the three-tower score are blended before the final top-K selection. Candidates beyond 40K move on to later stages with their two-tower score kept.
• Cutting the candidate subset with a heuristic can distort calibration (how well predicted probabilities match actual rates), and the authors address this concern explicitly verified.
  • They state that on auction candidates, the three-tower model actually reduced over-calibration, bringing predicted CTR closer to measured CTR.

• The four serving optimizations together cut P99 model inference time from over 200ms to about 30ms, and the final model is only 1–2ms slower than production verified.

  "Together, these changes reduced P99 model inference from over 200 ms to about 30 ms, leaving the final model only 1–2 ms slower than production." — Beyond Two Towers Part 2

  • The authors measured the effect of each component separately. Of the four optimizations, late expansion and BF16 gave the largest gains.
  • They moved Pin preprocessing into the Pin tower. The cross module originally ran an MLP per candidate; they attached a 256-dimension projection to the Pin tower and moved it to an offline cache. At 100K candidates, this is -2ms.
  • They changed FP32 to BF16. Only the three-tower path lowers precision, going from 103ms to 56ms at 50K candidates and from 45ms to 26ms at 20K candidates. Model quality is neutral.
  • They applied late expansion of user features. The point at which user features are expanded (broadcast) to the candidate batch size was pushed deeper into the attention, rather than right before the attention keys and values are built. At 100K candidates, this is -15ms.
  • They switched to pre-layer normalization. In cross-attention, LayerNorm had been applied after the attention to a large sequence; they changed it to pre-norm, applied to the input (batch 1) before the attention. At 100K candidates, this is -3ms.

• On held-out standard ads data, the loss of the three-tower main prediction fell by ~30% in relative terms compared with the existing production two-tower verified.
  • The co-train two-tower prediction was roughly on par with the production two-tower. Only the gCTR30 co-train loss got slightly worse, which the authors judge acceptable given the main task gain.
  • Based on this result, the authors state explicitly that "the standalone production two-tower engagement model can be deprecated and the co-train head relied on with no quality loss".

• In the online A/B test on standard ad surfaces (Home Feed, Related Pins, Search), the three engagement metrics each rose about 1% verified. Cost-per-click fell about 1%, and GPU spend increased by a "modest" amount verified.

  "In online A/B experiments on standard ads, the three-tower co-train model delivered around 1% gains in CTR, gCTR30, and oCTR, along with nearly 1% reductions in CPC, with only a modest increase in GPU spend." — Beyond Two Towers Part 2

  • Because the title is "Launching" and the body states the deprecation of the production two-tower model it replaces, we judge this to be an actual deployment.

• It is the first production statement that extends the two-tower in ads lightweight ranking.
  • Until now the corpus had several ads ranking statements (Meta ads scaling laws, Snap CAMIE, Tencent ANGLE and ChronicleRec, TikTok Shop ReST, and others). However, most statements aimed at the lightweight/pre-rank role focused on candidate reduction and KD.
  • Pinterest presents a practical pattern that adds expressiveness in the lightweight ranking role while avoiding a latency blowup with the candidate count. Its core is the idea of "building query tokens from candidate features to control the attention query axis".
  • This pattern may be reusable outside ad surfaces with large candidate counts.
• The co-train pattern, which trains a fast head and a slow head together, appears for the first time in the lightweight ranking role.
  • Earlier statements in the repository with similar ideas were UNIQUE (Baidu), UniRec (Kuaishou), and OneTrans-V2. All of them aimed to unify different stages of a cascade (a recommendation pipeline that narrows candidates over several stages).
  • Pinterest 3-Tower puts a fast head and a slow head in one model within a single stage (lightweight ranking) and re-scores only the candidate subset with the slow head.
  • Splitting fast and slow within a stage, rather than unifying across stages, is a new role in the repository.
• An ads lightweight ranking axis is added to the Pinterest statements the repository has covered.
  • While Pinterest CLR covered the candidate generation axis of the organic Home Feed and PinDCO covered the ads creative selection axis, 3-Tower Co-Train is a statement on the ads lightweight ranking axis.
  • With this source, it becomes clear how Pinterest's production recommendation and ads stack divides into three roles in the repository (organic retrieval / ads creative / ads lightweight ranking).
  • The three statements were published between 2026-07-21 (PinDCO arXiv v1) and 2026-09-17 (3-Tower blog post). This is a point at which Pinterest is visibly increasing its public communication about the ads ranking pipeline.

Some open questions remain:

• The repository has not yet covered how candidate generation and scoring were separated in Part 1 (Nexus). If the next batch reads the Nexus details closely, the "platform the 3-Tower sits on" itself will enter the corpus, and this statement will be placed more precisely.
• The three directions the authors state explicitly as follow-ups are also worth tracking.
  • Expansion to objectives other than engagement (for example, web conversion)
  • Longer sequence modeling
  • Stronger offline, online, and infrastructure feedback loops

Lookup summary

The only source that passed today is Pinterest Engineering's 3-Tower post. The other sources had no new commits and no posts that passed.

• X (xai-org/x-algorithm): There are 0 new commits. The latest commit is 4c5cfe8 (2026-09-26), already covered in yesterday's daily.
• Netflix Tech Blog: There are 0 relevant new posts. The latest post (2026-09-25, "Trading a Cloud Identity for Your Own") is on a security and infrastructure topic.
• engineering.fb.com / ai.meta.com: There are 0 relevant new posts.
• Google Research blog (Monday scan): There are 0 relevant new posts. The latest post (2026-09-24) is on a video generation topic.
• Pinterest Engineering (Monday scan): One post passed, the 3-Tower post covered in the main text. We also saw 2026-09-11 "Evolving Pinterest's Embedding Retrieval Platform" in the list. However, the repository's Pinterest CLR thread already covers the organic Home Feed retrieval axis, so folding it into the main text is deferred to the What to check next items.
• arXiv cs.IR triage: We parsed 30 and 0 passed.
  • Beyond a Scalar: Distributional Serving Interfaces for Watch-Time Prediction (2609.28383) was rejected. Checking the HTML version, all four authors are affiliated with Shanghai Jiao Tong University and Rice University, that is, academia, and there is no production deployment or A/B statement. The datasets are also the public KuaiRec, KuaiRand-1K, and WeChat21 rather than company production logs. It is an academic method, separate from the trend of watch-time distribution prediction being stated by a second production company after VK HEGM.
  • The other 29 are either leftovers of the batch already triaged in yesterday's 09-27 daily (submitted 2026-09-23–24) or academic RAG, QA, and retrieval methods. The newly parsed batch overlapped with yesterday's because arXiv cs.IR has had no new submissions since 2026-09-25. 2026-09-25 is a Friday but this category had no new submissions, 2026-09-26 and 2026-09-27 are a weekend, and the 2026-09-28 batch is not yet public.

Radar

Added the Pinterest 3-Tower Engagement Co-Train entry to new this month.

What to check next