0x00 한

Research / Daily reports

  • Sources read: 1
  • Canon files updated: 1
  • Radar entries: 1
In one lineShopify unified search and recommendation into a single product semantic token (SID) generation model and reported a 7.95% increase in orders in a Shop app home feed A/B test.

1Search and recommendation unified in one SID generation model — Shopify

Source Generative Search and Recommendation at Shopify — Shopify, 2026-11-05 v1, paper.

Background Product search and recommendation are usually run separately with different models and indexes. With SID (semantic ID, an identifier that represents an item as a short token sequence extracted from its content), both tasks can be recast as the same problem: "generate the next SID." Shopify named SID as the next step in a 2026-02 blog post, and in 2026-10 it also published a counter-paper finding that an SID-LLM is worse than a small dense encoder on internal data. This paper reports the result of building an SID-LLM that handles search and recommendation together and putting it into Shopify's production search and recommendation stack.

What we learnedA. Adding the SID-LLM to the home feed candidate pool raised orders and conversion together

  • In the Shop app home feed A/B test, the arm with the SID-LLM added saw orders on the affected surface rise 7.95%, purchasing users 8.60%, and conversion rate 9.85% verified — paper §9.2.2.
    • It writes that all three metrics remain significant after Dunnett correction (a multiple-comparison correction used when comparing several treatment arms with one control).
    • The home feed experiment used an autoregressive model with a joint-probability decoder.
  • The SID-LLM on the recommendation path does not replace the existing candidate generators; it converts the generated SIDs into 200 products and adds them to the candidate pool verified — paper §8.
    • The best model used in the offline comparison is based on LFM2-2.6B.
  • Offline, the SID-LLM contributed more than the other candidate generators in the pool verified — paper §9.2.1.

    • The metric is incremental pool recall (how much the overall pool recall drops when that generator is removed from the pool). It was measured on next-week purchases of 70,747 users with purchases.

    • The values with each generator producing 200 candidates are:

      Candidate generator Recall@50 (%) incremental pool recall (pp)
      SID-LLM (LFM2-2.6B) 24.61 2.65
      Recent interactions 17.28 1.64
      HSTU 17.21 0.78

"On the impacted surface, adding SID-LLM raised attributed orders by 7.95%, converting users by 8.60% and conversion rate by 9.85%. All three lifts are significant after Dunnett correction" — Generative Search and Recommendation at Shopify (paper), 2026-11-05

What we learnedOther changes

  • The search path serves a multi-token prediction generator that predicts several tokens at once together with a lightweight ranker that re-ranks 250 candidates verified — paper §8.
  • It writes that SID-LLM-only search matched the quality of the entire existing multi-model search stack within the latency requirements verified — paper abstract.
  • The conclusion section writes that the search stack will increasingly move to a single model, but does not disclose a plan to replace the recommendation side verified — paper §10.

Canon update radar.md added one line of 2026-11-10 additional evidence to the ### Generative retrieval / semantic ID thread and added a Shopify SID-LLM entry to ## New this month.

Why it matters SID generative retrieval produced production gains at Shopify by adding to the candidate pool rather than replacing the existing candidate generators. We judge that it does not conflict with the same company's October counter-paper and keep it as an inference. The evidence is that the counter-paper compared replacement, while this paper measured the parallel effect.

Corrections

  • The 2026-11-09 daily wrote that all 30 arXiv responses were a batch triaged earlier, but the Shopify paper (2611.08804) among them was in no record. Today we reapplied the acceptance criteria and added it to the radar.

Radar

What to check next