- Sources read: 2 (X Gen4 release: 1
- arXiv triage: 1 watching of 30)
- Canon files updated: 1 (x.md heavily revised)
- Diagrams updated: 1 (x-pipeline.svg)
- Radar entries updated: 1
Other changes reviewed today:
- The same release published the visibility-filtering sub-repo for visibility verdicts and the labeling code of 15 subsystems, including grox. Some rules are missing.
- SimClusters entered home-mixer (the service that assembles the For You feed) as a source for finding posts from accounts the user does not follow.
- Of the 30 new arXiv cs.IR papers, one video recommendation ensemble ranking paper, DrEM, was added to the radar as a watching item.
1X released ranking weights and a safety verdict pipeline in Gen4 code — X
Source xai-org/x-algorithm @ 47c1bcd — "Open-source X Recommendation Algorithm" — xai-org public repository, commit of 2026-08-13 17:09 UTC, public source code. These notes call this release public-source Gen4. A commit that tidied .gitattributes, a389166, followed. One commit changed +363,246 / −11,640 lines. The files number more than 300 in total; because of the GitHub API pagination limit, the exact file count was reconstructed from the tree. The main evidence documents are README.md #latest-update--august-13th-2026 and docs/BIDIRECTIONAL_BOOST_CHANGE.md. The main code files are home-mixer/params/param.rs and visibility-filtering/rules/registry.rs.
Background For the past 3 months, this repository's services/x.md has kept "public-source Gen3" as the canon. The three generations are:
- 2023 Scala first release
- 2025 Scala refresh:
c54bec0, introduced tweet-mixer - 2026 Rust rewrite:
aaa167b·e414c17·0bfc279, 216 files in total
The canon had two especially empty sections:
- First, weights. It was confirmed that the objective function is a weighted sum, but the actual values appeared in none of the three generations.
- In the 2026 Rust code, the
crate::paramsmodule itself was missing from the repo, so symbols such asFavoriteWeightappeared only at call sites and no definition could be found. - The widely quoted "reply 13.5 / good_profile_click 12.0 / report −369" table comes from a single 2023-04-05 snapshot. Even that document carried the author's own caveat that it "may differ slightly from production".
- In the 2026 Rust code, the
- Second, the safety family, that is, the visibility-filtering and labeling pipelines.
- The home-mixer code applied
VFFilterandAncillaryVFFilteras fail-closed, but who attaches the labels those filters consult, and when and how, was a complete black box. - The
grox/(nearline content classifier) sub-repo had been released only partially, as an annotation pipeline skeleton. - Account-level judgments, PageRank, inauthentic behavior detection, NSFW media classification, the rule engine, and the enforcement service were not even mentioned.
- The home-mixer code applied
This commit overturned both sections: the new README opens with an item saying it added key configuration parameters, including weights: "Adds key configuration parameters (including weights used to blend predicted action values into a score for a post)". Accordingly, 1084 lines of home-mixer/params/param.rs were newly added; together with the following item "Adds code for systems that impact whether a post is filtered from the For You feed", 15 or more subsystems were introduced. Phoenix (X's Grok-based Transformer recommendation model) also received training code, a synthetic data generator, and a QUICKSTART, so it can be run end-to-end. The new SimClusters sub-repo is wired in as an OON (out-of-network, accounts the viewer does not follow) retrieval source of home-mixer.
This note reads the character of this release as having shifted from "showing the shape of the logic" to "presenting values and rules in a verifiable form" inferred; this assessment is not in the README or the commit message (the single line "Open-source X Recommendation Algorithm"). The basis is that caveat list #1 of the Gen3 canon (services/x.md at this repository's e65f30f) wrote "what was released is the shape of the logic; the values and wiring were missing", and this release filled in those missing values. Paired with this, x.com/i/under_the_hood, an account-level label transparency tool, was released as a pilot. According to the new README's claim, users can look up the visibility labels attached to their account and posts, and the tool is designed so they can check those labels against this repo's code to see why they were treated that way.
What we learnedA. The weights are real values in a mirror file that carries production defaults
The numbers in the newly released weights file are not "examples" or "demo defaults"; they are a mirror that a cron job builds by pushing production primary values every day. The upper bound on accuracy is the last sync time, 2026-08-12 04:09 UTC. Representative values are reply 5.0, favorite 0.5, report −234, and share via copy link 20.0 verified.
home-mixer/params/param.rs holds the production defaults stated in the code. The comment in the file header defines the nature of this file precisely.
// mirrored from config feature-switch defaults; last sync 2026-08-12T04:09:22Z—home-mixer/params/param.rs:1
The new README's "Experiments and Configuration" section states the meaning of this mirroring as follows:
"To help people understand the production defaults, we run cron scripts that set the defaults in this repository's code to be the primary production values, for example in
home-mixer/params/param.rs." — README.md#experiments-and-configuration
There are 21 positive *Weight params; the table below adds 2 mutual-follow boost params to these and lists them in the order they appear in the file verified (source: the param.rs link above).
| Head | param symbol | Default | Notes |
|---|---|---|---|
| favorite | FavoriteWeight |
0.5 | Matches 0.5 in the 2023-04-05 snapshot |
| reply | ReplyWeight |
5.0 | Differs from 13.5 in the 2023 snapshot |
| ↳ + mutual-follow original-post boost | BidirectionalFollowReplyWeightBoost |
15.0 | Introduced at 20 on 2026-07-13, lowered to 15 on 2026-07-24 (see the case below) |
| retweet | RetweetWeight |
1.0 | Matches the 2023 snapshot |
| photo_expand | PhotoExpandWeight |
0.05 | |
| video_open | VideoOpenWeight |
0.05 | |
| click (post) | ClickWeight |
0.4 | Different scale from 2023 good_click 11 and good_click_v2 10. Possibly a separate head |
| open_link | OpenLinkWeight |
0.2 | |
| profile_click | ProfileClickWeight |
0.0 | Removed entirely from good_profile_click 12 in the 2023 snapshot |
| video quality view (vqv) | VqvWeight |
0.05 | |
| share | ShareWeight |
2.0 | |
| share via DM | ShareViaDmWeight |
5.0 | |
| share via copy link | ShareViaCopyLinkWeight |
20.0 | Highest positive individual head |
| dwell | DwellWeight |
0.0 | The actual dwell signal is in the continuous items below |
| ↳ + mutual-follow original-post boost | BidirectionalFollowDwellWeightBoost |
0.0 | Code exists but not deployed |
| quote | QuoteWeight |
5.0 | |
| quoted_click | QuotedClickWeight |
0.05 | |
| quoted_vqv | QuotedVqvWeight |
0.0 | |
| follow_author | FollowAuthorWeight |
4.0 | |
| post_unexplored | PostUnexploredWeight |
0.02 | New-author boost, see below. PostUnexploredWeightInNetworkOnly = true |
| continuous dwell time | ContDwellTimeWeight |
0.004 | |
| continuous click dwell time | ContClickDwellTimeWeight |
0.0 | |
| continuous active_secs 5-minute residual | ContActiveSecs5mResidualNormWeight |
0.0 | "Attention" category in the new README |
There are 5 negative head weights verified.
| Head | param symbol | Default | 2023 snapshot counterpart |
|---|---|---|---|
| not_interested | NotInterestedWeight |
−43.2 | No counterpart (head did not exist then) |
| block_author | BlockAuthorWeight |
−31.2 | No counterpart |
| mute_author | MuteAuthorWeight |
−58.8 | No counterpart |
| report | ReportWeight |
−234.0 | 2023 report −369 → toward milder |
| not_dwelled | NotDwelledWeight |
−0.02 | No counterpart (the only new negative head the canon had confirmed) |
The weights reflect both the value an action has in ranking and the frequency (propensity) with which that action typically occurs on the network verified. The three comment lines directly above the weight block say so, and they do not write the formula for combining the two factors.
"These weights reflect a combination of how much an action is valued in ranking and typical propensities of these actions across the X network (e.g. negative feedback is overall rare)." —
home-mixer/params/param.rs:279-281
This note reads the magnitude of report −234 not as the "relative importance of one report" but as the product value × 1/propensity inferred. The basis for thinking the absolute value is large because reports are so rare is "negative feedback is overall rare" in the comment.
What we learnedB. The OON multiplier, diversity, and VMRanker constants were all released too
The OON multiplier, diversity, and VMRanker (a remote service that reorders the candidate bundle to send to the feed) constants, which the canon had long recorded as "all four values undisclosed", have all been released.
- The decay factor multiplied into OON posts equals the value in the 2023-04-05 snapshot verified.
OonWeightFactor = 0.75, and the snapshot'sout_of_network_scale_factoris also 0.75.
- For topic requests, the OON decay is applied more strictly verified.
TopicOonWeightFactor = 0.5
- The new-user routing threshold is 0 seconds verified. A threshold of 0 seconds means that this code path is effectively mirrored as inactive inferred.
NewUserAgeThresholdSecs = 0
- Author-diversity decay is on, and the decay rate and floor values were released verified.
AuthorDiversityDecay = 0.5,AuthorDiversityFloor = 0.25,EnableAuthorDiversity = true- The 2nd post by the same author is multiplied by
(1−0.25)·0.5¹+0.25=0.625, and the kth post by (1−0.25)·0.5ᵏ⁻¹+0.25. - The exponent in the code is the number of earlier posts by the same author, and the first post is 0 verified —
home-mixer/scorers/ranking_scorer.rs:614-665.
- VMRanker is on, uses DPP (a selection technique that prevents mutually similar candidates from being picked together), and has no fallback verified. Therefore the canon's observation that on failure it is silently skipped as fail-open remains valid.
- This is a direct answer to the canon's open question "Is VMRanker/DPP on?".
VMRankerDppTheta = 0.65,VMRankerDppMaxSelectedRank = 150,EnableVMRanker = trueVMRankerValueModelId = "dpp",VMRankerClusterId = "Experiment3",VMRankerEnableFallback = false
- The legacy tweet-mixer is mirrored as off by default, and SimClusters replaces it verified.
- Maximum results per source:
PhoenixMaxResults = 1000,ThunderMaxResults = 1200,TweetMixerMaxResults = 800 - Source switches:
EnableTweetMixerSource = false,EnableSimclustersSource = true,EnablePhoenixSource = true
- Maximum results per source:
- Shadow traffic of 30% flows to the new Phoenix cluster by default verified.
ShadowTrafficDefaultPercent = 0.3
- The point where the canon recorded "no mechanism for new items" is actually defined by an impression threshold verified.
ColdStartImpressionThreshold = 1000- A post with fewer than 1000 impressions is classified as a "new author/post" and boosted by
PostUnexploredWeightin C below.
What we learnedC. There are three score adjustment factors, and the third, absent from the canon, is the new-author boost
The new README states three scoring adjustment factors explicitly verified. The third, the New-Author Boost, lifts posts from authors whose impressions are below the threshold toward a target position verified. The three factors are written in the README.md #scoring-and-ranking section as follows:
"Author Diversity ... Out-of-Network Discount ... New-Author Boost: posts from authors whose impressions are below a threshold are lifted toward a target position." — README.md
#scoring-and-ranking
The canon had recorded only two adjustment factors:
- Author-diversity decay
- OON multiplier
A third one, the new-author boost, is added to these.
- By default it is an additive boost; the multiplicative form is disabled; and it applies only in-network verified.
- Definition in param.rs:
PostUnexploredWeight = 0.02·PostUnexploredWeightInNetworkOnly = true EnableMultiplicativePostUnexplored = false·MultiplicativePostUnexploredAlpha = 0.0ColdStartImpressionThreshold = 1000
- Definition in param.rs:
What we learnedD. SimClusters came back as a candidate source for posts from accounts the user does not follow
The candidate sources were reorganized into the following four:
- Thunder (an in-network source that keeps recent posts from followed accounts in memory)
- Phoenix retrieval
- SimClusters
- The legacy tweet-mixer, set to off by default
For a long time the canon recorded "SimClusters, TwHIN, UTEG/GraphJet, FRS, Earlybird, Lists, Communities — none of these names appear anywhere in home-mixer/"; of these, SimClusters was overturned. The new README describes the candidate sources as follows:
"In-Network
thunder/keeps recent posts from the accounts a viewer follows in memory · Out-of-Networkphoenix/retrieval andsimclusters/find posts from accounts the viewer does not follow" — README.md#overview
This README statement means that a SimClusters sub-repo is newly created and that the core algorithm of simclusters/ (2020 KDD) is included here. The core finds KnownFor communities not with SVD but with Sparse Binary Factorization based on Metropolis-Hastings verified — UpdateKnownForSBFRunner.scala:5-9.
- This file imports
com.twitter.sbf.core.MHAlgorithmandSparseBinaryMatrixverified. topics/embedding-retrieval.md also describes the same stage as Metropolis-Hastings community detection.
Confirming the details requires a deep dive into all of simclusters/.
What we learnedE. Filters increased, and there was also one filter the canon missed
Filters grew from 14 to 17 verified — README.md #filtering. Three filters were newly added, and one more filter that the canon missed is listed here.
OONRetweetReplyFilterfilters out retweets and replies from accounts the user does not follow, as well as replies without a parent post. It is separate from the existingRetweetDeduplicationFilterverified (README).OONNsfwSimclustersFilterdrops posts from adult-content accounts that come from the SimClusters source for users who do not follow them. It was introduced alongside SimClusters verified (README).NewUserMinEngagementFilterremoves, from OON posts shown to new users, those below an engagement threshold verified (README).- The related params are
NewUserMinEngagementThreshold,NewUserMinEngagementFilterMetric,NewUserMinEngagementFilterUseRatio,NewUserMinEngagementFilterMaxAccountAgeSecs, andNewUserMinEngagementFilterMaxResurrectionAgeSecs.
- The related params are
InventoryHoldoutFilterdeterministically hashes user × post and holds out the configured share of inventory (sets it aside from impressions). It is a filter the canon missed and did not list verified (README).
The canon's observations, such as the no-op behavior of BackupFilter and the hardcoded supertopic in TopicIdsFilter, remain valid.
What we learnedF. There are 24 prediction heads in the documentation
The new README rearranged the heads into 5 categories, 24 in total verified — README.md #scoring-and-ranking.
- Engagement (7): favorite · reply · repost · quote · share · share via DM · share via copy link
- Clicks (6): post · profile · link · photo expand · video open · quoted post
- Attention (5): video quality view · dwell · dwell time · click dwell time · active seconds
- Author (1): follow author
- Negative (5): not interested · mute author · block author · report · not dwelled
The canon had recorded that the head count splits three ways: 19 in the released checkpoint, 22 in the Rust scorer, and 21 in the VMRanker proto; the new README states 24 in the documentation. There are 26 *Weight params in param.rs (21 positive, 5 negative), and removing the two that are not in the README list matches 24 verified. The two removed are PostUnexploredWeight, a heuristic weight rather than a head prediction, and QuotedVqvWeight, whose name is not in the README verified. The actual count in the source code can be confirmed only through a deep dive into the new phoenix training code.
What we learnedG. Visibility rules apply 28 to following posts and 54 to recommended posts
The rules file of the visibility-filtering sub-repo defines three policies verified.
FilterAlldrops everything verified.TimelineHomeis the default rule applied to following targets verified.TimelineHomeRecommendationsapplies to recommendation targets and adds OON-only drops to the default rules verified.- The rules file is
visibility-filtering/rules/registry.rs, and it is 912 lines verified.
There are 28 rules applied in common to following and recommendations verified. These rules in base_home_rules() run in the following order:
- Account status:
SUSPENDED_AUTHOR_DROP,DEACTIVATED_AUTHOR_DROP,ERASED_AUTHOR_DROP,OFFBOARDED_AUTHOR_DROP,ProtectedAuthorDropRule - Viewer relationship:
ViewerBlocksAuthorRule,ViewerMutesAuthorRule,MutedRetweetsRule - Tweet labels:
PDNA_DROP,BOUNCE_DROP,SPAM_DROP,FOR_EMERGENCY_USE_ONLY_DROP,FOSNR_HATEFUL_CONDUCT_DROP,FOSNR_VIOLENT_SPEECH_DROP,FOSNR_ABUSE_DROP,FOSNR_CIVIC_INTEGRITY_DROP - Tweet status:
NullcastedTweetDropRule,DropStaleTweetsRule,DropLegalTakendownPostRule,DropLocalLawsTakendownPostRule - NSFW viewer gating:
SensitiveViewerLoggedOutDropRule,SensitiveViewerUnderageDropRule,SensitiveViewerNoStatedAgeDropRule - Exclusive tweets:
DropExclusiveTweetContentRule - NSFW interstitials (handling that leaves the post in the feed but overlays a warning screen):
NSFW_HIGH_PRECISION_INTERSTITIAL,GORE_AND_VIOLENCE_INTERSTITIAL,NSFW_CARD_IMAGE_INTERSTITIAL,NsfwAuthorInterstitialRule
Recommended posts get the 28 above plus 26 OON-only rules, for 54 verified. The rules added by timeline_home_recommendations_policy() run in the following order:
- 2 DMCA and geo-restricted media rules
- 4 NSFW account rules
- 9 rules for tweets carrying NSFW, violence, DoNotAmplify, and spam labels
- 11 user-label rules related to NSFW, spam, impersonation, deactivated, and adult
This is the substance of the "different safety levels for in-network and OON" that the "Worth noting" bullet in the filter section of the canon (x.md) had already observed. Following users get only the 28 rules, and OON recommendations get 54. Evaluation is short-circuit, ending at the first drop verdict, and the OON-only rule group can issue only catch-up drops, with no interstitial or allow verdict (consistent with the same bullet in the canon).
We confirmed what the fail-closed bullet in the canon's filter section pointed to. The policy return value in visibility-filtering/rules/registry.rs is a Verdict struct, and the README states that interstitial is passed to UI handling and drop counts as actual filtering verified — README.md #labeling-path.
"interstitial ──► the post stays in the feed; nothing in this repository draws the interstitial"
What we learnedH. The roles of the subsystems that attach labels were officially documented for the first time
The new README organizes the label-attaching process as a 4-stage "Labeling Path". The stages run in the order Content Understanding → Labeling Rules → Storage → Visibility Filtering. The role of each subsystem was officially documented for the first time verified.
grox/: a nearline content classifier. It was already partially reflected in the canon.media-model-proxy/: a proxy that serves image and video models (NSFW, violence/gore, hate symbols, topic classification, known-media matching).clip/: trains CLIP image and text embedding models. They become the input to the media classifiers.adult-content/: trains and calibrates the adult media classifier.pnsfwmedia/: an adult media discriminator that combines CLIP embeddings with account scores.agatha/: an offline batch. It attaches labels based on how other accounts reacted to this account's posts, using the ratio of favorites to blocks, reports, and spam reports.bdsm/: extracts inauthentic/abusive signals from an account's time-series behavior sequence. It consists of Rust, proto, training, and tests subdirectories.user-cred-v2/: runs PageRank over follow and engagement edges to produce an account credibility score.scarecrow/: an event-reactive rule runner. It embedsbotmaker/as its rule engine.botmaker/: holds the rule DSL, compiler, and runtime.botmaker-rules/: the actual rules scarecrow loads. Some were not released, to prevent gaming.abuse-enforcement-service/: decides account actions (labeling, challenge, suspension) based on model scores.safety-label-user-agg/: aggregates the labels received by posts an account owns to the account level.visibility-filtering-client/: the client library of the visibility-filtering service.under-the-hood/: an account-level label transparency report builder. It consists of a daily batch and a serving-layer aggregation.
Of these 15, the subsystems other than grox had not been reflected in the canon by even one line until today.
What we learnedI. Only Phoenix can now be run from training to serving
The statement "the whole repo does not build" that the canon observed in "How to read the source — caveat list" #1 was partially resolved. Only Phoenix was shipped in a form that can run end-to-end through training and serving. The other crates still have undisclosed dependencies verified.
- Examples of the remaining undisclosed dependencies are
xai_service_runnerandxai_kafka. crate::params,crate::util, andcrate::clients, which Gen3 lacked, were released this time together with their files verified —home-mixer/lib.rs.lib.rsdeclarespub mod params;,pub mod clients;, andpub mod util;, and the tree has files underhome-mixer/params/,util/, andclients/.
- The new commit added several
Cargo.tomlfiles, concentrated in the phoenix subtree.
The new README states:
"Elsewhere, code may not necessarily include build- or deployment-related files or generally self-explanatory infrastructure imports (e.g.
xai_service_runnerorxai_kafka). ...phoenix/ships a Cargo workspace, apyproject.toml, aquickstartand synthetic data generation, so a small model can be trained and served end-to-end." — README.md#deployment-related-code
What we learnedJ. The Phoenix retrieval index consults the visibility verdict before a post is added
A new index service for Phoenix retrieval was also released verified — README.md #retrieval-index.
phoenix-rankall/maintains the post index that Phoenix retrieval looks up. It is updated each time an event arrives verified (README).phoenix-rankall-strato/is the event layer that decides which index a post belongs to. This layer consults visibility-filtering first verified (README).
The indexing pipeline is already synced with the ranking-side visibility rules, so a visibility judgment is attached before a post enters the index. This is an architectural fact the canon had not grasped at all.
What we learnedK. Five caveats must always be stated when citing the statements above
- The weights are a mirror that a cron job pushes daily, so there is a time lag. Changes made in production after the sync time in the header may not yet be reflected verified.
- Propensity must be considered when interpreting the sign and magnitude of the weights. The file comment (
param.rs:279-281) says only that the weights reflect both the value of the action and propensity verified. Reading this as value × 1/propensity is this note's interpretation inferred.- The ratio of absolute values, report −234 to favorite 0.5 (468×), does not directly mean that "a report is 468 times more important than a favorite".
- The mirroring scope is limited to params inside
home-mixer.- The internal constants of the VMRanker remote service, phoenix training hyperparameters, and botmaker rule constants are still outside the repo verified.
- The score normalizer that the Gen3 canon expected to be in
crate::utilis not in the Gen4 tree, andhome-mixer/util/mod.rsdoes not declare it either verified. vqv_weight, which the same canon pointed out, was released this time verified —home-mixer/util/candidates_util.rs:19.
- Phoenix training code is "trainable", not "identical to production".
- The README makes clear that the scale differs: "a small model can be trained and served end-to-end".
- Actual production checkpoints, data, and infrastructure are not public.
- Some botmaker rules were not released, to prevent gaming. The README states this explicitly. It means that for some of the labels visibility-filtering consults, the rules are not public verified.
What we learnedL. The process of changing one parameter was documented from the A/B start to the downward adjustment
One new document is a model for how parameter changes will be documented as code diffs going forward. The document is docs/BIDIRECTIONAL_BOOST_CHANGE.md, and the case it covers is the bidirectional follow reply boost of 2026-07.
- On 2026-07-10, an A/B started.
BidirectionalFollowReplyWeightBoostwas randomly assigned from 5/10/15/20, and 0 was applied to most users. - On 2026-07-13, the initial broad rollout was run, expanding to 20.
- On 2026-07-24, it was lowered from 20 to 15 in response to user feedback received on X. The feedback pointed out that relevant posts were scarce while the World Cup was under way.
The document carries this summary as is:
"In plain language, the bidirectional follow boost boosts original posts from people you mutually follow by increasing the weight on the predicted probability that you'll reply to a post from one of those authors who you mutually follow." —
docs/BIDIRECTIONAL_BOOST_CHANGE.md
- In the code, the boost is attached only to original posts, and replies and retweets are not targets verified. This rule is very specific logic that the canon lacked verified.
home-mixer/candidate_hydrators/bidirectional_follow_hydrator.rs(new): looks up mutual-follow status via SocialGraphClient and sets it inPostCandidate.is_mutual_follow_author.home-mixer/models/candidate.rs: added one field.home-mixer/scorers/ranking_scorer.rs: definesbidirectional_boost_eligible(c)asin_reply_to_tweet_id.is_none() && retweeted_tweet_id.is_none() && is_mutual_follow_author == Some(true), and when the condition holds, adds the boost to the reply and dwell weights.- The condition applies only to pure originals; replies and retweets are excluded, and anything that is not a mutual follow passes through unchanged.
Canon update
- Substantially revised services/x.md. The main changes are as follows.
- Added a Gen4 row to the "Public source Gen3" table (2026-08-13, +363k/−12k lines, more than 300 files, 15 new visibility-filtering and labeling sub-repos, including
phoenix/training code). - Added SimClusters to the candidate source table, marked
TweetMixeras off by default, and inserted the valuesEnableSimclustersSource=trueandEnablePhoenixSource=true. - Updated the candidate allocation section. Inserted the values
TweetMixerMaxResults=800,PhoenixMaxResults=1000, andThunderMaxResults=1200, plusOonWeightFactor=0.75andTopicOonWeightFactor=0.5. Rewrote the canon's "all four values undisclosed" as "three values public (the one for new users has a threshold of 0, so it is effectively inactive)". - In the ranking section, rewrote the Phoenix prediction head table to match the new README's 24-category layout and mapped it to the 26
*Weightparams in param.rs (21 positive, 5 negative). - Fully rewrote the weights section. Overturned "Conclusion: values undisclosed" into "Conclusion: production mirror published after 2026-08-13". Inserted a detail table (favorite 0.5, reply 5.0, +mutual-follow original 15.0, retweet 1.0, ..., report −234) and added a section contrasting the 2023 snapshot with the details.
- In the VMRanker section, inserted the values
EnableVMRanker=true,VMRankerDppTheta=0.65,VMRankerDppMaxSelectedRank=150, andVMRankerValueModelId="dpp". Moved the open question "Is VMRanker/DPP on?" to the resolved items. - Updated the filter section from 14 to 17 filters. Added the entries
OONRetweetReplyFilter,OONNsfwSimclustersFilter,NewUserMinEngagementFilter, andInventoryHoldoutFilter. - Introduced a new-author boost subsection. It is the third adjustment factor after Author Diversity Decay and OON Discount.
- Introduced a visibility-filtering / labeling pipeline section. The rule count is base 28 + OON extra 26 = 54, and the sub-repo table lists the names below.
agatha·bdsm·user-cred-v2·pnsfwmedia·scarecrow·botmaker·abuse-enforcement-service·safety-label-user-agg·under-the-hood·clip·media-model-proxy
- Introduced a Phoenix training and index section. It covers the Cargo workspace, pyproject.toml, QUICKSTART, synthetic data, and phoenix-rankall / phoenix-rankall-strato. It also records the architectural fact that indexing consults the visibility verdict first.
- Folded the
docs/BIDIRECTIONAL_BOOST_CHANGE.mdcase into a "How the weights change" subsection (2026-07-10 A/B → 07-13 rollout → 07-24 20→15). - Caveat #1 (the repo does not build) in "How to read the source — caveat list" in the canon is half resolved. Phoenix now builds.
- Resolved many open questions in bulk: the actual weight values, VMRanker on/off, the share of
has_cached_posts(still unresolved), the number of Phoenix heads (confirmed as 24), and whether SimClusters was revived (confirmed as revived). - Added new open questions.
- Core algorithm details of the SimClusters sub-repo (SVD pipeline and k)
- Scoring details and update cycles of agatha, bdsm, and user-cred-v2
- Loss, negative sampling, and optimizer of the
phoenix/training pipeline - The label taxonomy that the Under the Hood tool exposes
- The holdout-ratio param of InventoryHoldoutFilter
- Added a Gen4 row to the "Public source Gen3" table (2026-08-13, +363k/−12k lines, more than 300 files, 15 new visibility-filtering and labeling sub-repos, including
- Updated
assets/x-pipeline.svg.- Added SimClusters to the candidate sources and marked tweet-mixer as off by default.
- Corrected the filter count from 14 to 17.
- Changed "values undisclosed" in the weights section to "see canon (e.g., reply 5.0 + mutual-follow boost 15.0)".
- Updated the prediction heads to 24.
- Added the values "DPP θ=0.65 · K=150 · on by default" to VMRanker's "silently skipped on failure".
- Updated radar.md. Because this release applies directly to a tracked service (X), it went straight into the canon without passing through the radar. Only one item, related to the arXiv triage below, was added to the radar.
Why it matters ⚠ Mass claim change. Two of the biggest gaps in the X canon until today (the weight values and the labeling pipeline) were resolved on the same day. A canon rewrite of this scale is rare for this repository. More than 300 lines of the x.md canon were changed or newly written today.
- The first implication is a realignment of the research direction.
- Over the past 3 months, the open questions in x.md have driven the observation of other services (Meta, Netflix). Questions such as "How much of the X ads stack is public?", "Is SimClusters actually revived?", and "What is the deployment state of VMRanker DPP?" set the direction of several topics/ notes.
- Because these questions were answered today, this repository now moves to the next-stage question: "why are the values X published what they are, and what are the counterpart checkpoints at other companies?"
- For example, X's report weight of −234 and Netflix GenRec's catalog softmax are two extremes of entirely different loss functions. The fact that these two numbers now sit side by side in the canon is itself a reference point for the next topics expansion.
- The second implication is a signal that industry standardization of transparency architecture is beginning.
- The documentation approach shown by
docs/BIDIRECTIONAL_BOOST_CHANGE.mdpublishes everything from the A/B start through result observation and parameter adjustment as a code diff within 3 weeks. No other company (Meta, Netflix, or YouTube) has had a case like this so far. - If X calls this an "example algorithm diff" and announced it will repeat it, then a comparable production diff time series would exist for the first time.
- In that case, the value of this repository tracking X's commit log daily also grows considerably. This is because which param changes in the next commit, and how, will serve as a lagging indicator of A/B results.
- The documentation approach shown by
- The third implication is that part of the canon caveat list must be adjusted again.
- The Phoenix part of caveat #1 (the repo does not build) in "How to read the source — caveat list" is resolved, but the rest is only partially resolved.
crate::paramswas filled in by param.rs, andcrate::utilandcrate::clientswere also published in the same commit verified. - Caveat #2 (the three scorers named by the README are dead files) still needs verification. They may have been cleaned up in the new release.
- Caveat #6 (the share of has_cached_posts) remains unresolved.
- The canon caveat list will be re-verified file by file in the next batch (deep dive needed).
- The Phoenix part of caveat #1 (the repo does not build) in "How to read the source — caveat list" is resolved, but the rest is only partially resolved.
The items below need a deep dive, and a human must trigger them and check them in separate sessions. Given the scale of this release, file-by-file verification exceeds today's batch budget.
simclusters/sub-repo: core algorithm (resolved: it is Metropolis-Hastings SBF, see D above), index format, the API surface thathome-mixercalls, and the diff against the 2020 KDD paperagatha//bdsm//user-cred-v2/: the actual model and update cycle of each account scorer, and label thresholdspnsfwmedia/calibration: how it combines the CLIP embedding and the agatha scorephoenix/training code: loss, negative sampling, optimizer, and synthetic data formatphoenix-rankall/andphoenix-rankall-strato/: the indexing pipeline and the visibility pre-check logicabuse-enforcement-service/: which score maps to which action (label/challenge/suspension)scarecrow/,botmaker/,botmaker-rules/: the rule DSL and compiler, and the ratio and categories of public versus undisclosed rulesunder-the-hood/: how the label taxonomy exposed to users connects to the rule code- Remaining new
home-mixer/files: new hydrators, filters, scorers, and side-effects (re-verify the canon caveat list) - Individual rule files in
visibility-filtering/rules/: the verdict condition of each rule and final verification of the safety-level bullets in the canon filter section
2One video recommendation paper was added to the radar as watching, with 0 canon updates — arXiv
Source arXiv API cs.IR latest 30 (2026-08-12–13) — arXiv API, new paper list.
Background The daily batch skims the 30 new arXiv cs.IR papers by title and abstract. It then checks the original text only for papers that hit at least one of: production authors, an A/B claim, mechanism novelty, or scale results. Today, most of the 30 were academic increments such as RAG/QA/text-to-SQL/multimodal benchmarks, or studies limited to a domain such as music, POI, or medicine. This triage checked the original text of 3 candidates that passed.
What we learnedA. Among the candidates whose original text was checked, only DrEM passed into watching
- STAR (2608.12986) was rejected. A competition framework is not a production claim.
- The title is "Structured Tokenization and Target-Aware Interest Representation for PCVR Prediction".
- The abstract itself describes it as a "framework for the KDD Cup 2026 Tencent UniRec Challenge". It has no production deployment claim.
- HCGRec (2608.11980) was rejected. It is an academic increment in the SID (semantic ID, an ID that represents an item as a sequence of meaning-based codes) family.
- The title is "Hint-Conditioned Generative Recommendation with Semantic IDs".
- The author affiliation is not public (no affiliation on the arXiv page). It is marked as CIKM 2026 accepted and has a GitHub link. It has no production deployment claim.
- DrEM (2608.12778) passed into watching. The author affiliation is not public, but the abstract claims actual deployment.
- The title is "Dual-Side Robust Ensemble Ranking from Noisy User Preference Predictions in Video Recommendation".
- As the basis for the deployment claim, the abstract writes: "large-scale online A/B tests".
- The author names (Xiaoxiao Xu, Kaiqiao Zhan, and others) are likely to statistically match the Kuaishou family inferred.
- Added a 3–4 line pointer to "New this month" in radar.md (no deep dive was done). The company and the production surface will be confirmed by cross-checking in the next batch.
Canon update
- Added a new DrEM (video rec, unknown company) entry to "New this month" in radar.md. The 3–4 line pointer notes that there is an A/B claim, that the author affiliation could not be confirmed, and that it is likely a Kuaishou line inferred.
Why it matters The radar's daily pass count falls within the expected 0–2. This week the SID and generative ranking family keeps adding increments (the Alibaba, Kuaishou, and Yandex lines are already being watched in the canon), but there is not yet any statement about a new company or a new mechanism.
Corrections
- The x.md canon statement "there is no primary source for weight values in any generation. The only numeric source is a single snapshot as of 2023-04-05" was overturned on 2026-08-13.
home-mixer/params/param.rs, as a production cron mirror, published all actual values: the 26*Weightparams (21 positive, 5 negative), 2 mutual-follow boosts, the OON multiplier, and the diversity constants.
- The x.md canon statement "SimClusters ... these names appear nowhere in
home-mixer/" was partially overturned on 2026-08-13. Thesimclusters/sub-repo was connected to home-mixer as an OON retrieval source.- However, the canon sentence "effectively only tweet-mixer survives as retrieval from the 2023 generation" was rewritten as "the retrieval that survives from the 2023 generation is tweet-mixer (mirrored as off by default) + SimClusters (revived)".
- The pipeline diagram (assets/x-pipeline.svg) that held the x.md canon statement "weight values and the params module are outside the repo — undisclosed" had to be updated, and the update is done.
- For the x.md canon open question "Is VMRanker/DPP on?", we confirmed from the param that it is on, and DPP is θ=0.65, K=150. This item was removed from the open question list.
- Updated the x.md canon's "17 filters (14 before scoring + 3 after)" to 20 (17 before scoring + 3 after).
- The x.md canon statement "the 2026 code has no mechanism at all for [new item cold start]" was overturned.
PostUnexploredWeight = 0.02,ColdStartImpressionThreshold = 1000, andNewUserMinEngagementFilterfor new users each exist. The canon's observation of "absence" was based only on a snapshot from 3 months ago.
Radar
- Added a new DrEM (2608.12778) watching entry to "New this month" in radar.md. It has an A/B claim in the video rec field, and the author affiliation could not be confirmed.
What to check next
- Deep dive on the X Gen4 release — among the 10 deep-dive items above, the priority order is as follows.
- Re-verifying the canon caveat list with the new
home-mixer/files - Settling the canon's SimClusters "revived" statement with the
simclusters/sub-repo - Reconstructing the safety pipeline from
abuse-enforcement-service/and the individual rules invisibility-filtering/rules/
- Re-verifying the canon caveat list with the new
- Confirm the DrEM author affiliation — if it is the Kuaishou family, it is a company already watched in the canon; if it is elsewhere, it becomes a candidate for a new thread in the radar.
- Tracking X commits after BIDIRECTIONAL_BOOST_CHANGE — check when and in what form the next param diff appears. The repository's daily batch already checks X commits, so it will be detected automatically.
- EU DSA transparency report — see what X newly submits or removes after this 8-13 release. Whether the regulatory documents and the code are in sync becomes a new verification angle.