- Sources read: 1
- Canon files updated: 1
- Radar entries: 0
1Home-mixer's experimental dwell-regret alternative scoring was rolled back — X
Source xai-org/x-algorithm@8b25829 — GitHub commit, author commit 2026-09-18T23:55:39Z, 41 files, +2,877/−1,828. These notes number the commits that came after X's public source Gen4 (2026-08-13, 47c1bcd) in order; this commit is Gen4+25.
Background This commit was registered 22 hours after the commit covered yesterday, Gen4+24 (c279172), on the same date (2026-09-18). The 1,828 deleted lines roughly balance the 2,877 added lines, but the deletions are concentrated in three intertwined files (item A below). The commit rolls back one experimental scoring mode in home-mixer and also touches several other areas.
The commit covers five axes.
- A. In the ranking scoring of home-mixer (the service that receives X's feed requests, gathers candidates, and ranks them), one experimental scoring mode introduced early in Gen4 was dropped.
- Deleting the whole file
home-mixer/scorers/value_model_gate.rs(−537) also cuthome-mixer/scorers/ranking_scorer.rssubstantially, by −465 lines. home-mixer/params/param.rslost 18 related parameters, at −110 lines.
- Deleting the whole file
- B. A cold-start slice, distinguished by a dataset_type tag, was formally wired into Phoenix (X's For You ranking and candidate retrieval model stack) retrieval.
- C. Phoenix bundle export was extended from ranking only to also support two-tower (a retrieval model that embeds users and posts separately and matches them) retrieval.
- In the Phoenix serving/train line,
recsys_bundle_export.pywas substantially reworked at +559/−107.
- In the Phoenix serving/train line,
- D. VF (visibility-filtering, a service that decides per post whether it may be shown to this viewer) gained a server implementation of the
evaluate_tweetsgRPC, which was a stub until yesterday. A hydrator also appeared that replaces the visibility fields TES (Tweet Entity Service, an X-internal tweet metadata service) used to supply. This hydrator fetches fields from Strato (an X-internal system that code goes through to read or store data) columns.- On the addition side, two new files appeared:
visibility-filtering/evaluate_tweets.rs(+258) andvisibility-filtering/hydration/tes_composite.rs(+607).
- On the addition side, two new files appeared:
- E. In infrastructure cleanup, the CUDA kernels were split out of the xrex repo into the external package
xrex_cuda_kernels. Also, fav-count-based reasoning gating was added to the adult spam clone detection flow of grox (a Grok-based nearline classifier sub-repo).- The name of this detection flow starts with
media_injected_adult_infrared.
- The name of this detection flow starts with
Of the five axes, A and D are events in different directions: the unwinding and the maturing of experiment infrastructure that has built up in Gen4. A is the cleanup of a separate scoring mode introduced early in Gen4, because the measured results were poor (no evidence is public; it is observed only as the deletion of whole files). D is a maturing flow in which a stub that had only the client interface open in Gen4+24 yesterday now extends to server logic.
What we learnedA. The experimental scoring mode gated dwell-regret and its user gate were rolled back entirely
The mode that used dwell-regret scoring through a gate, and the file that defined its per-user gate, were both rolled back verified.
- The evidence file
home-mixer/scorers/value_model_gate.rsdoes not exist at 8b25829. - The mode is
gated_dwell_regret, and the file isvalue_model_gate. - In the earlier Gen4 open-source history, this file defined
GateModel::from_params(). - GateModel normalized 19 user-state features with log1p, then produced a gate score as a logistic linear combination. It then flipped the assignment by request hash only within a hysteresis band.
- The feature list is
seq_len·n_fav·n_reply·n_rt_quote·n_vqv·n_click·n_bm_share·n_profile_follow·n_photo·n_negfb·n_7d·n_1d·active_days·active_days_7d·days_since_last·span_days·followers·followings·account_age_years. - The patch of the rollback commit shows this entire logic deleted, at −537 lines.
- The evidence file
GateModel fired in
ranking_scorer.rsonly when theValueModelModeparameter was in gating mode. At that point, depending on per-user conditions, it decided whether to use a dwell-regret-family alternative score based on dwell time.ValueModelModehad three values (weighted/dwell_regret_sigmoid/gated_dwell_regret).
8b25829 removed all three value branches.
- let use_dwell_regret = match query.params.get(ValueModelMode).as_str() { - DWELL_REGRET_SIGMOID_MODE => true, - GATED_DWELL_REGRET_MODE => GateModel::from_params(&query.params) - .is_some_and(|gate| gate.serve_new_scoring(query)), - _ => false, - };—
home-mixer/scorers/ranking_scorer.rs, the deleted portion of 2026-09-18 (the actual source is gone)There is no replacement logic. The conditional branch itself was removed, and only the
use_dwell_regret == falsebranch remains in theweighted_partsandweighted_scorescomputation.- Home-mixer scoring was simplified once more after Gen4+8 (2026-08-25) moved client-side head-weight generation to the server, and settled on a single weighted path.
Related elements were also deleted from
ranking_scorer.rsverified —home-mixer/scorers/ranking_scorer.rs(+12/−465).- The
DwellRegretWeightsstruct and itsfrom_params()constructor.- The struct's 13 tuning fields were
alpha_favorite·alpha_reply·alpha_retweet·alpha_quote·alpha_share·alpha_share_via_dm·alpha_share_via_copy_link·neg_not_interested·neg_block_author·neg_mute_author·neg_report·temperature·dwell_floor.
- The struct's 13 tuning fields were
- The whole
compute_dwell_regret_base_scores()function. This function averaged each parameter within the candidate set, then turned each candidate's relative preference intocentered_ratio = p / mean − 1. It multipliedpositiveandnegativebysigmoid(positive/T)andexp(min(negative,0)/T)respectively, and produced the final score asdwell × modulation. In other words, it is an alternative scoring function that "modulates dwell time by engagement signals". - The
centered_ratio()andsigmoid()helpers. - The logic that skipped
MultiplierPreOffsetunder theuse_dwell_regretbranch. - Seven regression tests, the
GATE_WEIGHTS_ALL_ZEROconstant, and the"rust_home_mixer_value_model_mode": "weighted"override line in the existing regression tests.- Test names include
dwell_regret_neutral_post_scores_dwell,dwell_regret_above_average_likeable_is_boosted,dwell_regret_report_sinks_candidate,dwell_regret_floor_applies_to_low_dwell,dwell_regret_per_request_normalization_amplifies_for_low_engagement_set, and others.
- Test names include
- The
The related parameters were also deleted symmetrically in the parameter mirror verified —
home-mixer/params/param.rs(+8/−110).- This file is a mirror that periodically carries over production config values. This mirror sync time changed to
2026-09-17T20:46:44Z → 2026-09-18T16:21:20Z. - 18 params were deleted.
- This file is a mirror that periodically carries over production config values. This mirror sync time changed to
| Name | Feature switch key | Default |
|---|---|---|
ValueModelMode |
rust_home_mixer_value_model_mode |
"weighted" |
DwellRegretTemperature |
..._dwell_regret_temperature |
10.0 |
DwellRegretDwellFloor |
..._dwell_regret_dwell_floor |
1.0 |
DwellRegretAlphaFavorite |
..._dwell_regret_alpha_favorite |
1.0 |
DwellRegretAlphaReply |
..._dwell_regret_alpha_reply |
1.0 |
DwellRegretAlphaRetweet |
..._dwell_regret_alpha_retweet |
1.0 |
DwellRegretAlphaQuote |
..._dwell_regret_alpha_quote |
1.0 |
DwellRegretAlphaShare |
..._dwell_regret_alpha_share |
1.0 |
DwellRegretAlphaShareViaDm |
..._dwell_regret_alpha_share_via_dm |
1.0 |
DwellRegretAlphaShareViaCopyLink |
..._dwell_regret_alpha_share_via_copy_link |
1.0 |
DwellRegretNegNotInterested |
..._dwell_regret_neg_not_interested |
−10,000 |
DwellRegretNegBlockAuthor |
..._dwell_regret_neg_block_author |
−8,000 |
DwellRegretNegMuteAuthor |
..._dwell_regret_neg_mute_author |
−15,000 |
DwellRegretNegReport |
..._dwell_regret_neg_report |
−60,000 |
DwellRegretGateWeights |
..._dwell_regret_gate_weights |
19-feature logistic weight string |
DwellRegretGateBias |
..._dwell_regret_gate_bias |
1.033918 |
DwellRegretGateThreshold |
..._dwell_regret_gate_threshold |
−0.634264 |
DwellRegretGateHysteresisBand |
..._dwell_regret_gate_hysteresis_band |
0.0 |
The production default of the rolled-back parameter confirms that the gate held actually trained weights. This is because the default string of
DwellRegretGateWeightswas still present at the time of deletion."seq_len:0.530298,n_fav:-0.082139,n_reply:0.485541,n_rt_quote:0.056561,n_vqv:-0.072778,n_click:-0.176675,n_bm_share:-0.167574,n_profile_follow:-0.221285,n_photo:-0.106004,n_negfb:0.031839,n_7d:-0.075799,n_1d:-0.241730,active_days:0.047017,active_days_7d:-0.126896,days_since_last:-0.034238,span_days:-0.052186,followers:-0.066642,followings:0.064140,account_age_years:-0.045455"- Strong positive weights were placed on two axes,
seq_len(current session sequence length) andn_reply(28-day reply count). - Negative weights were placed on the remaining interaction counts (
n_click,n_bm_share,n_profile_follow,n_1d). - In other words, the goal appears to have been serving dwell-regret alternative scoring to users who are "long-session active and have strong recent reply activity".
- These coefficients are no longer applied to any user.
- Strong positive weights were placed on two axes,
The experimental alternative function that sat beside ranking scoring in Gen4 (modulating dwell time by engagement signals) and its user gating were removed all at once, and ranking scoring narrowed back to a single weighted path. How to reflect the dwell signal here appears to have moved to adding heads at training time and redefining score combination inferred. Examples are the click_dwell_time sigmoid BCE (threshold=10s) formally wired in Gen4+22 and the EnableCdwellOnImpr (click × dwell product) of Gen4+23 inferred. The evidence is that the timing of the two flows is adjacent, and the observation that once the circuit modulating dwell through a separate scoring function disappeared, only circuits treating dwell as a head itself and as a combination term remain.
What we learnedB. A cold-start slice was wired into Phoenix retrieval, but it is not requested by default
A cold-start slice distinguished by a dedicated tag was formally wired into Phoenix retrieval verified.
- The evidence file is
home-mixer/sources/phoenix_source.rs(+67/−19). - The tag is
HOME_COLD_DATASET_TYPE=13.
- The evidence file is
A new parameter
PhoenixColdStartMaxResultswas added, so two caps are passed together when calling the retrieval RPC.- The feature switch is
rust_home_mixer_phoenix_cold_start_max_results, and the default is0.
quality_factor::apply(query.params.get(PhoenixMaxResults)), + query.params.get(PhoenixColdStartMaxResults),- Because the default is 0, production by default still does not request the cold-start slice.
- The feature switch is
Response candidates carry dataset_type, and served_type (the source label attached to a candidate) differs by that value.
- The response candidate (
ScoredCandidate) carries adataset_type: u32field. - If that value is
HOME_COLD_DATASET_TYPE = 13, served_type maps to the new enumForYouPhoenixRetrievalCold; otherwise it maps to the existingForYouPhoenixRetrieval.
const HOME_COLD_DATASET_TYPE: u32 = 13; fn served_type_for_dataset(dataset_type: u32) -> pb::ServedType { if dataset_type == HOME_COLD_DATASET_TYPE { pb::ServedType::ForYouPhoenixRetrievalCold } else { pb::ServedType::ForYouPhoenixRetrieval } }- The response candidate (
A matching observation line was attached on the side-effect side too verified —
home-mixer/side_effects/scored_stats_side_effect.rs(+25/−0).- The new function
record_phoenix_retrieval_cold_statsemits size statistics of selected/non-selected candidates to the stat receiver under the prefixPhoenixRetrievalColdTweets. - It is the third observation axis alongside the existing MoE slice (
PhoenixRetrievalMoeTweets). MoE (Mixture of Experts) is a model structure that, for each input, picks and uses some of several expert subnetworks.
- The new function
The MoE source and Topics source do not share this cold-start slice logic (confirmed in the patch).
phoenix_moe_source.rsandphoenix_topics_source.rseach also grew by 1 line, but both are new import line additions.- The two sources keep using the
ForYouPhoenixRetrievalMoeandForYouPhoenixRetrievalTopicsserved_type as before.
Until now, X's cold-start handling had two axes:
- Users whose history is below a threshold are routed to a separate model fleet. Retrieval and ranking each have
Phoenix*NewUserInferenceClusterId. - New authors (at or below
ColdStartImpressionThreshold=1000) get aPostUnexploredWeight=0.02boost, and there is the Thompson sampling option from Gen4+1.
This wiring opens a third axis at the candidate level. Within the candidate set returned by the same Phoenix retrieval RPC, a separate cap is placed on the "cold" subset by the dataset_type tag, and it is observed. That is, Phoenix retrieval itself appears to have been extended to always return a cold candidate slice together, regardless of user history inferred. The evidence is that the default is 0, so it has not yet entered the production default, but the observation stat lines were also wired symmetrically, which indicates that the A/B infrastructure is complete.
What we learnedC. Phoenix bundle export can now export the forward program of two-tower retrieval as well as ranking
Phoenix bundle export was extended to export the two-tower retrieval forward in addition to the ranking forward verified.
- The evidence file is
phoenix/xrex/train/recsys_bundle_export.py(+559/−107). - This file was introduced at 634 lines in Gen4+13 (2026-09-01, 7ba7768). It exported the ranking model (
RecsysAggregatedModelConfig), lowered withjax.export, to StableHLO MLIR bytecode,.jax_export, andMANIFEST.json. - This commit attached retrieval two-tower program export alongside it. Two new axes appeared.
- The evidence file is
Axis 1: the number of program kinds grew to two verified. Two constants distinguish the program kind within the export bundle.
KIND_RANKING = "recsys_ranking_forward" KIND_RETRIEVAL = "recsys_retrieval_forward"Axis 2: because the retrieval export must carry additional resources in the bundle, new data structures and wire keys were introduced.
POST_TABLE_KEY = "post_embeddings.embeddings" POST_DATASET_TYPES_KEY = "post_embeddings.dataset_types" POST_IDS_KEY = "post_embeddings.post_ids" AUTHOR_IDS_KEY = "post_embeddings.author_ids"- The new dataclass
RetrievalExportholds five attributes that must be extracted from the trainer'sRetrievalModelRunnerat export time. The attributes arelarge_k,retrieval_dataset_types,enable_async_topk,enable_radix_select_topk, andenable_dataset_slice_topk. - The manifest block carries the items below.
- large_k
- target dataset types (a list of name·value pairs)
- post table geometry (rows·width·dtype)
- three data keys (POST_TABLE_KEY·POST_DATASET_TYPES_KEY·POST_IDS_KEY·AUTHOR_IDS_KEY)
topk_unorderedflag
- The new dataclass
The retrieval forward extracts a user embedding with the user tower, matches it against the whole post table, and returns top-K for each target dataset type verified — new function
_make_retrieval_forward_fn.def forward_fn( batch: Any, merged_embeddings: jax.Array, post_table: jax.Array, dataset_types: jax.Array ): recsys_embeddings = _recsys_embeddings_from_merged( merged_embeddings, embedding_slices, packed_geometry ) model = model_config.make(sharding_context=make_legacy_sharding_context(mesh)) results = model.forward( batch, recsys_embeddings, post_table, dataset_types, retrieval.large_k, target_values, dataset_ranges=None, use_async_topk=retrieval.use_async_topk, use_radix_select_topk=retrieval.use_radix_select_topk, post_scales=None, ) flat: list[jax.Array] = [] for indices, scores in results: flat.append(indices) flat.append(scores) return tuple(flat)- Each target dataset type yields two outputs (
indices_<name>,scores_<name>). This naming rule is defined in theRetrievalExport.output_namesproperty.
- Each target dataset type yields two outputs (
For two-tower, the rule for computing the serving sequence length differs.
_make_export_configalso gained two arguments,two_tower: bool = Falseandmesh_devices: int = 1.
def _user_tower_sequence_len( *, training_seq_len, training_history_seq_len, num_user_prefix_tokens, history_seq_len, ) -> int: prefix_tokens = training_seq_len - training_history_seq_len if prefix_tokens != num_user_prefix_tokens: raise ValueError( f"cannot derive the serving user-tower sequence_len: sequence_len " f"{training_seq_len} - history_seq_len {training_history_seq_len} = {prefix_tokens} " f"prefix tokens, but the model reserves num_user_prefix_tokens=" f"{num_user_prefix_tokens}. The user tower is built as prefix + history; if " "history_seq_len was overridden for serving, also override " "model_config.user_tower_config.model_config.sequence_len to " f"{num_user_prefix_tokens + history_seq_len} (prefix + serving history), as the " "jax slot does — otherwise the attention kernel cannot be sized for serving" ) return prefix_tokens + history_seq_len- The ranking export's serving sequence is
prefix + history + candidate, but the retrieval export has onlyprefix + history. There is no candidate axis, and it emits only the user embedding. - The overrides of
_make_export_configalso usemodel_config.user_tower_config.model_config.sequence_leninstead ofmodel_config.model_config.sequence_len.
The third new addition is distributed export. The retrieval bundle is exported over a mesh and can be distributed across multiple devices.
if not two_tower and mesh_devices != 1: raise NotImplementedError( "ranking programs are single-device (one executable per GPU); " f"mesh_devices={mesh_devices} is only for retrieval bundles" )- The reason is also stated explicitly. Ranking is served with one executable per GPU, but retrieval has to split the post table across several devices, so it is exported with SPMD.
_export_meshchecksexport_cfg.parallel_config.mesh_shape()to build the devices._retrieval_shardingsspecifiesNamedShardingfor batch/params/post_table/dataset_types.- Embedding rows are partitioned across the data axis, and params are replicated.
Several constraints are imposed explicitly.
enable_int8_post_table,enable_bloom_filter, andenable_topic_filterare not supported. These options are supported only by the native runtime, and StableHLO export uses a bf16 post table with no per-post filter.enable_dataset_slice_topk=Trueis ignored and a warning is raised. The export program always compiles only the mask-then-topk-over-whole-table path.- The
using_seqpack + two_towercombination is prohibited. Packing does not fit the SPMD model. - With
DEBUG_ALLOW_RANDOM_INIT=1, the export itself is refused. This is because exporting without a mask would emit an invalid program.
build_bundlenow accepts both kinds of models.two_tower = isinstance(model_config, RecsysTwoTowerModelConfig) if not two_tower and ( not isinstance(model_config, RecsysAggregatedModelConfig) or isinstance(model_config, RecsysGenRecsModelConfig) ): raise NotImplementedError( "StableHLO bundle export supports ranking (RecsysAggregatedModelConfig) and " f"retrieval (RecsysTwoTowerModelConfig) only, got {type(model_config).__name__}" )
Until now, Phoenix serving carried only the ranking forward in a StableHLO bundle. With this commit, preparation for carrying retrieval two-tower in the same bundle frame is complete. Because infrastructure for exporting retrieval distributed across several devices via SPMD was attached too, the direction is for Phoenix retrieval's serving stack to be unified under the same bundle format as ranking. It may not be a coincidence that this landed at the same time as the PhoenixColdStartMaxResults wiring in B above inferred. In effect, while the retrieval serving axis was being tightened, the slice observation interface via the dataset_type tag settled alongside it inferred.
What we learnedD. VF now has a server that receives verdict requests, and takes the visibility fields for verdicts from a Strato column instead of TES
- A server implementation of the gRPC that receives verdict requests was attached to VF, and the visibility fields TES used to supply moved to a Strato column hydrator verified.
- New files:
visibility-filtering/evaluate_tweets.rs(+258),visibility-filtering/hydration/tes_composite.rs(+607) - Files changed together:
visibility-filtering/server.rs(+86/−2)visibility-filtering/filter.rs(+336/−13)visibility-filtering/hydration/tes_hydrator.rs(+113/−320)visibility-filtering/hydration/mod.rs(+78/−22)
- This gRPC is
evaluate_tweets.
- New files:
This axis splits into two sub-axes.
D1. The verdict-request RPC that was a stub until yesterday was filled in with server logic by this commit.
daily/2026-09-18.md records the observation that Gen4+23 (42266f3) on 2026-09-17 opened only the client interface of the
evaluate_tweetsRPC and the Thrift decoder for the response payload. According to the same observation, the server implementation wasErr(Status::unimplemented("EvaluateTweets is not served yet")).This commit fills that
evaluate_tweetsstub with server logic. Theserver.rspatch directly shows the unimplemented stub being replaced with a real implementation.async fn evaluate_tweets( &self, - _: Request<vf_pb::EvaluateTweetsRequest>, + request: Request<vf_pb::EvaluateTweetsRequest>, ) -> Result<Response<vf_pb::EvaluateTweetsResponse>, Status> { - Err(Status::unimplemented("EvaluateTweets is not served yet")) + self.evaluate_tweets.handle(request).await }The new server endpoint
EvaluateTweetsEndpoint(258 new lines) receives a request, runs the internal filter, and fills the response with a per-tweetOutcome.- It receives the request's three fields (
viewer_id,country_code,safety_level) and a tweet list. - safety_level is converted from the Thrift enum to the internal Rust
SafetyLevelenum. Only three values are allowed:FILTER_ALL,TIMELINE_HOME, andTIMELINE_HOME_RECOMMENDATIONS. - It then calls
FilterTweets::run.
let candidates = req.tweets.iter() .filter(|o| o.quote_context.is_none()) .map(|o| RawCandidate { tweet_id: TweetId(o.tweet_id), request_author_id: None }) .collect(); let response = self.filter_tweets.run(FilterRequest { viewer_id, country_code, safety_level, candidates }).await;- It receives the request's three fields (
Three metrics (
evaluate_tweets_requests,evaluate_tweets_latency_ms,evaluate_tweets_batch_size) are recorded for each request.The
evaluate_tweets_loopbackintegration test pins the fact that the server logic is wired in. This test starts the gRPC server on a tokio TCP listener and makes a round-trip call with XaiVfClient.
D2. Besides the verdict, an evaluation status is attached to the result.
An
EvaluationStatusaxis was added toFilterOutcome. A new enum appears infilter.rs.#[derive(Clone, Copy, Debug, PartialEq, Eq)] pub enum EvaluationStatus { Evaluated, UnresolvedAuthor, Failed, }Previously the filter could emit only one
Verdictper tweet. Now one of three statuses is attached next to the verdict.- A tweet whose hydration succeeded is
Evaluated. - A tweet whose author did not come out of hydration is
UnresolvedAuthor, and its verdict isVerdict::unresolved_author(). - A tweet that hit the rules with insufficient fields because hydration itself failed is
Failed. HydrationOutputalso gained a new fieldfailed_ids: HashSet<TweetId>to distinguish the third case (Failed).
- A tweet whose hydration succeeded is
The axis names line up with the client-side
EvaluationResult { Evaluated(Box<Action>), NotEvaluated, Failed }(introduced in Gen4+23), and we infer that the server and client axes mesh as follows inferred.EvaluationStatus::UnresolvedAuthor + Verdict::unresolved_author()→EvaluationResult::NotEvaluatedEvaluationStatus::Failed→EvaluationResult::FailedEvaluationStatus::Evaluated + Verdict::*→EvaluationResult::Evaluated(Action)- It is left as an inference because the patch does not show the mapping directly, only the assembly of
EvaluateTweetsResponse. However, the value names of the two axes are symmetric.
D3. The seven visibility field kinds that TES handled moved to a separate Strato column hydrator.
tes_hydrator.rsshrank substantially, by −320 lines, and most of the deleted part is theTweetHydrationstruct.-#[derive(Default)] -pub(crate) struct TweetHydration { - pub(crate) nullcast: TweetHydrationBatch<bool>, - pub(crate) community: TweetHydrationBatch<i64>, - pub(crate) nsfw_user: TweetHydrationBatch<bool>, - pub(crate) nsfw_admin: TweetHydrationBatch<bool>, - pub(crate) takedown_reasons: TweetHydrationBatch<Vec<TakedownReason>>, - pub(crate) edit_control: TweetHydrationBatch<EditControl>, - pub(crate) media: TweetHydrationBatch<MediaFeature>, -}These seven axes were unified into a single
TweetHydrationBatch<TweetForVisibility>.TweetForVisibilityis an 11-field struct defined in the new filetes_composite.rs.pub(crate) struct TweetForVisibility { pub(crate) author_id: u64, pub(crate) source_tweet_id: Option<u64>, pub(crate) is_nullcast: bool, pub(crate) nsfw_user: bool, pub(crate) nsfw_admin: bool, pub(crate) has_takedown: bool, pub(crate) takedown_reasons: Vec<TakedownReason>, pub(crate) media: MediaFeature, pub(crate) is_community_tweet: bool, pub(crate) edit_control: Option<EditControl>, pub(crate) exclusive_conversation_author_id: Option<u64>, }The source that fetches this data is a fetch operation on a Strato column, not TES verified —
visibility-filtering/hydration/tes_composite.rs.- The column is
tweetypie/federated/tweetForVisibility.Tweet, and the operation isfetch.
const COLUMN: &str = "tweetypie/federated/tweetForVisibility.Tweet"; const OPERATION: &str = "fetch";ProdTweetForVisibilitySource::get_tweets_for_visibilitysends the list of tweet_ids as a batch call to this column. The response is decoded as MVal (Strato), and the Thrift-definedTweetis flattened withTweetForVisibility::project()and returned.- In other words, a VF-specific Strato column interface separate from TES was opened.
- The column is
TesHydratorstill exists, but it now orchestrates two sources.- From TES it still receives
PureCoreData(corpus-dimension signals such as author_id, text, and source_tweet_id). - From
TweetForVisibilitySourceit receives the flag/label axes that the visibility rules require. - That is, corpus data and visibility data were split onto different Strato backends.
- From TES it still receives
Until now, VF had referenced TES's broad tweet fields as they are. It now breaks from that practice and looks up only the minimum field set needed for visibility rules from a separate Strato column. Three axes settled together:
- In Gen4+13, the VF rules were restructured into a two-way tweet/author split.
- Gen4+23 deleted its own twemcache subtree of 3,234 lines entirely and moved to the company-wide
xai_cache, while opening the evaluate_tweets client interface as a stub. - Today the server implementation and the TES separation are completed.
That is, in Gen4 VF was reorganized layer by layer in the order rules → dependencies → serving interface.
What we learnedE. grox's adult spam video detection turns on reasoning mode only for posts whose like count is at or above a threshold
Reasoning mode was attached to the grox detection flow through fav-count (like count) gating verified.
- The evidence file is
grox/flows/ptos/task_safety_ptos_media_injected_adult_infrared_video_spam_detection.py(+14/−9). - The target is the
media_injected_adult_infrared_video_spam_detectionflow. It was introduced in Gen4+24 and compares a reference video bundle with the VLM storyboard of the candidate video.
use_reasoning = post.get_fav_count() >= MEDIA_INJECTED_REASONING_FAV ... prompt = media_injected_adult_infrared_video_spam_detection_prompt() system_prompt = ( _strip_thinking_restrictions(prompt) if use_reasoning else prompt ) ... assistant_content = [] if use_reasoning else [THINKING_CONTROL_END]- The evidence file is
For posts whose fav count is at or above the threshold, a path where the model thinks explicitly opens.
- It strips the "thinking restrictions" from the system prompt and leaves the
THINKING_CONTROL_ENDmarker of the assistant response empty.
- It strips the "thinking restrictions" from the system prompt and leaves the
The value of the threshold
MEDIA_INJECTED_REASONING_FAVwent intogrox/flows/ptos/constants.py, but the value is not exposed in the patch.The
SafetyPtosPolicyCrossValidatorof Gen4+7 (2026-08-24) turned on reasoning only for the categoriesUSE_REASONING_CATEGORIES = {ViolentSpeech, SuicideOrSelfHarm}. This time, reasoning is gated on the individual post axis by fav count.
This gating is a typical binary branch for spending cost only on threads with high fav counts. It is in the same family as the oai-gemma4-26b-reply-spam binary branch of Gen4+22 (2026-09-16) (threads over 150k followers). The deluxe_fav_count_threshold special-video entry gate of Gen4+12 (2026-09-01) is in the same family too.
What we learnedOther changes
- The MoE inference cluster ID of Phoenix retrieval was rolled forward verified.
- The evidence is
param.rs, and the value changed:Experiment2Memy04 → Experiment3Memy04.
- The evidence is
- The loaders of three CUDA kernels were changed to load the external package
xrex_cuda_kernelsinstead of in-repo sources verified —phoenix/xrex/cuda/adler32/__init__.py(+21/−6).Other files:
phoenix/xrex/cuda/top_k_by_key/__init__.py(+26/−6),phoenix/xrex/cuda/unique/__init__.py(+15/−5),phoenix/xrex/cuda/__init__.py(+11/−4)Until now they were imported with
from xrex.cuda.<name>.src import <name>_api(a subfolder in the repo). Now this changes toimport xrex_cuda_kernels.<name>_api as <name>_api(an external package). For example,adler32/__init__.pychanged like this:try: import xrex_cuda_kernels.adler32_api as adler32_api except ModuleNotFoundError: adler32_api = None else: jax.ffi.register_ffi_target("xrex_adler32", adler32_api.adler32(), platform="CUDA") _FFI_BACKENDS.add("gpu")If a module is missing, it silently falls back instead of throwing an exception.
if _MISSING: _log.warning( "xrex.cuda.adler32: xrex_cuda_kernels.%s not installed; the zlib pure_callback " "reference path runs instead of the compiled kernel(s)", ", ".join(_MISSING), )In environments where the package is not installed, the pure-Python reference path of zlib's
pure_callbackruns instead.top_k_by_keyanduniquefollow the same pattern.Until now the xrex repo even contained the CUDA kernel sources (
src/). With this commit, the kernel binaries were split into a separately installed Python package (not included in the open-source release).Open-source users run the reference path, and only the production xrex environment uses the compiled kernels. This is cleanup in the same direction as Gen4+21 (2026-09-15) changing the H100 attn kernel (
pallas_ranker_varlen_attn → cutedsl_ranker_varlen_attn).
Canon update Added a Gen4+25 row to the top-level timeline table of ../services/x.md and created a detailed section at the bottom.
Why it matters X's open source, which had been moving in an additive direction through yesterday's commit (Gen4+24), today for the first time in Gen4 also carries a rollback. In addition, Gen4 is "maturing" on three axes.
- The rolled-back
value_model_gate(the 19-feature user gate) and the two scoring modesdwell_regret_sigmoidandgated_dwell_regretwere introduced early in Gen4. They coexisted with the removal of client-side head weights in Gen4+8 (2026-08-25), but the reason to roll them forward appears to have disappeared in the measured metrics (no A/B results are public). - What was rolled back was an experimental path not in the canon, so the canon gets no new claim, only the observation that "this axis was tried and then cleaned up".
- The first axis that is maturing is the dataset_type axis of Phoenix retrieval (B above). The server is complete through dataset_slice_topk, and the open-source code exposes only the candidate slice cap and observation stats.
- The second axis is Phoenix bundle export (C above). The format, which could previously carry only ranking, was extended to carry the retrieval two-tower as well, and SPMD distributed export was attached.
- The third axis is VF (D above). Three layers were organized in this order: rules (Gen4+13) → deps (Gen4+23) → serving interface (today). Through Gen3, VF was a "label consumer"; in Gen4, VF is taking shape as "a separate service that stores, looks up, and serves labels".
Corrections
The server implementation of the verdict service that yesterday's daily predicted was confirmed today, and we found a commit attribution error in the source summary of the same daily.
- The prediction in the
## What to check nextsection of yesterday's daily (2026-09-19.md) was partly confirmed. The item "server-side implementation of the VFevaluate_tweetsRPC" was confirmed as being implemented, exactly that server, in today's commit (D1 above).- However, the source summary at the top of yesterday's daily wrongly attributed evaluate_tweets to c279172 (yesterday's commit). In c279172, VF actually had only a 2-line addition to
visibility-filtering/hydration/gizmoduck_hydrator.rs(confirmed with gh api). - The evaluate_tweets client stub shipped in Gen4+23 (2026-09-17, 42266f3), and daily/2026-09-18.md cited it correctly as 42266f3.
- The canon is not affected. The Gen4+23 row in x.md already states 42266f3 explicitly.
- However, the source summary at the top of yesterday's daily wrongly attributed evaluate_tweets to c279172 (yesterday's commit). In c279172, VF actually had only a 2-line addition to
- The fact that home-mixer scoring had a separate
ValueModelModeexperiment was not in the canon (services/x.md) and was mentioned in only one place, daily/2026-09-10.md. Now that it is rolled back, there is no need to fold this axis into the canon again.
What to check next
There are five things to follow up on from today's commit.
- Check when the cold-start cap
PhoenixColdStartMaxResultsis turned on — the open-source default of this parameter is 0. Which training and serving config supplies the actual value, and in what ratio it splits againstPhoenixMaxResults=1000, determine the actual share of the dataset_type=13 cold-start slice. - Check whether the bundle export of the two-tower model config (
RecsysTwoTowerModelConfig) is actually used — Phoenix retrieval serving has so far run on the native runtime (RetrievalModelRunner). The checkpoint to watch is whether it moves to StableHLO bundle serving.- Check which batch job (the
recsys_export.pyfamily) actually triggers the second kind supported by the new build_bundle, and which retrieval serving engine loads the exported bundle.
- Check which batch job (the
- Check whether the schema of the VF-only Strato column stabilizes — this column may settle into an interface exposing only the minimal field set needed for VF's visibility rules, or it may widen again toward TES over time. Which way it goes determines whether the VF-TES separation lasts.
- The target column is
tweetypie/federated/tweetForVisibility.Tweet.
- The target column is
- Check the actual value of the reasoning threshold
MEDIA_INJECTED_REASONING_FAV— this value, added togrox/flows/ptos/constants.py, is still undisclosed. Also check at what scale it compares withGROK_GEMMA_FOLLOWER_SPLIT(150k) in Gen4+22 ordeluxe_fav_count_thresholdin Gen4+12. - Check whether the new served_type
ForYouPhoenixRetrievalColdappears in production A/B — confirm when the new stat line shows up with measured values on the dashboard. Also check whether the ranking scorer recognizes this served_type and handles it separately (a continuation of the Gen4+16is_phoenix_moe(c)condition expansion).
Radar
We triaged 30 new papers today, and judging by titles and abstracts, none satisfies all three conditions. The three conditions are platform-company authors, a production deployment or A/B result claim, and novelty in a ranking or retrieval mechanism. The target is arXiv cs.IR, the count is based on 30 parsed <entry> elements, and the period is 2026-09-15–09-17. This period largely overlaps the window yesterday's daily covered, and the few items that looked worth a brief review are also only academic increments.
- "Single-Token Expected-Value Scoring for Cold-Start Candidate Ranking"(2026-09-16): The title is about cold-start ranking, but the authors are academic and the abstract has no production claim. We did not pass it.
- "SURF: Subtractive Updates for Recommender Forgetting"(2026-09-16): It addresses handling deletion requests (unlearning) and has no production deployment claim.
- "One-Step Retrieval Framework for Real-Time Sponsored Search Ads Using Hierarchical Text Representations"(2026-09-16): It is platform sponsored search ads retrieval, but the company affiliation needs confirmation. The abstract alone does not state an A/B claim explicitly. We defer adding it to watching.
- "LIGE-GR"(2026-09-16) was already added to yesterday's radar.
Because today is Sunday, we skipped the industry blog sweep (done on Mondays only).
Zero passed.