0x00 한

Research / Daily reports

  • Sources read: 1
  • Canon files updated: 1
  • Radar entries: 0
In one lineX rolled back both the experimental dwell-regret alternative scoring in home-mixer, its feed-assembly service, and the 19-feature user gate that decided entry to it. Only the weighted path remains for score computation.

1Home-mixer's experimental dwell-regret alternative scoring was rolled back — X

Source xai-org/x-algorithm@8b25829 — GitHub commit, author commit 2026-09-18T23:55:39Z, 41 files, +2,877/−1,828. These notes number the commits that came after X's public source Gen4 (2026-08-13, 47c1bcd) in order; this commit is Gen4+25.

Background This commit was registered 22 hours after the commit covered yesterday, Gen4+24 (c279172), on the same date (2026-09-18). The 1,828 deleted lines roughly balance the 2,877 added lines, but the deletions are concentrated in three intertwined files (item A below). The commit rolls back one experimental scoring mode in home-mixer and also touches several other areas.

The commit covers five axes.

  • A. In the ranking scoring of home-mixer (the service that receives X's feed requests, gathers candidates, and ranks them), one experimental scoring mode introduced early in Gen4 was dropped.
    • Deleting the whole file home-mixer/scorers/value_model_gate.rs (−537) also cut home-mixer/scorers/ranking_scorer.rs substantially, by −465 lines.
    • home-mixer/params/param.rs lost 18 related parameters, at −110 lines.
  • B. A cold-start slice, distinguished by a dataset_type tag, was formally wired into Phoenix (X's For You ranking and candidate retrieval model stack) retrieval.
  • C. Phoenix bundle export was extended from ranking only to also support two-tower (a retrieval model that embeds users and posts separately and matches them) retrieval.
    • In the Phoenix serving/train line, recsys_bundle_export.py was substantially reworked at +559/−107.
  • D. VF (visibility-filtering, a service that decides per post whether it may be shown to this viewer) gained a server implementation of the evaluate_tweets gRPC, which was a stub until yesterday. A hydrator also appeared that replaces the visibility fields TES (Tweet Entity Service, an X-internal tweet metadata service) used to supply. This hydrator fetches fields from Strato (an X-internal system that code goes through to read or store data) columns.
    • On the addition side, two new files appeared: visibility-filtering/evaluate_tweets.rs (+258) and visibility-filtering/hydration/tes_composite.rs (+607).
  • E. In infrastructure cleanup, the CUDA kernels were split out of the xrex repo into the external package xrex_cuda_kernels. Also, fav-count-based reasoning gating was added to the adult spam clone detection flow of grox (a Grok-based nearline classifier sub-repo).
    • The name of this detection flow starts with media_injected_adult_infrared.

Of the five axes, A and D are events in different directions: the unwinding and the maturing of experiment infrastructure that has built up in Gen4. A is the cleanup of a separate scoring mode introduced early in Gen4, because the measured results were poor (no evidence is public; it is observed only as the deletion of whole files). D is a maturing flow in which a stub that had only the client interface open in Gen4+24 yesterday now extends to server logic.

What we learnedA. The experimental scoring mode gated dwell-regret and its user gate were rolled back entirely

  • The mode that used dwell-regret scoring through a gate, and the file that defined its per-user gate, were both rolled back verified.

    • The evidence file home-mixer/scorers/value_model_gate.rs does not exist at 8b25829.
    • The mode is gated_dwell_regret, and the file is value_model_gate.
    • In the earlier Gen4 open-source history, this file defined GateModel::from_params().
    • GateModel normalized 19 user-state features with log1p, then produced a gate score as a logistic linear combination. It then flipped the assignment by request hash only within a hysteresis band.
    • The feature list is seq_len·n_fav·n_reply·n_rt_quote·n_vqv·n_click·n_bm_share·n_profile_follow·n_photo·n_negfb·n_7d·n_1d·active_days·active_days_7d·days_since_last·span_days·followers·followings·account_age_years.
    • The patch of the rollback commit shows this entire logic deleted, at −537 lines.
  • GateModel fired in ranking_scorer.rs only when the ValueModelMode parameter was in gating mode. At that point, depending on per-user conditions, it decided whether to use a dwell-regret-family alternative score based on dwell time.

    • ValueModelMode had three values (weighted / dwell_regret_sigmoid / gated_dwell_regret).
  • 8b25829 removed all three value branches.

    -        let use_dwell_regret = match query.params.get(ValueModelMode).as_str() {
    -            DWELL_REGRET_SIGMOID_MODE => true,
    -            GATED_DWELL_REGRET_MODE => GateModel::from_params(&query.params)
    -                .is_some_and(|gate| gate.serve_new_scoring(query)),
    -            _ => false,
    -        };
    

    — home-mixer/scorers/ranking_scorer.rs, the deleted portion of 2026-09-18 (the actual source is gone)

  • There is no replacement logic. The conditional branch itself was removed, and only the use_dwell_regret == false branch remains in the weighted_parts and weighted_scores computation.

    • Home-mixer scoring was simplified once more after Gen4+8 (2026-08-25) moved client-side head-weight generation to the server, and settled on a single weighted path.
  • Related elements were also deleted from ranking_scorer.rs verified — home-mixer/scorers/ranking_scorer.rs (+12/−465).

    • The DwellRegretWeights struct and its from_params() constructor.
      • The struct's 13 tuning fields were alpha_favorite·alpha_reply·alpha_retweet·alpha_quote·alpha_share·alpha_share_via_dm·alpha_share_via_copy_link·neg_not_interested·neg_block_author·neg_mute_author·neg_report·temperature·dwell_floor.
    • The whole compute_dwell_regret_base_scores() function. This function averaged each parameter within the candidate set, then turned each candidate's relative preference into centered_ratio = p / mean − 1. It multiplied positive and negative by sigmoid(positive/T) and exp(min(negative,0)/T) respectively, and produced the final score as dwell × modulation. In other words, it is an alternative scoring function that "modulates dwell time by engagement signals".
    • The centered_ratio() and sigmoid() helpers.
    • The logic that skipped MultiplierPreOffset under the use_dwell_regret branch.
    • Seven regression tests, the GATE_WEIGHTS_ALL_ZERO constant, and the "rust_home_mixer_value_model_mode": "weighted" override line in the existing regression tests.
      • Test names include dwell_regret_neutral_post_scores_dwell, dwell_regret_above_average_likeable_is_boosted, dwell_regret_report_sinks_candidate, dwell_regret_floor_applies_to_low_dwell, dwell_regret_per_request_normalization_amplifies_for_low_engagement_set, and others.
  • The related parameters were also deleted symmetrically in the parameter mirror verified — home-mixer/params/param.rs (+8/−110).

    • This file is a mirror that periodically carries over production config values. This mirror sync time changed to 2026-09-17T20:46:44Z → 2026-09-18T16:21:20Z.
    • 18 params were deleted.
Name Feature switch key Default
ValueModelMode rust_home_mixer_value_model_mode "weighted"
DwellRegretTemperature ..._dwell_regret_temperature 10.0
DwellRegretDwellFloor ..._dwell_regret_dwell_floor 1.0
DwellRegretAlphaFavorite ..._dwell_regret_alpha_favorite 1.0
DwellRegretAlphaReply ..._dwell_regret_alpha_reply 1.0
DwellRegretAlphaRetweet ..._dwell_regret_alpha_retweet 1.0
DwellRegretAlphaQuote ..._dwell_regret_alpha_quote 1.0
DwellRegretAlphaShare ..._dwell_regret_alpha_share 1.0
DwellRegretAlphaShareViaDm ..._dwell_regret_alpha_share_via_dm 1.0
DwellRegretAlphaShareViaCopyLink ..._dwell_regret_alpha_share_via_copy_link 1.0
DwellRegretNegNotInterested ..._dwell_regret_neg_not_interested −10,000
DwellRegretNegBlockAuthor ..._dwell_regret_neg_block_author −8,000
DwellRegretNegMuteAuthor ..._dwell_regret_neg_mute_author −15,000
DwellRegretNegReport ..._dwell_regret_neg_report −60,000
DwellRegretGateWeights ..._dwell_regret_gate_weights 19-feature logistic weight string
DwellRegretGateBias ..._dwell_regret_gate_bias 1.033918
DwellRegretGateThreshold ..._dwell_regret_gate_threshold −0.634264
DwellRegretGateHysteresisBand ..._dwell_regret_gate_hysteresis_band 0.0
  • The production default of the rolled-back parameter confirms that the gate held actually trained weights. This is because the default string of DwellRegretGateWeights was still present at the time of deletion.

    "seq_len:0.530298,n_fav:-0.082139,n_reply:0.485541,n_rt_quote:0.056561,n_vqv:-0.072778,n_click:-0.176675,n_bm_share:-0.167574,n_profile_follow:-0.221285,n_photo:-0.106004,n_negfb:0.031839,n_7d:-0.075799,n_1d:-0.241730,active_days:0.047017,active_days_7d:-0.126896,days_since_last:-0.034238,span_days:-0.052186,followers:-0.066642,followings:0.064140,account_age_years:-0.045455"

    • Strong positive weights were placed on two axes, seq_len (current session sequence length) and n_reply (28-day reply count).
    • Negative weights were placed on the remaining interaction counts (n_click, n_bm_share, n_profile_follow, n_1d).
    • In other words, the goal appears to have been serving dwell-regret alternative scoring to users who are "long-session active and have strong recent reply activity".
    • These coefficients are no longer applied to any user.

The experimental alternative function that sat beside ranking scoring in Gen4 (modulating dwell time by engagement signals) and its user gating were removed all at once, and ranking scoring narrowed back to a single weighted path. How to reflect the dwell signal here appears to have moved to adding heads at training time and redefining score combination inferred. Examples are the click_dwell_time sigmoid BCE (threshold=10s) formally wired in Gen4+22 and the EnableCdwellOnImpr (click × dwell product) of Gen4+23 inferred. The evidence is that the timing of the two flows is adjacent, and the observation that once the circuit modulating dwell through a separate scoring function disappeared, only circuits treating dwell as a head itself and as a combination term remain.

What we learnedB. A cold-start slice was wired into Phoenix retrieval, but it is not requested by default

  • A cold-start slice distinguished by a dedicated tag was formally wired into Phoenix retrieval verified.

  • A new parameter PhoenixColdStartMaxResults was added, so two caps are passed together when calling the retrieval RPC.

    • The feature switch is rust_home_mixer_phoenix_cold_start_max_results, and the default is 0.
             quality_factor::apply(query.params.get(PhoenixMaxResults)),
    +        query.params.get(PhoenixColdStartMaxResults),
    
    • Because the default is 0, production by default still does not request the cold-start slice.
  • Response candidates carry dataset_type, and served_type (the source label attached to a candidate) differs by that value.

    • The response candidate (ScoredCandidate) carries a dataset_type: u32 field.
    • If that value is HOME_COLD_DATASET_TYPE = 13, served_type maps to the new enum ForYouPhoenixRetrievalCold; otherwise it maps to the existing ForYouPhoenixRetrieval.
    const HOME_COLD_DATASET_TYPE: u32 = 13;
    
    fn served_type_for_dataset(dataset_type: u32) -> pb::ServedType {
        if dataset_type == HOME_COLD_DATASET_TYPE {
            pb::ServedType::ForYouPhoenixRetrievalCold
        } else {
            pb::ServedType::ForYouPhoenixRetrieval
        }
    }
    

    — home-mixer/sources/phoenix_source.rs

  • A matching observation line was attached on the side-effect side too verified — home-mixer/side_effects/scored_stats_side_effect.rs (+25/−0).

    • The new function record_phoenix_retrieval_cold_stats emits size statistics of selected/non-selected candidates to the stat receiver under the prefix PhoenixRetrievalColdTweets.
    • It is the third observation axis alongside the existing MoE slice (PhoenixRetrievalMoeTweets). MoE (Mixture of Experts) is a model structure that, for each input, picks and uses some of several expert subnetworks.
  • The MoE source and Topics source do not share this cold-start slice logic (confirmed in the patch).

    • phoenix_moe_source.rs and phoenix_topics_source.rs each also grew by 1 line, but both are new import line additions.
    • The two sources keep using the ForYouPhoenixRetrievalMoe and ForYouPhoenixRetrievalTopics served_type as before.

Until now, X's cold-start handling had two axes:

  • Users whose history is below a threshold are routed to a separate model fleet. Retrieval and ranking each have Phoenix*NewUserInferenceClusterId.
  • New authors (at or below ColdStartImpressionThreshold=1000) get a PostUnexploredWeight=0.02 boost, and there is the Thompson sampling option from Gen4+1.

This wiring opens a third axis at the candidate level. Within the candidate set returned by the same Phoenix retrieval RPC, a separate cap is placed on the "cold" subset by the dataset_type tag, and it is observed. That is, Phoenix retrieval itself appears to have been extended to always return a cold candidate slice together, regardless of user history inferred. The evidence is that the default is 0, so it has not yet entered the production default, but the observation stat lines were also wired symmetrically, which indicates that the A/B infrastructure is complete.

What we learnedC. Phoenix bundle export can now export the forward program of two-tower retrieval as well as ranking

  • Phoenix bundle export was extended to export the two-tower retrieval forward in addition to the ranking forward verified.

    • The evidence file is phoenix/xrex/train/recsys_bundle_export.py (+559/−107).
    • This file was introduced at 634 lines in Gen4+13 (2026-09-01, 7ba7768). It exported the ranking model (RecsysAggregatedModelConfig), lowered with jax.export, to StableHLO MLIR bytecode, .jax_export, and MANIFEST.json.
    • This commit attached retrieval two-tower program export alongside it. Two new axes appeared.
  • Axis 1: the number of program kinds grew to two verified. Two constants distinguish the program kind within the export bundle.

    KIND_RANKING = "recsys_ranking_forward"
    KIND_RETRIEVAL = "recsys_retrieval_forward"
    
  • Axis 2: because the retrieval export must carry additional resources in the bundle, new data structures and wire keys were introduced.

    POST_TABLE_KEY = "post_embeddings.embeddings"
    POST_DATASET_TYPES_KEY = "post_embeddings.dataset_types"
    POST_IDS_KEY = "post_embeddings.post_ids"
    AUTHOR_IDS_KEY = "post_embeddings.author_ids"
    
    • The new dataclass RetrievalExport holds five attributes that must be extracted from the trainer's RetrievalModelRunner at export time. The attributes are large_k, retrieval_dataset_types, enable_async_topk, enable_radix_select_topk, and enable_dataset_slice_topk.
    • The manifest block carries the items below.
      • large_k
      • target dataset types (a list of name·value pairs)
      • post table geometry (rows·width·dtype)
      • three data keys (POST_TABLE_KEY·POST_DATASET_TYPES_KEY·POST_IDS_KEY·AUTHOR_IDS_KEY)
      • topk_unordered flag
  • The retrieval forward extracts a user embedding with the user tower, matches it against the whole post table, and returns top-K for each target dataset type verified — new function _make_retrieval_forward_fn.

    def forward_fn(
        batch: Any, merged_embeddings: jax.Array, post_table: jax.Array, dataset_types: jax.Array
    ):
        recsys_embeddings = _recsys_embeddings_from_merged(
            merged_embeddings, embedding_slices, packed_geometry
        )
        model = model_config.make(sharding_context=make_legacy_sharding_context(mesh))
        results = model.forward(
            batch, recsys_embeddings, post_table, dataset_types,
            retrieval.large_k, target_values,
            dataset_ranges=None,
            use_async_topk=retrieval.use_async_topk,
            use_radix_select_topk=retrieval.use_radix_select_topk,
            post_scales=None,
        )
        flat: list[jax.Array] = []
        for indices, scores in results:
            flat.append(indices)
            flat.append(scores)
        return tuple(flat)
    
    • Each target dataset type yields two outputs (indices_<name>, scores_<name>). This naming rule is defined in the RetrievalExport.output_names property.
  • For two-tower, the rule for computing the serving sequence length differs.

    • _make_export_config also gained two arguments, two_tower: bool = False and mesh_devices: int = 1.
    def _user_tower_sequence_len(
        *, training_seq_len, training_history_seq_len, num_user_prefix_tokens, history_seq_len,
    ) -> int:
        prefix_tokens = training_seq_len - training_history_seq_len
        if prefix_tokens != num_user_prefix_tokens:
            raise ValueError(
                f"cannot derive the serving user-tower sequence_len: sequence_len "
                f"{training_seq_len} - history_seq_len {training_history_seq_len} = {prefix_tokens} "
                f"prefix tokens, but the model reserves num_user_prefix_tokens="
                f"{num_user_prefix_tokens}. The user tower is built as prefix + history; if "
                "history_seq_len was overridden for serving, also override "
                "model_config.user_tower_config.model_config.sequence_len to "
                f"{num_user_prefix_tokens + history_seq_len} (prefix + serving history), as the "
                "jax slot does — otherwise the attention kernel cannot be sized for serving"
            )
        return prefix_tokens + history_seq_len
    
    • The ranking export's serving sequence is prefix + history + candidate, but the retrieval export has only prefix + history. There is no candidate axis, and it emits only the user embedding.
    • The overrides of _make_export_config also use model_config.user_tower_config.model_config.sequence_len instead of model_config.model_config.sequence_len.
  • The third new addition is distributed export. The retrieval bundle is exported over a mesh and can be distributed across multiple devices.

    if not two_tower and mesh_devices != 1:
        raise NotImplementedError(
            "ranking programs are single-device (one executable per GPU); "
            f"mesh_devices={mesh_devices} is only for retrieval bundles"
        )
    
    • The reason is also stated explicitly. Ranking is served with one executable per GPU, but retrieval has to split the post table across several devices, so it is exported with SPMD.
    • _export_mesh checks export_cfg.parallel_config.mesh_shape() to build the devices. _retrieval_shardings specifies NamedSharding for batch/params/post_table/dataset_types.
    • Embedding rows are partitioned across the data axis, and params are replicated.
  • Several constraints are imposed explicitly.

    • enable_int8_post_table, enable_bloom_filter, and enable_topic_filter are not supported. These options are supported only by the native runtime, and StableHLO export uses a bf16 post table with no per-post filter.
    • enable_dataset_slice_topk=True is ignored and a warning is raised. The export program always compiles only the mask-then-topk-over-whole-table path.
    • The using_seqpack + two_tower combination is prohibited. Packing does not fit the SPMD model.
    • With DEBUG_ALLOW_RANDOM_INIT=1, the export itself is refused. This is because exporting without a mask would emit an invalid program.
  • build_bundle now accepts both kinds of models.

    two_tower = isinstance(model_config, RecsysTwoTowerModelConfig)
    if not two_tower and (
        not isinstance(model_config, RecsysAggregatedModelConfig)
        or isinstance(model_config, RecsysGenRecsModelConfig)
    ):
        raise NotImplementedError(
            "StableHLO bundle export supports ranking (RecsysAggregatedModelConfig) and "
            f"retrieval (RecsysTwoTowerModelConfig) only, got {type(model_config).__name__}"
        )
    

Until now, Phoenix serving carried only the ranking forward in a StableHLO bundle. With this commit, preparation for carrying retrieval two-tower in the same bundle frame is complete. Because infrastructure for exporting retrieval distributed across several devices via SPMD was attached too, the direction is for Phoenix retrieval's serving stack to be unified under the same bundle format as ranking. It may not be a coincidence that this landed at the same time as the PhoenixColdStartMaxResults wiring in B above inferred. In effect, while the retrieval serving axis was being tightened, the slice observation interface via the dataset_type tag settled alongside it inferred.

What we learnedD. VF now has a server that receives verdict requests, and takes the visibility fields for verdicts from a Strato column instead of TES

This axis splits into two sub-axes.

D1. The verdict-request RPC that was a stub until yesterday was filled in with server logic by this commit.

  • daily/2026-09-18.md records the observation that Gen4+23 (42266f3) on 2026-09-17 opened only the client interface of the evaluate_tweets RPC and the Thrift decoder for the response payload. According to the same observation, the server implementation was Err(Status::unimplemented("EvaluateTweets is not served yet")).

  • This commit fills that evaluate_tweets stub with server logic. The server.rs patch directly shows the unimplemented stub being replaced with a real implementation.

         async fn evaluate_tweets(
             &self,
    -        _: Request<vf_pb::EvaluateTweetsRequest>,
    +        request: Request<vf_pb::EvaluateTweetsRequest>,
         ) -> Result<Response<vf_pb::EvaluateTweetsResponse>, Status> {
    -        Err(Status::unimplemented("EvaluateTweets is not served yet"))
    +        self.evaluate_tweets.handle(request).await
         }
    
  • The new server endpoint EvaluateTweetsEndpoint (258 new lines) receives a request, runs the internal filter, and fills the response with a per-tweet Outcome.

    • It receives the request's three fields (viewer_id, country_code, safety_level) and a tweet list.
    • safety_level is converted from the Thrift enum to the internal Rust SafetyLevel enum. Only three values are allowed: FILTER_ALL, TIMELINE_HOME, and TIMELINE_HOME_RECOMMENDATIONS.
    • It then calls FilterTweets::run.
    let candidates = req.tweets.iter()
        .filter(|o| o.quote_context.is_none())
        .map(|o| RawCandidate { tweet_id: TweetId(o.tweet_id), request_author_id: None })
        .collect();
    let response = self.filter_tweets.run(FilterRequest { viewer_id, country_code, safety_level, candidates }).await;
    
  • Three metrics (evaluate_tweets_requests, evaluate_tweets_latency_ms, evaluate_tweets_batch_size) are recorded for each request.

  • The evaluate_tweets_loopback integration test pins the fact that the server logic is wired in. This test starts the gRPC server on a tokio TCP listener and makes a round-trip call with XaiVfClient.

D2. Besides the verdict, an evaluation status is attached to the result.

  • An EvaluationStatus axis was added to FilterOutcome. A new enum appears in filter.rs.

    #[derive(Clone, Copy, Debug, PartialEq, Eq)]
    pub enum EvaluationStatus {
        Evaluated,
        UnresolvedAuthor,
        Failed,
    }
    
  • Previously the filter could emit only one Verdict per tweet. Now one of three statuses is attached next to the verdict.

    • A tweet whose hydration succeeded is Evaluated.
    • A tweet whose author did not come out of hydration is UnresolvedAuthor, and its verdict is Verdict::unresolved_author().
    • A tweet that hit the rules with insufficient fields because hydration itself failed is Failed.
    • HydrationOutput also gained a new field failed_ids: HashSet<TweetId> to distinguish the third case (Failed).
  • The axis names line up with the client-side EvaluationResult { Evaluated(Box<Action>), NotEvaluated, Failed } (introduced in Gen4+23), and we infer that the server and client axes mesh as follows inferred.

    • EvaluationStatus::UnresolvedAuthor + Verdict::unresolved_author() → EvaluationResult::NotEvaluated
    • EvaluationStatus::Failed → EvaluationResult::Failed
    • EvaluationStatus::Evaluated + Verdict::* → EvaluationResult::Evaluated(Action)
    • It is left as an inference because the patch does not show the mapping directly, only the assembly of EvaluateTweetsResponse. However, the value names of the two axes are symmetric.

D3. The seven visibility field kinds that TES handled moved to a separate Strato column hydrator.

  • tes_hydrator.rs shrank substantially, by −320 lines, and most of the deleted part is the TweetHydration struct.

    -#[derive(Default)]
    -pub(crate) struct TweetHydration {
    -    pub(crate) nullcast: TweetHydrationBatch<bool>,
    -    pub(crate) community: TweetHydrationBatch<i64>,
    -    pub(crate) nsfw_user: TweetHydrationBatch<bool>,
    -    pub(crate) nsfw_admin: TweetHydrationBatch<bool>,
    -    pub(crate) takedown_reasons: TweetHydrationBatch<Vec<TakedownReason>>,
    -    pub(crate) edit_control: TweetHydrationBatch<EditControl>,
    -    pub(crate) media: TweetHydrationBatch<MediaFeature>,
    -}
    
  • These seven axes were unified into a single TweetHydrationBatch<TweetForVisibility>. TweetForVisibility is an 11-field struct defined in the new file tes_composite.rs.

    pub(crate) struct TweetForVisibility {
        pub(crate) author_id: u64,
        pub(crate) source_tweet_id: Option<u64>,
        pub(crate) is_nullcast: bool,
        pub(crate) nsfw_user: bool,
        pub(crate) nsfw_admin: bool,
        pub(crate) has_takedown: bool,
        pub(crate) takedown_reasons: Vec<TakedownReason>,
        pub(crate) media: MediaFeature,
        pub(crate) is_community_tweet: bool,
        pub(crate) edit_control: Option<EditControl>,
        pub(crate) exclusive_conversation_author_id: Option<u64>,
    }
    
  • The source that fetches this data is a fetch operation on a Strato column, not TES verified — visibility-filtering/hydration/tes_composite.rs.

    • The column is tweetypie/federated/tweetForVisibility.Tweet, and the operation is fetch.
    const COLUMN: &str = "tweetypie/federated/tweetForVisibility.Tweet";
    const OPERATION: &str = "fetch";
    
    • ProdTweetForVisibilitySource::get_tweets_for_visibility sends the list of tweet_ids as a batch call to this column. The response is decoded as MVal (Strato), and the Thrift-defined Tweet is flattened with TweetForVisibility::project() and returned.
    • In other words, a VF-specific Strato column interface separate from TES was opened.
  • TesHydrator still exists, but it now orchestrates two sources.

    • From TES it still receives PureCoreData (corpus-dimension signals such as author_id, text, and source_tweet_id).
    • From TweetForVisibilitySource it receives the flag/label axes that the visibility rules require.
    • That is, corpus data and visibility data were split onto different Strato backends.

Until now, VF had referenced TES's broad tweet fields as they are. It now breaks from that practice and looks up only the minimum field set needed for visibility rules from a separate Strato column. Three axes settled together:

  • In Gen4+13, the VF rules were restructured into a two-way tweet/author split.
  • Gen4+23 deleted its own twemcache subtree of 3,234 lines entirely and moved to the company-wide xai_cache, while opening the evaluate_tweets client interface as a stub.
  • Today the server implementation and the TES separation are completed.

That is, in Gen4 VF was reorganized layer by layer in the order rules → dependencies → serving interface.

What we learnedE. grox's adult spam video detection turns on reasoning mode only for posts whose like count is at or above a threshold

  • Reasoning mode was attached to the grox detection flow through fav-count (like count) gating verified.

    use_reasoning = post.get_fav_count() >= MEDIA_INJECTED_REASONING_FAV
    ...
    prompt = media_injected_adult_infrared_video_spam_detection_prompt()
    system_prompt = (
        _strip_thinking_restrictions(prompt) if use_reasoning else prompt
    )
    ...
    assistant_content = [] if use_reasoning else [THINKING_CONTROL_END]
    
  • For posts whose fav count is at or above the threshold, a path where the model thinks explicitly opens.

    • It strips the "thinking restrictions" from the system prompt and leaves the THINKING_CONTROL_END marker of the assistant response empty.
  • The value of the threshold MEDIA_INJECTED_REASONING_FAV went into grox/flows/ptos/constants.py, but the value is not exposed in the patch.

  • The SafetyPtosPolicyCrossValidator of Gen4+7 (2026-08-24) turned on reasoning only for the categories USE_REASONING_CATEGORIES = {ViolentSpeech, SuicideOrSelfHarm}. This time, reasoning is gated on the individual post axis by fav count.

This gating is a typical binary branch for spending cost only on threads with high fav counts. It is in the same family as the oai-gemma4-26b-reply-spam binary branch of Gen4+22 (2026-09-16) (threads over 150k followers). The deluxe_fav_count_threshold special-video entry gate of Gen4+12 (2026-09-01) is in the same family too.

What we learnedOther changes

  • The MoE inference cluster ID of Phoenix retrieval was rolled forward verified.
    • The evidence is param.rs, and the value changed: Experiment2Memy04 → Experiment3Memy04.
  • The loaders of three CUDA kernels were changed to load the external package xrex_cuda_kernels instead of in-repo sources verified — phoenix/xrex/cuda/adler32/__init__.py (+21/−6).
    • Other files: phoenix/xrex/cuda/top_k_by_key/__init__.py (+26/−6), phoenix/xrex/cuda/unique/__init__.py (+15/−5), phoenix/xrex/cuda/__init__.py (+11/−4)

    • Until now they were imported with from xrex.cuda.<name>.src import <name>_api (a subfolder in the repo). Now this changes to import xrex_cuda_kernels.<name>_api as <name>_api (an external package). For example, adler32/__init__.py changed like this:

      try:
          import xrex_cuda_kernels.adler32_api as adler32_api
      except ModuleNotFoundError:
          adler32_api = None
      else:
          jax.ffi.register_ffi_target("xrex_adler32", adler32_api.adler32(), platform="CUDA")
          _FFI_BACKENDS.add("gpu")
      
    • If a module is missing, it silently falls back instead of throwing an exception.

      if _MISSING:
          _log.warning(
              "xrex.cuda.adler32: xrex_cuda_kernels.%s not installed; the zlib pure_callback "
              "reference path runs instead of the compiled kernel(s)",
              ", ".join(_MISSING),
          )
      
    • In environments where the package is not installed, the pure-Python reference path of zlib's pure_callback runs instead. top_k_by_key and unique follow the same pattern.

    • Until now the xrex repo even contained the CUDA kernel sources (src/). With this commit, the kernel binaries were split into a separately installed Python package (not included in the open-source release).

    • Open-source users run the reference path, and only the production xrex environment uses the compiled kernels. This is cleanup in the same direction as Gen4+21 (2026-09-15) changing the H100 attn kernel (pallas_ranker_varlen_attn → cutedsl_ranker_varlen_attn).

Canon update Added a Gen4+25 row to the top-level timeline table of ../services/x.md and created a detailed section at the bottom.

Why it matters X's open source, which had been moving in an additive direction through yesterday's commit (Gen4+24), today for the first time in Gen4 also carries a rollback. In addition, Gen4 is "maturing" on three axes.

  • The rolled-back value_model_gate (the 19-feature user gate) and the two scoring modes dwell_regret_sigmoid and gated_dwell_regret were introduced early in Gen4. They coexisted with the removal of client-side head weights in Gen4+8 (2026-08-25), but the reason to roll them forward appears to have disappeared in the measured metrics (no A/B results are public).
  • What was rolled back was an experimental path not in the canon, so the canon gets no new claim, only the observation that "this axis was tried and then cleaned up".
  • The first axis that is maturing is the dataset_type axis of Phoenix retrieval (B above). The server is complete through dataset_slice_topk, and the open-source code exposes only the candidate slice cap and observation stats.
  • The second axis is Phoenix bundle export (C above). The format, which could previously carry only ranking, was extended to carry the retrieval two-tower as well, and SPMD distributed export was attached.
  • The third axis is VF (D above). Three layers were organized in this order: rules (Gen4+13) → deps (Gen4+23) → serving interface (today). Through Gen3, VF was a "label consumer"; in Gen4, VF is taking shape as "a separate service that stores, looks up, and serves labels".

Corrections

The server implementation of the verdict service that yesterday's daily predicted was confirmed today, and we found a commit attribution error in the source summary of the same daily.

  • The prediction in the ## What to check next section of yesterday's daily (2026-09-19.md) was partly confirmed. The item "server-side implementation of the VF evaluate_tweets RPC" was confirmed as being implemented, exactly that server, in today's commit (D1 above).
    • However, the source summary at the top of yesterday's daily wrongly attributed evaluate_tweets to c279172 (yesterday's commit). In c279172, VF actually had only a 2-line addition to visibility-filtering/hydration/gizmoduck_hydrator.rs (confirmed with gh api).
    • The evaluate_tweets client stub shipped in Gen4+23 (2026-09-17, 42266f3), and daily/2026-09-18.md cited it correctly as 42266f3.
    • The canon is not affected. The Gen4+23 row in x.md already states 42266f3 explicitly.
  • The fact that home-mixer scoring had a separate ValueModelMode experiment was not in the canon (services/x.md) and was mentioned in only one place, daily/2026-09-10.md. Now that it is rolled back, there is no need to fold this axis into the canon again.

What to check next

Radar

We triaged 30 new papers today, and judging by titles and abstracts, none satisfies all three conditions. The three conditions are platform-company authors, a production deployment or A/B result claim, and novelty in a ranking or retrieval mechanism. The target is arXiv cs.IR, the count is based on 30 parsed <entry> elements, and the period is 2026-09-15–09-17. This period largely overlaps the window yesterday's daily covered, and the few items that looked worth a brief review are also only academic increments.

  • "Single-Token Expected-Value Scoring for Cold-Start Candidate Ranking"(2026-09-16): The title is about cold-start ranking, but the authors are academic and the abstract has no production claim. We did not pass it.
  • "SURF: Subtractive Updates for Recommender Forgetting"(2026-09-16): It addresses handling deletion requests (unlearning) and has no production deployment claim.
  • "One-Step Retrieval Framework for Real-Time Sponsored Search Ads Using Hierarchical Text Representations"(2026-09-16): It is platform sponsored search ads retrieval, but the company affiliation needs confirmation. The abstract alone does not state an A/B claim explicitly. We defer adding it to watching.
  • "LIGE-GR"(2026-09-16) was already added to yesterday's radar.

Because today is Sunday, we skipped the industry blog sweep (done on Mondays only).

Zero passed.