2026-10-02

Other changes reviewed today:

• In the same X commit (2026-10-01, b79b947), abuse-enforcement-service gained a module that restarts the service when the Kafka config changes in operation.
• In the same commit, the phoenix-rankall config was split into a separate module, which revealed an SID (an ID that represents an item as a sequence of semantic tokens) backfill QPS cap of 800k.
• The GEAR paper from Douyin (ByteDance) stated that it deployed generative ads retrieval to Douyin Ads production.

1VF now decides the blocked actions and notice text for action limits — X

VF has a new limited-actions policy system, made up of two new files totaling 703 lines. There are five restriction reasons: ConversationControl, ReadonlyViewer, BlockedViewer, RootAuthorBlockedViewer, and StaleTweet. In code, these five are variants of LimitedEngagementReason.

• The new policy file takes a restriction reason and resolves, through FeatureSwitches, the list of actions to block for that reason and the combination of copy settings verified — limited_actions_policy.rs:14-115.

  • The file is VF visibility-filtering/params/limited_actions_policy.rs, with 306 new lines.
  • The input is LimitedEngagementReason. What it resolves is the LimitedActionType list attached to that reason and the combination of copy namespace / prompt_type / learn_more_url.
  • The return value is LimitedActionsPolicies (reason → Vec<LimitedAction>).

• The tests pin the list of actions to block for each of the five reasons verified — limited_actions_policy.rs:143-230.

  • ConversationControl → [Reply]
  • BlockedViewer → 19 actions (Reply·Retweet·QuoteTweet·Like·React·SendViaDm·AddToBookmarks·AddToMoment·PinToProfile·ShareTweetVia·Follow·ListsAddRemove·Embed·CopyLink·VoteOnPoll·ShowRetweetActionMenu·ReplyDownVote·EditTweet·Highlight)
  • RootAuthorBlockedViewer → [Reply]
  • ReadonlyViewer → 14 actions (all write actions when the viewer is in read-only mode)
  • StaleTweet → 15 actions (including copy, share, and view engagement)

• The new copy file loads the notice text from the localized UI text bundle, and it also has a mechanism that compares the bundle hash against pinned metadata verified — limited_actions_copy.rs:1-85.

  • The file is VF visibility-filtering/limited_actions_copy.rs, with 397 new lines.
  • It loads the StringCenter bundle, which holds the localized UI text, from /config/stringcenter/visibility-ltd-actions.json. The mechanism that compares the bundle hash against pinned metadata is DriftCheck.
  • The bundle loader assembles Prompt { headline, subtext, kind } from LimitedActionType and PromptCopy.
  • LimitedActionType is a 26-variant enum: Reply·Retweet·QuoteTweet·Like·React·SendViaDm·AddToBookmarks·AddToMoment·PinToProfile·ViewTweetActivity·ShareTweetVia·Follow·ListsAddRemove·MuteConversation·Embed·ViewHiddenReplies·HideCommunityTweet·CopyLink·VoteOnPoll·RemoveFromCommunity·ShowRetweetActionMenu·ReplyDownVote·Autoplay·EditTweet·Highlight·ViewPostEngagements.
  • PromptCopy consists of a namespace, a prompt_type, and an optional learn_more_url.
  • kind has three variants: Basic, SeeConversation (CTA), and LearnMore { language, link_text, url }.

• The signature changed so that the function converting a verdict result into a Thrift action takes this policy as its third argument verified.

  • The changed function is thrift_action(verdict, level, policies) in treatment.rs, and the added argument is &LimitedActionsPolicies.

  • The rules → thrift/proto projection layer isolated in the previous commit (Gen4+32, which introduced treatment.rs) appears to have its role extended in this commit.

  • The changed signature:

    pub(crate) fn thrift_action(
        verdict: &Verdict,
        level: SafetyLevel,
        policies: &LimitedActionsPolicies,
    ) -> Action {
    

    — treatment.rs:16-20

• The response for a restricted tweet is now assembled as a new Thrift type, and the wire changes from passing "only the restriction reason enum" to a "payload that also carries a localized prompt per action" verified.

  • Nine symbols were added to the Thrift imports, including BasicLimitedActionPrompt · CtaLimitedActionPrompt · LimitedAction · LimitedActionCtaType · LimitedActionPrompt · LimitedActionsPolicy · LocalizedMessage · LocalizedMessageLimitedActionPrompt · MessageLink.
  • The assembly path is Verdict's Shown{engagement: Some(...)}, and the result is LimitedEngagements.

• VF's EvaluateTweets endpoint holds three fields, and one of them is the limited-action copy verified — evaluate_tweets.rs:22-40.

  • The EvaluateTweetsEndpoint in VF evaluate_tweets.rs has the fields filter_tweets, client_switches, and limited_actions_copy.

• The restriction information changed from a structure that wrapped a single reason to one that accumulates multiple reasons verified.

  • LimitedEngagement in VF models/verdict.rs switched from a single-reason wrapper to a structure that accumulates a first reason and a rest Vec.

    #[derive(Clone, Debug, PartialEq, Eq)]
    pub struct LimitedEngagement {
        first: LimitedEngagementReason,
        rest: Vec<LimitedEngagementReason>,
    }
    
    impl LimitedEngagement {
        pub fn add(&mut self, reason: LimitedEngagementReason) {
            if !self.reasons().any(|held| held == reason) {
                self.rest.push(reason);
            }
        }
    }
    

    — verdict.rs:67-98

• On the rules side too, when a rule meets a new reason on a tweet that is already restricted, it appends the reason to the Vec verified — rules/registry.rs.

  • When collect_specs matches ActionSpec::LimitedEngagement(reason) and an engagement already exists, it appends to the Vec with limit.value.add(*reason).
  • In other words, if BlockedViewer and ReadonlyViewer both apply to the same tweet, VF, not downstream, builds the union of the action sets under the two reasons verified — limited_engagements in treatment.rs.

• The Brazilian electoral court label was added in two places in the country-specific NSFW content gating policy tree of VF's rules verified — registry.rs:513-517, 648-652.
  • The added label is brazil_election_legal/drop/undesirable, and the policy tree is country_specific_nsfw_content_gating in rules/registry.rs.
  • The two places are the general drop list and the label drop list.
• In the previous commit (Gen4+32), the home-mixer Brazil 2026 election handle list grew by 9, from 2,786 to 2,795, and now VF also registers the Brazilian electoral court label as an official drop label.
• This means that when VF receives from the labeling service a tweet carrying the brazil_election_legal label, it routes the tweet to the drop/undesirable path whether the label was generated by Grok or assigned by a person.

• A new file extracts the calling service's identity from the x509 certificate of the gRPC request peer verified — caller_identity.rs:1-56.
  • VF caller_identity.rs is new, at 126 lines.
  • Of the SAN URI and the subject CN, it takes the one that starts with the twtr:svc: prefix and uses it as the service-to-service identity.
  • The metric vf_requests_by_caller is aggregated by three labels (caller_identity, rpc: FilterTweets|EvaluateTweets|GetSafetyLabels, deadline: present|absent).
• The tests pin three cases verified.
  • The SAN URI takes priority.
  • If there is no SAN, it falls back to the CN.
  • If neither starts with the twtr:svc: prefix, there is no identity.
• Every EvaluateTweets request is counted in the caller identity metric verified (the endpoint file linked in card A).
  • The request handler calls caller_identity::record(Endpoint::EvaluateTweets, &request) at the very start.

abuse-enforcement-service is the service that decides actual actions (label/challenge/suspension), and its Rust rewrite was released on 2026-09-23. Starting with this commit, the service captures its boot-time config (boot_config) and builds the Kafka producers on top of it. The captured value is a serde_json::Value. Previously, lib.rs pulled the producers out with DynamicConfig.kafka_producers(). A dedicated module (restart_on_config_change.rs) that restarts the service when the Kafka config changes in operation was also added. As a result, the kafka producer binding has narrowed to react only to the "boot-time snapshot".

• The new module compares the config captured at boot with the live config in operation, and prompts a restart of the service if a designated path changes verified — restart_on_config_change.rs:1-85.
  • restart_on_config_change.rs in abuse-enforcement-service is new, at 173 lines.
  • The value captured at boot is a serde_json::Value, and the watched targets are designated JSON pointer paths.
  • There are three default watched paths: /kafka/consumer/topic_labels, /topic_labels, and /kafka/producer.
  • The result has three states (Check::{Unchanged, Invalid, Restart(Vec<String>)}). If nothing changed, it is Unchanged.
  • If something changed, it runs validate(live). If validation passes, the result is Restart (the list of changed paths); if it fails, the result is Invalid. On Invalid, it does not restart and only logs.
• A metric signaling a pending restart is reported with two labels, valid and invalid verified.
  • The metric name is CONFIG_RESTART_PENDING.
• The function that builds the Kafka producers was changed to take the boot-time config snapshot instead of the live config verified — lib.rs:831-840, 1385-1392.
  • In lib.rs (+325/−55), the signature changed from build_kafka_producers(cfg, dynamic_config) to build_kafka_producers(cfg, boot_config).
  • boot_config is the boot-time config snapshot captured with dynamic_config.config().unwrap_or(serde_json::Value::Null). Producer building refers only to the kafka_producers section of this snapshot.
  • growthbook.rs (+132/−32) gained a new function, kafka_producers_from_config(Option<&Value>). As a result, the way producer config is obtained changed from "calling a DynamicConfig method" to "parsing the JSON Value directly".
  • In rules.rs (+65/−51), the lifetime was fine-tuned from rules_cache.resolve(entity_type, rules_override.as_deref()) to as_ref() (no change in meaning).

phoenix-rankall is a Kafka consumer that handles SID backfill and ranking scores. Previously, kafka.rs alone held the topic, group, bootstrap, SASL, SSL, wily, and SID settings. In this commit that responsibility moved to phoenix-rankall/src/config/mod.rs, which has existed since the Gen4 release (+84 lines), leaving only a thin adapter in kafka.rs that calls KafkaConsumerBuilder. The config sources are declared explicitly in both env vars and the CLI (binary name xai-recsys-rankall). The SID backfill constants already in this module also stand out: a QPS cap of 800k, a batch of 5,000, and a minimum post age of 180 seconds.

• The Kafka config that kafka.rs held moved to the existing CLI module config/mod.rs verified — config/mod.rs:1-98.

  • phoenix-rankall src/config/mod.rs is a +84/−0 modification. This file has existed since the Gen4 release (47c1bcd), and the 2026-08-16 daily also quotes it.

  • The binary name is xai-recsys-rankall, and there are about 20 CLI arguments that take their values from environment variables.

    #[derive(Debug, Parser)]
    #[command(name = "xai-recsys-rankall")]
    pub struct Cli {
        #[arg(long, value_enum, env = "PIPELINE")]
        pub pipeline: PipelineKind,
        #[arg(long, default_value = "/s/kafka/main-2:kafka-tls", env = "KAFKA_BOOTSTRAP")]
        pub bootstrap: String,
        #[arg(long, default_value_t = 12, env = "SEEK_HOURS")]
        pub seek_hours: u64,
        #[arg(long, default_value = "phoenix.HomeExperiment1Lap7.", env = "SCORE_PREFIX")]
        pub score_prefix: String,
        #[arg(long, default_value_t = 180, env = "DUMP_INTERVAL_SECS")]
        pub dump_interval_secs: u64,
        #[arg(long, default_value_t = 180.0, env = "SID_MIN_POST_AGE_SECONDS")]
        pub sid_min_post_age_seconds: f64,
        #[arg(long, default_value_t = 5000, env = "SID_BACKFILL_BATCH_SIZE")]
        pub sid_backfill_batch_size: usize,
        #[arg(long, default_value_t = 800_000, env = "SID_BACKFILL_QPS_CAP")]
        pub sid_backfill_qps_cap: usize,
        #[arg(long, default_value_t = 1.0, env = "SID_BACKFILL_MAX_AGE_HOURS")]
        pub sid_backfill_max_age_hours: f64,
    

    — config/mod.rs:9-96

  • The figures worth noting here are SID_BACKFILL_QPS_CAP = 800_000 and SCORE_PREFIX = "phoenix.HomeExperiment1Lap7.".

  • The former states explicitly that the SID (Semantic ID) backfill QPS cap is 800k.

  • The latter means that phoenix-rankall consumes the ranking scores of Phoenix (a Grok-based Transformer ranker) for a specific experiment group, Experiment1 Lap7 on the Home surface.

  • There are more figures. SID backfill runs with a batch of 5,000 at 30-second intervals, a minimum post age of 180 seconds, and a maximum post age of 1 hour.

  • The Kafka dump interval is 180 seconds, 3 versions are retained, and the upper bound on how far back in Kafka time to rewind on restart is 12 hours.

• VF's fallback cache, which managed its 64 shards itself, was completely rewritten to delegate this to a quick_cache sync Cache verified — fallback_cache.rs:1-50.
  • The changed file is VF hydration/fallback_cache.rs (+69/−173).
  • The earlier structure of 64 shards × Mutex-wrapped unsync Cache (with a generation counter and a CacheEntry wrapper) was replaced by a quick_cache sync Cache and OptionsBuilder. Shard distribution and creation logic are delegated to the library.
  • The generic parameter also narrowed from FallbackCache<K, V> to FallbackCache<V>. The key is now hardcoded as u64.
  • Calls that took the Mutex each time through the with_locked_shard pattern are gone, so this refactor removes all Mutex contact at the fallback cache.
• There are also some minor changes.
  • In VF rules/golden_corpus/oon_tweet_label.rs (+76/−3), the golden corpus for the oon_tweet label group was expanded slightly.
  • The test fixture in VF params/client_switches.rs (+63/−13) was extended to load two files, scala_client_switches.yml and limited_actions_policy.yml.
  • VF hydration/execute.rs (+27/−24) was fine-tuned following the Fetcher change.
  • In Phoenix python/training/xai-checkpointing/xai_checkpointing/orbax_encrypted.py (+42/−11), checkpoint encryption was refactored. It is omitted because it is not a feed pipeline.
  • grox (Grok-based nearline classifier) flows/reply_spam/task_write.py (+13/−1) changed.
  • home-mixer query_hydrators/served_history_query_hydrator.rs (+5/−7) changed.
  • In home-mixer params/param.rs (+4/−4), only the cron sync time changed (no substantive change in value). We will check again in the next commit whether it was reflected.

1. The authority over the "restriction reason" has been pulled into VF. Until now, the only information VF put on the engagement axis of the Verdict was a single reason enum, and which actions that reason blocks and what the UI text says were the job of downstream (home-mixer, the client). From this commit, the following three things happen inside VF:

   • LimitedActionsPolicies holds the reason → action list mapping, based on FeatureSwitches.
   • LimitedActionsCopy holds (action, user locale, country) → localized Prompt.
   • treatment.rs::thrift_action combines the two and puts the per-action Prompt list directly inside the Thrift payload LimitedEngagements.

   As a result, the client does not need to maintain its own localization or action mapping tables, and every decision of reason → action → text is declared as server-side data. This change pairs with Gen4+32, where treatment.rs was isolated as the rules → thrift/proto/metric projection layer. The previous commit "gathered the wire projection point into one file," and this commit "raised the richness of the data passing through that projection point up to the UI level."

2. As LimitedEngagement changed to a structure that accumulates multiple reasons, the situation where the same tweet has several restriction reasons at once is now modeled explicitly.

   • Previously, once rules decided the engagement, any later ActionSpec::LimitedEngagement match was dropped (a dead-code path).
   • From this commit, if rules decide the engagement several times, the reasons accumulate in the Vec through limit.value.add(*reason).
   • In other words, when BlockedViewer (19 actions restricted) and ReadonlyViewer (14 actions restricted) overlap, VF merges the action lists of the two reasons into a union by action type and puts it in the payload together with the first reason verified — treatment.rs:128-148 and the test a_limit_sends_its_first_reason_and_the_union_of_its_reasons_policies.
   • Combinations such as ConversationControl with StaleTweet, or RootAuthorBlockedViewer with BlockedViewer, also become possible.

3. abuse-enforcement-service switched to a "boot-time config snapshot" model.

   • After 2026-09-23, when the Rust rewrite was released, this service had been using live config through a DynamicConfig object. From this commit, the Kafka producer binding depends only on the boot-time JSON Value snapshot.
   • Instead, when the Kafka config in operation (topics, SASL authentication, bootstrap servers) changes, the restart_on_config_change module restarts the service to apply the new config.
   • This pattern keeps the kafka producer lifecycle simple (no hot swap).
   • It also makes config changes clearly visible in logs and metrics (restart events).
   • Finally, it provides a fail-safe that does not restart if config validation (validate(live)) fails.
   • This approach applies only to changes in core config (topic labels, kafka producer config), and everything else is still a live reload path.

4. phoenix-rankall exposed an experiment group identifier and a QPS cap. Splitting out the CLI config module looks like an organizational refactor, but two figures are exposed in the process.

   • The first is SCORE_PREFIX = "phoenix.HomeExperiment1Lap7.". It means that the Phoenix ranking service publishes the scores of a specific experiment group on the Home surface, "Experiment1 Lap7", to a Kafka topic, and phoenix-rankall consumes only the scores with that specific prefix.
   • The name "Lap7" appears to be an experiment iteration number inferred.
   • The second is SID_BACKFILL_QPS_CAP = 800_000. It is the figure saying that the SID backfill QPS cap is 800k.
   • SID backfill is designed not to apply to all posts but to process only posts in a specific window (at most 1 hour, at least 3 minutes). SID_BACKFILL_MAX_AGE_HOURS = 1 appeared in an earlier commit and is kept this time as well.
   • Considering that the posts shown per day number in the hundreds of millions to billions, and reading this QPS cap together with the window design, the picture is that the SID embedding table is kept only for the subset of posts within the last 1 hour of freshness that the 800k QPS range covers.
   • Posts outside the freshness window remain without an SID or in a stale state.

Registering the Brazilian electoral court label in VF is an expansion of label vocabulary, not added logic. The Brazil 2026 election filter (handles hardcoded on the home-mixer side) was introduced in Gen4+1. This registration means that when VF receives from the labeling service a tweet carrying the brazil_election_legal label, it handles the tweet through the drop path regardless of the label's origin. The rule looks at the label type (BRAZIL_ELECTION_LEGAL) and the request country (br), and exempts the author themselves verified — tweet_rules.rs:570-579. The home-mixer handle list and the VF label, two axes, work together. After the previous commit raised the Brazil handle list to 2,795, adding the VF label registration means that the declaration that the two paths cooperate during the election period is now explicit in code.

2Generative ads retrieval GEAR deployed in Douyin Ads production — radar

GEAR is a framework that jointly optimizes the tokenizer, generator, and reranker end to end. It has two core mechanisms:

• BasisVQ reparameterizes the codebook as an orthogonal basis. This enables global gradient sharing and rigid spatial rotation of latent space, and stabilizes gradient dynamics without ad-hoc heuristics.
  • It extends BasisVQ to a prefix-aware BasisRQ, which greatly raises the expressiveness of the codebook within the same asymptotic time complexity.
• The context-conditioned reranking head sits inside the generation process and disambiguates colliding items with minimal computational overhead.

"It currently serves hundreds of millions of daily active users on Douyin Ads, yielding substantial empirical improvements in extensive online A/B tests." — arXiv 2609.39327, 2026-09-30

• (a) Snap GRP v0.1 (2026-09-29) unified retrieval, ranking, and reward in a single encoder-decoder, and achieved serving latency −69% and A/B view time +0.82%.
• (b) Tencent PCG TGR (2026-09-01) bundled ranking, retrieval, slate generation, and reason tokens into one frame and deployed it on 5 surfaces.
• (c) Kuaishou OneTrans-V2 (2026-09-23) unified a 3-stage cascade into a single Transformer, and achieved GMV +9.74%.
• (d) Today's GEAR explicitly addresses two structural bottlenecks of generative retrieval itself (representation collapse, item collision), and is serving hundreds of millions of DAU on Douyin Ads.

Unlike (a), (b), and (c), GEAR focuses only on retrieval, and targets the two known problems of that role with BasisVQ (codebook orthogonal basis reparameterization) and a context-conditioned reranking head. An earlier work on generative retrieval in the ads domain is Taobao TransRetrieval (2026-08-28). However, TransRetrieval is on the Transformer scaling side, treating the scaling problem as field heterogeneity, not on the SID-generation side. GEAR opens a Douyin line of SID-generation retrieval in ads, so whether ads retrieval converges toward generative approaches is the next checkpoint to watch.

Radar

• GEAR (Douyin / ByteDance) deployed generative ad retrieval in Douyin Ads production for hundreds of millions of DAU. The affiliation, deployment scale, and mechanism are confirmed facts verified — arXiv 2609.39327.
  • It resolves the two bottlenecks end to end: representation collapse with BasisVQ (orthogonal basis codebook), and item collision with a context-conditioned reranking head.
  • It states that it obtained "substantial empirical improvements" in online A/B tests.
  • Added to radar.md under new this month.

What to check next