2026-08-20

1Ads brand safety V2 verdict redesigned within one day — X

• Gen4+1 (2026-08-14, c65aa17) partially released the weight interpretation rules, the Brazil election filter, Thompson sampling, three Phoenix bools, and the BDSM enforcement note. It spans 21 files, +2,271/−32, and the details are in daily/2026-08-16.md.
• Gen4+2 (2026-08-17, b089ce6) added the ads brand safety V2 verdict function and the adult content Grok 4.5 judge layer, and removed the Phoenix retrieval indexing gate. The V2 verdict side included compute_verdict_v2, strip_v1_grok_written, and 7 PROMPT_OWNED_RULES. It spans 34 files, +751/−162, and the details are in daily/2026-08-19.md.

Today's source is the next release (2026-08-18, 11a71f8). The migration helper that yesterday's canon (a document holding the current understanding of each service) recorded in detail was deleted entirely in this commit. What was deleted is strip_v1_grok_written and the 7 PROMPT_OWNED_RULES rule ids. This is the first time this has happened in the only "daily commit" open-source service this repository covers. The observation left in yesterday's daily, that "which param changes how in the next commit becomes a lagging indicator of A/B results", found real evidence within a day.

This commit contains a dual-write redesign that overturns, within a day, a fact folded in yesterday. Dual-write is an approach that records the same data in both the old and the new format during a migration. The commit also cleaned up three layers, pipeline, indexing, and grox (a Grok-based content classification service), and five directions, including this redesign, affect the canon.

The way ads brand safety V2 handles dual-write labels was redesigned within a day. In the Gen4+2 original, when the V2 verdict logic looked at V1 labels, it filtered out only the dual-write labels that belonged to 7 specific botmaker (X's rules engine) rule ids, and then judged with the V2 label set. The constant holding these 7 is PROMPT_OWNED_RULES. Today's commit deleted this helper (strip_v1_grok_written) and even the Cow import. Instead, it uses a new constant, V2_WRITTEN_LABELS (4 V2 labels), as the verdict condition.

• At its start, the verdict function compute_verdict_v2 checks whether the label set contains any of these 4.
  • If it contains none, it falls back straight to compute_verdict(v1).
  • If it contains at least one, it judges with the pure V2 label set, which is the set with NSFA_HIGH_PRECISION and NSFA_LIMITED_INVENTORY removed.
• The key changes are the new constant and the new gate at the start of the verdict function verified — home-mixer/models/brand_safety.rs

  • The new constant holds the 4 V2 labels.

    const V2_WRITTEN_LABELS: &[SafetyLabelType] = &[
        SafetyLabelType::GROK_SFA_V2,
        SafetyLabelType::GROK_NSFA_V2,
        SafetyLabelType::GROK_NSFA_LIMITED_V2,
        SafetyLabelType::GROK_NSFA_EXPANDED_V2,
    ];
    

  • The new gate calls the V1 verdict function as is when no V2 label is present.

    pub(crate) fn compute_verdict_v2(
        labels: &HashMap<SafetyLabelType, SafetyLabel>,
        tweet_id: u64,
    ) -> BrandSafetyVerdict {
        if !V2_WRITTEN_LABELS.iter().any(|l| labels.contains_key(l)) {
            return compute_verdict(labels, tweet_id);
        }
        if MEDIUM_RISK_LABELS_V2.iter().any(|l| labels.contains_key(l)) {
            return BrandSafetyVerdict::MediumRisk;
        }
        // ... low risk check, otherwise Safe
    }
    

  • The V2 label set also shrank. NSFA_HIGH_PRECISION was removed from MEDIUM_RISK_LABELS_V2, where it was one of the previous 16. NSFA_LIMITED_INVENTORY was removed from LOW_RISK_LABELS_V2, where it was one of the previous 3.

• The decisive evidence for this change is one case asserted by the new test case v2_defers_to_v1_when_v2_has_not_ruled.
  • V1 judges the label set NSFA_HIGH_PRECISION + GROK_NSFA + GROK_SFA_V2 as MediumRisk, but V2 judges it as Safe.
  • In other words, once V2 sends a verdict signal through GROK_SFA_V2 alone, the V2 decision fully dominates, ignoring even NSFA_HIGH_PRECISION, V1's traditional heuristic label.

The new test v2_defers_to_v1_when_v2_has_not_ruled spells out this verdict approach in 6 cases.

let v1_safe = labels_with(&[SafetyLabelType::GROK_SFA]);
assert_eq!(compute_verdict_v2(&v1_safe, PRE_CUTOFF_ID), compute_verdict(&v1_safe, PRE_CUTOFF_ID));
assert_eq!(compute_verdict_v2(&v1_safe, PRE_CUTOFF_ID), BrandSafetyVerdict::Safe);

// Decisive case: when V2 sends an SFA label, even V1's heuristic is ignored
let freed = labels_with(&[
    SafetyLabelType::NSFA_HIGH_PRECISION,
    SafetyLabelType::GROK_NSFA,
    SafetyLabelType::GROK_SFA_V2,
]);
assert_eq!(compute_verdict(&freed, PRE_CUTOFF_ID), BrandSafetyVerdict::MediumRisk);
assert_eq!(compute_verdict_v2(&freed, PRE_CUTOFF_ID), BrandSafetyVerdict::Safe);

— home-mixer/models/brand_safety.rs:290-322

Compared with the Gen4+2 original folded into the canon yesterday, the following was overturned:

The direction of this redesign is much clearer. It adopts the standard dual-write migration approach as is: "if V2 has ruled, V2 dominates; otherwise keep the V1 safety net." The 7-rule-id filter approach folded in yesterday was, in effect, a one-day experiment.

The reverse-chron pipeline that builds the Following feed branched to a dedicated visibility-verdict hydrator. VF (visibility-filtering), which handles the visibility verdict, is a service that decides whether a post may be shown to the current viewer. A hydrator is a stage that fills in the field values a candidate post needs.

The new file home-mixer/candidate_hydrators/vf_following_candidate_hydrator.rs (95 lines) was wired in, and reverse_chron_posts_pipeline.rs uses this new hydrator instead of the existing VFCandidateHydrator. The new hydrator's signature is nearly identical to the existing one, but the PostCandidate it returns fills only the two fields visibility_reason and drop_ancillary_posts and initializes the rest with ..Default::default(). In other words, the Following pipeline narrows the VF hydration output to these two fields, and other hydrators fill the remaining candidate fields. The should_drop_ancillary helper is made pub(crate) in the existing file and reused in the new file.

• The new hydrator was newly created as a separate file verified — home-mixer/candidate_hydrators/vf_following_candidate_hydrator.rs

  pub struct VFFollowingCandidateHydrator {
      pub strato_vf_client: Arc<dyn VfClient + Send + Sync>,
      pub xai_vf_client: Arc<dyn VfClient + Send + Sync>,
  }
  
  #[async_trait]
  impl Hydrator<ScoredPostsQuery, PostCandidate> for VFFollowingCandidateHydrator {
      async fn hydrate(&self, query: &ScoredPostsQuery, candidates: &[PostCandidate])
          -> Vec<Result<PostCandidate, String>> {
          // ... collect post_ids, client.get_result(post_ids, TimelineHome, query.user_id, context)
          let hydrated = match primary_result {
              Some(Err(err)) => Err(err.to_string()),
              _ => Ok(PostCandidate {
                  visibility_reason,
                  drop_ancillary_posts: Some(drop_ancillary),
                  ..Default::default()
              }),
          };
          // ...
      }
  }
  

• reverse_chron_posts_pipeline.rs wires in the new hydrator instead of the existing one verified.

  -            Box::new(VFCandidateHydrator::new(strato_vf_client, xai_vf_client).await),
  +            Box::new(VFFollowingCandidateHydrator::new(
  +                strato_vf_client,
  +                xai_vf_client,
  +            )),
  

• SafetyLevel::TimelineHome is kept as is. This means Following and For You share the safety level, and we also reconfirmed that it is not TimelineHomeRecommendations.

• The should_drop_ancillary helper was promoted to pub(crate) in the existing vf_candidate_hydrator.rs and is shared by both hydrators.

• The return values differ. The existing hydrator fills and returns all candidate fields, but the new one fills only two fields (visibility_reason and drop_ancillary_posts) and leaves the rest at default. That is, it reads as the Following pipeline adding only the VF result on top of fields that other hydrators have already filled inferred.

Seven SID (semantic ID, an ID that represents an item as discrete codes at several levels) fields were newly wired into the request sent to VMRanker, which handles re-ranking. From Gen3 through Gen4+2, SID appeared only in Phoenix retrieval window names (video_14day, video_48h, nsfw_video_48h/168h/14day). This change is the first trace of VMRanker also consuming SID explicitly. The architectural fact that a 3-level codebook (coarse → fine) exists was also newly confirmed by this wiring.

build_request in home-mixer/scorers/vm_ranker.rs began sending new fields alongside each source's k, pool_rank, and pool_rank_gap. The new fields are sid_known, sid_k1/k2/k3, the 3-level hierarchical SID codewords, and sid_gap1/gap2/gap3, the neighbor gap at each level.

• For each source, build_request sends the existing rank field set together with seven SID fields verified — home-mixer/scorers/vm_ranker.rs:194-206

  Some(Source {
      k: s.k,
      pool_rank: s.pool_rank,
      pool_rank_gap: s.pool_rank_gap,
      sid_known: s.sid_known,
      sid_k1: s.sid_k_l1,
      sid_k2: s.sid_k_l2,
      sid_k3: s.sid_k_l3,
      sid_gap1: s.sid_gap_l1,
      sid_gap2: s.sid_gap_l2,
      sid_gap3: s.sid_gap_l3,
  }),
  

  • sid_k1/k2/k3 and sid_gap1/gap2/gap3 in the request copy the source's sid_k_l1/2/3 and sid_gap_l1/2/3 values as is.

• The SID is 3-level hierarchical. Each candidate source carries three codewords together. sid_k_l1 is coarse, sid_k_l2 is mid, and sid_k_l3 is fine.

• sid_gap_l{1,2,3} is the neighbor gap in each level's codebook. Because the field names pair with pool_rank_gap, they appear to be indicators standing in for confidence or sparsity inferred.

• sid_known is a bool flag indicating whether the SID is known (valid). If it is unknown, VMRanker appears to fall back to other fields inferred.

• The new fields are wired alongside the existing k, pool_rank, and pool_rank_gap. These three are the candidate source rank information confirmed in Gen4. In other words, VMRanker now has three axes of information when judging diversity and re-ranking.

  • Collaborative rank per candidate source
  • Position in the 3-level semantic ID
  • Neighbor gap

• The next commit must show whether the SID window on the Phoenix retrieval side and these vm_ranker fields share the same codebook. This SID window is the SID to which video_14day was added in the 2026-08-14 commit (Gen4+1).
• DPP (determinantal point process) is a diversity technique that selects at the set level so that scores are reduced when similar items are chosen together. Whether VMRanker uses the SID gap as the similarity component of the DPP kernel also needs to be checked in the next commit.
• How this corresponds to the SID-family 3-level codebook approach observed in Kuaishou HD-Rec and PushDualGen also needs to be checked. Both cases are being watched in radar.md. X did not disclose the codebook composition, training method, or vocabulary.

The upstream event stream that Phoenix retrieval receives was migrated to the unified posts topic V3. What moved is Phoenix's postCreationEventForwarder, and the following changed:

• The Kafka topic changed from content_understanding_realtime_unified_posts to ..._v3.
• The Kafka cluster moved from X's main main-2 to the separate Phoenix-dedicated cluster phoenix-kafka-external-bootstrap.
  • The full paths are /s/kafka/main-2:kafka-tls and /s/kafka/phoenix-kafka-external-bootstrap.
• The mapping moved from ValueOnly to KeyValue(key=Long). Previously it carried only the Thrift value; now it states postId explicitly as the Kafka key.
  • The new mapping is KeyValue({key = Long, keyEncoding = NativeEncoding, ...}).
• The consumer group was also newly created, from post_creation_event to post_creation_event_v3, and develConsumerGroupId = Some("post_creation_event_v3_devel") was introduced as well.

The signature of postCreationEventProcessor.strato also changed from Op.execute[Event, Unit] to Op.execute[(Long, Event), Unit], so it prefers the postId that comes from the tuple.

• The forwarder configuration changed as in the following diff verified — phoenix-rankall-strato/stream_forwarders/postCreationEventForwarder.strato

  -import <content_understanding/sharedConfig>
  +import <hydra/sharedConfig>
  
   StreamForwarder({
     source = KafkaTopic({
  -    topic = "content_understanding_realtime_unified_posts",
  -    dest = "/s/kafka/main-2:kafka-tls",
  -    consumerGroupId = "post_creation_event",
  -    develConsumerGroupId = None,
  -    mapping = ValueOnly({
  +    topic = "content_understanding_realtime_unified_posts_v3",
  +    dest = "/s/kafka/phoenix-kafka-external-bootstrap",
  +    consumerGroupId = "post_creation_event_v3",
  +    develConsumerGroupId = Some("post_creation_event_v3_devel"),
  +    mapping = KeyValue({
  +      key = Type(Long),
  +      keyEncoding = NativeEncoding,
         value = Type(...ContentUnderstandingMetadataV2),
  -      encoding = ThriftEncoding
  +      valueEncoding = ThriftEncoding
       })
     }),
  

• The signature of postCreationEventProcessor.strato changed to match verified.

  -val executeOp = Op.execute({ idempotent = true })[Event, Unit] { ctx =>
  +val executeOp = Op.execute({ idempotent = true })[(Long, Event), Unit] { ctx =>
  +  val (postId, _) = ctx.arg
     val request = {
  -    postId = ctx.arg.postMetadata.post.postId,
  +    postId = postId,
       eventSource = PostCreation
     }
  

• Raised the content understanding schema from v1 to v3.
• Moved from X's main Kafka cluster to an isolated Phoenix-dedicated cluster.
• Promoted postId to the Kafka key in the mapping, so partitioning, dedup, and inverted-index lookup can be handled purely by key.

Moving the import to hydra/sharedConfig is a signal that the Phoenix family is being brought under the hydra infrastructure. This change is in the same direction as the "removal of the indexing gate" in the 2026-08-17 commit (Gen4+2). It is consistent to read it as a separate Kafka cluster having become necessary as indexing throughput grew inferred.

grox's reply_spam removed its separate flow and was compressed into a single unified flow. Previously two generators sat side by side. PostStreamTaskGenerator detected spam in the entire post stream, and ReplyRankingTaskGenerator ranked replies.

• Today's commit deleted PostStreamTaskGenerator and also removed the TOPIC_UNIFIED_POSTS (v1 topic) constant.

• Instead, ReplyRankingTaskGenerator now injects three plans together. PlanReplyRanking is the existing plan, and PlanSpamComment and PlanCoordinatedSpam are the newly absorbed plans.

• In task_filter.py, the follower threshold was raised 1.5× from 40k to 60k verified — grox/flows/reply_spam/task_filter.py.

  • This is the third increase, following 15k → 30k in the 2026-08-14 commit (Gen4+1) and 30k → 40k in the 2026-08-17 commit (Gen4+2).

• That is, the set of targets for grok spam verdicts widened and the set scored for reply ranking narrowed verified — task_filter.py:77-92, 214-225.

  • Replies where both the root and the reply target are at or below the threshold get a spam verdict, and replies where either one exceeds the threshold get reply ranking scoring.
  • The scoring pipeline itself was also consolidated into one flow on a single topic (v3).

• The generator cleanup is as in the following diff verified — grox/flows/reply_spam/generators.py

  -@register
  -class PostStreamTaskGenerator(StreamTaskGenerator):
  -    TASK_GENERATOR_TYPE = POST_STREAM
  -    PLANS_TO_INJECT = {PlanSpamComment.KEY, PlanCoordinatedSpam.KEY}
  -
  -    def _get_loader(self):
  -        return KafkaPostLoader(TOPIC_UNIFIED_POSTS)
  -
  
   @register
   class ReplyRankingTaskGenerator(StreamTaskGenerator):
       TASK_GENERATOR_TYPE = REPLY_RANKING
  -    PLANS_TO_INJECT = {PlanReplyRanking.KEY}
  +    PLANS_TO_INJECT = {
  +        PlanReplyRanking.KEY,
  +        PlanSpamComment.KEY,
  +        PlanCoordinatedSpam.KEY,
  +    }
  
       def _get_loader(self):
           return KafkaPostLoader(TOPIC_UNIFIED_POSTS_V3)
  

• The two threshold constants in task_filter.py were also readjusted verified.

  • TaskSpamFilter.FOLLOWER_COUNT_THRESHOLD_FOR_SPAM_DETECTION: 40,000 → 60,000
  • TaskReplyRankingFilter.FOLLOWER_COUNT_THRESHOLD_FOR_REPLY_RANKING: 40,000 → 60,000

• constants.py was also cleaned up verified. The POST_STREAM and TOPIC_UNIFIED_POSTS constants were deleted.

  • Four constants remain: REPLY_RANKING, REPLY_RANKING_RECOVERY, TOPIC_UNIFIED_POSTS_V3, and TOPIC_REPLY_RANKING_RECOVERY.

Cold start instrumentation became simpler. The change is in count_tracked_ids in author_cold_start.rs, and the tracked filter is gone.

• The counting logic changed as follows verified — home-mixer/scorers/author_cold_start.rs:100-107

       let source = c.served_type.map(|t| t as i32).unwrap_or(0);
  -    if tracked.contains(&c.tweet_id) {
  -        *counts.entry((c.tweet_id, source)).or_insert(0) += 1;
  -    }
  -    if tracked.contains(&c.author_id) {
  -        *counts.entry((c.author_id, source)).or_insert(0) += 1;
  -    }
  +    *counts.entry((c.tweet_id, source)).or_insert(0) += 1;
  

• The tracked list filter was removed and the author_id count was deleted, so it now always counts the tweet_id of every candidate.

• That is, the nature of the home_mixer.cold_start_tracked_ids_total{id, source} instrumentation changed. It is now a full census of how many candidates the cold start pipeline passes through per source, not the number of appearances of a specific tracked ID.

• The ColdStartTrackedIds param itself remains, but its connection to where this function is used changed. The purpose of the observation hook moved from "A/B on specific accounts" to "overall traffic per source".

This commit also contains 4 minor changes.

• Logging for the grox adult content cross-validation was improved verified. In the judge task folded in yesterday, the outcome attribute of Metrics.counter(metric) changed from the two labels "hard"|"soft" to the raw verdict result policyType.value. Even if the judge returns adult categories at finer granularity, they can be instrumented without loss.
• The vf_following_candidate_hydrator module was registered in home-mixer/candidate_hydrators/mod.rs verified.
• In visibility-filtering/server_deps.rs, the timeout of the Gizmoduck (X's user information service) Strato client was set explicitly to 80ms verified. Previously no timeout was stated.
  • Created a new constant const GIZMODUCK_STRATO_REQUEST_TIMEOUT_MS: u64 = 80 and wired it into the client config as request_timeout_ms: Some(GIZMODUCK_STRATO_REQUEST_TIMEOUT_MS).
• home-mixer/candidate_hydrators/ads_brand_safety_vf_hydrator.rs changed by +1/−1 verified. It slightly adjusted the parent hydrator and kept the wiring in line with the V2 verdict redesign.

• One-line summary section: last updated 2026-08-19 → 2026-08-20, and a Gen4+3 statement was added to the summary paragraph.
• Open-source Gen4 table: added a Gen4+3 row (11a71f8).
• Ads section (brand_safety V2): expanded to describe Gen4+2 and Gen4+3 separately. Detailed the deletion of strip_v1_grok_written and the redesign of the V2_WRITTEN_LABELS gate, and quoted the decisive case of the new test v2_defers_to_v1_when_v2_has_not_ruled.
• Ranking section (VMRanker and DPP): added a table summarizing the 7 SID fields and folded in the 3-level hierarchical codebook fact.
• Cold start section: folded in the simplification of count_tracked_ids.
• Phoenix training and index section: added a detailed table of the postCreationEventForwarder V3 migration.
• Grox section: folded in the Gen4+3 readjustment and flow merge (PostStreamTaskGenerator deleted, follower threshold 60k).
• Grox → PtOS adult content cross-validation section: folded in the logging improvement of the Gen4+2 judge task.
• Feed surface section (Following): folded in the VFFollowingCandidateHydrator branch.
• Open question section: introduced "What was resolved / overturned by the Gen4+3 release (2026-08-20)". As a ⚠ Claim changed (within a day) item, it states explicitly that the PROMPT_OWNED_RULES line folded in yesterday was overturned.
• Sources section: added links to 6 files related to Gen4+3.

The question that yesterday's daily flagged as needing a deep dive, "PROMPT_OWNED_RULES 7 rule ids ↔ real botmaker-rules/ rule file mapping", was not resolved by itself; the question itself no longer exists. That is because the constant was deleted from the code.

This leaves two kinds of observation evidence in the canon.

1. The release cadence of X's Gen4 was confirmed as the observation that "it commits daily, and one-day redesigns are common". Gen4 (8-13) → Gen4+1 (8-14) → Gen4+2 (8-17) → Gen4+3 (8-18) are 4 releases within five days (8-13–8-18). The last two (yesterday's and today's) are a one-day experiment in which today completely replaced the dual-write logic added yesterday. The practice introduced in Gen4 by docs/BIDIRECTIONAL_BOOST_CHANGE.md, "documenting parameter changes through code diffs", has produced two real cases in five days.
2. When the canon folds in new dual-write logic, it is right to also state that "this logic itself may change in the next commit". Yesterday's daily statement of strip_v1_grok_written was perfect in its detailed evidence, its rule id listing, and its list of undisclosed items. But it implicitly carried the implication that "this logic will be the stable version". From now on, whenever we see a new dual-write wiring, we will always note that "a specific helper in this commit may be deleted in the next generation".
3. The origin of the 4 V2_WRITTEN_LABELS labels, the new verdict condition, is a new deep dive target. The four, GROK_SFA_V2·GROK_NSFA_V2·GROK_NSFA_LIMITED_V2·GROK_NSFA_EXPANDED_V2, are the 3 labels that entered the label set yesterday (GROK_NSFA_V2·GROK_NSFA_EXPANDED_V2·GROK_NSFA_LIMITED_V2) plus GROK_SFA_V2. GROK_SFA_V2 is not a new label; it is the label compute_verdict_v2 has used since Gen4+2 to decide whether Grok had scored verified — brand_safety.rs:120-130@b089ce6. We have not yet been able to confirm the following three things:
   • Whether GROK_SFA_V2 means an explicit positive label, "safe for ads V2"
   • Whether Grok prompt retraining was deployed between yesterday and today
   • Or whether the V2 labeler comes from another pipeline of the asset

Corrections

⚠ The statement of the Gen4+2 dual-write logic was overturned. The detailed quote in yesterday's daily/2026-08-19 is included as well. The strip_v1_grok_written and PROMPT_OWNED_RULES 7-rule-id lines were deleted from the code today.

• The canon did not delete the Gen4+2 statement. Instead it attached the comment "redesigned within a day" and kept it as evidence for comparing generations, stating it explicitly in two places: the open question section and the Gen4 table.
• This looks like a slight departure from the CLAUDE.md principle "the canon is compressed to hold only conclusions". Still, the one-day dual-write experiment itself is decisive evidence for observing X's release cadence, so it is kept for generation comparison.

Radar and scope of checks

No new radar entries were added. The places checked today and the results are as follows.

Of the 30 items triaged from arXiv, 0 passed as new, because none has a production or A/B claim. UniDot (arXiv 2608.16797, 2026-08-17) covers a unified industrial recommender architecture but does not meet the radar acceptance criteria. It has only a KDD Cup 2026 runner-up result claim, and none of production deployment, author affiliation, or A/B metrics is public.

What to check next