2026-08-16

Other changes reviewed today:

• In the same commit, X added a filter that removes from For You the posts of accounts reported to the electoral court in connection with Brazil's 2026 election, unless the viewer follows the account.
• Three bool inputs of the X ranking model started receiving real values for the first time.
• The author affiliation confirmed that DrEM, Kuaishou's ensemble ranking paper, is a production paper.

1Weight-reading rule stated in code and Brazil election filter added — X

• The header comment of the parameter file of home-mixer (the service that serves the For You feed) puts this interpretation rule first verified — home-mixer/params/param.rs:281-306.

  "Each weight multiplies the predicted probability of that action (P(favorite), P(repost), …) or a continuous value e.g. watch time -- the weights do not multiply raw engagement counts. One common misinterpretation is that you can read these weight ratios as count equivalences, e.g. the incorrect statement that 'one report cancels 468 likes' -- this is incorrect because the weights apply to the predicted probabilities rather than raw counts." — home-mixer/params/param.rs:283-292

• The canon has used statements such as "reply 13.5 vs favorite 0.5 → 27×", quoted through Gen3, in several places (e.g., multi-task-ranking.md). X's own comment settles that this statement is wrong in principle.

  • Weights multiply predicted probabilities, so they are not ratios of raw counts.
  • In effect, by releasing the parameter mirror in Gen4, X explicitly acknowledged that without a reading guide these values cause misunderstanding.

• The same comment goes on to explain why the absolute value of the report weight looks so large.

  "And the baseline probability of a Report is more than 1000x lower than a Like, so it's weighted more to allow the prediction to affect the final ranking at all."

  • The size of the report value −234 is the product of value × 1/propensity. Propensity is the base probability that the action occurs.
  • The absolute value appears large because the propensity of a report is 1/1000 or less of that of a like.
  • Yesterday's canon update also noted a caveat in the same direction; this time X described it itself in the code.

• The same comment also rejects the misconception about mass reporting, for two reasons.

  "It's predicting your likelihood of the action, not summing up raw weights on counts. Also, recommendations are personalized, so reports from bad actors will primarily affect recommendations for users who are similar to the bad actors, rather than having the same effect on the post's ranking to everyone."

  • In sum, even if bot accounts file reports in bulk, two things hold.
    1. The score is your predicted probability. So mass reports primarily affect the recommendations of users similar to those bots.
    2. Only actions taken on posts served in the Home Timeline count as ranking signals. So reactions generated by sharing a link in a group chat do not affect ranking.

• The same comment block was also inserted in home-mixer/scorers/ranking_scorer.rs:415-446, and a paragraph with the same content was inserted in the README #scoring-and-ranking section.

  • By placing the same statement in three places, X made sure readers are not misled whether they read the code or the documents.

• X announced that, under Brazilian electoral law, it removes from For You the posts of accounts reported to Brazil's electoral court while keeping accounts the viewer follows directly verified — README.md #august-14th-2026.

  "As announced by X, in accordance with Brazilian electoral law, For You now runs Brazil2026ElectionFilter, which removes posts from accounts reported to Brazil's Electoral Court for the 2026 election, unless the viewer explicitly follows the account. A benefit of open-source is that you can see that changes like this exist, and exactly how they work — take a look at the code." — README.md, announcement link @XBR

• This is the first case in which a jurisdictional filter can be observed directly in code. Until now, responses to regulations such as the DSA, GDPR, and NetzDG could mostly be confirmed only through regulatory filings (the EU transparency report).

• The filter is named Brazil2026ElectionFilter. It hardcodes 665 accounts reported to Brazil's electoral court (TSE) as a code constant and leaves the account name in a comment next to each ID verified — brazil_2026_election_filter.rs:9-20.

  • The comment above the constant reads "User ids reported to the Electoral Court for the Brazil 2026 election", and it links the TSE candidate dataset (candidatos-2026) as the underlying data.
  • The constant's type is FxHashSet<u64>, and the comments take the form @username.
  • If the viewer follows the account, it is treated as an exception (applied symmetrically, regardless of political orientation).

• The filter body, home-mixer/filters/brazil_2026_election_filter.rs, is 1,573 lines, most of which are the account ID constants and the usernames noted in comments.

  • The core logic is a little over 10 lines.

    static BRAZIL_2026_ELECTION_USER_IDS: LazyLock<FxHashSet<u64>> = ...;  // 665 accounts
    
    impl Brazil2026ElectionFilter {
        fn is_excluded_author(user_id: u64, followed_user_ids: &FxHashSet<u64>) -> bool {
            BRAZIL_2026_ELECTION_USER_IDS.contains(&user_id) && !followed_user_ids.contains(&user_id)
        }
        fn should_remove(candidate: &PostCandidate, followed_user_ids: &FxHashSet<u64>) -> bool {
            let is_excluded = |user_id: u64| Self::is_excluded_author(user_id, followed_user_ids);
            if is_excluded(candidate.author_id) { return true; }
            if candidate.retweeted_user_id.is_some_and(is_excluded) { return true; }
            if candidate.quoted_user_id.is_some_and(is_excluded) { return true; }
            candidate.ancestor_users.iter().copied().any(is_excluded)
        }
    }
    

• If any of the candidate's author, retweet original author, quote original author, or reply-thread ancestor authors is on the list and the viewer does not follow that account, the filter removes the candidate.

  • The follow exception is based on the user's own follow graph, so it works symmetrically regardless of the political spectrum.

• The file header comment states the legal basis verified.

  "Application providers that use a recommendation system for users must exclude from the results the channels and profiles reported to the Electoral Court under the terms of § 1º of this article and, except in cases of paid boosting, the content posted on them." — home-mixer/filters/brazil_2026_election_filter.rs:9-11

• The dataset behind the list is also noted in a comment verified.

  "https://dadosabertos.tse.jus.br/dataset/candidatos-2026"

• This jurisdictional filter is applied right after the social graph filter handles mutes and blocks. The connection point is phoenix_candidate_pipeline.rs:356-368.

  • It sits right after AuthorSocialgraphFilter and right before VideoFilter.

• The list includes people from several parties. A few excerpts from the code:

  • 40053694 // @FlavioBolsonaro (Flavio Bolsonaro, Liberal Party senator)
  • 21069302 // @marcelvanhattem (Marcel van Hattem, New Party member of the lower house)
  • 48298703 // @eduardopaes (Eduardo Paes, Social Democratic Party, mayor of Rio de Janeiro)
  • 25858078 // @ManuelaDavila (Manuela D'Avila, Communist Party)
  • 33374761 // @cirogomes (Ciro Gomes, Democratic Labour Party)

• There is one unusual comment at the top of the code verified.

  "OmarAzizSenador deleted his account at the time this code was written." — home-mixer/filters/brazil_2026_election_filter.rs:15

  • The documentation is detailed enough to separately note, in a comment, a person who is on the TSE list but whose X account was deleted.

• If this practice continues, similar filters will be visible in code for each country's election cycle. It is also a signal that the following phrase X stressed in the README of the 2026-08-14 commit (Gen4+1) applies to regulatory responses too: "a benefit of open-source is that you can see that changes like this exist, and exactly how they work".

• Three things to watch next:

  1. How the filter is cleaned up after the election cycle ends. The Brazil 2026 election is an event that ends at a specific point, so this filter is also temporary. Whether X removes the filter, empties only the constant and keeps the code, or moves it to a separate archive will be an indicator of how long the open-source practice lasts.
  2. Whether filters responding to other countries' elections and regulations are added in the same way. Whether similar filters appear in the code for EU parliamentary elections, Indian general elections, or US presidential elections is what to check over the next six months.
  3. How this connects to the diff-documentation practice announced by docs/BIDIRECTIONAL_BOOST_CHANGE.md. That document, covered yesterday, was a model for documenting parameter changes as code diffs from now on. How later commits document additions to or deletions from the Brazil filter's 665-account list will also show the maturity of this practice.

• The cold start slot selector was updated verified — home-mixer/scorers/author_cold_start.rs:151-190. Now the way of choosing "which candidate fills the slot" from the cold start candidate set splits into two branches: greedy max-score and Beta reward Thompson sampling.

  • The changed file is author_cold_start.rs.
  • Thompson sampling draws one sample per candidate from its Bayesian posterior and selects by that value; it is a standard bandit technique that induces exploration probabilistically.

• Six parameters were newly added verified — home-mixer/params/param.rs:687-724.

  • EnableColdStartThompsonSampling = false: the default is off. Actual production still runs greedy.
  • ColdStartBetaAlpha0 = 0.75: the α of the Beta(α₀, β₀) prior; it counts as the prior observation of likes.
  • ColdStartBetaBeta0 = 49.25: β₀; it counts as the prior observation of non-likes. Since α₀ + β₀ = 50 is the effective sample size of the prior, it roughly means the prior knowledge "0.75 likes out of 50 impressions".
  • ColdStartTsTopK = 5: take the top K by Beta sampling, then break ties with the actual score.
  • ColdStartImpressionScale = 1.0: the scale multiplied into view_count.
  • ColdStartTrackedIds = "": a comma-separated list of account and post IDs that specifies the targets for which serving statistics are instrumented.

• The sampling rule is written in the same file verified.

  • For each candidate i, one sample is drawn from the posterior Beta(α₀ + likes, β₀ + max(views − likes, 0)). The top K are chosen by that value, and within them the candidate with the highest original score is selected.

    fn sample_reward<R: Rng + ?Sized>(candidate: &PostCandidate, alpha0: f64, beta0: f64, scale: f64, rng: &mut R) -> f64 {
        let n = scale * candidate.view_count.unwrap_or(0) as f64;
        let x = (candidate.fav_count.unwrap_or(0).max(0) as f64).min(n);
        let alpha = alpha0 + x;
        let beta = beta0 + (n - x).max(0.0);
        Beta::new(alpha, beta).map(|d| d.sample(rng)).unwrap_or(0.5)
    }
    

• The fact that the default is off matters. As of this release, the cold start applied to real users is still max-score greedy.

  • Thompson sampling is only wired into the A/B infrastructure, and whether it is actually adopted will be decided by upcoming A/B results.

• If a specific account or post ID is registered in ColdStartTrackedIds, a counter tracks how many times that account appeared from which source in the cold start pipeline. It is a per-item observation hook that lets a specific account be observed directly in A/B experiments.

  • The counter name is home_mixer.cold_start_tracked_ids_total{id, source}.
  • For example, if one politician's account is registered, one can count how many times that account's posts were shown through which cold start path.
  • The existence of this instrumentation hook points the same way as Gen4's under-the-hood transparency approach, which opened account-level label lookup. However, this is X-internal instrumentation, not material released externally.

• The input features of Phoenix (the Transformer ranking model of X For You) were populated for the first time. This input field had been classified as a "dead field" in the canon and remained on the caveat list.
  • Through early Gen4, the field observed in code was bool_features: vec![false; ...].
  • This "dead field" bool_features was entirely filled with vec![false; ...] from Gen3 through early Gen4, so it had no effect as an input.
• In this commit (Gen4+1), three of them were populated for the first time verified — phoenix/crates/common/xai-recsys/src/util.rs:456-540, 728-900, phoenix/crates/common/xai-recsys/src/model_config.rs:40.
  • The newly populated ones are IS_STALE_POST14D and IS_AUTHOR_FOLLOWED/FOLLOWING_BY_VIEWER_SEQ, three bool sequences in all. They indicate whether a post is stale at 14 days, and the two follow directions between author and viewer.
  • The core of the code diff is two places where vec![false; candidate_seq_len * n_post_bool] was replaced with candidate_bool_features. It applies to both the candidate sequence and the history sequence.

• Two signals that handle stale posts are inserted as a pair verified.

  • If stale, the counts are set to 0 and the bool is raised; if not stale, the actual counts are put in.

    let creation_valid = candidate_post_creation_ts_sec[j] > 0;
    let original_age_sec = now_sec as i64 - candidate_post_creation_ts_sec[j] as i64;
    let is_stale = stale_post_enabled && creation_valid && original_age_sec > STALE_POST_14D_TTL_SEC;
    candidate_is_stale_post[j] = is_stale;
    if is_stale {
        stamp_engagement_counts(..., 0, 0, 0, 0, 0);
    } else {
        stamp_engagement_counts(..., candidate.fav_count, candidate.reply_count, ...);
    }
    

  • The ranker can explicitly learn a candidate with "count=0 + is_stale=true" and one with "count=0 + is_stale=false" as different cases.

• It is especially interesting that the stale post signal is paired with count zeroing. It attaches an explicit label to tell the ranker "this count is low not because the post is unpopular but because it is old and its exposure has stopped".
  • This pairing is a design that lets the ranker work around, through training, the snowball problem (a self-reinforcing loop in which a low count on an old post leads to a low ranking, and that ranking in turn leads to a low count) that the canon has observed through Gen3.
• It appears to be a design that lets the ranker work around, through training, the problem the canon has observed since Gen3: "a low count on an old post leads to a low ranking, and that ranking in turn leads to a low count, a self-reinforcing loop" inferred. X did not document the purpose of this design directly, but the structure pairing count zeroing with a bool set strongly suggests that direction.
• The is_mutual_follow boost (reply +15) quoted in Gen4 was a condition hardcoded in the scorer. It also matters that the model now receives follow directions as separate input bools and can pick up a similar pattern through training.
  • The shift from hardcoded boosts to learned signals matches the overall direction seen in the move from Gen3 to Gen4: the weight moves from manual gating to model training.

• The enforcement note generator of BDSM (the scorer that detects abusive accounts from account behavior sequences) was partially released.

  • From Gen3 through early Gen4, build_enforcement_note at bdsm/runtime/score_results_sink_focal.py:78 was a stub containing only return None.

• In this commit (Gen4+1), the gate logic and the dominant-head (the bot head with the highest score) selection logic were released, and only the template strings were left as sentinel values verified — bdsm/runtime/score_results_sink_focal.py:78-128.

  • The sentinel value is "<redacted>".

  • The released function body is as follows.

    def build_enforcement_note(head_scores_list, action_hist_list):
        if not head_scores_list: return None
        total = sum(h["cnt"] for h in (action_hist_list or ()))
        if total < 30: return None                       # MIN_ACTIONS gate
        bot_heads = [(h["head_name"], h["score"]) for h in head_scores_list
                     if h["head_name"] != "LegitimateUser" and h["score"] > 0.5]
        if not bot_heads: return None
        bot_heads.sort(key=lambda x: x[1], reverse=True)
        dominant_name, dominant_score = bot_heads[0]
        note = _ENFORCEMENT_TEMPLATES.get(dominant_name, _REDACTED_TEMPLATE)
        note += f" [model: {dominant_name}={dominant_score:.2f}"
        if len(bot_heads) > 1:
            secondary = ", ".join(f"{n}={s:.2f}" for n, s in bot_heads[1:3])
            note += f", also: {secondary}"
        note += "]"
        return note
    

• The names of the seven bot heads were officially released, and each head's template string is replaced with a sentinel.

  • The head names are FollowBot, LikeBot, EngagementAmplifier, ReplySpamBot, TweetSpamBot, RTBot, and MultiActionBot.
  • The replacement value is _REDACTED_TEMPLATE = "<redacted>".

• The note outputs as is which bot family was scored and how highly, but the sentences explaining why it was scored that way (which action patterns caused it) are not public.

  • The final note takes a form such as "<redacted> [model: FollowBot=0.91, also: ReplySpamBot=0.60, LikeBot=0.53]".

• The README states the reason for releasing only half verified — bdsm/README.md.

  "The public package keeps the gates (MIN_ACTIONS, dominant-head pick) and the enforcement_note proto field. The template strings and the per-head key_actions interpolator are the sentinel '<redacted>' — same idea as the 9.99 operating points." — bdsm/README.md

• The principle of releasing the structure, names, and gates but not the tuned real values and prompt strings is the same as the operating point 9.99 sentinel in sink_policy.yaml.

  • The 9.99 sentinel is a value outside the probability range, so a verdict threshold containing this value never triggers.
  • Yesterday's canon update folded in the SafetyLevel 3-tier and 54 rules, and today the note generation logic of the upper-level account scorer that references those rules is about half open.

• The description text of the proto field was also updated verified — bdsm/proto/abuse_inference.proto:98-102.

  "Optional human-readable note. In this release templates and key_actions are the sentinel '<redacted>' (see README); production fills the real appeal paragraph. ActionName enums are unchanged."

• In this commit (Gen4+1), X kept very consistently to the principle of "release the structure, but not the prompts, strings, and tuned real values".

  • The sink_policy 9.99 sentinel, the <redacted> template of the enforcement note, and the partial masking of anti-gaming parts in botmaker-rules were all laid out on the same principle.
  • The direction is to preserve resistance to gaming while widening as much as possible the room to verify the structure, names, gates, and head combinations.

• In the flow where abuse-enforcement-service receives this note and decides on actual actions (label/challenge/suspension), the part corresponding to its input was about half opened today.

• grox (the Grok-based comment pipeline) doubled the follower-count threshold it uses for reply spam verdicts and reply ranking scoring from 15,000 to 30,000 verified — grox/flows/reply_spam/task_filter.py:17, 185.
  • The changed constants are TaskSpamFilter.FOLLOWER_COUNT_THRESHOLD_FOR_SPAM_DETECTION and TaskReplyRankingFilter.FOLLOWER_COUNT_THRESHOLD_FOR_REPLY_RANKING.
  • Replies where both the thread root and the reply target have 30k followers or fewer get a spam verdict, and replies where either exceeds 30k get reply ranking scoring (task_filter.py:77-92, 214-225).
  • So this increase widens the set subject to spam verdicts and narrows the set subject to reply ranking scoring. It is a change that adjusts grok resource allocation between the two tasks.
• phoenix-rankall gained video_14day, a new video SID (semantic ID, representing an item as hierarchical codes) window with a two-week time span verified — phoenix-rankall/src/config/mod.rs:177-180.
  • video_14day was added alongside the existing video_48h, nsfw_video_48h/168h/14day, and video_96h.
  • It means that the SID-based popularity signal of Phoenix retrieval now includes videos from a two-week window.
• The mock fixture of abuse-enforcement-service changed by 1 line verified. It only changed one rule field value and does not affect the actual logic.

• In the "In one line" section, added a summary of the Gen4+1 follow-up release and updated the filter total from 20 to 21.
• Introduced a Gen4+1 row in the "Public source Gen4" generation table (commit c65aa17, +2,271/−32).
• Inserted a quote of the interpretation rule that prevents the raw-count misconception into the "Weights — values released with the Gen4 release" section.
• Added a Brazil2026ElectionFilter entry to the "21 filters" section and stated explicitly where it connects in the pipeline. The heading of this section was changed from the earlier "20 filters".
  • This filter was folded into the services/x.md 21 filters section as one new Gen4+1 entry.
• Added the six Thompson sampling parameters and the sampling rule to the "Cold start" section.
• Added the three bool features and the stale post count zeroing logic to the "Phoenix training and index" section.
• Changed the heading "Grox (LLM comments) — unchanged from Gen3" to "+ 2026-08-14 threshold increase" and added the threshold adjustment.
• Created the section "BDSM (abusive-account detection from account behavior sequences) — enforcement note generation logic partially released in Gen4+1".
• Introduced the subsection "Resolved by the Gen4+1 release (2026-08-16)" in the open question section and put in 5 items.
• Added 6 entries to the sources section (the c65aa17 commit, README #latest-updates, the Brazil filter, the TSE dataset, the @XBR announcement, the BDSM sink).

In assets/x-pipeline.svg, changed the filter label to "17 → 18" and the new label to "4 → 5 (including Brazil2026)".

• The README #latest-updates now has a structure of date-based subsections. It is in reverse chronological order, with August 13 following August 14.
• Each subsection contains three elements.
  1. Summary of the change
  2. Links to related files
  3. Links to supporting documents and announcements
• If a param changes in the next commit, that change itself becomes an indicator that reveals an A/B result after the fact.
• If a filter is newly attached or removed, it becomes an indicator for observing regulatory responses in real time.

Deep-dive items left unprocessed today:

• Keep watching when the Thompson sampling A/B experiment is actually activated and what the result metrics look like.
• Separately track in the commit history how the Brazil election filter's 665-account list is updated.
• Check whether bot heads other than the seven, such as FollowBot, are later added to BDSM _ENFORCEMENT_TEMPLATES.
• Check when the rest of the Phoenix bool feature slots are filled.

2DrEM, a noise-robust ensemble ranking paper, is confirmed as Kuaishou's — Kuaishou

• The HTML version confirms that this paper is a collaboration between Kuaishou Technology Beijing and Shenzhen University.
  • Kaiqiao Zhan, Xiaoxiao Xu, Tiantian He, Jun Zhang, and Chunjie Chen use a Kuaishou email domain.
  • Weike Pan, Canwei Huang, and Ziran Deng are affiliated with Shenzhen University.
• Kuaishou researchers already published two papers this month (HD-Rec 2026-08-11 radar.md, PushDualGen 2026-08-12). This paper is the third.
• Of these, PushDualGen is the one that states a production deployment, while HD-Rec reports only public benchmark results verified — arXiv 2608.06997.
  • The HD-Rec abstract's experiments cover only "three public cross-domain recommendation benchmarks", and it has no deployment or A/B statement.
• The three papers share one trait: they all belong to different algorithm families. They address SID cross-domain, push notifications combining SID and CoT, and noise robustness of ensemble ranking, respectively.
  • In effect, this confirms that the company has the resources for several teams and several research lines to publish production SOTA papers at the same time.

• The paper addresses a problem specific to the ensemble ranking stage. In the multi-stage pipeline of industrial video recommendation, it assumes that a downstream ensemble ranker uses the predictions of the upstream multi-task ranker both as input features and as the source of proxy preference labels.
  • Here the predicted probabilities that multiple tasks output, such as pctr, pvtr, pcmtr, and pftr, are collectively called pxtr.
• Because pxtr is the output of an upstream model, it inevitably contains noise. This noise acts in two ways.
  • It flips the proxy preference labels (supervision-side noise).
  • It flows into the feature-side input and shakes the ranking score (feature-side noise).
• Existing ensemble ranking methods treated pxtr as a reliable signal and ignored the noise. DrEM corrects this noise on both sides at the same time.
  • On the label side, it estimates the probability that the true preference of each pair was flipped (pairwise flip probability, shortened below to flip probability) and introduces a risk-denoising robust loss that corrects the empirical risk with that probability. It proves theoretically that this loss is superior to the basic pairwise loss even when the flip probability estimate has errors (Theorem 1).
  • On the input side, it samples perturbations from the prediction noise distribution of pxtr and trains the output to stay stable under perturbation with a preference-preserving ranking consistency regularizer.
• The key point is that both corrections are derived from a shared noise model. Because the correction targets of supervision and features are aligned to the same noise source, parameters can be shared to perform synchronized corrections.

• The experiment used two groups, with DrEM attached to the production backbones EMER and EASQ respectively; each group was randomly assigned 5.1% of the main traffic at the user level for 7 days verified — HTML version section 5.4 (RQ3).

  "To answer RQ3 and validate real-world performance, we conduct online A/B tests on the industrial short-video platform. We set up two parallel experiment groups to verify the generalization of our DrEM on both representative backbones, applying our DrEM on top of production-grade EMER and EASQ backbone. The experiment runs for 7 days with 5.1% of the main traffic randomly split at the user level via hash-based assignment to each group." — arXiv 2608.12778, section 5.4

The main A/B results are in the following table (section 5.4, Table 2, all metrics p < 0.005) verified.

• Summarizing what the authors observe in the same section, the gain on sparse behaviors is larger than on dense behaviors. The improvements of +1.20% for follow and +1.39% for comment are far larger than +0.69% for view and +0.40% for long view.
• This is consistent with the theory. The pxtr of sparse tasks (pcmtr, pftr) has high prediction variance, so the flip probability is also high, and the robust correction has correspondingly more room to improve.
• The heatmap stratified by flip probability in section 5.5 (RQ4) shows the same conclusion. In the low flip-probability range, the improvement for dense tasks (pctr, pvtr) is zero or slightly negative, while in the high range the gain rises sharply to positive values.
• The authors also explain why EASQ's gain is smaller than EMER's. EASQ already partially mitigates supervision-side noise with its own questionnaire-signal alignment, so there is less room for DrEM's supervision correction to improve.
  • In other words, this can be understood hierarchically: supervision correction yields much larger gains, and on a backbone that already handles that part, only the feature-side correction remains.

However, the literature does not confirm whether the specific problem DrEM addresses, pxtr noise at the ensemble ranking stage, exists as a separate component at X, Meta, and Netflix, or is mitigated in another way. If a similar statement appears from a second company, it becomes a promotion candidate for a new section in topics/multi-task-ranking.md.

What to check next