Do conformal guarantees hold when bots change tactics?

Links: CS224W L14 (advanced topics: conformal prediction, the L14 proof workshop on split conformal), L21 (trust), notebooks/project/retweet_thread.py §L14 (the same detector with 95% prediction sets). Atlas: uncertainty quantification; trustworthy ML on social data.

Question

Split conformal prediction promises that a prediction set contains the true label with probability at least 1 − α, for any model, as long as calibration and test data are exchangeable. A moderation team that publishes "95% coverage" makes that promise about next month's accounts, but next month's coordinated campaign may use different tactics. Does the guarantee survive, for whom does it break, and how cheaply can it be repaired?

Hypotheses (written before running)

1. In distribution. On new worlds of the training type, both marginal and class-conditional (Mondrian) conformal cover at least 0.93 of accounts, and Mondrian covers at least 0.90 of bots.

2. Marginal coverage hides the failure. Under every shift, marginal coverage stays at least 0.93, because humans are 97% of accounts.

3. A tactic change breaks the bot guarantee. When rings switch from reciprocal to co-amplification, Mondrian bot coverage falls below 0.50.

4. Cheap repair. Recalibrating Mondrian on 20 labelled bots (and 200 humans) of the new type restores bot coverage to at least 0.85.

Setup

• Detector: the label-free model (gl.exp.label_free_features, logistic regression), trained on one reciprocal world.
• Calibration: three other reciprocal worlds (90 bots). Score: 1 − p̂(true label). α = 0.05.

• Test worlds (new seeds) of six types: same (reciprocal), co-amplification, naive, heavy camouflage (30 camouflage retweets per bot), large rings (3 × 20), and early (reciprocal observed for 10 of 30 days).

• Reported: overall, bot and human coverage; share of bots flagged as {bot} alone; share of uninformative sets {human, bot}.

Run: uv run marimo edit experiments/conformal-shift/conformal_shift.py and press the run button.

Results (summary; full table in results.md)

• H1–H4 all supported. The guarantee holds in distribution; under the two damaging shifts (a tactic change to co-amplification, and catching a campaign after 10 days) marginal coverage still reads 0.94–0.96 while bot coverage is 0.13–0.23. Mondrian makes no difference without exchangeability.

• Not every shift hurts: heavier camouflage, bigger rings and naive bots are all still covered, because the detector still separates them. Conformal fails exactly when the model fails, and says nothing about it.

• Recalibrating on 20 labelled bots of the new type restores bot coverage (0.90–1.00) by making sets honest: 55% (co-amplify) and 75% (early) of accounts get {human, bot}.

• Side effect: recalibrated human coverage falls to 0.84–0.92 in several types. A reading, not yet tested: one sibling world is one draw of world-level randomness, so its accounts are not exchangeable with another world's.

Real-data extension: Twitch countries (added after the first run; hypotheses written before running it)

The synthetic shifts were designed. A real platform has shifts nobody designed: Twitch's six country graphs (gl.datasets.twitch) share one label (does the streamer broadcast mature content?) and the same account fields, but the communities, the share of mature streamers (0.25 in RU to 0.61 in DE) and the follow patterns differ. Detector: logistic regression on account fields that mean the same everywhere (log views, log account age, partner, log degree, clustering) and their neighbour means, trained on 40% of England (ENGB), calibrated on another 30%, tested on the last 30% ("same") and on DE, ES, FR, PTBR and RU. α = 0.05; "positive" = mature.

8. In distribution. On held-out ENGB accounts, marginal and Mondrian coverage are both at least 0.93.

9. A real shift breaks class-conditional coverage. In at least 2 of the 5 other countries, Mondrian coverage of one class falls below 0.90.

10. Marginal coverage hides it. In those countries marginal coverage stays at least 0.90.

11. Cheap repair. Recalibrating Mondrian on 200 labelled accounts of the target country gives both classes at least 0.90 coverage in every country.

Run: the second button in the notebook (under a minute; downloads 3 MB). Results go to twitch_results.json and a separate section of results.md.

Twitch results (summary; full table in results.md)

• H8 supported. Held-out England is covered at 0.947 (marginal and Mondrian).

• H9 supported. Under the country shift the non-mature class loses coverage under Mondrian: 0.87 (DE), 0.89 (ES), 0.79 (FR). Mature streamers stay over-covered (0.97–0.99); why the shift hits one class and not the other is not tested here.

• H10 refuted. Unlike the synthetic shift, marginal coverage does not hide this one: France falls to 0.863 and Spain to 0.918. A real shift moved the whole score distribution, not one small class, so the aggregate shows it. The synthetic finding (marginal coverage 0.94–0.96 while bots drop to 0.13–0.23) depended on bots being 3% of accounts.

• H11 supported. Recalibrating on 200 labelled accounts of the country restores both classes to 0.94–0.96 everywhere.

• Caveat. The detector is weak (AUC 0.55–0.66 from account fields and neighbour means), so 72–88% of sets are {mature, non-mature}: the guarantee is honest because it is mostly uninformative. The synthetic study's lesson holds: conformal repairs honesty, not accuracy.

Next steps

1. Recalibrate from several small samples of different worlds (test the side-effect reading).

2. Weighted conformal under covariate shift (Tibshirani et al. 2019) with density-ratio weights from unlabelled new-world accounts, which needs no new labels.

3. Shift detection: a conformal martingale over the stream of new accounts' p-values (Vovk), to raise an alarm when exchangeability breaks.

In the repository: experiments/conformal-shift/ · README.md config.yaml conformal_shift.py results.json results.md twitch_results.json