Training-Free Adaptive Sampling Scheduler for Discrete-Time Consistency Distillation
Qi Wang · Jinjia Zhou
Graduate School of Science and Engineering, Hosei University, Tokyo, Japan
In one sentence. Few-step consistency models fail not because the student is
weak but because the schedule is rigid: we score every timestep by an SNR-derived
Importance, spend the same number of steps where the trajectory actually moves, and add a
matching sampler and colour-balancing pass — no retraining, drop-in for LCM, PCM,
TCD and TDD. At 2 steps FID falls by up to 86%.
1. Why few-step sampling still breaks
Scheduler rigidity. Every discrete-time distillation method ships a timestep schedule
welded to its own training framework and step budget. Deterministic-target methods (LCM, PCM) and
stochastic-target methods (TCD, TDD) alike degrade as soon as the scheduler or the step count changes at
inference time — the schedule was never chosen on the merits of the timesteps themselves.
CFG compounding. Classifier-free guidance multiplies with the inner distillation scale,
pushing pixels into saturation and drifting the colour balance. The fewer steps you take, the less
opportunity the model has to correct it.
2. Timesteps are not equally informative
Forward diffusion (DDPM). Up to the content is still fully discernible;
past it is essentially unrecognisable. Almost all structural change is packed into the
middle of the trajectory.
We quantify that observation with an Importance measure : the rate of change of
log-SNR, inverted and normalised to . A large marks a timestep where the SNR moves fast —
where the ODE trajectory turns. The complementary skip rate says how much of that
step can safely be skipped.
Importance & skip rate (Eq. 8) | ,
Fig. 3. (a) peaks near ; the dashed line is the threshold
, whose two crossings bracket the band worth sampling densely. (b) The skip rate
mirrors it.
3. Adaptive schedule
Let be the original uniformly spaced timesteps and the timesteps that attain the highest
Importance within predefined intervals. We keep the uniform-grid steps where change is slow and substitute
high-Importance candidates where it is fast. The step count never changes — only its
spacing. That is what makes the method free: no retraining, no extra parameters, no change to the sampler
loop's cost.
Adaptive schedule (Eq. 9),
4. The -I sampler
CTM re-injects noise with a fixed at every step. We instead scale the re-injection by the
local Importance, so exploration widens exactly where the trajectory turns fastest and narrows
where it is nearly straight.
Fig. 4. (a) Dense sampling in high-Importance regions, sparse elsewhere, at a preserved
budget. (b) Importance-modulated forward and backward transitions lift sampling quality precisely where
the SNR changes most rapidly.
5. Smoothing & colour balancing
To undo CFG compounding we squash outliers with , then re-centre the channel-wise and global means
so values return to the function's linear region instead of pinning at . Unlike prior mapping methods
this needs no prior deformation, and more information survives under strong guidance.
Soft clamp & re-centring (Eq. 12–13),
6. Quantitative results
MS-COCO 2014 validation set, Karpathy 30k captions, CFG . Columns are 2 / 4 / 8
sampling steps. MPI = mean improvement over the paired baseline.
Method
FID ↓
CLIP Score ↑
Inception Score ↑
2
4
8
2
4
8
2
4
8
SDXL — 1024×1024
PCM
372.82
112.65
31.73
18.44
24.18
30.44
1.71
11.67
25.78
PCM + Ours
65.83
29.40
23.21
30.22
30.40
31.52
16.54
24.59
31.80
TCD
363.50
103.66
53.72
18.73
26.05
30.44
1.82
12.47
17.83
TCD + Ours
62.89
28.51
27.44
28.88
31.71
32.06
16.33
28.02
32.17
TDD
58.45
29.80
27.72
29.27
31.12
31.47
17.18
27.02
29.72
TDD + Ours
55.71
27.88
26.60
29.49
31.47
31.74
17.62
29.36
32.88
△ MPI
203.45
53.44
11.97
7.38
4.08
0.99
9.93
10.27
7.84
SD v1.5 — 512×512
LCM
86.33
88.20
109.84
28.02
26.48
25.19
14.28
11.73
8.71
LCM + Ours
58.01
30.04
47.67
30.18
30.71
30.79
17.18
28.81
19.84
PCM
424.04
89.99
38.82
18.99
26.51
30.02
1.76
12.49
21.07
PCM + Ours
60.66
23.11
22.47
29.93
30.37
31.03
16.86
28.93
31.65
△ MPI
195.85
62.52
39.26
6.55
4.05
3.31
9.00
16.76
10.86
Scroll the table sideways to see all nine columns.
Reading the table
All five baselines improve on every metric at every budget, and the gains land where
few-step sampling is weakest — at 2 steps FID falls 82–86% for PCM and TCD.
The most telling case is LCM: on its own it gets worse with more steps
(86.33 → 88.20 → 109.84 FID), the signature of CFG overexposure compounding. Our scheduler
reverses that trend. The pattern holds across both resolutions and all four distilled students, not just on
the metric we optimised for.
7. Qualitative comparison
Fig. 5. At 2 and 4 steps the baselines lose global structure and wash the colour out;
ours keeps the layout coherent and the detail sharper on both backbones. By 8 steps the gap narrows, as
expected once the student has budget enough to correct itself.
8. Ablations
no
CTM's
ours (-I)
Fig. 7 — the sampler. Scaling noise re-injection by the local Importance, instead
of CTM's fixed , gives more stable shapes and fewer structural errors at the same step budget.
Threshold sweep: best .Fig. 8 — smoothing & colour balance. Top row: the clipping used by prior work.
Bottom row: ours — natural colour, fewer saturated regions and more surviving detail at high
CFG.
9. What to take away
Diagnosis. The schedule, not the student, is the bottleneck under cross-method or
variable-step transfer.
Mechanism. An SNR-derived Importance says where computation is worth spending;
adaptive selection, the -I sampler and a colour pass do the rest.
Cost. No retraining, no extra parameters, negligible runtime overhead — a
drop-in scheduler swap on top of an existing distilled student.
Where it helps most. The 2–4 step regime, consistently across SDXL and
SD v1.5.