Conventional DDCoarse
Same-class collapse
Samples inherit nearly identical optimization signals, limiting local diversity.
Accepted at ECCV 2026
Fine-grained recognition · Dataset distillation
Distill the details
that make a difference.
FD2 preserves subtle, localized cues while building a compact synthetic dataset—turning fine-grained variation from a distillation obstacle into a learning signal.
01 / Motivation
Fine-grained data asks a harder question.
Conventional decoupled distillation is guided by coarse class labels. On fine-grained datasets, that can blur the tiny distinctions between classes—and make distilled images within a class too alike.
A bird is not just a bird. The answer may live in the wing, the beak, or a patch of color.
FD2Fine-grained
Discriminative diversity
Class prototypes guide identity while attention constraints preserve distinct cues.
02 / Framework
One framework.
Three deliberate stages.
FD2 slots into the established decoupled dataset-distillation pipeline without rewriting the recipe.
01
Model pretraining
Counterfactual attention learning discovers discriminative regions and aggregates their representations into class prototypes.
Build prototypes02
Sample distillation
A characteristic constraint pulls each sample toward its own prototype and away from others. A similarity constraint diversifies same-class attention.
Distill details03
Soft-label generation
The pretrained fine-grained model converts distilled samples into informative soft labels for efficient downstream training.
Transfer knowledgeFramework overview Figure 2
03 / Design
Identity,
without imitation.
Two complementary objectives keep the synthetic set faithful to its class and meaningfully varied within it.
Fine-grained characteristic
Be closer to your class.
Align each representation with its class prototype while repelling prototypes from other classes.
Attention similarity
Look somewhere different.
Reduce overlap between current and previous attention maps from the same class.
04 / Takeaway
The result
A compact dataset that remembers where to look.
Across fine-grained and general datasets, FD2 integrates seamlessly with decoupled dataset distillation and improves performance in most settings—showing strong transferability.
05 / Citation
@inproceedings{ma2026fd2,
title = {FD$^2$: A Dedicated Framework for
Fine-Grained Dataset Distillation},
author = {Ma, Hongxu and Li, Guang and Wang, Shijie
and Zhou, Dongzhan and Sun, Baoli and
Ogawa, Takahiro and Haseyama, Miki and
Wang, Zhihui},
booktitle = {European Conference on Computer Vision (ECCV)},
year = {2026}
}