In transfer learning, one fundamental problem is transferability estimation, where a metric measures transfer performance without training. Existing metrics face two issues: 1) requiring target domain labels, and 2) only focusing on task speciality but ignoring equally important domain commonality. To overcome these limitations, we propose TranSAC, a Transferability metric based on task Speciality And domain Commonality, capturing the separation between classes and the similarity between domains. Its main advantages are: 1) unsupervised, 2) fine-tuning free, and 3) applicable to source-dependent and source-free transfer scenarios. To achieve this, we investigate the upper and lower bounds of transfer performance based on fixed representations extracted from the pre-trained model. Theoretical results reveal that unsupervised transfer performance is characterized by entropy-based quantities, naturally reflecting task specificity and domain commonality. These insights motivate the design of TranSAC, which integrates both factors to enhance transferability. Extensive experiments are performed across 12 target datasets with 36 pre-trained models, including supervised CNNs, self-supervised CNNs, and ViTs. Results demonstrate the importance of domain commonality and task speciality, allowing TranSAC as superior to state-of-the-art metrics for pre-trained model ranking, target domain ranking, and source domain ranking.
扫码关注我们
求助内容:
应助结果提醒方式:
