Error-Driven Triplet-Based Online Fine-Tuning for Cross-Background Image Classification

Abstract

Image classification is a fundamental task in computer vision with numerous real-world applications. However, classification models trained on one set of images may not perform well when tested on another set, especially when the two sets differ in terms of the background environment. To address this challenge, we propose an error-driven triplet-based online fine-tuning approach that leverages misclassified samples to refine the classification model at intervals when enough misclassified samples are collected. Our approach builds on the triplet network architecture, which learns to represent images in a low-dimensional space where images with the same label are clustered together. We use a pre-trained classification model that was trained on a collection of 180 types of images from one background scene. However, when we apply the model to a new background scene with additional types of images, its performance is compromised due to the domain shift. Our proposed approach leverages the misclassified samples by contrast them with positive and negative samples as triplet data to fine-tune the model in new background scene. We use a loss function that combines the triplet loss and the classification loss to update the model weights. We evaluate our approach on two challenging image classification datasets with different background environments. The experimental results demonstrate that our approach achieves significant improvements in accuracy compared to the baseline classification model without fine-tuning. Overall, our error-driven triplet-based online fine-tuning approach shows promising results for adapting classification models to changing background with additionally new classification types, where the performance of pre-trained models is limited.