Ziwei Wu, Lecheng Zheng, Yuancheng Yu, Ruizhong Qiu, John Birge, Jingrui He
{"title":"针对不平衡群体的公平异常检测","authors":"Ziwei Wu, Lecheng Zheng, Yuancheng Yu, Ruizhong Qiu, John Birge, Jingrui He","doi":"arxiv-2409.10951","DOIUrl":null,"url":null,"abstract":"Anomaly detection (AD) has been widely studied for decades in many real-world\napplications, including fraud detection in finance, and intrusion detection for\ncybersecurity, etc. Due to the imbalanced nature between protected and\nunprotected groups and the imbalanced distributions of normal examples and\nanomalies, the learning objectives of most existing anomaly detection methods\ntend to solely concentrate on the dominating unprotected group. Thus, it has\nbeen recognized by many researchers about the significance of ensuring model\nfairness in anomaly detection. However, the existing fair anomaly detection\nmethods tend to erroneously label most normal examples from the protected group\nas anomalies in the imbalanced scenario where the unprotected group is more\nabundant than the protected group. This phenomenon is caused by the improper\ndesign of learning objectives, which statistically focus on learning the\nfrequent patterns (i.e., the unprotected group) while overlooking the\nunder-represented patterns (i.e., the protected group). To address these\nissues, we propose FairAD, a fairness-aware anomaly detection method targeting\nthe imbalanced scenario. It consists of a fairness-aware contrastive learning\nmodule and a rebalancing autoencoder module to ensure fairness and handle the\nimbalanced data issue, respectively. Moreover, we provide the theoretical\nanalysis that shows our proposed contrastive learning regularization guarantees\ngroup fairness. Empirical studies demonstrate the effectiveness and efficiency\nof FairAD across multiple real-world datasets.","PeriodicalId":501301,"journal":{"name":"arXiv - CS - Machine Learning","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-09-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Fair Anomaly Detection For Imbalanced Groups\",\"authors\":\"Ziwei Wu, Lecheng Zheng, Yuancheng Yu, Ruizhong Qiu, John Birge, Jingrui He\",\"doi\":\"arxiv-2409.10951\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Anomaly detection (AD) has been widely studied for decades in many real-world\\napplications, including fraud detection in finance, and intrusion detection for\\ncybersecurity, etc. Due to the imbalanced nature between protected and\\nunprotected groups and the imbalanced distributions of normal examples and\\nanomalies, the learning objectives of most existing anomaly detection methods\\ntend to solely concentrate on the dominating unprotected group. Thus, it has\\nbeen recognized by many researchers about the significance of ensuring model\\nfairness in anomaly detection. However, the existing fair anomaly detection\\nmethods tend to erroneously label most normal examples from the protected group\\nas anomalies in the imbalanced scenario where the unprotected group is more\\nabundant than the protected group. This phenomenon is caused by the improper\\ndesign of learning objectives, which statistically focus on learning the\\nfrequent patterns (i.e., the unprotected group) while overlooking the\\nunder-represented patterns (i.e., the protected group). To address these\\nissues, we propose FairAD, a fairness-aware anomaly detection method targeting\\nthe imbalanced scenario. It consists of a fairness-aware contrastive learning\\nmodule and a rebalancing autoencoder module to ensure fairness and handle the\\nimbalanced data issue, respectively. Moreover, we provide the theoretical\\nanalysis that shows our proposed contrastive learning regularization guarantees\\ngroup fairness. Empirical studies demonstrate the effectiveness and efficiency\\nof FairAD across multiple real-world datasets.\",\"PeriodicalId\":501301,\"journal\":{\"name\":\"arXiv - CS - Machine Learning\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-09-17\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"arXiv - CS - Machine Learning\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/arxiv-2409.10951\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"\",\"JCRName\":\"\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"arXiv - CS - Machine Learning","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/arxiv-2409.10951","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"","JCRName":"","Score":null,"Total":0}
Anomaly detection (AD) has been widely studied for decades in many real-world
applications, including fraud detection in finance, and intrusion detection for
cybersecurity, etc. Due to the imbalanced nature between protected and
unprotected groups and the imbalanced distributions of normal examples and
anomalies, the learning objectives of most existing anomaly detection methods
tend to solely concentrate on the dominating unprotected group. Thus, it has
been recognized by many researchers about the significance of ensuring model
fairness in anomaly detection. However, the existing fair anomaly detection
methods tend to erroneously label most normal examples from the protected group
as anomalies in the imbalanced scenario where the unprotected group is more
abundant than the protected group. This phenomenon is caused by the improper
design of learning objectives, which statistically focus on learning the
frequent patterns (i.e., the unprotected group) while overlooking the
under-represented patterns (i.e., the protected group). To address these
issues, we propose FairAD, a fairness-aware anomaly detection method targeting
the imbalanced scenario. It consists of a fairness-aware contrastive learning
module and a rebalancing autoencoder module to ensure fairness and handle the
imbalanced data issue, respectively. Moreover, we provide the theoretical
analysis that shows our proposed contrastive learning regularization guarantees
group fairness. Empirical studies demonstrate the effectiveness and efficiency
of FairAD across multiple real-world datasets.