Pub Date : 2023-04-30DOI: 10.48550/arXiv.2305.00593
Maohao Shen, S. Ghosh, P. Sattigeri, Subhro Das, Yuheng Bu, G. Wornell
Due to privacy or commercial constraints, large pre-trained language models (PLMs) are often offered as black-box APIs. Fine-tuning such models to downstream tasks is challenging because one can neither access the model’s internal representations nor propagate gradients through it. This paper addresses these challenges by developing techniques for adapting PLMs with only API access. Building on recent work on soft prompt tuning, we develop methods to tune the soft prompts without requiring gradient computation. Further, we develop extensions that in addition to not requiring gradients also do not need to access any internal representation of the PLM beyond the input embeddings. Moreover, instead of learning a single prompt, our methods learn a distribution over prompts allowing us to quantify predictive uncertainty. Ours is the first work to consider uncertainty in prompts when only having API access to the PLM. Finally, through extensive experiments, we carefully vet the proposed methods and find them competitive with (and sometimes even improving on) gradient-based approaches with full access to the PLM.
{"title":"Reliable Gradient-free and Likelihood-free Prompt Tuning","authors":"Maohao Shen, S. Ghosh, P. Sattigeri, Subhro Das, Yuheng Bu, G. Wornell","doi":"10.48550/arXiv.2305.00593","DOIUrl":"https://doi.org/10.48550/arXiv.2305.00593","url":null,"abstract":"Due to privacy or commercial constraints, large pre-trained language models (PLMs) are often offered as black-box APIs. Fine-tuning such models to downstream tasks is challenging because one can neither access the model’s internal representations nor propagate gradients through it. This paper addresses these challenges by developing techniques for adapting PLMs with only API access. Building on recent work on soft prompt tuning, we develop methods to tune the soft prompts without requiring gradient computation. Further, we develop extensions that in addition to not requiring gradients also do not need to access any internal representation of the PLM beyond the input embeddings. Moreover, instead of learning a single prompt, our methods learn a distribution over prompts allowing us to quantify predictive uncertainty. Ours is the first work to consider uncertainty in prompts when only having API access to the PLM. Finally, through extensive experiments, we carefully vet the proposed methods and find them competitive with (and sometimes even improving on) gradient-based approaches with full access to the PLM.","PeriodicalId":73025,"journal":{"name":"Findings (Sydney (N.S.W.)","volume":"1 1","pages":"2371-2384"},"PeriodicalIF":0.0,"publicationDate":"2023-04-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42953055","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
We investigate whether office workers will continue to work from home after COVID-19 pandemic restrictions. Office workers at Swedish public agencies were surveyed concerning the pre-pandemic experience of work from home, current practices, and willingness to continue work from home post-pandemic. Results indicate that willingness increased greatly during the pandemic. Most surveyed workers wanted to continue to work from home at least three days per week, and the greater the experience of remote work, the greater the inclination to increase the number of days of working from home. Job type, housing, public transport commuting, and living region affected the willingness.
{"title":"Sustained Work from Home Post-Pandemic? A Swedish Case","authors":"B. Vilhelmson, Eva Thulin, Louise Brundin","doi":"10.32866/001c.74470","DOIUrl":"https://doi.org/10.32866/001c.74470","url":null,"abstract":"We investigate whether office workers will continue to work from home after COVID-19 pandemic restrictions. Office workers at Swedish public agencies were surveyed concerning the pre-pandemic experience of work from home, current practices, and willingness to continue work from home post-pandemic. Results indicate that willingness increased greatly during the pandemic. Most surveyed workers wanted to continue to work from home at least three days per week, and the greater the experience of remote work, the greater the inclination to increase the number of days of working from home. Job type, housing, public transport commuting, and living region affected the willingness.","PeriodicalId":73025,"journal":{"name":"Findings (Sydney (N.S.W.)","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-04-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44419661","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-20DOI: 10.48550/arXiv.2304.10346
Julia Rozanova, Marco Valentino, Lucas C. Cordeiro, André Freitas
Probing strategies have been shown to detectthe presence of various linguistic features inlarge language models; in particular, seman-tic features intermediate to the “natural logic”fragment of the Natural Language Inferencetask (NLI). In the case of natural logic, the rela-tion between the intermediate features and theentailment label is explicitly known: as such,this provides a ripe setting for interventionalstudies on the NLI models’ representations, al-lowing for stronger causal conjectures and adeeper critical analysis of interventional prob-ing methods. In this work, we carry out newand existing representation-level interventionsto investigate the effect of these semantic fea-tures on NLI classification: we perform am-nesic probing (which removes features as di-rected by learned linear probes) and introducethe mnestic probing variation (which forgetsall dimensions except the probe-selected ones).Furthermore, we delve into the limitations ofthese methods and outline some pitfalls havebeen obscuring the effectivity of interventionalprobing studies.
{"title":"Interventional Probing in High Dimensions: An NLI Case Study","authors":"Julia Rozanova, Marco Valentino, Lucas C. Cordeiro, André Freitas","doi":"10.48550/arXiv.2304.10346","DOIUrl":"https://doi.org/10.48550/arXiv.2304.10346","url":null,"abstract":"Probing strategies have been shown to detectthe presence of various linguistic features inlarge language models; in particular, seman-tic features intermediate to the “natural logic”fragment of the Natural Language Inferencetask (NLI). In the case of natural logic, the rela-tion between the intermediate features and theentailment label is explicitly known: as such,this provides a ripe setting for interventionalstudies on the NLI models’ representations, al-lowing for stronger causal conjectures and adeeper critical analysis of interventional prob-ing methods. In this work, we carry out newand existing representation-level interventionsto investigate the effect of these semantic fea-tures on NLI classification: we perform am-nesic probing (which removes features as di-rected by learned linear probes) and introducethe mnestic probing variation (which forgetsall dimensions except the probe-selected ones).Furthermore, we delve into the limitations ofthese methods and outline some pitfalls havebeen obscuring the effectivity of interventionalprobing studies.","PeriodicalId":73025,"journal":{"name":"Findings (Sydney (N.S.W.)","volume":"1 1","pages":"2444-2455"},"PeriodicalIF":0.0,"publicationDate":"2023-04-20","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48665579","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This study analyzed data from a 2022 shared mobility survey to investigate the impact of gender on the potential improvements to on-demand transportation services, particularly shared-ride services such as ridesharing through transportation network companies (TNCs) and microtransit. Results revealed that male respondents were more inclined to opt for financial incentives such as pretax benefits, direct subsidies, and subsidizing certain trips. Employer-related programs such as parking cash-out programs and flexible working hours were also found to be more appealing to male respondents. In contrast, female respondents placed a greater emphasis on safety-related measures, indicating that safety concerns may be too significant for them to overlook, even when presented with financial incentives.
{"title":"Gender Gaps in Improvements to Shared-Ride Services: Insights from a Shared Mobility Survey","authors":"I. Sener, Austin Sibu, Todd Hansen","doi":"10.32866/001c.74249","DOIUrl":"https://doi.org/10.32866/001c.74249","url":null,"abstract":"This study analyzed data from a 2022 shared mobility survey to investigate the impact of gender on the potential improvements to on-demand transportation services, particularly shared-ride services such as ridesharing through transportation network companies (TNCs) and microtransit. Results revealed that male respondents were more inclined to opt for financial incentives such as pretax benefits, direct subsidies, and subsidizing certain trips. Employer-related programs such as parking cash-out programs and flexible working hours were also found to be more appealing to male respondents. In contrast, female respondents placed a greater emphasis on safety-related measures, indicating that safety concerns may be too significant for them to overlook, even when presented with financial incentives.","PeriodicalId":73025,"journal":{"name":"Findings (Sydney (N.S.W.)","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-04-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49602316","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2023-04-10DOI: 10.48550/arXiv.2304.04836
Weiwei Sun, Pengjie Ren, Z. Ren
Knowledge selection is the key in knowledge-grounded dialogues (KGD), which aims to select an appropriate knowledge snippet to be used in the utterance based on dialogue history. Previous studies mainly employ the classification approach to classify each candidate snippet as “relevant” or “irrelevant” independently. However, such approaches neglect the interactions between snippets, leading to difficulties in inferring the meaning of snippets. Moreover, they lack modeling of the discourse structure of dialogue-knowledge interactions. We propose a simple yet effective generative approach for knowledge selection, called GenKS. GenKS learns to select snippets by generating their identifiers with a sequence-to-sequence model. GenKS therefore captures intra-knowledge interaction inherently through attention mechanisms. Meanwhile, we devise a hyperlink mechanism to model the dialogue-knowledge interactions explicitly. We conduct experiments on three benchmark datasets, and verify GenKS achieves the best results on both knowledge selection and response generation.
{"title":"Generative Knowledge Selection for Knowledge-Grounded Dialogues","authors":"Weiwei Sun, Pengjie Ren, Z. Ren","doi":"10.48550/arXiv.2304.04836","DOIUrl":"https://doi.org/10.48550/arXiv.2304.04836","url":null,"abstract":"Knowledge selection is the key in knowledge-grounded dialogues (KGD), which aims to select an appropriate knowledge snippet to be used in the utterance based on dialogue history. Previous studies mainly employ the classification approach to classify each candidate snippet as “relevant” or “irrelevant” independently. However, such approaches neglect the interactions between snippets, leading to difficulties in inferring the meaning of snippets. Moreover, they lack modeling of the discourse structure of dialogue-knowledge interactions. We propose a simple yet effective generative approach for knowledge selection, called GenKS. GenKS learns to select snippets by generating their identifiers with a sequence-to-sequence model. GenKS therefore captures intra-knowledge interaction inherently through attention mechanisms. Meanwhile, we devise a hyperlink mechanism to model the dialogue-knowledge interactions explicitly. We conduct experiments on three benchmark datasets, and verify GenKS achieves the best results on both knowledge selection and response generation.","PeriodicalId":73025,"journal":{"name":"Findings (Sydney (N.S.W.)","volume":"1 1","pages":"2032-2043"},"PeriodicalIF":0.0,"publicationDate":"2023-04-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44014821","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The purpose of the study is to explore the relative influence of different measures of accessibility, public transport quality and local density on trip chain mode choice. This study uses data from the National Travel Survey in Norway and develops a logistic regression model on the choice between public transport and car on daily trip chains. Results show that the most important factors in explaining the use of public transport are 1) the travel time competitiveness of public transport versus the car, 2) parking restrictions, 3) centrality of trip chain destinations, and 4) waiting time between departures.
{"title":"Public Transport Use on Trip Chains: Exploring Various Mode Choice Determinants","authors":"E. B. Lunke, Ø. Engebretsen","doi":"10.32866/001c.74112","DOIUrl":"https://doi.org/10.32866/001c.74112","url":null,"abstract":"The purpose of the study is to explore the relative influence of different measures of accessibility, public transport quality and local density on trip chain mode choice. This study uses data from the National Travel Survey in Norway and develops a logistic regression model on the choice between public transport and car on daily trip chains. Results show that the most important factors in explaining the use of public transport are 1) the travel time competitiveness of public transport versus the car, 2) parking restrictions, 3) centrality of trip chain destinations, and 4) waiting time between departures.","PeriodicalId":73025,"journal":{"name":"Findings (Sydney (N.S.W.)","volume":"134 17","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-04-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41247634","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This study investigates the association between bikeshare program usage and neighborhoods with different socio-demographic information. This study investigated 2023 Citi Bike bikeshare data in New York City. For the analysis, Generalized Linear Regression (GLR) in ArcGIS Pro was used. The results revealed that Citi Bike bikeshare ridership increases in neighborhoods with higher income, more workers, more males, higher density, and lower African American and Hispanic population in New York City.
{"title":"Equity Analysis of Bikeshare Access: A Case Study of New York City","authors":"R. Javid, E. Sadeghvaziri","doi":"10.32866/001c.73906","DOIUrl":"https://doi.org/10.32866/001c.73906","url":null,"abstract":"This study investigates the association between bikeshare program usage and neighborhoods with different socio-demographic information. This study investigated 2023 Citi Bike bikeshare data in New York City. For the analysis, Generalized Linear Regression (GLR) in ArcGIS Pro was used. The results revealed that Citi Bike bikeshare ridership increases in neighborhoods with higher income, more workers, more males, higher density, and lower African American and Hispanic population in New York City.","PeriodicalId":73025,"journal":{"name":"Findings (Sydney (N.S.W.)","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-04-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46506996","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper examines changes in car-ownership levels before and after the COVID-19 pandemic in the US. In contrast to the two years before the pandemic, the propensity of households to be carless decreased for all households considered, as well as for low- and middle-income, and minority households. There is also evidence of an increase in the average number of vehicles for low-income households. The results highlight the additional financial burden faced by households during the pandemic as a result of higher levels of car-ownership, and that the recovery of public transportation ridership may be negatively impacted with the rise in car-ownership among transit-using groups.
{"title":"Exploring Car-Ownership and Declining Carlessness in the United States during the COVID-19 Pandemic","authors":"P. Thakuriah","doi":"10.32866/001c.72773","DOIUrl":"https://doi.org/10.32866/001c.72773","url":null,"abstract":"This paper examines changes in car-ownership levels before and after the COVID-19 pandemic in the US. In contrast to the two years before the pandemic, the propensity of households to be carless decreased for all households considered, as well as for low- and middle-income, and minority households. There is also evidence of an increase in the average number of vehicles for low-income households. The results highlight the additional financial burden faced by households during the pandemic as a result of higher levels of car-ownership, and that the recovery of public transportation ridership may be negatively impacted with the rise in car-ownership among transit-using groups.","PeriodicalId":73025,"journal":{"name":"Findings (Sydney (N.S.W.)","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-03-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45962195","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
During the COVID-19 early outbreak, the effects of built environment on individuals’ urban park visitation are understudied. We employed structural equation modeling to disentangle such complex relationships and investigate the impacts over time at the census tract level across the 100 largest American cities. The results show that density and design both explain park visitation, while only the dimension of density has major influences on the distance to urban parks.
{"title":"The Effect of Built Environment on Urban Park Visits during the Early Outbreak of COVID-19","authors":"Xiaoxiao Liu, Wei Zhai","doi":"10.32866/001c.73734","DOIUrl":"https://doi.org/10.32866/001c.73734","url":null,"abstract":"During the COVID-19 early outbreak, the effects of built environment on individuals’ urban park visitation are understudied. We employed structural equation modeling to disentangle such complex relationships and investigate the impacts over time at the census tract level across the 100 largest American cities. The results show that density and design both explain park visitation, while only the dimension of density has major influences on the distance to urban parks.","PeriodicalId":73025,"journal":{"name":"Findings (Sydney (N.S.W.)","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-03-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43798401","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Will mass adoption of remote work lead to a mass exodus from Silicon Valley? A survey of over 660 high-tech workers in the San Francisco Bay Area ran from November 2021 to March 2022, asking about relocation interest and intended destinations. 53% of respondents expressed interest in moving or already moved, with post-pandemic preferences shifting towards suburbs and stand-alone homes. However, most moves merely represented a dispersal into Bay Area suburbs with a median relocation distance of 33.68km. The number of expected days in the office, dictated by employer remote work policies, significantly impacted relocation interest.
{"title":"Post-Pandemic Relocation Preferences of Remote Tech Workers","authors":"Simon Tan, Kevin Fang, T. Lester","doi":"10.32866/001c.73259","DOIUrl":"https://doi.org/10.32866/001c.73259","url":null,"abstract":"Will mass adoption of remote work lead to a mass exodus from Silicon Valley? A survey of over 660 high-tech workers in the San Francisco Bay Area ran from November 2021 to March 2022, asking about relocation interest and intended destinations. 53% of respondents expressed interest in moving or already moved, with post-pandemic preferences shifting towards suburbs and stand-alone homes. However, most moves merely represented a dispersal into Bay Area suburbs with a median relocation distance of 33.68km. The number of expected days in the office, dictated by employer remote work policies, significantly impacted relocation interest.","PeriodicalId":73025,"journal":{"name":"Findings (Sydney (N.S.W.)","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-03-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48279320","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：