Knowledge and implementation of open science principles and behaviors remains uneven across the sciences, despite over 10 years of intensive education and advocacy. One reason for the slow and uneven progress of the open science movement is a set of closely held myths about the implications of open science practices, bolstered by recurring objections and arguments that have long been addressed. This paper covers three of these major recurring myths: 1) that open science conflicts with prioritizing diversity, 2) that “open data” is a binary choice between fully open and accessible and completely closed off, and 3) that preregistration is only appropriate for certain types of research designs. Putting these myths to rest is necessary as we work towards improving our scientific practice.
{"title":"Three Persistent Myths about Open Science","authors":"Moin Syed","doi":"10.36850/mr11","DOIUrl":"https://doi.org/10.36850/mr11","url":null,"abstract":"Knowledge and implementation of open science principles and behaviors remains uneven across the sciences, despite over 10 years of intensive education and advocacy. One reason for the slow and uneven progress of the open science movement is a set of closely held myths about the implications of open science practices, bolstered by recurring objections and arguments that have long been addressed. This paper covers three of these major recurring myths: 1) that open science conflicts with prioritizing diversity, 2) that “open data” is a binary choice between fully open and accessible and completely closed off, and 3) that preregistration is only appropriate for certain types of research designs. Putting these myths to rest is necessary as we work towards improving our scientific practice.","PeriodicalId":275817,"journal":{"name":"Journal of Trial and Error","volume":"34 17","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140728062","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Serendipity refers to the combination of “accident” and “sagacity”; an unexpected and unpredicted event which is noticed by an agent with the right skills to make the most of it. Famous examples include Jocelyn Bell’s discovery of pulsars which was made after she noticed an unusual output from a radio telescope (Arfini, 2023). Bell noticed and unpredicted output on the graphical trace and followed it up, eventually discovering the existence of pulsars. The rate of serendipitous discovery in science is unclear, although it has been estimated to be high (Thagard, 2012). This series is meant not only to add to the repertoire of serendipity stories, but to begin treating these tales as members in a growing archive, in which we attend to the role of chance and the unexpected in our rational pursuits of knowledge. Scientists here will share how accidents and reason intertwined in their practice, and researchers of serendipity will unpack how that happens.
{"title":"Serendipity in Scientific Research","authors":"Wendy Ross, Samantha Copeland, Stuart Feinstein","doi":"10.36850/v91j-7541","DOIUrl":"https://doi.org/10.36850/v91j-7541","url":null,"abstract":"Serendipity refers to the combination of “accident” and “sagacity”; an unexpected and unpredicted event which is noticed by an agent with the right skills to make the most of it. Famous examples include Jocelyn Bell’s discovery of pulsars which was made after she noticed an unusual output from a radio telescope (Arfini, 2023). Bell noticed and unpredicted output on the graphical trace and followed it up, eventually discovering the existence of pulsars. The rate of serendipitous discovery in science is unclear, although it has been estimated to be high (Thagard, 2012). This series is meant not only to add to the repertoire of serendipity stories, but to begin treating these tales as members in a growing archive, in which we attend to the role of chance and the unexpected in our rational pursuits of knowledge. Scientists here will share how accidents and reason intertwined in their practice, and researchers of serendipity will unpack how that happens.","PeriodicalId":275817,"journal":{"name":"Journal of Trial and Error","volume":"12 4","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-04-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140727903","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Seventy-six-year-old Rose was referred to me for music therapy with a diagnosis of residual schizophrenia. Rose was very passive and only wanted to listen to French chansons. After two years, I ended the therapy out of a belief that our music therapy sessions were not meaningful for her. About a year later, I took on more work hours and Rose surprisingly returned to the therapy room and requested to listen to music. I now realize that I was wrong to believe that the absence of active participation indicated a lack of meaning. A receptive state is not a passive state, and a relationship with a client can also be formed by listening to music together.
{"title":"The Music Must Play On: The Music Therapy Sessions that Should not Have Stopped","authors":"Ayelet Dassa","doi":"10.36850/mbmc-r441","DOIUrl":"https://doi.org/10.36850/mbmc-r441","url":null,"abstract":"Seventy-six-year-old Rose was referred to me for music therapy with a diagnosis of residual schizophrenia. Rose was very passive and only wanted to listen to French chansons. After two years, I ended the therapy out of a belief that our music therapy sessions were not meaningful for her. About a year later, I took on more work hours and Rose surprisingly returned to the therapy room and requested to listen to music. I now realize that I was wrong to believe that the absence of active participation indicated a lack of meaning. A receptive state is not a passive state, and a relationship with a client can also be formed by listening to music together.","PeriodicalId":275817,"journal":{"name":"Journal of Trial and Error","volume":"29 S1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2023-12-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"139166172","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Folco Panizza, P. Ronzani, Carlos Martini, Lucia Savadori, Matteo Motterlini
In this report we outline the null findings of a pre-registered experiment on vaccine hesitancy in the United Kingdom. The experiment targeted vaccine misconceptions common among participants by presenting a correction to such claims endorsed by a group of medical experts. The experiment had the aim to increase vaccination intention and actual uptake during the 2021 COVID-19 vaccination campaign. Our results revealed that, contrary to a similar study conducted with Italian residents, our intervention was unsuccessful in changing participants’ attitudes and behaviour towards COVID-19 vaccines. The report concludes with a discussion of the potential reasons for these null findings.
{"title":"Medical Expert Endorsement Fails to Reduce Vaccine Hesitancy in U.K. Residents","authors":"Folco Panizza, P. Ronzani, Carlos Martini, Lucia Savadori, Matteo Motterlini","doi":"10.31234/osf.io/gh5vy","DOIUrl":"https://doi.org/10.31234/osf.io/gh5vy","url":null,"abstract":"In this report we outline the null findings of a pre-registered experiment on vaccine hesitancy in the United Kingdom. The experiment targeted vaccine misconceptions common among participants by presenting a correction to such claims endorsed by a group of medical experts. The experiment had the aim to increase vaccination intention and actual uptake during the 2021 COVID-19 vaccination campaign. Our results revealed that, contrary to a similar study conducted with Italian residents, our intervention was unsuccessful in changing participants’ attitudes and behaviour towards COVID-19 vaccines. The report concludes with a discussion of the potential reasons for these null findings.","PeriodicalId":275817,"journal":{"name":"Journal of Trial and Error","volume":"8 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-09-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127980591","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Arielle Bennett, Daniel Garside, Cassandra Gould van Praag, T. Hostler, Ismael Kherroubi Garcia, E. Plomp, Antonio Schettino, S. Teplitzky, Hao Ye
The Scientific Reform Movement has highlighted the need for large research teams with diverse skills. This has necessitated the growth of professional team infrastructure roles (TIRs) who support research through specialised skills, but do not have primary responsibility for conceiving or leading research projects. TIRs such as Lab Technicians, Project Managers, Data Stewards, Community Managers, and Research Software Engineers all play an important role in ensuring the success of a research project, but are commonly neglected under current reward and recognition procedures, which focus on the individual academic researcher instead of the teams involved. Without meaningful identification and recognition of TIR contributions, we risk reinforcing the conceptual and practical division between academic researchers and TIRs. This situation is inequitable and detrimental to the research enterprise: the limited potential for career advancement for TIRs may cause them to leave for other occupations, ultimately leading to a loss of institutional skill, expertise, and memory. This contribution explores the evolution of specialist TIRs and the status of these positions in various settings. We provide three case study descriptions of TIR activities, so that readers may become more familiar with the breadth and depth of their work. We then propose system level changes designed to embed meaningful recognition of all contributions. Acknowledging the contributions of all research roles will help retain skill and expertise, and lead to collaborative research ecosystems that are well-positioned to address complex research challenges.
{"title":"A Manifesto for Rewarding and Recognizing Team Infrastructure Roles","authors":"Arielle Bennett, Daniel Garside, Cassandra Gould van Praag, T. Hostler, Ismael Kherroubi Garcia, E. Plomp, Antonio Schettino, S. Teplitzky, Hao Ye","doi":"10.36850/mr8","DOIUrl":"https://doi.org/10.36850/mr8","url":null,"abstract":"The Scientific Reform Movement has highlighted the need for large research teams with diverse skills. This has necessitated the growth of professional team infrastructure roles (TIRs) who support research through specialised skills, but do not have primary responsibility for conceiving or leading research projects. TIRs such as Lab Technicians, Project Managers, Data Stewards, Community Managers, and Research Software Engineers all play an important role in ensuring the success of a research project, but are commonly neglected under current reward and recognition procedures, which focus on the individual academic researcher instead of the teams involved.\u0000\u0000Without meaningful identification and recognition of TIR contributions, we risk reinforcing the conceptual and practical division between academic researchers and TIRs. This situation is inequitable and detrimental to the research enterprise: the limited potential for career advancement for TIRs may cause them to leave for other occupations, ultimately leading to a loss of institutional skill, expertise, and memory.\u0000\u0000This contribution explores the evolution of specialist TIRs and the status of these positions in various settings. We provide three case study descriptions of TIR activities, so that readers may become more familiar with the breadth and depth of their work. We then propose system level changes designed to embed meaningful recognition of all contributions. Acknowledging the contributions of all research roles will help retain skill and expertise, and lead to collaborative research ecosystems that are well-positioned to address complex research challenges.","PeriodicalId":275817,"journal":{"name":"Journal of Trial and Error","volume":"43 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-08-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"126137662","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Crystal N. Steltenpohl, Hilary Lustick, M. S. Meyer, L. E. Lee, Sondra M. Stegenga, Laurel Standiford Reyes, R. Renbarger
Discussions around transparency in open science focus primarily on sharing data, materials, and coding schemes, especially as these practices relate to reproducibility. This fairly quantitative perspective of transparency does not align with all scientific methodologies. Indeed, qualitative researchers also care deeply about how knowledge is produced, what factors influence the research process, and how to share this information. Explicating a researcher’s background and role allows researchers to consider their impact on the research process and interpretation of the data, thereby increasing both transparency and rigor. Researchers may engage in positionality and reflexivity in a variety of ways, and transparently sharing these steps allows readers to draw their own informed conclusions about the results and study as a whole. Imposing a limited, quantitatively-informed set of standards on all research can cause harm to researchers and the communities they work with if researchers are not careful in considering the impact of such standards. Our paper will argue the importance of avoiding strong defaults around transparency (e.g., always share data) and build upon previous work around qualitative open science. We explore how transparency in all aspects of our research can lend itself toward projecting and confirming the rigor of our work.
{"title":"Rethinking Transparency and Rigor from a Qualitative Open Science Perspective","authors":"Crystal N. Steltenpohl, Hilary Lustick, M. S. Meyer, L. E. Lee, Sondra M. Stegenga, Laurel Standiford Reyes, R. Renbarger","doi":"10.36850/mr7","DOIUrl":"https://doi.org/10.36850/mr7","url":null,"abstract":"Discussions around transparency in open science focus primarily on sharing data, materials, and coding schemes, especially as these practices relate to reproducibility. This fairly quantitative perspective of transparency does not align with all scientific methodologies. Indeed, qualitative researchers also care deeply about how knowledge is produced, what factors influence the research process, and how to share this information. Explicating a researcher’s background and role allows researchers to consider their impact on the research process and interpretation of the data, thereby increasing both transparency and rigor. Researchers may engage in positionality and reflexivity in a variety of ways, and transparently sharing these steps allows readers to draw their own informed conclusions about the results and study as a whole. Imposing a limited, quantitatively-informed set of standards on all research can cause harm to researchers and the communities they work with if researchers are not careful in considering the impact of such standards. Our paper will argue the importance of avoiding strong defaults around transparency (e.g., always share data) and build upon previous work around qualitative open science. We explore how transparency in all aspects of our research can lend itself toward projecting and confirming the rigor of our work.","PeriodicalId":275817,"journal":{"name":"Journal of Trial and Error","volume":"2012 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-17","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"127390544","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Clinical trials are routinely preregistered. In psychology and the social sciences, however, only a small percentage of studies are preregistered, and those preregistrations often contain ambiguities. As advocates strive for broader uptake and effective use of preregistration, they can benefit from drawing on the experience of preregistration in clinical trials and adapting some of those successes to the psychology and social sciences context. We recommend that individuals and organizations who promote preregistration: (1) Establish core preregistration criteria required to consider a preregistration complete; (2) Award preregistered badges only to articles that meet the badge criteria; and (3) Leverage complementary workflows that provide a similar function as preregistration.
{"title":"Reflections on Preregistration: Core Criteria, Badges, Complementary Workflows","authors":"Robert T. Thibault, C. Pennington, M. Munafo","doi":"10.36850/mr6","DOIUrl":"https://doi.org/10.36850/mr6","url":null,"abstract":"Clinical trials are routinely preregistered. In psychology and the social sciences, however, only a small percentage of studies are preregistered, and those preregistrations often contain ambiguities. As advocates strive for broader uptake and effective use of preregistration, they can benefit from drawing on the experience of preregistration in clinical trials and adapting some of those successes to the psychology and social sciences context. We recommend that individuals and organizations who promote preregistration: (1) Establish core preregistration criteria required to consider a preregistration complete; (2) Award preregistered badges only to articles that meet the badge criteria; and (3) Leverage complementary workflows that provide a similar function as preregistration.","PeriodicalId":275817,"journal":{"name":"Journal of Trial and Error","volume":"131 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"134430480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
It is acknowledged that conducting open research requires additional time and effort compared to conducting ‘closed’ research. However, this additional work is often discussed only in abstract terms, a discourse which ignores the practicalities of how researchers are expected to find the time to engage with these practices in the context of their broader role as multifaceted academics. In the context of a sector that is blighted by stress, burnout, untenable workloads, and hyper-competitive pressures to produce, there is a clear danger that additional expectations to engage in open practices add to the workload burden and increase pressure on academics even further. In this article, the theories of academic capitalism and workload creep are used to explore how workload models currently exploit researchers by mismeasuring academic labour. The specific increase in workload resulting from open practices and associated administration is then outlined, including via the cumulative effects of administrative burden. It is argued that there is a high chance that without intervention, increased expectations to engage in open research practices may lead to unacceptable increases in demands on academics. Finally, the individual and systematic responsibilities to mitigate this are discussed.
{"title":"The Invisible Workload of Open Research","authors":"T. Hostler","doi":"10.36850/mr5","DOIUrl":"https://doi.org/10.36850/mr5","url":null,"abstract":"It is acknowledged that conducting open research requires additional time and effort compared to conducting ‘closed’ research. However, this additional work is often discussed only in abstract terms, a discourse which ignores the practicalities of how researchers are expected to find the time to engage with these practices in the context of their broader role as multifaceted academics. In the context of a sector that is blighted by stress, burnout, untenable workloads, and hyper-competitive pressures to produce, there is a clear danger that additional expectations to engage in open practices add to the workload burden and increase pressure on academics even further. In this article, the theories of academic capitalism and workload creep are used to explore how workload models currently exploit researchers by mismeasuring academic labour. The specific increase in workload resulting from open practices and associated administration is then outlined, including via the cumulative effects of administrative burden. It is argued that there is a high chance that without intervention, increased expectations to engage in open research practices may lead to unacceptable increases in demands on academics. Finally, the individual and systematic responsibilities to mitigate this are discussed.","PeriodicalId":275817,"journal":{"name":"Journal of Trial and Error","volume":"42 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130294316","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Håvard Toft, Alexey Sirotkin, Markus Landrø, R. Engeset, J. Hendrikx
Outdoor recreation continues to increase in popularity. In Norway, several avalanche fatalities are recorded every year, but the accurate calculation of a fatal accident rate is impossible without knowing how many people are exposed. We attempted to employ signaling data from telecom network data to enumerate backcountry travelers in avalanche terrain. Each signaling data event contains information about which coverage area the phone is connected to and timestamp. There is no triangulation, making it impossible to know whether the associated phone is moving or stationary within the coverage area. Hence, it's easier to track the phone's movement through different coverage areas. We utilize this by enumerating the number of people with phones traveling to avalanche-prone terrain for the 2019/2020 winter season. We estimated that 13,666 phones were in avalanche terrain during the season, ranging from 0 to 118 phones/day with an average of 75 phones/day. We correlated the number of phones per day against amount of daylight (R2=0.186, p-value <0.01), weekends and holidays (R2=0.073, p-value <0.01), number of bulletin views (R2=0.045, p-value <0.01). Unfortunately, the validation revealed discrepancies between the estimated positions in the mobile network and the true reference positions as collected with a GPS. We attribute this to the algorithm being designed to measure urban mobility and the long distance between the base transceiver stations in mountainous areas. This lack of coherence between the signaling data and GPS records for rural areas in Norway has implication for the utility of signaling data outside of urban regions.
{"title":"Challenges of Using Signaling Data From Telecom Network in Non-Urban Areas","authors":"Håvard Toft, Alexey Sirotkin, Markus Landrø, R. Engeset, J. Hendrikx","doi":"10.36850/e14","DOIUrl":"https://doi.org/10.36850/e14","url":null,"abstract":"Outdoor recreation continues to increase in popularity. In Norway, several avalanche fatalities are recorded every year, but the accurate calculation of a fatal accident rate is impossible without knowing how many people are exposed. We attempted to employ signaling data from telecom network data to enumerate backcountry travelers in avalanche terrain. Each signaling data event contains information about which coverage area the phone is connected to and timestamp. There is no triangulation, making it impossible to know whether the associated phone is moving or stationary within the coverage area. Hence, it's easier to track the phone's movement through different coverage areas. We utilize this by enumerating the number of people with phones traveling to avalanche-prone terrain for the 2019/2020 winter season. We estimated that 13,666 phones were in avalanche terrain during the season, ranging from 0 to 118 phones/day with an average of 75 phones/day. We correlated the number of phones per day against amount of daylight (R2=0.186, p-value <0.01), weekends and holidays (R2=0.073, p-value <0.01), number of bulletin views (R2=0.045, p-value <0.01). Unfortunately, the validation revealed discrepancies between the estimated positions in the mobile network and the true reference positions as collected with a GPS. We attribute this to the algorithm being designed to measure urban mobility and the long distance between the base transceiver stations in mountainous areas. This lack of coherence between the signaling data and GPS records for rural areas in Norway has implication for the utility of signaling data outside of urban regions.","PeriodicalId":275817,"journal":{"name":"Journal of Trial and Error","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-05-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"130124838","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Questionable research practices may reduce the public’s trust in science. The present article considers some questionable metascience practices (QMPs) that may threaten scientists’ trust in metascience. A QMP is a research practice, assumption, or perspective that has been questioned by several commentators as being potentially problematic for the credibility of metascience and/or the science reform movement. The present article reviews 10 QMPs that relate to criticism, replication, bias, generalization, and the characterization of science. Specifically, the following QMPs are considered: (1) rejecting or ignoring self-criticism; (2) a fast ‘n’ bropen scientific criticism style; (3) overplaying the role of replication in science; (4) assuming a replication rate is “too low” without specifying an “acceptable” rate; (5) an unacknowledged metabias towards explaining the replication crisis in terms of researcher bias; (6) assuming that researcher bias can be reduced; (7) devaluing exploratory results as more “tentative” than confirmatory results; (8) presuming that QRPs are problematic research practices; (9) focusing on knowledge accumulation as an index of scientific progress; and (10) focusing on specific scientific methods. It is stressed that only some metascientists engage in some QMPs some of the time, and that these QMPs may not always be problematic. Research is required to estimate the prevalence and impact of QMPs. In the meantime, QMPs should be viewed as invitations to ask “questions” about how we go about doing metascience rather than as grounds for mistrusting the credibility of metascience.
{"title":"Questionable Metascience Practices","authors":"Mark Rubin","doi":"10.36850/mr4","DOIUrl":"https://doi.org/10.36850/mr4","url":null,"abstract":"Questionable research practices may reduce the public’s trust in science. The present article considers some questionable metascience practices (QMPs) that may threaten scientists’ trust in metascience. A QMP is a research practice, assumption, or perspective that has been questioned by several commentators as being potentially problematic for the credibility of metascience and/or the science reform movement. The present article reviews 10 QMPs that relate to criticism, replication, bias, generalization, and the characterization of science. Specifically, the following QMPs are considered: (1) rejecting or ignoring self-criticism; (2) a fast ‘n’ bropen scientific criticism style; (3) overplaying the role of replication in science; (4) assuming a replication rate is “too low” without specifying an “acceptable” rate; (5) an unacknowledged metabias towards explaining the replication crisis in terms of researcher bias; (6) assuming that researcher bias can be reduced; (7) devaluing exploratory results as more “tentative” than confirmatory results; (8) presuming that QRPs are problematic research practices; (9) focusing on knowledge accumulation as an index of scientific progress; and (10) focusing on specific scientific methods. It is stressed that only some metascientists engage in some QMPs some of the time, and that these QMPs may not always be problematic. Research is required to estimate the prevalence and impact of QMPs. In the meantime, QMPs should be viewed as invitations to ask “questions” about how we go about doing metascience rather than as grounds for mistrusting the credibility of metascience.","PeriodicalId":275817,"journal":{"name":"Journal of Trial and Error","volume":"1 1","pages":"0"},"PeriodicalIF":0.0,"publicationDate":"2023-04-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"122987699","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：