Frontiers in research metrics and analytics最新文献

Integrating generative AI (GenAI) in qualitative research offers innovation but intensifies core epistemological, ontological, and ethical challenges. This article conceptualizes the meta-crisis of generativity-a convergence of Denzin and Lincoln's three crises: representation (blurring human/AI authorship), legitimation (questioning trust in AI-generated claims), and praxis (ambiguity in non-human participation). We examine how human-GenAI collaboration challenges researchers' voice, knowledge validity, and ethical agency across research paradigms. To navigate this, we propose strategic approaches: preserving positionality via voice annotation and reflexive bracketing (representation); ensuring trustworthiness through algorithmic audits and adapted validity checklists (legitimation); and redefining agency via participatory transparency and posthuman ethics (praxis). Synthesizing these, we expand qualitative rigor criteria-such as credibility and reflexivity-into collaborative frameworks that emphasize algorithmic accountability. The meta-crisis is thus an invitation to reanimate the critical ethos of qualitative research through interdisciplinary collaboration, balancing the potential of GenAI with ethical accountability while preserving humanistic foundations.

Traditional bibliometric approaches to research impact assessment have predominantly relied on citation counts, overlooking the qualitative dimensions of how research is received and discussed. Altmetrics have expanded this perspective by capturing mentions across diverse platforms, yet most analyses remain limited to quantitative measures, failing to account for sentiment. This study aimed to introduce a novel artificial intelligence-driven sentiment analysis framework designed to evaluate the tone and intent behind research mentions on social media, with a primary focus on X (formerly Twitter). Our approach leverages a bespoke sentiment classification system, spanning seven levels from strong negative to strong positive, to capture the nuanced ways in which research is endorsed, critiqued, or debated. Using a machine learning model trained on 5,732 manually curated labels (ML2024) as a baseline (F1 score = 0.419), we developed and refined a Large Language Model (LLM)-based classification system through three iterative rounds of expert evaluation. The final AI-driven model demonstrated improved alignment with human assessments, achieving an F1 score of 0.577, significantly enhancing precision and recall over traditional methods. These findings underscore the potential of advanced AI methodologies in altmetric analysis, offering a richer, more context-aware understanding of research reception. This study laid the foundation for integrating sentiment analysis into Altmetric platforms, providing researchers, institutions, and policymakers with deeper insights into the societal discourse surrounding scientific outputs.

Introduction: The Washington Group Short Set (WG-SS) is a widely used tool for identifying disability in national and international population-based surveys. However, results from cognitive testing revealed key differences in response patterns between individuals who self-report and those with a proxy respondent. Considering proxy reporting is frequently used in national surveys, discrepancies between reporting sources could affect the accuracy of disability prevalence estimates and have important implications for health equity and policy.

Methods: A binary logistic regression was conducted to examine the relationship between proxy respondents and WG-SS disability status after controlling for sociodemographic characteristics, using pooled data from the 2010-2018 National Health Interview Survey (NHIS).

Results: After controlling for sociodemographic characteristics, proxy respondents were 4.48 times more likely to be classified as having a WG-SS disability compared to those who self-reported.

Discussion: Differences in proxy reporting have real implications for equity, access, and policy accountability. If proxy reporting systematically increases the likelihood of disability classification, prevalence estimates may be distorted. This is especially problematic when proxies are more likely to report for populations already at risk of under- or overrepresentation in disability data, such as older adults, people with cognitive disabilities, and children and adolescents. Future studies using the WG-SS should treat the reporting source, i.e., proxy response, not as a procedural footnote, but as a central variable in assessing data quality and equity.

Introduction: The primary objective of this study was to investigate the association between journal Impact Factor (IF) and study quality in real-world observational studies. The secondary objective was to explore whether the association changes as a function of different study factors (study design, funding type and geographic location).

Methods: Study quality was assessed using the Newcastle-Ottawa Scale (NOS). IFs were obtained from journal websites. The association between journal IF and NOS score was evaluated firstly using Spearman's correlation coefficient, and secondly using one-way Analysis of Variance (ANOVA).

Results: We selected 457 studies published in 208 journals across 11 consecutive systematic literature reviews (SLRs) conducted at our organization over the last 5 years. Most studies were cross-sectional and from North America or Europe. Mean (SD) NOS score was 6.6 (1.03) and mean (SD) IF was 5.2 (4.5). Overall, there was a weak positive correlation between NOS score and IF (Spearman's coefficient (ρ) = 0.23 [95% CI: 0.13-0.31]; p < 0.001). There was no correlation between NOS score and IF for prospective cohort studies (ρ = 0.07 [95% CI:-0.12-0.25]) and industry-funded studies (ρ = 0.06 [95% CI:-0.09-0.21]). Based on ANOVA, the effect size, eta squared (η²), was 0.04 (95% CI: 0.01-0.08), indicating a small effect.

Discussion: While there is some correlation between journal quality and study quality, our findings indicate that high-quality research can be found in journals with lower IF, and assessing study quality requires careful review of study design, methodology, analysis, interpretation, and significance of the findings. Notably, in industry-funded studies, no correlation was found between methodological quality and IF.

Background: Leucine-rich repeat-containing G-protein-coupled receptor 5 (LGR5) is involved in canonical Wnt/β-catenin signaling and is a marker of stem cells in several tissues. It plays an important role in self-renewal, metastasis, and chemoresistance of tumors. The plasticity and potential of LGR5 (+) cancer stem cells could provide therapeutic targets for cancer. However, the data in this field is very limited and requires further investigation.

Methods: This study aimed to explore the status and evolutionary trends of LGR5 research using bibliometric analysis. In total, 2,187 publications were retrieved from the Web of Science Core Collection. The period covered by the articles was from 1999 to 2023. CiteSpace, VOSviewer, R software, and Bibliometric Online Analysis Platform were used for bibliometric analysis and visualization.

Results: The USA was the most productive country, with the highest centrality and largest single-country publications. The Netherlands was the earliest country to conduct LGR5 research. Cleavers, H from the Hubrecht Institute (KNAW) of the Netherlands was the most influential author as measured by H, G, and M-index values and contributions to the number of publications and citations. Intestinal stem cells were a hot topic, while keywords "LGR5 (+) stem cells," "inflammation," and "tumor microenvironment" exhibited the strongest burst in recent years, indicating a significant research focus in the future. Additionally, targeting LGR5 (+) stem cells in a specific phase of cancer and in combination with tumor microenvironment (TME) combination could be a future hotspot.

Conclusion: Research on LGR5 continues to develop through active global efforts. This study offers a comprehensive analysis of LGR5 using bibliometric and visual methods, highlighting current research hotspots and potential directions for researchers interested in the field.

Global citation-based databases, such as Stanford University's Top 2% Scientists (SUD2%) database, offer powerful tools for tracking high-impact researchers. Despite Saudi Arabia's growing investment in scientific research, a longitudinal analysis of its presence in these elite citation rankings has been lacking. This study provides the first 5-year analysis (2019-2023) of Saudi-affiliated scientists listed in the SUD2% (single-year category), evaluating their growth in numbers, performance indicators, disciplinary distribution, and gender representation. Data were extracted from Elsevier's Mendeley-hosted SUD2% dataset. The key bibliometric metrics under analysis included the average composite citation score (C-score), citation rank, total citations, and h-index. A one-way repeated measures ANOVA on ranks was used to assess statistical differences between Saudi-affiliated and global scientists. Gender classification was performed using NamSor, based on validated confidence thresholds. The number of Saudi-affiliated scientists in the SUD2% nearly tripled from 556 in 2019 to 1,684 in 2023. Significant gains were also observed in average C-scores (p = 0.003), citation ranks (p = 0.002), total citations (p = 0.001), and h-indices (p = 0.025). Disciplinary analysis revealed continued dominance in clinical medicine, chemistry, and biomedical research. Gender analysis revealed male dominance (93.9%) over the 5-year period, although female representation increased from 5.0% in 2019 to 7.3% in 2023. Saudi Arabia's scientific community is making statistically significant progress in high-impact research, evidenced by increasing representation and improved citation metrics in global SUD2% rankings. While gaps remain-particularly in gender representation and individual citation ranks-trends point toward sustained momentum and broadening institutional participation in global research excellence.

We describe the Dimensions Search Language (DSL), a domain-specific language for bibliographic and scientometrics analysis. The DSL is the main component of the Dimensions API (version 2.12.0), which provides end-users with a powerful, yet simple-to-learn and use, tool to search, filter, and analyze the Dimensions database using a single entry point and query language. The DSL is the result of an effort to model the way researchers and analysts describe research questions in this domain, as opposed to using established paradigms commonly used by software developers e.g., REST or SOAP. In this article, we describe the API architecture, the DSL main features, and the core data model. We describe how it is used by researchers and analysts in academic and business settings alike to carry out complex research analytics tasks, like calculating the H-index of a researcher or generating a publications' citation network.

This article adopts a constructivist grounded theory approach based on the principle of intersubjective relations and the co-construction of interpretations. Reflecting on the author's experiences as a tutor, supervisor, examiner, and reviewer to demystify qualitative data analysis (QDA), this article emphasizes member checking with participants to confirm the researchers' interpretations and collaboratively constructed meanings, addressing reflexivity, peer debriefing, and triangulation. QDA is framed as an iterative, dynamic process of extracting meaning from diverse data forms (field text, narrative, voice, reflective note, text, audio, and video). The procedures of data analysis and/or writing as a process of inquiry, such as data immersion, initial impressions, codes, categories, and theme developments, are explored across methods or methodologies, including autoethnography, participatory action research (PAR), narrative inquiry, grounded theory, phenomenology, ethnography, case study, and other alternative research methods or methodologies, with data saturation as the final stepping point when no new data and/or insights from field text, narrative, voice, reflective note, text, audio, or video are extracted. Challenges like generating an intersubjective construction of meaning with research participants and achieving data saturation are addressed through methods such as reflexivity, peer debriefing, member checking and triangulation. This article provides a practical guide for scholars on data analysis, incorporating reflections, procedures, and some points for consideration to ensure rigor and meaningful analysis.

Background: The Clinical and Translational Science Award (CTSA) program, funded by the National Center for Advancing Translational Sciences (NCATS), has supported over 65 hubs, generating 118,490 publications from 2006 to 2021. Measuring the impact of these outputs remains challenging, as traditional bibliometric methods fail to capture patents, policy contributions, and clinical implementation. The Translational Science Benefits Model (TSBM) provides a structured framework for assessing clinical, community, economic, and policy benefits, but its manual application is resource-intensive. Advances in Natural Language Processing (NLP) and Artificial Intelligence (AI) offer a scalable solution for automating benefit extraction from large research datasets.

Objective: This study presents an NLP-driven pipeline that automates the extraction of TSBM benefits from research outputs using Latent Dirichlet Allocation (LDA) topic modeling to enable efficient, scalable, and reproducible impact analysis. The application of NLP allows the discovery of topics and benefits to emerge from the very large corpus of CTSA documents without requiring directed searches or preconceived benefits for data mining.

Methods: We applied LDA topic modeling to publications, patents, and grants and mapped the topics to TSBM benefits using subject matter expert (SME) validation. Impact visualizations, including heatmaps and t-SNE plots, highlighted benefit distributions across the corpus and CTSA hubs.

Results: Spanning CTSA hub grants awarded from 2006 to 2023, our analysis corpus comprised 1,296 projects, 127,958 publications and 352 patents. Applying our NLP-driven pipeline to deduplicated data, we found that clinical and community benefits were the most frequently extracted benefits from publications and projects, reflecting the patient-centered and community-driven nature of CTSA research. Economic and policy benefits were less frequently identified, prompting the inclusion of patent data to better capture commercialization impacts. The Publications LDA Model proved the most effective for benefit extraction for publications and projects. All patents were automatically tagged as economic benefits, given their intrinsic focus on commercialization and in accordance with TSBM guidelines.

Conclusion: Automated NLP-driven benefit extraction enabled a data-driven approach to applying the TSBM at the scale of the entire CTSA program outputs.

Introduction: In a global landscape characterized by intense competition and stringent funding criteria, researchers face the dual challenges of limited resources and high demand for innovation-a challenge that Brazil is no exception to. This study aimed to explore the perceptions, barriers, and challenges faced by researchers during the project submission process for approval by funding agencies, with a focus on schools within the Federal Network of Professional, Scientific, and Technological Education Institutions.

Methods: A quantitative cross-sectional approach was used to examine the characteristics of researchers at a Brazilian institution in 2023. The sample comprised eighty three researchers who completed an online questionnaire containing eighty three questions on demographic characteristics, factors associated with project submission and approval, and reasons for non-submission or non-approval. The data were analyzed using descriptive statistics, including the Kolmogorov-Smirnov, Pearson's chi-square, and Mann-Whitney U-tests, followed by post hoc analysis and Yates' correction. Logistic regression was applied using the backward elimination method, and significant parameters (p < 0.20) free from multicollinearity were selected.

Results: This study revealed that most researchers were men (61.4%) with doctoral degrees (91.6%), highlighted the critical role of proposal clarity and relevance in the project evaluation process. Gender (p = 0.011) and academic level (p = 0.025) were significant factors influencing project submission rates, with Brazilian National Council for Scientific and Technological Development (CNPq) fellows and researchers involved in graduate programs submitting more projects. The participants identified "search for funding" and "desire to expand research impact" as their primary motivations while citing "complex funding calls" and "funding limitations" as major barriers. Additionally, age and the number of children were found to affect project approval (p ≤ 0.018), with "proposal clarity" and "researchers' experience" having been critical factors for submission approval (p ≤ 0.03).

Conclusion: The study results highlighted a gender disparity, with lower participation among women, and identified key factors influencing project submission, including the search for funding, curriculum development, and structural challenges. Additionally, the findings suggest the adoption of gender-sensitive and early-career grant criteria, targeted support for underrepresented researchers, and flexible mechanisms for those with caregiving responsibilities. These findings underscore the importance of public policies and institutional strategies in promoting equitable and inclusive funding opportunities.