COVID-19 has created additional challenges for the analysis of non-randomized interventions in health system settings. Our objective is to evaluate these challenges and identify lessons learned from the analysis of a medically tailored meals (MTM) intervention at Kaiser Permanente Northwest (KPNW) that began in April 2020.
We identified both a historical and concurrent comparison group. The historical comparison group included patients living in the same area as the MTM recipients prior to COVID-19. The concurrent comparison group included patients admitted to contracted non-KPNW hospitals or admitted to a KPNW facility and living outside the service area for the intervention but otherwise eligible. We used two alternative propensity score methods in response to the loss of sample size with exact matching to evaluate the intervention.
We identified 452 patients who received the intervention, 3873 patients in the historical comparison group, and 5333 in the concurrent comparison group. We were able to mostly achieve balance on observable characteristics for the intervention and the two comparison groups.
Lessons learned included: (a) The use of two different comparison groups helped to triangulate results; (b) the meaning of utilization measures changed pre- and post-COVID-19; and (c) that balance on observable characteristics can be achieved, especially when the comparison groups are meaningfully larger than the intervention group. These findings may inform the design for future evaluations of interventions during COVID-19.
The Veterans Health Administration (VHA) is the largest integrated health system in the U.S. and has identified the learning health system as a strategic priority. Clinicians and staff engaging in active learning through continuous quality improvement (QI) is a key pillar for learning system maturity. An interdisciplinary frontline team at a VHA medical center participated in the Learn. Engage. Act. Process. (LEAP) virtual coaching program to learn how to conduct multidisciplinary team-based QI cycles of change. These clinicians lead and deliver the MOVE! weight management program, an evidence-based comprehensive lifestyle intervention. The team worked to continuously improve patient weight loss by engaging in incremental learning cycles of change. The aim of this study is to tell the story of this team's learning experience and the resulting positive reinforcing loop with patient outcomes.
This is a mixed methods case study description of one team that participated in the LEAP Program that provides hands-on QI learning for frontline teams with virtual coaching and a structured curriculum. Autoethnographic qualitative descriptions of team experiences over time illustrate this team's continued engagement in learning loops. Multilevel linear modeling was used to assess patient outcomes before vs after the team's participation in LEAP.
The team's participation in LEAP provided a set of fundamental QI skills and established a commitment to continual learning. Incremental improvements led to significant weight loss for patients who participated in MOVE! after the team completed LEAP (mean = 9.80 pounds; SD 10.43) compared to the pre-LEAP time period (mean = −6.83 pounds; SD 9.63).
Despite competing priorities and time limitations, this team's experiences provide a positive vision of how team engagement in data-driven continuous learning is feasible at the frontline and can lead to higher job satisfaction and stronger teams. These types of team activities provide much-needed backbone to being a mature learning health system.
The persisting and evolving COVID-19 pandemic has made apparent that no singular policy of mitigation at a regional, national or global level has achieved satisfactory and universally acceptable results. In the United States, carefully planned and executed pandemic policies have been neither effective nor popular and COVID-19 risk management decisions have been relegated to individual citizens and communities. In this paper, we argue that a more effective approach is to equip and strengthen community coalitions to become local learning health communities (LLHCs) that use data over time to make adaptive decisions that can optimize the equity and well-being in their communities.
We used data from the North Carolina (NC) county and zip code levels from May to August 2020 to demonstrate how a LLHC could use statistical process control (SPC) charts and simple statistical analysis to make local decisions about how to respond to COVID-19.
We found many patterns of COVID-19 progression at the local (county and zip code) levels during the same time period within the state that were completely different from the aggregate NC state level data used for policy making.
Systematic approaches to learning from local data to support effective decisions have promise well beyond the current pandemic. These tools can help address other complex public health issues, and advance outcomes and equity. Building this capacity requires investment in data infrastructure and the strengthening of data competencies in community coalitions to better interpret data with limited need for advanced statistical expertise. Additional incentives that build trust, support data transparency, encourage truth-telling and promote meaningful teamwork are also critical. These must be carefully designed, contextually appropriate and multifaceted to motivate citizens to create and sustain an effective learning system that works for their communities.
The learning health system (LHS) aligns science, informatics, incentives, stakeholders, and culture for continuous improvement and innovation. The Agency for Healthcare Research and Quality and the Patient-Centered Outcomes Research Institute designed a K12 initiative to grow the number of LHS scientists. We describe approaches developed by 11 funded centers of excellence (COEs) to promote partnerships between scholars and health system leaders and to provide mentored research training.
Since 2018, the COEs have enlisted faculty, secured institutional resources, partnered with health systems, developed and implemented curricula, recruited scholars, and provided mentored training. Program directors for each COE provided descriptive data on program context, scholar characteristics, stakeholder engagement, scholar experiences with health system partnerships, roles following program completion, and key training challenges.
To date, the 11 COEs have partnered with health systems to train 110 scholars. Nine (82%) programs partner with a Veterans Affairs health system and 9 (82%) partner with safety net providers. Clinically trained scholars (n = 87; 79%) include 70 physicians and 17 scholars in other clinical disciplines. Non-clinicians (n = 29; 26%) represent diverse fields, dominated by population health sciences. Stakeholder engagement helps scholars understand health system and patient/family needs and priorities, enabling opportunities to conduct embedded research, improve outcomes, and grow skills in translating research methods and findings into practice. Challenges include supporting scholars through roadblocks that threaten to derail projects during their limited program time, ranging from delays in access to data to COVID-19-related impediments and shifts in organizational priorities.
Four years into this novel training program, there is evidence of scholars' accomplishments, both in traditional academic terms and in terms of moving along career trajectories that hold the potential to lead and accelerate transformational health system change. Future LHS training efforts should focus on sustainability, including organizational support for scholar activities.
Enterprise data warehouses (EDWs) serve as foundational infrastructure in a modern learning health system, housing clinical and other system-wide data and making it available for research, strategic, and quality improvement purposes. Building on a longstanding partnership between Northwestern University's Galter Health Sciences Library and the Northwestern Medicine Enterprise Data Warehouse (NMEDW), an end-to-end clinical research data management (cRDM) program was created to enhance clinical data workforce capacity and further expand related library-based services for the campus.
The training program covers topics such as clinical database architecture, clinical coding standards, and translation of research questions into queries for proper data extraction. Here we describe this program, including partners and motivations, technical and social components, integration of FAIR principles into clinical data research workflows, and the long-term implications for this work to serve as a blueprint of best practice workflows for clinical research to support library and EDW partnerships at other institutions.
This training program has enhanced the partnership between our institution's health sciences library and clinical data warehouse to provide support services for researchers, resulting in more efficient training workflows. Through instruction on best practices for preserving and sharing outputs, researchers are given the tools to improve the reproducibility and reusability of their work, which has positive effects for the researchers as well as for the university. All training resources have been made publicly available so that those who support this critical need at other institutions can build on our efforts.
Library-based partnerships to support training and consultation offer an important vehicle for clinical data science capacity building in learning health systems. The cRDM program launched by Galter Library and the NMEDW is an example of this type of partnership and builds on a strong foundation of past collaboration, expanding the scope of clinical data support services and training on campus.
The Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program produces evidence reports to assist health systems in improving patient care. To make EPC evidence reports more accessible and usable, AHRQ convened a panel of learning health system (LHS) senior leaders to develop two web-based tools—a table-based tool presenting high-level results, and a graphical display that presented detailed data—to disseminate the reports and test the tools. Researchers examined (1) the context under which interview participants reviewed and used the evidence reported by the tools, (2) their experiences using the tools, (3) the tools' influence on clinical practice, and (4) how the tools could be improved.
Researchers collected and inductively analyzed qualitative data from tool implementation meetings with six LHSs and interviews with 27 LHS leaders and clinical staff who used the tools. Researchers used website utilization metrics to augment qualitative results.
The tools were efficient, complementary, and useful sources of summarized evidence to promote system change, educate trainees and clinicians, inform research, and support shared decision making with patients and families. Clinical leaders appreciated the evidence review thoroughness and quality and viewed AHRQ as a trusted source of information. However, many felt the tools were not practical for bedside use because of their complex content. Participants also noted the reports had limited evidence strength and robustness. They suggested optimizing the tools for mobile device use to facilitate tool uptake and developing training resources about tool navigation and statistical content interpretation.
LHSs found the tools to be useful resources for making the EPC Program reports more accessible to and usable for health system leaders. The tools have the potential to meet some, but not all, LHS evidence needs. Their value depends on reports' usefulness, which ultimately depends on the evidence quality.
Texas Health Resources (THR), a large, nonprofit health care system based in the Dallas-Fort Worth area, has collaborated with the University of Texas Southwestern Medical Center (UTSW) to develop and operate a unique, integrated approach for Learning Health System (LHS) workforce development. This training model centers on academic health system faculty members conducting later-stage translational research within a partnering regional care delivery organization.
The THR Clinical Scholars Program engages early career UTSW faculty members to conduct studies that are likely to have an impact on care delivery at the health system level. Interested candidates submit formal applications to the program. A joint committee comprised of senior research faculty from UTSW and THR clinical leadership reviews proposals with a focus on the shared LHS needs of both institutions—developing high quality research output that can be applied to enhance care delivery. A key prioritization criterion for funding is the degree to which the research addresses a question relevant to THR as a high-volume network with multiple channels for consumers to access care. The program emphasis is on supporting embedded research initiatives using health system data to generate knowledge that will improve the quality and efficiency of care for the patient populations served by the participant organizations.
We discuss specific strategic and tactical components of the THR Clinical Scholars Program including an overview of the academic affiliation agreement between the collaborating organizations, criteria for successful program applications, data sharing, and funding. We also share project summaries from selected clinical scholars as examples of the LHS research done in the program to date.
This experience report provides an implementation framework for other academic health systems interested in adopting similar LHS workforce training models with community partners.
To address Australian workforce needs, we developed a Learning Healthcare System (LHS) Academy fellowship program for clinicians. In the Academy, fellows complete foundational coursework, an LHS project, and other professional development deliverables to foster their future as digital health champions within their organizations. In this paper, we describe the 11-month-long program, as well as our evaluation results from the first 2 months of the program.
In the first week of the program, we sent all fellows an open-ended survey asking fellows to describe their digital health professional identities and what they expected to achieve from the fellowship program. At 2 months, we sent a follow-up open-ended survey that captured identical measures, their perceived barriers to participation in the program, perceived use of topics in the workplace and to their projects, and recommendations for program improvement. We analyzed the open text responses using qualitative content analysis, to identify categories of responses.
Overall, 2 months into the program, it was evident that participants were finding the teaching model engaging, useful, valuable, and applicable to their work and projects. Fellows perceived barriers to engagement in the program as balancing other commitments, lacking technical expertise, and having difficulty seeing themselves as leaders. Fellows expected that the program will allow them to implement new models of care, provide them with enough expertise to become leaders and champions in digital health, and become mentors for future generations. As far as changes in their professional identity, there was a notable increase in the number of fellows perceiving themselves as leaders.
Fellowship programs are one promising means of developing the healthcare workforce in LHS capabilities. Future studies should describe and evaluate LHS programs, to provide insights and recommendations for other educators interested in implementing similar programs of work within their own institutions.
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：