Methods of Information in Medicine最新文献

Background: Deep generative models (DGMs) present a promising avenue for generating realistic, synthetic data to augment existing health care datasets. However, exactly how the completeness of the original dataset affects the quality of the generated synthetic data is unclear.

Objectives: In this paper, we investigate the effect of data completeness on samples generated by the most common DGM paradigms.

Methods: We create both cross-sectional and panel datasets with varying missingness and subset rates and train generative adversarial networks, variational autoencoders, and autoregressive models (Transformers) on these datasets. We then compare the distributions of generated data with original training data to measure similarity.

Results: We find that increased incompleteness is directly correlated with increased dissimilarity between original and generated samples produced through DGMs.

Conclusions: Care must be taken when using DGMs to generate synthetic data as data completeness issues can affect the quality of generated data in both panel and cross-sectional datasets.

Introduction: Health care information systems can generate and/or record huge volumes of data, some of which may be reused for research, clinical trials, or teaching. However, these databases can be affected by data quality problems; hence, an important step in the data reuse process consists in detecting and rectifying these issues. With a view to facilitating the assessment of data quality, we developed a taxonomy of data quality problems in operational databases.

Material: We searched the literature for publications that mentioned "data quality problems," "data quality taxonomy," "data quality assessment," or "dirty data." The publications were then reviewed, compared, summarized, and structured using a bottom-up approach, to provide an operational taxonomy of data quality problems. The latter were illustrated with fictional examples (though based on reality) from clinical databases.

Results: Twelve publications were selected, and 286 instances of data quality problems were identified and were classified according to six distinct levels of granularity. We used the classification defined by Oliveira et al to structure our taxonomy. The extracted items were grouped into 53 data quality problems.

Discussion: This taxonomy facilitated the systematic assessment of data quality in databases by presenting the data's quality according to their granularity. The definition of this taxonomy is the first step in the data cleaning process. The subsequent steps include the definition of associated quality assessment methods and data cleaning methods.

Conclusion: Our new taxonomy enabled the classification and illustration of 53 data quality problems found in hospital databases.

Objective: To provide high-quality data for coronavirus disease 2019 (COVID-19) research, we validated derived COVID-19 clinical indicators and 22 associated machine learning phenotypes, in the Mass General Brigham (MGB) COVID-19 Data Mart.

Methods: Fifteen reviewers performed a retrospective manual chart review for 150 COVID-19-positive patients in the data mart. To support rapid chart review for a wide range of target data, we offered a natural language processing (NLP)-based chart review tool, the Digital Analytic Patient Reviewer (DAPR). For this work, we designed a dedicated patient summary view and developed new 127 NLP logics to extract COVID-19 relevant medical concepts and target phenotypes. Moreover, we transformed DAPR for research purposes so that patient information is used for an approved research purpose only and enabled fast access to the integrated patient information. Lastly, we performed a survey to evaluate the validation difficulty and usefulness of the DAPR.

Results: The concepts for COVID-19-positive cohort, COVID-19 index date, COVID-19-related admission, and the admission date were shown to have high values in all evaluation metrics. However, three phenotypes showed notable performance degradation than the positive predictive value in the prepandemic population. Based on these results, we removed the three phenotypes from our data mart. In the survey about using the tool, participants expressed positive attitudes toward using DAPR for chart review. They assessed that the validation was easy and DAPR helped find relevant information. Some validation difficulties were also discussed.

Conclusion: Use of NLP technology in the chart review helped to cope with the challenges of the COVID-19 data validation task and accelerated the process. As a result, we could provide more reliable research data promptly and respond to the COVID-19 crisis. DAPR's benefit can be expanded to other domains. We plan to operationalize it for wider research groups.

Background: Although the process-mining algorithms have evolved in the past decade, the lack of attention to extracting event logs from raw data of databases in an automatic manner is evident. These logs are available in a process-oriented manner in the process-aware information systems. Still, there are areas where their extraction is a challenge to address (e.g., trauma registries).

Objective: The registry data are recorded manually and follow an unstructured ad hoc pattern; prone to high noises and errors; consequently, registry logs are classified at a maturity level of one, and extracting process-centric information is not a trivial task therein. The experiences made during the event log building from the trauma registry are the subjects to be studied.

Results: The result indicates that the three-phase self-service registry log builder tool can withstand the mentioned issues by filtering and enriching the raw data and making them ready for any level of process-mining analysis. This proposed tool is demonstrated through process discovery in the National Trauma Registry of Iran, and the encountered challenges and limitations are reported.

Conclusion: This tool is an interactive visual event log builder for trauma registry data and is freely available for studies involving other registries. In conclusion, future research directions derived from this case study are suggested.

Objective: Our objective was to phenotype periodontal disease (PD) diagnoses from three different sections (diagnosis codes, clinical notes, and periodontal charting) of the electronic dental records (EDR) by developing two automated computer algorithms.

Methods: We conducted a retrospective study using EDR data of patients (n = 27,138) who received care at Temple University Maurice H. Kornberg School of Dentistry from January 1, 2017 to August 31, 2021. We determined the completeness of patient demographics, periodontal charting, and PD diagnoses information in the EDR. Next, we developed two automated computer algorithms to automatically diagnose patients' PD statuses from clinical notes and periodontal charting data. Last, we phenotyped PD diagnoses using automated computer algorithms and reported the improved completeness of diagnosis.

Results: The completeness of PD diagnosis from the EDR was as follows: periodontal diagnosis codes 36% (n = 9,834), diagnoses in clinical notes 18% (n = 4,867), and charting information 80% (n = 21,710). After phenotyping, the completeness of PD diagnoses improved to 100%. Eleven percent of patients had healthy periodontium, 43% were with gingivitis, 3% with stage I, 36% with stage II, and 7% with stage III/IV periodontitis.

Conclusions: We successfully developed, tested, and deployed two automated algorithms on big EDR datasets to improve the completeness of PD diagnoses. After phenotyping, EDR provided 100% completeness of PD diagnoses of 27,138 unique patients for research purposes. This approach is recommended for use in other large databases for the evaluation of their EDR data quality and for phenotyping PD diagnoses and other relevant variables.

Background: Generative pretrained transformer (GPT) models are one of the latest large pretrained natural language processing models that enables model training with limited datasets and reduces dependency on large datasets, which are scarce and costly to establish and maintain. There is a rising interest to explore the use of GPT models in health care.

Objective: We investigate the performance of GPT-2 and GPT-Neo models for medical text prediction using 374,787 free-text dental notes.

Methods: We fine-tune pretrained GPT-2 and GPT-Neo models for next word prediction on a dataset of over 374,000 manually written sections of dental clinical notes. Each model was trained on 80% of the dataset, validated on 10%, and tested on the remaining 10%. We report model performance in terms of next word prediction accuracy and loss. Additionally, we analyze the performance of the models on different types of prediction tokens for categories. For comparison, we also fine-tuned a non-GPT pretrained neural network model, XLNet (large), for next word prediction. We annotate each token in 100 randomly sampled notes by category (e.g., names, abbreviations, clinical terms, punctuation, etc.) and compare the performance of each model by token category.

Results: Models present acceptable accuracy scores (GPT-2: 76%; GPT-Neo: 53%), and the GPT-2 model also performs better in manual evaluations, especially for names, abbreviations, and punctuation. Both GPT models outperformed XLNet in terms of accuracy. The results suggest that pretrained models have the potential to assist medical charting in the future. We share the lessons learned, insights, and suggestions for future implementations.

Conclusion: The results suggest that pretrained models have the potential to assist medical charting in the future. Our study presented one of the first implementations of the GPT model used with medical notes.

Objectives: The TMF (Technology, Methods, and Infrastructure for Networked Medical Research) Data Protection Guide (TMF-DP) makes path-breaking recommendations on the subject of data protection in research projects. It includes comprehensive requirements for applications such as patient lists, pseudonymization services, and consent management services. Nevertheless, it lacks a structured, categorized list of requirements for simplified application in research projects and systematic evaluation. The 3LGM2IHE ("Three-layer Graphbased meta model - Integrating the Healthcare Enterprise [IHE] " ) project is funded by the German Research Foundation (DFG). 3LGM2IHE aims to define modeling paradigms and implement modeling tools for planning health care information systems. In addition, one of the goals is to create and publish 3LGM² information system architecture design patterns (short "design patterns") for the community as design models in terms of a framework. A structured list of data protection-related requirements based on the TMF-DP is a precondition to integrate functions (3LGM² Domain Layer) and building blocks (3LGM² Logical Tool Layer) in 3LGM² design patterns.

Methods: In order to structure the continuous text of the TMF-DP, requirement types were defined in a first step. In a second step, dependencies and delineations of the definitions were identified. In a third step, the requirements from the TMF-DP were systematically extracted. Based on the identified lists of requirements, a fourth step included the comparison of the identified requirements with exemplary open source tools as provided by the "Independent Trusted Third Party of the University Medicine Greifswald" (TTP tools).

Results: As a result, four lists of requirements were created, which contain requirements for the "patient list", the "pseudonymization service", and the "consent management", as well as cross-component requirements from the TMF-DP chapter 6 in a structured form. Further to requirements (1), possible variants (2) of implementations (to fulfill a single requirement) and recommendations (3) were identified. A comparison of the requirements lists with the functional scopes of the open source tools E-PIX (record linkage), gPAS (pseudonym management), and gICS (consent management) has shown that these fulfill more than 80% of the requirements.

Conclusions: A structured set of data protection-related requirements facilitates a systematic evaluation of implementations with respect to the fulfillment of the TMF-DP guidelines. These re-usable lists provide a decision aid for the selection of suitable tools for new research projects. As a result, these lists form the basis for the development of data protection-related 3LGM² design patterns as part of the 3LGM2IHE project.

Background: Management of child health care can be negatively affected by incomplete recording, low data quality, and lack of data integration of health management information systems to support decision making and public health program needs. Given the importance of identifying key determinants of child health via capturing and integrating accurate and high-quality information, we aim to address this gap through the development and testing requirements for an integrated child health information system.

Subjects and methods: A five-phase design thinking approach including empathizing, defining, ideation, prototyping, and testing was applied. We employed observations and interviews with the health workers at the primary health care network to identify end-users' challenges and needs using tools in human-centered design and focus group discussion. Then, a potential solution to the identified problems was developed as an integrated maternal and child health information system (IMCHIS) prototype and tested using Software Quality Requirements and Evaluation Model (SQuaRE) ISO/IEC 25000.

Results: IMCHIS was developed as a web-based system with 74 data elements and seven maternal and child health care requirements. The requirements of "child disease" with weight (0.26), "child nutrition" with weight (0.20), and "prenatal care" with weight (0.16) acquired the maximum weight coefficient. In the testing phase, the highest score with the weight coefficient of 0.48 and 0.73 was attributed to efficiency and functionality characteristics, focusing on software capability to fulfill the tasks that meet users' needs.

Conclusion: Implementing a successful child health care system integrates both maternal and child health care information systems to track the effect of maternal conditions on child health and support managing performance and optimizing service delivery. The highest quality score of IMCHIS in efficiency and functionality characteristics confirms that it owns the capability to identify key determinants of child health.