We are sharing notes from our regular collaborative journal club meetings—here are some updates from our last couple of meetings:
October 4, 2021:
In today’s journal club, we discussed a paper by Rajkomar et al (2018) in npj digital medicine titled “Scalable and accurate deep learning with electronic health records.” We discussed the approach taken within this study of using the entire electronic medical record data as the basis for deep learning prediction of medical events including inpatient mortality, extended length of stay, unplanned readmission, and discharge diagnoses. (The analysis was performed by Google with data from UCSF and UCM). The models leveraged FHIR-representations of the EHR data, avoiding specific selection of variables to use for prediction. They showed improvement over existing EHR-based predictive models that were used as baseline comparisons for the various prediction tasks.
We discussed our interest in learning more about how the performance of such models might vary for patients of different subgroups. The datasets on which the models were evaluated consisted of hospitalizations for broad sets of patients hospitalized for many different reasons. It would be helpful to compare these more general models’ performance to that of predictive models for patients with particular conditions or receiving particular treatments. In addition, it might be valuable to know whether particular subgroups of patients (by age, other demographics, reason for hospitalization) might present greater challenges for the models to predict outcomes. Another point of discussion was whether or not such models, once trained on data from a particular institution, might be generalizable to data at other institutions.
September 27, 2021:
In this session, we discussed a manuscript titled “Deep neural network improves fracture detection by clinicians” published in Proceedings of the National Academy of Sciences by Lindsey et. Al (collaboration between Imagen Tech and Hospital for Special Surgery).
Since Emergency medicine clinicians at the ER might not have the expertise to detect specific joint fractures, the authors decided to develop a deep learning model to detect wrist-related fractures from the X-ray images with the purpose of using the system in the emergency departments. Authors used the model’s prediction side-to-side the original image to help the clinicians, and demonstrated that the model performs both increases the detection accuracy, and speeds up the time-spent for diagnosis.
A point was raised in the discussion that how this model can be utilized, or even if it is needed. We discussed that some advanced hospitals like MGH have access to facilities and orthopedic surgeons that supervise clinicians before finalizing their decisions, and therefore, the fracture missing rate is smaller than other hospitals. We also talked about how model’s prediction might bias the clinicians to rely on the computer’s judgment instead of their own judgement, which makes having a highly-accurate and robust system even more important. We also discussed that the main limitation of studies such as this one is the closed-system (having in-house codes and datasets), which makes the 3rd party model validations impossible.