• Dear Readers,

    We’re delighted to present Atomic Academia.

    In autumn 🍂 and spring 🌸, we publish preprints to feature innovative research.
    In winter ❄️ expert authors write Atomic Articles reviewing the lastest developments in their fields.

    ⓘ Interested in contributing?
    Guidelines for authors
    📰 ISSN 3050-029X (Print) | 🖥️ ISSN 3050-0303 (Online)

Implications of Large Language Models in Medical Education

Literature Review
Recent literature has showcased Large Language Models (LLMs) like ChatGPT as increasingly capable tools within medical education. For example, Kung et al. (2023) demonstrate that ChatGPT, without specialized training, performs near the passing threshold for the USMLE, indicating its potential utility in medical education and perhaps even in clinical decision-making. Similarly, Gilson et al. (2023) evaluate ChatGPT's performance on USMLE Step 1 and Step 2, noting that it exceeds a 60% threshold on certain test sets, a performance equating to that of a third-year medical student. This study also highlights ChatGPT's ability to provide logical justifications and integrate internal and external information in its answers, which supports its potential use as an interactive medical education too.
Comparison to Existing Knowledge
The article reviewed aligns well with the cited works by discussing the utility of LLMs like ChatGPT in medical education. However, it appears to rely heavily on the perceived benefits without adequately addressing the significant limitations highlighted in prior studies, such as the potential for generating inaccurate or misleading medical advice and the challenges of keeping the AI's knowledge base current with the latest research and clinical guidelines, there are medical specific solutions that claim to have solved this (not GPT or Google based). While it recognises the issue of AI hallucinations, this acknowledgment is brief and lacks depth in discussing the implications for clinical practice and education.
Research Question and Conceptual Framework
The primary question the article seems to address is the feasibility and effectiveness of using LLMs in medical education and clinical settings. The article provides a general discussion but lacks a clear theoretical framework or specific hypotheses. This omission limits the ability to rigorously evaluate the claims made. The discussion would benefit from a more structured theoretical approach, perhaps by incorporating models of educational psychology or adult learning theories that could offer insights into how LLMs might enhance cognitive retention in medical students.
Methodology
The article's methodology is not explicitly defined, as it functions more as an overview or opinion piece rather than empirical research. This is a significant limitation, as it does not allow for replication or rigorous scientific evaluation. The findings and discussions presented are based on a narrative review of existing literature and anecdotal evidence rather than systematic data collection or analysis.
Novel Contributions and Implications
The article adds to the discourse by compiling examples of practical applications of LLMs in medical education, which could inform future research directions and educational policy. The insights into the potential of LLMs to assist in educational settings are valuable; however, they need to be substantiated with empirical evidence.
Constructive Feedback
To enhance the article's impact and credibility, the author could consider including a section on methodological approaches that could be used to study the impact of LLMs in medical education empirically. Additionally, a more balanced view that critically engages with both the capabilities and limitations of LLMs in clinical and educational settings would provide a more comprehensive overview of the subject. Future research should focus on longitudinal studies that can measure the long-term outcomes of LLM integration into medical curricula. Finally how were the papers used in this discussion identified?
Limitations
The article does not clearly differentiate its narrative review nature from empirical research, which may lead to misinterpretations about its conclusiveness. The theoretical and conceptual underpinnings are inadequately defined, which could be addressed by aligning the discussion with established educational theories.
Conclusion
While the article provides a pertinent overview of the potential applications of LLMs in medical education, it falls short in rigour and depth. The integration of a stronger theoretical framework, a clear methodology, and a balanced discussion of limitations would substantially enhance its value to the academic community. Nevertheless it is a good narrative review.
References
Kung, T.H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., & Tseng, V. (2023). Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health, 2(2), e0000198. https://doi.org/10.1371/journal.pdig.0000198

Gilson, A., Safranek, C.W., Huang, T., Socrates, V., Chi, L., Taylor, R.A., & Chartash, D. (2023). How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Medical Education, 9, e45312. https://doi.org/10.2196/45312
Credibility
An academic with a multiyear history in coding and development. Cryptography and AI active and the developer of the Atomic Academic AI. The comments left on this article are more general about replicability and rigour rather than subject specific to AI as it is a non-technical preprint review.
Last edited:
Back
Top